QA/Sync/Test Plan/grinder tests

Overview

We would like to have a test framework that tests the sync servers for load and functionality. For this, we will use Grinder in order to mimic the action of many firefox clients simultaneously.

We are trying to answer the following questions:

• How does the app fail under high load conditions? How does it fail?
  • From a large data input?
  • From a lot of connections?
• Which functions create a particularly high amount of load?
• Are any functions effected sooner by high load?
• What kind of load is reasonable to run these servers at?

Test Cases

• Sustained Load - Long periods of load until read time becomes unbearably large because we are no longer caching data
• Spike Load - How do we deal with a ton of requests at once or in a very short time (say >15 min)
• Generating Data (and verification)
  • Bookmarks
  • History
  • Bookmarks and History
  • Account creation
  • Account deletion
  • Tabs syncing - P1
  • Passwords Syncing - P2

These test cases need to mimic the actions of firefox as closely as possible. We will use grinder to simulate the actions of a firefox client.

Issues

• In order to validate data we need to store that data. In order to validate a sync servers worth of data we will need a lot of storage. (Some optimizations may be able to take place).
• We need to parallelize the load generation over multiple machines to match the capability of the server
  • Grinder supports multiple hosts
  • We need help to get the adequate hardware for this task

Priorities

• First priority is Bookmarks and history - this effects the most users.

What are the next steps? Do you agree with the Test Cases that we are trying to address?