Tuesday, January 24, 2012

Using Loadrunner Pacing to Hit a Specific Transaction Rate

It is often necessary to apply load at a specific transaction rate per second (TPS).  For example, it may be necessary to test every build or patch of an application by running a fixed benchmark of 20 queries per second against the application, measuring the response time and server utilization at that fixed load.  By then examining the trend over time of response time and server resource utilization under uniform load, performance regressions can be easily identified.

One way to set a specific transaction rate using Loadrunner is by using the pacing feature.
Pacing specified how often an iteration starts.  For example, if an iteration runs in 300 - 500 milliseconds, setting pacing to 1 second for that script will case a user to run the iteration once every second as illustrated below:
Each second the iteration starts and then ends after 400 ms or so.  At the next second interval, the next iteration starts.  An exact transaction rate of 1 iteration per second is reached in this way.  By increasing the user count in this case to 10, a transaction rate of 10 TPS can be achieved.  

An iteration can have multiple transactions per iteration.  If the above example was an iteration that included three transactions (query1, query2, and query3), a single user would run three transactions per second and 10 users would give 30 TPS.

The following formula provides a calculator for determining what the pacing in seconds should be set to given three input parameters, transactions per iteration, user count, and target transaction rate:
  • transactionsPerIteration  (the number of transactions included in an iteration)
  • users (the number of users to be run)
  • tps (the target transaction rate per second)

For example, suppose transactionsPerIteration = 3, users = 50, and target transaction rate = 10 TPS.  The equation gives a pacing in seconds of (3 * 50)/10 = 1.5 seconds.

One thing that can prevent the target transaction rate from being reached is when the iteration slows down when run under multiple users such that the time an iteration runs is longer than the pacing time.  In this case a warning is generated and the transaction rate is not reached.  This can be sometimes be avoided by increasing the user count somewhat.

However, the problem could also be due to limitations in the application's vertical scalability.  The application may have concurrency problems preventing concurrent users from executing the transaction without blocking other users.  In a worst case, the application's concurrency problems may make it impossible to reach the target transaction rate regardless of the number of users.  In any case, it is worthwhile to investigate what the bottlenecks are that are limiting scalability and resolving those issues.

No comments:

Post a Comment