<About />

My name is Kevin. I am a web professional living in Massachusetts. I build websites mostly using Drupal and jQuery. I use Vim even when I don't need to. When I'm not on the computer, I'm usually hanging with my wife, Melissa.

<Search />

siege

More regression testing

Submitted by Kevin Hankens on Thu, 02/19/2009 - 16:55

Very interesting test results from some regression testing I did today for a client. I was using Siege and Apache AB for anonymous user testing, and attempting to simulate authenticated users by specifying cookie data with AB. The tests were done remotely from an EC2 instance.

Here were the basic commands:

> ab -t 300 -kc 50 http://example.com/
> ab -t 300 -kc 50 -CSESS[...]=[...] http://example.com/
> siege -c 50 -t 300s -f url_list.txt

Rough Results:

No Drupal caching; No opcode caching
AB Anonymous 5.23 req./sec.
AB Authenticated 3.20 req./sec.
Siege Anonymous 7.0 req./sec.
Normal Drupal caching; No opcode caching
AB Anonymous 121.34 req./sec.
AB Authenticated 6.40 req./sec.
Siege Anonymous 218.83 req./sec.
Normal Drupal caching; eaccelerator enabled
AB Anonymous 338.31 req./sec.
AB Authenticated 14.49 req./sec.
Siege Anonymous 543.50 req./sec.

I ran each test a few times and selected the median value. So, these aren't 100% conclusive, but there is definitely a trend here. The interesting thing is the huge difference between anonymous and authenticated users.

Regression testing Drupal with Siege

Submitted by Kevin Hankens on Tue, 02/10/2009 - 12:59

So, I was doing some playing with Siege today which can be used to do regression testing and benchmarking on a webserver.

Install on RedHat was easy:

$ yum install siege

Quick test of 100 concurrent users for 10 seconds randomly hitting the urls from url_list.txt (one URL per line):

$ siege -c 100 -t 10s -f url_list.txt

The interesting thing was turning on normal caching in Drupal (no block caching because I'm using the node privacy byrole module). Check out the difference:

Before caching:

** SIEGE 2.66
** Preparing 100 concurrent users for battle.
The server is now under siege...
Lifting the server siege..      done.
Transactions:           27 hits
Availability:       100.00 %
Elapsed time:         9.77 secs
Data transferred:         0.77 MB
Response time:         5.56 secs
Transaction rate:         2.76 trans/sec
Throughput:         0.08 MB/sec
Concurrency:        15.37
Successful transactions:          27
Failed transactions:            0
Longest transaction:         9.26
Shortest transaction:         0.00

After caching:

** SIEGE 2.66
** Preparing 100 concurrent users for battle.
The server is now under siege...
Lifting the server siege..      done.
Transactions:          179 hits
Availability:       100.00 %
Elapsed time:        10.27 secs
Data transferred:         3.45 MB
Response time:         4.67 secs
Transaction rate:        17.43 trans/sec
Throughput:         0.34 MB/sec
Concurrency:        81.40
Successful transactions:         179
Failed transactions:            0
Longest transaction:        10.01
Shortest transaction:         3.65

Third time running after caching:

** SIEGE 2.66
** Preparing 100 concurrent users for battle.
The server is now under siege...
Lifting the server siege..      done.
Transactions:         1353 hits
Availability:       100.00 %
Elapsed time:        10.50 secs
Data transferred:        18.73 MB
Response time:         0.74 secs
Transaction rate:       128.86 trans/sec
Throughput:         1.78 MB/sec
Concurrency:        95.98
Successful transactions:        1353
Failed transactions:            0
Longest transaction:         1.40
Shortest transaction:         0.03

I hit my max_clients limit with plenty of memory to spare the first run, but only had a concurrency of 15. However check out the difference in transactions after turning on caching. 27 before caching and 179 immediately after. Then 563 and 1353 after running a second and third time. Plus, the last run, I hit 95 concurrency!!

Now, Drupal caching isn't going to be as great for authenticated users, but for a site that has mostly anonymous visitors, the caching is pretty impressive. So, moral of the story, turn on normal caching when you install a new Drupal site :)

On a side note, you should also install the devel module and log queries - again quite impressive when you turn on caching.