Load Testing with Selenium and BrowserMob

Getting into new territory as a developer is often exciting but also a little bit scary. So when I started my new role just last month one of the first tasks I was given was to look at how to load test the company’s e-commerce web site which is used by a small number of customers. The problem was that the company had gained a new customer who have around 500 staff that will be using the site though thankfully not all at the same time. To be honest, it shouldn’t be a problem but this particular web site has its fair share of issues, and there’s a lack of confidence in it for anything other than a small handful of users which is the reason for attempting to stress it, just to see if it is still standing at the end.

This gave me the opportunity to research and evaluate a number of different ways to achieve this goal. First up was Visual Studio 2010 Ultimate Edition which, with its built in support for load testing I would have thought would have been ideal, but to be honest I found it unintuitive and a little bit painful. Then I started looking at a free plug-in for Fiddler called StresStimulus which seemed to do the job simply enough but seemed a little bit lacking in some respects. It’s early days for the app though and the developer seems very active with updates so I’ll definitely revisit it in the future to see how it’s coming along.

Finally, I looked at Selenium IDE, a firefox plug-in that records what you do as you browse your web site, creating a script along the way. Initially, I’d ignored it as according to the Selenium F.A.Q. it isn’t designed for load testing but for functional testing instead. It was only when I came across the BrowserMob web site that I was able to go back and start using Selenium in anger. BrowserMob is a “cloud” service that let’s you upload your Selenium scripts and then pay to have them run using real web browsers against your site. Whilst it runs you get various reports updating in real-time showing the response times of your web server or any errors that occur. There’s also a MySQL database created which you can then download and run your own queries against to interpret the results any way you like.

BrowserMob looks expensive  but I guess if your company’s web site is vital to the business then ensuring it works properly under load is worth the cost. However, for smaller companies/sites the free plan is enough to get a feel for how it works and provide some useful data. It allows a maximum of 25 concurrent users running multiple scripts (each script exercising a different path through your site) for up to an hour. When I say free, I mean they credit you with $10 which is enough to run a couple of small tests. After that you can purchase additional “cloud dollars” to run more tests.

So, having created a script or two the next job is to parameterize them so that you can have different users logging in or searching for different parts, etc. When the script is uploaded it gets turned into a javascript file that uses the BrowserMob API. For my script I needed two csv files, one containing usernames and passwords and the other containing parts which, of course need to exist in the database of the site.

The first thing to do is make sure my csv files are loaded at the beginning of the script:

var partsCsv = browserMob.getCSV("partParams.csv");
var loginsCsv = browserMob.getCSV("loginParams.csv");

Each “browser user” gets its own identifier when the test is running and can be retrieved by a call to the getUserNum() method. I used this to ensure that each user would login with different credentials  by using the return value as an index into the csv file:

var currentLoginRow = loginsCsv.get(browserMob.getUserNum());
var username = currentLoginRow.get("username");
var password = currentLoginRow.get("password");

The retrieved values are then used when filling in the login controls on the form:

selenium.type("ctl00__rightContent__login_UserName", username);
selenium.type("ctl00__rightContent__login_Password", password);

and the same for the parts:

var currentPartRow = partsCsv.get(browserMob.getUserNum());
var partNo = currentPartRow.get("part");
selenium.type("ctl00_ctl00__defaultContent__defaultContent__filter__codeTextBox", partNo);

The use of the getUserNum() method here ensures that each browser user will get a different part number to search for.

Once I had this in place, I was able to schedule a test then sit back and watch it run. I think the combination of Selenium and BrowserMob is a great idea. It turned out to be the best way for us, a small shop, to learn about how our site performs (and there’s plenty of work to do to make it better) but whether or not it’s worth the kind of money charged for the paid plans is debatable although I’m sure BrowserMob know what the market will stand. Anyway, with the free plan there’s nothing to lose so if you’re thinking of doing load testing it’s definitely worth checking out.

Advertisements
Load Testing with Selenium and BrowserMob

2 thoughts on “Load Testing with Selenium and BrowserMob

  1. […] The Eventually Consistent developer blog reports that after researching on-demand load testing tools, EC loves BrowserMob best. Why? (1) The ability to upload Selenium and run real browsers against your site, (2) real-time reports showing response times/server errors and (3) the downloadable database which lets you run your own queries and analyze them in numerous ways. EC said BrowserMob was “the best way for us, a small shop, to learn about how our site performs.”  Thanks EC and glad we could help.  Read the full post. […]

Comments are closed.