I spend a lot of time thinking about test automation. We all know that, when it’s done right, it can save everyone time, increase test coverage, and eliminate a lot of tedious, repetitive grunt work. The trouble is, when it’s done wrong, it becomes just as much of a millstone as the manual testing it replaces. Witness:
- The massive maintenance burden of keeping automated tests working when the products they are hard-coded to look for suddenly disappear from the site under test because they’re sold out.
- The pain of debugging a test that’s given you an incomprehensible stack trace instead of a sensible error message.
- The constant fine-tuning of automated tests to cope with slight changes to the user interface that prevent them from “seeing” the objects they are looking for.
…and so on. I’m sure you can think of some more examples. So, to sum up, if you’re going for test automation in a big way, you’ve got to plan carefully. And you have to employ the same kind of programming best-practices as the developers do. And, of course, you must be prepared to spend significant time on maintenance. This is daunting, and, if you’re just dabbling in automation rather than being a full-time Test Automator, you probably aren’t prepared to make that kind of commitment for fear that it won’t deliver enough value to justify the time invested. Game over.
It’s a shame to give up at this point, because, as valid as those concerns are, you can still make good use of automation techniques to achieve real efficiency gains in this situation. The trick is to “think small” and break things down into little pieces. When we think about test automation, there’s a tendency to try and tackle complete test cases in one go. But if you can automate just one fairly simple, yet repetitive, part of the case, it’s often possible to score a significant time saving, even if you’re completing the rest of the case manually.
I’ve recently been doing some testing of the search functionality on an ecommerce website. The change was technical only: the idea was that the end users wouldn’t notice any difference after it was deployed. Clearly, the easist way to test this is to hook up two sites to the same database, put the new code on one and the old code on the other, then check that any search you run returns the same results on both sites.
IRB is a great tool for this. When you’re viewing search results, there’s almost certainly a repeating HTML element that contains the product ID. All you have to do is work out how to identify those elements, and you can get a list of every product ID in a couple of seconds. Identifying the elements is pretty easy with a tool like Firebug. In this example, I’m assuming we’re shopping on New Look.
As you can see in the screenshot, the product ID can be found in a hyperlink that resides in a list item with class “c5-1”. Bear in mind that I don’t work for New Look, so I had to work this out. If I did work for New Look, I could get this information from a developer. Anyway, all I have to do is get Ruby to iterate through every element on the page with class “c5-1” and print the relevant part of the link address. The following snippet does just that:
# at this point you can go to your browser, navigate to the site under test, and run the search
$browser.elements(:class_name,'c5-1').each do |el|
Let’s just go through the lines that do the real work to see how it’s done:
Dead simple: tells Watir to look at the page and pick out any HTML element with the class ‘c5-1’.
$browser.elements(:class_name,'c5-1').each do |el|
Tells Ruby to iterate through each of the matching elements, passing each one in turn to the next code block as an object called el.
- el.links just picks out the first link in the element. There’s only one, anyway.
- .href gets the target URL of the link.
- .match(/d+($|?)/) finds the product ID within the link. What’s that regex doing? Well, d+ looks for one or more digits (i.e. the bit of the string we want) and ($|?) just specifies that we only want sequences of digits that are immediately followed either by the end of the line (represented by $) or a question mark (which we must escape using a backslash, otherwise it would be treated as a wildcard). I noticed that some of the links had an argument like “?productFind=search” after the ID, hence why the ‘?’ is needed.
- .to_s.chomp(‘?’) extracts the string from the match results, and removes the ‘?’ from the end, if there was one.
Actually, chances are I could have got away with a much simpler regex that simply returned any string of digits, regardless of what followed them. This would probably have worked, but since I don’t know the site very well, it seemed safer to be a bit more specific.
So, we run that once in IRB, return to our browser and run the search on the other site, then repeat the last three lines in IRB to get another list. Copy/paste the lists into Excel, write a couple of swift vlookup functions, and you can instantly spot any differences between the two results.
Boom! Job done. And we don’t have to bother about source control, nor do we have to do any complicated setting up of environments if we want to share it with colleagues: a simple email containing a few lines of code will do the trick. Even better, if we later decide to automate this test case fully, we’ve already written the most tricky bit of the algorithm.
The trick here is to treat test automation not as a way of life, nor as a paradigm that must be followed totally or not at all, but as a tool in your armoury that you can use as little or as much of depending on what the situation demands. If you don’t know Ruby, I’m sure you could do something similar with Selenium IDE. The point is to recognise the situations where a bit of innovation can help you work smarter, not harder.