I spend a lot of time thinking about test automation. We all know that, when it’s done right, it can save everyone time, increase test coverage, and eliminate a lot of tedious, repetitive grunt work. The trouble is, when it’s done wrong, it becomes just as much of a millstone as the manual testing it replaces. Witness:
- The massive maintenance burden of keeping automated tests working when the products they are hard-coded to look for suddenly disappear from the site under test because they’re sold out.
- The pain of debugging a test that’s given you an incomprehensible stack trace instead of a sensible error message.
- The constant fine-tuning of automated tests to cope with slight changes to the user interface that prevent them from “seeing” the objects they are looking for.
…and so on. I’m sure you can think of some more examples. So, to sum up, if you’re going for test automation in a big way, you’ve got to plan carefully. And you have to employ the same kind of programming best-practices as the developers do. And, of course, you must be prepared to spend significant time on maintenance. This is daunting, and, if you’re just dabbling in automation rather than being a full-time Test Automator, you probably aren’t prepared to make that kind of commitment for fear that it won’t deliver enough value to justify the time invested. Game over.