It was with some disbelief that I sat through Cal Henderson’s otherwise highly entertaining (and to be recommended) Building Flickr Workshop, to hear that automated testing didn’t feature highly in their list of priorities (as an interesting aside, Cal also considers object oriented programming to be ‘insane’, which all ‘sane’ code existing in the space between one-giant-function programing and OO). “Testing web applications” Cal said, “is hard”. This is a common misconception. Like any type of application, if you structure it for testability, testing is easy. And if you invest in testing tools written specifically for testing web applications (FIT and FITnesse, Selenium are good examples) even those applications written without testing in mind have no excuse for going without.
But with Flickr being such a high profile and successful application I assumed there would be a more mature approach towards automated testing. Is this laissez faire approach to automated testing common?
Lets look at this another way. If you don’t have automated tests, you either have to test manually (which means longer time to release) or you don’t test. If you are in the business of delivering quality products, it probably means you do end up performing lots of manual tests which could otherwise be done quickly, automatically and even if you want on every checkin.
The architecture of the web makes it idea for an agile release early, release often process. With the web, the only destination environment you have to worry about is your server and the client’s browsers. Pushing out a release really can be as simple as the one-click deployment Flickr uses. With a mature testing framework, releasing directly to your user can be managed multiple times a day if required.
Testing might not seem like fun – but once you realise it means you can release cool functionality to your users more often, and you’ll spend less of your time fixing bugs, perhaps we can all start to love writing tests just that little bit more.
magpiebrain 
6 Responses to “Web 2.0 needs testing”
FIT and FITnesse weren’t written specifically for testing web applications though Selenium and Watir (which I prefer) were. It’s hard to fault Flickr given how often they are actually able to deploy (once every half hour) though I think they are able to exploit the relatively low sophistication and criticality of their application. Of course, I’m still thinking that with automated tests and OO, they’d be even more capable.
This raises a particularly ugly question: are they deploying every half an hour because the only way they have to test things is to push it out to the public?
Are those multiple deployments really delivering value/new functionality or are they caught in the fire-fighting cycle of: release -> discover bug -> release again?
I’ve never heard of anyone complaining about Flickr’s bugginess… This doesn’t mean that bugs aren’t there, but the count and severity is low enough that people are still considering the overall experience superior. There’s also changing expectations by labeling your product BETA.
It actually occured to me after I wrote this that what I might of done is give a good argument _against_ the need for automated testing. “See? Flickr is successful, and they do need automated functional tests!”. Of course as “Carlos mentioned(Why doesn’t Flickr do automated tests?)”:http://www.lixo.org/archives/2005/10/18/why-doesnt-flickr-do-automated-tests/ it is probably the case that the Flickr guys worked out where to spend their time and money and automated testing wasn’t it.
It will be interesting to see if as they grow they consider them important or not. As many of us have seen, the costs of not having a comprehensive automated test suite gets greater over time (as more functionality is added). As Flickr matures, they may find themselves with longer time betweeen releases if they still trly on performing most of their QA function manually.
I have to admit this is a rather excellent coincidence. I found this article after searching for Web 2.0 testing after going to Thomas Fuch’s workshop also run by Carson Workshops. Thomas is keen on testing, his library has lots of unit tests. However, he just couldn’t suggest how to integrate testing into a web 2.0 application. As we all know browser differences make up a large part of web application bug issues (at least at the front end). As such it seems strange to methat these two big players in Web 2.0 don’t have, or haven’t emphasised the need for end to end repeatable tests.
We develop and test Web 2.0 applications and generally cater to retail market. What I feel is knowing your application is very important and Flicker with a 30 minutes release cycle are not coming up with earth shattering changes, hence no need for end to end testing. They probably understand their application and have a fair bit of idea about the impact areas of changes for every release. Hence they focus their QA efforts on the impact areas and have been able to deliver well so far.
In future they may grow into a complex application where testing maually may not be such a lucrative option and may opt for Automated testing. But till then none of us can complain as Flickr is doing it right and I see none of the users including me complaining.