A recent post by Cedric in response to an article by Michael Feathers (danger, community navel gazing alert! Obviously this blog is above such things) casts doubt on the necessity of unit tests.
Michael’s rules on what a unit test is seem overly draconian to some (not being able to touch the filesystem for example 99% of the time I’d agree, but if I’m testing my FileConfiguration class, what the hell am I supposed to do?) and despite many other people attempting to define what a unit test is, Cedric’s take on it comes the closest to my own understanding of what a unit test is:
…a unit test is traditionally defined as a set of test that exercises a single class in isolation of all others.
Cedric then goes on to say that 90% of the tests he sees written with JUnit are not unit tests. I for one would be wary of working on those codebases. He also states that he doesn’t care if the test he writes is a unit or functional test which immediately makes me worry about working on his codebase.
The point that gets missed here is that Unit and Functional tests are not different because of how they work, how long they take or the tools required to make them work, but they are different because they fulfill different roles in development.
Unit tests, because they focus on testing one class and one class only, make it much easier to create highly-focused loosely coupled code, as you create and test a class in isolation of other code. As you have good test coverage at a single class level, you can then safely refactor the class and know you haven’t changed the behavior of the unit. As you are writing code and testing at a small level, you can commit your code more frequently, enabling frequent, smaller-scale integration. If you are testing at a class level, the test themselves become a living document detailing how your code works.
Functional tests are supposed to test slices through your system – from UI to back-end and back if possible. You are asserting that your system conforms to the behavior expected by your users.
Unit tests tend to be fast because they limit the code they test. Functional tests tend to be slow, because they test more code, and also interact with slow components (e.g. databases, browsers, networks etc) but their speed doesn’t define what type of test they are.
Often there is the need to separate fast tests from slow tests – and TestNG’s categorisation of tests is perhaps the first feature of the TestNG that has interested me. But the categorisation is done in order to define fast feedback cycles (e.g. run a quick build on checkin, and longer build on the hour, everyday etc – see Speeding up the build and build pipelining for more details) this should not be used to confuse functional and unit tests. Developers need to understand the different requirements for, and benefits of, these two different types of tests.
magpiebrain 
6 Responses to “Why unit tests are important”
If you want to see how scary my code is, just sync to TetsNG’s CVS head and decide for yourself 🙂
Here is a follow-up challenge: among all the TestNG tests that you’ll find there, count how many are unit tests and how many are not.
I wrote most of these tests myself and to be honest, I have no idea what the answer to this question is, and actually, I believe that it doesn’t matter at all.
I add a functionality, I test it. Period.
It doesn’t matter to me whether this functionality is user-facing or just an internal detail. If it can prevent the product from working the way the user expects it or the way it’s documented, it needs to have a test. Any test.
Michael’s rules on what a unit test is seem overly draconian to some (not being able to touch the filesystem for example. 99% of the time I’d agree, but if I’m testing my FileConfiguration class, what the hell am I supposed to do?)?
–
By all means, write the test. The point is really to think about what could be tested independently of the file system. Defining ‘unit tests’ narrowly, like I do with teams, is really an invitation to think about that issue.
Cedric Wrote:
> I add a functionality, I test it. Period.
>
> It doesn’t matter to me whether this
> functionality is user-facing or just an
> internal detail. If it can prevent the
> product from working the way the user
> expects it or the way it’s documented, it
> needs to have a test. Any test.
But my point was that by not having a fine-grained unit test you loose much of the benifits of testing at this level (and therefore many of the benifits of TDD), namely:
* promoting loosly coupled, highly cohesive code
* small units of code with good test coverage promoting frequent checkins
* small units of code with good test coverage promote refactoring at a small level
* Testing at a smaller scope makes it easy to cover all execution paths
* Large scale tests seem to promote state-based testing rather than testing of the design – less mocking, more asking – which isn’t going to help a tell don’t ask OO design.
Without some mental note to “only test one class at a time” you’ll quickly find any team writing tests for a larger and larger scope.
Michael Wrote:
> By all means, write the test. The point is
> really to think about what could be tested
> independently of the file system. Defining
> ‘unit tests’ narrowly, like I do with teams,
> is really an invitation to think about that
> issue.
Agreed – it comes back to rules being there for the guidance of the wise and the obidience of fools I suppose.
I prefer to think/define unit tests in terms of testing a behaviour. Usually that behaviour is implemented in one method of one class, but this is not always the case.
Accessors are typically get/set pairs, and I always test them at the same time in the same test.
Master/Detail relationships are another. It’s easy to test the Detail in isolation, but how do I test the Master without a Detail or Two?
Just a thought. 🙂
So write your FileConfiguration class so it takes a io stream or a file name. Then you can test 99% of the functionality, probably the most important functionality, without touching the file system. As I recall one of Feather’s points was that if you cannot write a proper unit test, then maybe the design needs work.
Simon Wrote:
> Accessors are typically get/set pairs,
> and I always test them at the same time in
> the same test.
Which is fine as it goes, but I’m wary of describing get/set pairs as behaviour – a bucket of data has little in the way of behaviour 🙂
> Master/Detail relationships are another.
> It’s easy to test the Detail in isolation,
>but how do I test the Master without a Detail
> or Two?
>
> Just a thought. 🙂
And that is where mock objects come into play. When you test A, you want to test that it acts with B and C in an expected way (e.g. when I call email on a group object, I expect it to contact the report generator and then the email server).
> So write your FileConfiguration class so it
> takes a io stream or a file name. Then you can
> test 99% of the functionality, probably the most
> important functionality, without touching the
> file system.
> As I recall one of Feather’s points was
> that if you cannot write a proper unit
> test, then maybe the design needs work.
His points where to encourage discussion, to to enforce rigid rules. In an environment where people are unsure of what and how to test, rigid adherence to a set of guidelines is useful – one people understand the concepts they understand where it is ok to ignore them. If a single classes job is to locate a named property file on the classpath, and throw a runtime exception if it doesn’t exist and it needs the use of no other (non-core) classes, I have to test it actually loading a file. I could wrap and mock the IO system I suppose, but then I’d need to test the IO wrapper – and that seems overkill for a class that loads a property file.