I find your Level 1-Level 4 class categorization interesting but not really useful for the purposes of testability. I tend to mentally categorize in terms of newables and injectables (and then sub-categorize in DDD terms). I tried to explain this categorization in this post http://loosecouplings.blogspot.com/2011/01/how-to-write-testable-code-overview.html

I also find the practice of ‘using IoC containers all over your app’ alarming. I touched on a similar point [trying to preach the virtues of manual DI] in http://loosecouplings.blogspot.com/2011/01/dependency-injection-using-di-container.html

I think we (and others discussing similar issues in the blogosphere now) are all working toward the same goal here from different angles. If you figure it out, let me know! 🙂

Thanks for reply.

In my opinion unit tests is one of the tangible assets that good quality code has. Other ones such as easy maintenance, easy to read code and so on are too hypothetical for management to convince them that writing high quality code is important. By using BAT tests it is more difficult to get someone believe that code quality matters.

As for the slowness of BAT tests, there are many solutions available today that weren’t as readily available before that can make running BAT tests in parallel reduce to total time by a great deal. Obviously you wouldn’t be able to run these kinds of tests as quickly as traditional unit tests, but running them on a daily or even 2x a day schedule will provide an huge benefit.

As for someone new to TDD. I would suggest avoiding mocks and DI containers, but I would not suggest at all avoiding unit tests or TDD in general. In my next post in this series I planned to talk about how to refactor code in a such a way that a large portion of the code requires level 1 or level 2 unit tests. Using a TDD approach without allowing yourself mocks or DI naturally leads to this kind of code.

I agree that some teams may write BAT tests instead of caring about OOP and other best practices, but those teams are likely to have been teams that weren’t writing unit tests or any quality either. In the long run, if you have a complete end to end regression suite that you can run daily though, (as much as it pains me as a software craftsman to say this), some of the principles we hold dear are not nearly as important, since failures in functionality are found so quickly.

Another question: what would be your advice to someone new to TDD? Should they avoid mocks and DI containers? It’s very easy to use Watin even if you have very bad quality code which is difficult to test. Advocating usage of BAT tests might cause teams not caring about OOP and other best practices.

Thanks for the comment. Sorry for the late reply. I believe you get exactly what I am saying. Glad to hear that someone else has come to some of the same conclusions.

Your calculator example is spot on. That is exactly why I don’t like mocks anymore. Very good point.

Come one John – examples P-lease.

And as you’ve noted, I’ve also found that any moderate use of mocks, in the end, just couples the test to the implementation and certainly increases the friction of any future refactoring. I’m now avoiding those where I can completely.

The way I see it: If I go into my calculator class and completely change the way my internal add method works, so long as when 5 and 5 are input I get back 10, that’s all I care about. I certainly don’t want my new refactoring to break a test because my internal add method is no longer calling some other internal method that some mock had an expectation on just so the unit test could be “pure” and without dependencies.

Recently I’ve taken this same thinking (simple observations of state) to the BAT (to use your terminology here) level and started creating automated end-to-end tests that map pretty much directly to a feature. They are essentially my virtual customer who, multiple times per day, checks that every feature they have ever asked for still work in the way they expect. I think the most value comes from these tests.

Above everything else, I want to know that the software works in the way a user sitting in front it will expect. The only way to do that is to automate the sitting in front of it and checking.

Of course, the safety net argument still applies for unit tests and we need to be able to refactor code and have quick feedback about any broken behaviour (where a large set of constantly running BATs might not provide that quickly enough) – so a decent set of unit tests is still really important.

But focusing on the value that each and every unit test will give you is where I’m at now. To me, a unit test is more code – a form of debt, and I am going to evaluate the need for each one to make damn sure it’s paying for itself it terms of its reliability and usefulness. If it doesn’t negate its own debt, I don’t want it. I’m not going to test that my view has been passed the correct type of model any more.

Rant over. Great most and very thought provoking 🙂