Comments on: Back to Basics: Becoming BAT Man

By: Hannes Kochniß

Hannes Kochniß — Mon, 15 Oct 2012 23:40:30 +0000

“It’s simple in testing: if you can’t maintain it, meaning that you can locate the cause for an error (much more likely in BATs than in Unit tests) in few minutes, yu shouldn’t do it. But this time grows exponentially with the amount of code covered by a single test.”
Sorry, very late here in germany..

By: Hannes Kochniß

Hannes Kochniß — Mon, 15 Oct 2012 23:38:19 +0000

I know this is very very old. Just stumbled upon it.
I’m a automation-test programmer at a company with a very old legacy codebase, growing since 2005, working since 2010 on wholistic blackbox-integration tests (the ones you call BATs). I have written such BATs for over a year, now switching to SpecFlow-based component tests, which are still in the business domain, but expressed very verbose, are in-process, and such stable and as fast as unit tests. Also, they are direct usecase descriptions and thus living specifiations, used by the business, project leaders, used in communication with customers…
The BATs from before.. we had to throw them away, they were a maintenance burden.
It’s simple in testing: if you can’t maintain it, meaning that the cause for an error (much more likely in BATs than in Unit tests) has to be able to be pinpointed in minutes. But this time gorws exponentially with the odeamount covered by a single test. Thus, BATs are.. very very bad.

I actually can now get much better productivity in the longterm with SpecFlow based component tests as living documentation + very slim integration tests (mostly smoketests via automated LinqPad html output on points like Exchange server invocation, Sql Server invocation, etc.) on the “outside”. Beautyfull concept, will never look back at BATs in my life, I’m sure there..

By: jsonmez

jsonmez — Wed, 09 Feb 2011 16:20:41 +0000

In reply to MrDustpan.

Hmm, good question. I am not completely sure one that one.

I think to some degree that would be correct, but I think it depends on how good you are with automation tools and building a framework.
What I mean by this is that if your company already has the competency in BATs the cost to get BATs up for a new product might be very low. So even for non-complex software the ROI might be there, but on the other hand if your company is not already invested in BATs then the initial cost might be very heavy in relation to the benefit.

By: MrDustpan

MrDustpan — Wed, 09 Feb 2011 16:05:49 +0000

Hey John, would you say that BATs have a diminishing return on investment as the complexity of the software / business domain goes down? In other words, if I have to create a fairly simple site, would I be better off writing a few unit tests, or would full on automated tests still make sense?

Really looking forward to the new series!