Comments on: Getting up to BAT: Adding Smoke Tests to Your Build https://simpleprogrammer.com/getting-up-to-bat-adding-smoke-tests-to-your-build/ Fri, 13 Apr 2018 20:28:33 +0000 hourly 1 https://wordpress.org/?v=7.0 By: Phil Ruse https://simpleprogrammer.com/getting-up-to-bat-adding-smoke-tests-to-your-build/#comment-572 Tue, 05 Apr 2011 22:20:48 +0000 https://simpleprogrammer.com/?p=1295#comment-572 In reply to jsonmez.

Thanks for that, makes sense!

]]>
By: jsonmez https://simpleprogrammer.com/getting-up-to-bat-adding-smoke-tests-to-your-build/#comment-571 Tue, 05 Apr 2011 22:09:35 +0000 https://simpleprogrammer.com/?p=1295#comment-571 In reply to Phil Ruse.

I would say no. I think the answer here is to make sure the defect is recorded and then the test is disabled until that defect is marked as fixed and someone has ran the automated test to prove that it is indeed fixed.
The reason for doing this is to ensure that automated tests always indicate new failures, not existing ones, because we want failing automated tests to be treated as an urgent matter that has to be resolved immediately.

If we properly track the defect the automated test has proven, (I don’t say uncovered here, because surely the person writing the test that shows this defect would have uncovered it in the process of writing the test,) then we can avoid duplicating the test and reinstate the test once the defect is fixed.

I want to clarify also that I am not saying that we should pull automated tests from the build when they fail. I am saying we should never add failing automated test to the build. There is a very big difference in my mind, but easily confused.

Thanks for the question, helps me clarify what I was thinking. 🙂

]]>
By: Phil Ruse https://simpleprogrammer.com/getting-up-to-bat-adding-smoke-tests-to-your-build/#comment-570 Tue, 05 Apr 2011 22:03:14 +0000 https://simpleprogrammer.com/?p=1295#comment-570 Does this mean we have duplicated tests – one as pat of the automated build (proof of correctness), that is disabled if it fails, and one in our standard ‘test’ that we can use as part of our ‘is it fixed yet’ development (it fails until we fix it)?

]]>