The cost of fixing defects

I have many times been asked to estimate the test effort in a variety of test activities in different projects. In general test estimation is a difficult skill to master  and it sometimes requires several iterations before I am confident about the result – even when I have gathered some test metrics regarding time and effort on test activities in previous projects.

One of the test activities that can mess up your test estimates is the cost of fixing defects (from a test perspective it’s about defect management, retests and regression tests). You will not gain popularity or gather useful insight if you ask the project manager or the developers the question “How many defects do you expect to put in the product?”

This week I read a new book about test and again I see some estimates on how much it costs to fix defects depending on when the defect was discovered.

Continue reading “The cost of fixing defects”

Defects and bugs

It is always satisfying for a tester to execute test and learn something new about the software under test – including finding defects and bugs.

Sometimes during test execution testers can come across requirements that are not implemented exactly to the letter in the software and then we, the testers, starts to ask questions.

If you are agile the question might be registered (or not) on a post-it or email and handed over to somebody with an answer.

If you are using a structured approach then you might register a question in your test management system and assign it to somebody according to some defect/bug life cycle process.

In any case you have a question that needs an answer by somebody who knows better.

I have experienced many different reactions to defects/bugs from non-testers.
Apparently a tester asking questions is not welcome at all times and certainly not in all types of context.

One of my first and worst experiences was in a networking company in 1998-99. We were about 20 system test engineers (read:testers with a fancy title) that were testing network equipment and network adapters.
We had a new product which management had high expectations to and with potentially high market value.
During the test we found many problems and asked a lot of questions to the
developers of both hardware and software. And of course we registered many bugs.
Reason: it was a brand new ASIC design and brand new drivers.

The Management team did not like the test progress and hear about our many bugs and at some point the CEO felt it necessary to talk directly to us testers.
He told us (testers) to stop testing and just fix the problems and make the product work.

Since then it is my experience that management people have matured somewhat during the years gone by and also sees test as a profession handled by intelligent people. But they still don’t like to hear about defects and bugs. Sometimes I choose to call the defects/bugs “test observations” instead because the words “defects” or “bugs” can upset non-testers.

So what have I learned as a tester when I find the need to ask questions and
register defects/bugs?

To be a tester means learning about the software you test but also learning about the people in the project and the stakeholders. Not everybody likes to learn that the software contains defects/bugs or questionable areas of functionality.
The tester has to learn to be diplomatic and know about how to communicate that there is a defect/bug (equal to bad news) to people with another viewpoint than the tester.