How to estimate number of test iterations

How to estimate test iterations?

A common question that comes up in the Mind of a Software Tester trainings conducted by ZenTEST Labs is how to estimate number of test iterations. A good way to do that is to compute bug insertion rate and bug fix rate by the development team. Once this is done, you can easily estimate number of iterations to test.

E.g. : You have been asked to estimate the number of more test iterations required. You are at the end of round one. You can find the usual bug insert rate by developers in your organization when they fix bugs. As well as find the usual bug fix rate (number of bugs that usually get fixed when you report 100 bugs).  Thus in your organization if the bug fix rate is 50% and the bug insert rate is 10% (it is usually not this high), then this is how you shall calculate. Let us assume that the number of open bugs today at the end of iteration one is 100. Thus in round two, 50 bugs shall be open and 10 more shall be introduced. Thus, at the end of round two, you shall have 60 bugs. In round three, you shall have 30 bugs fixed and 6 introduced. Thus you shall be left with 36 bugs. Keep doing this calculation till you arrive at zero or one bug. That shall tell you the number of iterations.

The attached excel file shall be of help in this calculation Download estimation.xls

Consider the following as well:

a. You may be asked to estimate number of iterations at the beginning of the project and not at the end of round one. In that case, you shall estimate the number of bugs at the end of round one and perform the above calculation.

b. Consider finding averages in your organization for different rounds. It is possible that your average bug fix and insert rates are higher in the initial rounds.

c. You could apply further math to this basic idea and tailor this to your organization. For instance, Lloyd Raden from Grove Consultants recommends you to use nested rate as well.

d. Note that this is not a pure statistical way. But, in our experience we have found it simple and practical to come up with an estimate on number of iterations than use SWAG (Scientific Wild Ass Guess).

Comments

[Saritha Amenani]

Is there any method where we can estimate number of bugs at the beginning of the cycle

[Ravish Dakhneja]

Thankyou for the estimate number of test iterations, it really helped me a lot. I would like to know, How can we increase Web Application Reliability when number of new bugs insertion are more even after fixing majority of them. I'm working on a .NET based applications.

[shrini]

>>> You can find the usual bug insert rate by developers in your organization when they fix bugs

I am puzzled - how can one know about bug insert rates? If we are able to crack some of metric like this - Testing will become a child's play.

I would say - it is trying to make a 3 months old kid to scale mount everest all alone.

then What is this "usual"

For example bug insert rate is 30 bugs per day
( not sure what is the denominator your bug insert metric)
Would that help me in estimation - No way. If some gives me that metric and asks me to use it I would deny because that metric is meaning less - context free number. I can not believe it ....

I know making estimations in testing is a tough nut to crack - that is why you dont see much articles and blogs on this topic.

And there is no magic formula ....

Shrini