Why Manual Test Works at Begining But Fails Eventually?
I would like to describe why manual software test works at the beginning but fails eventually.
Software test automation is the best practice of the industry. Good engineers write good tests by themselves and keep maintaining a good quality of products. On the other hand, we sometimes face project which does not have enough test automation. These projects eventually fail for some quality problems. What is the best approach to fix the problem of these projects?
Begining of Manual Tests
The typical example reason not to implement tests is that “we are too busty to implement tests”. I can easily imagine that TDD folks respond “You are too busy because you are not implementing test”. But let’s put that discussion aside and see how these projects go.
Management will ask the external test team to maintain software quality. We need the external team to cover the development team since they have no time to work on enough testing.
A trap of Manual Tests
There is a trap of manual tests. It’s problematic because manual test works well by some point. The following diagram describes the “sweet spot” in which manual tests work.
The team tries to maintain the quality by reducing the release cycle if they face production issues. The idea is to increase the batch size of changes so that we can reduce the number of regression test runs, and can secure more time to test. This strategy works well at some level.
However, this strategy stops working eventually. As the batch size increases, the complexity of the change increases exponentially. The complexity of interaction among the three systems is much bigger than the one between the two. It’s because the number of links increases exponentially as the number of nodes increases.
Indeed, the number of regression tests decreases as we increase the batch size of changes. However, the complexity of the change increases exponentially increases. This results in more bugs injected which are harder to be detected. The past successful experience like increasing more resources or increase of test period no longer works.
Getting Away from the Trap
The value of test automation or CI/CD is not the cost reduction. It is the minimization of batch size. Effective IT organizations understand this and spend a lot of investment. They make smaller changes and frequent release as possible. This results in less possible to ship bugs to production.
How we can get away from the trap of manual tests? I have only one answer, unfortunately. We need to spend time to automate tests and minimize the batch size. There are no people against automation. The challenge is how we can handle the request until enough test is implemented. Your stakeholder will ask you not to stop the new development during implement more tests. However, there is no silver bullet.
You need effective leadership to get out of the trap. You need to somehow secure the time to promote the activity. You might need to talk to stakeholders, or you might need to protect the team from stakeholders depending on the situation. Promoting automation is actually the best approach to make product development successful.
