Or: This week Matt and I talk ourselves in circles about metrics:
- Why measure? Because everyone wants to improve.
- What to measure? Very very difficult to measure “soft skills” like testing.
- Several perspectives on this.
I’ve worked in places where they counted bugs and I’ve made the mistake of counting test cases.
- Power to change the need for overly simplistic metrics?
- Asked for metrics but allowed to choose what to measure?
- Try to identify why they want those metrics and use critical thinking to explain why it is flawed.
- Counting bugs – project that is near completion vs a new project?
- Counting test cases – no test case is equal, all written differently. What is a “test case”?
- To the junior testers – help your test lead
- To the test managers – don’t rely on metrics
- Consider the danger of KPIs, what behaviours do you want to encourage?
Testing could be measured by:
- Are customers happy?
- Is the product selling?
- Are there very few critical issues found in production?
- Are budgets and schedules being met?
- Are developers happy?
- Are product managers happy?
- Are testers happy?
- If you have to count bugs then can you add some context to those metrics?
Asked around testersio slack team:
Feelings of metrics not making sense for individuals, only departments
Could measure % of production deployments without a critical defect reported in 24 hours/week?