Today I’ve got a sort of mix of an episode, covering two different but related topics, that I tried to expand into two episodes, but couldn’t make them work, then I read Simon Prior’s blog post on the Testing Mindset:
https://priorsworld.wordpress.com/2015/08/19/the-testing-mindset/ and realised that the two topics were inter-related and maybe only needed one episode.
In the blog post, Simon talks about QA and testing as a process, starting as soon as possible, and that, combined with me reading about exploratory testing sparked this idea. I want to talk about testing and creativity, and testing as a creative process.
QA and testing often has a bit of a stiff, filling in boxes/checking things off a list image. Stuffy gatekeepers or killjoys saying things like ‘well technically’. And don’t get me wrong, sometimes I revel in the checking section of it, it’s good for me to do while I’ve got something more complex in the back of my mind, or to familiarise myself with a feature before I jump into the meaty part of actually testing it.
Then the more creative part of testing can happen. Exploratory testing involves creative, investigative thinking. Features are very rarely ‘correct’ but they can be right or wrong for what you are building for, and its looking at that, at how it works and why that is the creative part of testing. There are many bits that fall into this ‘rightness’, some of which is subjective enough that some bugs may come back from the client but it can also be things like consistency of user pathways, and how it flows. Patterns in usability and context on the feature I’m looking at, stuff that’s sometimes harder to write AC for, but important nevertheless.
There’s also looking at what you’re doing as a whole project. As well as testing each feature as it comes to you, you need to test how it fits into the project as a whole – the same principles of looking at flow and consistency apply, but on a different scale, and in different ways.
(As an aside, this kind of testing starts right at the beginning of the project.
I’ll test out the requirements, and the plans, even if it is just mentally, just to see what’s happening. It helps me get to grips with the project, provides some background and context. It might help the client refine requirements, or bring new ideas to the table. Then I’ll help writing the AC and I’ll be mentally testing them as well, all before we’re out of the initial planning).
This kind of links to another issue that’s been occupying my time at the moment, and that’s one of timeboxing my testing off. Balancing priorities and making sure I’ve tested all the parts of the feature and ensure I’ve not spent far too long on testing can be difficult.
Its hard to say ‘yes, this is done’ definitively. I often say to myself that I’ve found all the obvious bugs, but at the end of the day, the user is going to know how they’re going to use the system best, so I generally expect usability changes/bugs that are going to come from the way the client interacts with the system both from an individual point of view and from an internal process point of view.
But there’s always a niggle of doubt, of waking up at 3am with a scenario I haven’t tested in my head, which I then go on to test next day (no lie, I have realised at 3am that I misses a scenario, went to work, got back a story I had previously passed, tested my 3am scenario, found a bug that was deemed valid enough to reject the story. I am such a newbie sometimes). I’m responsible for carrying out the testing, and yeah, while this isn’t a gatekeeper scenario, I am still a tester, on the QA team. There are expectations there. And I want to contribute to good code. I want to make people admire our work.
But, I also need to not burn all the project’s budget by testing each feature too extensively, to the point of not being able to prioritise correctly. or focusing on the wrong priorities.
I generally stick to a rule of thumb of aiming to spend 20% of the time spent developing a feature testing it. So twelve minutes testing for every hour spent developing. Again, very vague, depends on a lot of things, but it helps. Because I could spend a lot of time testing, sometimes.
And there has to be a point when you decide that you’ve tested enough. You’ve gone through the most obvious test cases, you’ve gone away and come back to it a few minutes that if needed.
This is where a definition of done, and risk assessment comes in, I guess, otherwise I think I’d never be done.