Or the importance of context and domain knowledge.
This episode was inspired by an blog post on uTest that came into my inbox this week, and I wanted to chat and expand on the points made there1.
As a tester, not having any context bugs me (hah!).
I need some insight into the product, or the client for me to feel comfortable testing. Having the spec or AC is not necessarily enough (though can certainly be useful for the basic functionality testing and checking). But for the other types of testing, where the lines between QA and testing blur, around testing fit for purpose, and the actual solution to the issue itself? I need to know who we’ve produced this work for, and why.
What industry? B2B? B2C? Not commercial? What users are they aimed at? Is the functionality I’m testing just for the CMS users or is it for visitors or customers? What are they hoping to achieve with what we’re giving them?
All these things inform how I test things, the questions I ask, and the feedback I give.
I’ve spoken before about how my background informs my testing – I know I make certain assumptions about a website’s or app’s functionality, because I use them all the time and am a techy person. That is something I have to keep in mind when I’m testing, and plan it out. I’ve started looking into mindmapping a basic test a website mindmap, and I’m hoping to put some steps together to remind me to test for things like user journey as well as the functionality I am actually testing.
Having more tech experience than the users we are testing for but less domain knowledge is the worst of all worlds. I can’t even begin to figure out the best way to test the system. I try to mitigate the first by being aware of my knowledge and being aware of best UX and accessibility practices but the latter needs context.
I’m not saying I need to know everything about the industry – especially from an agency point of view – I don’t necessarily need to be an expert in all sectors we build for (currently ranging from charity, to government sites, through to various ecommerce sites – gardening equipment and plants to scientific consumables), but I need to know basics at least for testing and QA.
I also need context about the project itself – what’s out of the box and what’s a custom functionality that I might not have seen before? If I’m doing manual testing I need to be able to use the site effectively, and as a user would, in order to get you the best feedback possible.
I also test better if I’m connected to a project – if its something I feel like I know.
I have been challenged on this – while discussing this episode Mark Jones asked me if actually having domain knowledge – especially for an ecommerce or other project that will be marketing driven to get visitors – would hinder my testing, as I’d know about what the product owner would expect me to do. If I come to the site completely without any knowledge about what was expected of me as a user, would I do more effective exploratory testing, as I’d be relying on the site to tell me all I needed to know? Which is an interesting idea. I’d like to think that I can divorce myself from my knowledge (like I do with tech knowledge) and be able to say ‘yes I can do this this way, but I’d expect to be able to do it that way as well or instead’. But its something to keep an eye on, to see if this desire to know about the product is influencing my testing.
Mark also suggested that in addition to my mind map I also write down all my testing steps to slow myself down and ensure I’m looking at things properly. This might be something I have to start doing as part of an exploratory testing session, and just take copious notes, and see what comes out of it.
So, I prefer having domain knowledge, I think it helps me more than it hinders, especially if I try to keep aware of my biases, knowledge, and expectations of how websites work, but too much knowledge may be a bad thing. What do you think listeners? Do you have a preference? How much domain knowledge do you have. As someone who’s done far more agency work that SaaS or other project work, I’d be interested in how testers in those settings find their domain knowledge affects (or doesn’t) their testing.