Ep 62: Banishing the Permenantly Exhausted Pigeon

This week I talk to Neil Studd about motivation! We touch on a lot of stuff including but not limited to: burnout, 9-5 testers, neurodiversity, and more.

Summarised transcription provided by Matt Bretten:

  • In terms of trying to motivate myself and those around me, it’s a real challenge. People will brush it off as “just feelings” or “just emotions”, but it can have a profound physical impact on you if you bottle it up.
  • Change and the size of change can have a big effect on motivation. The role of testing is changing so much and our role boundaries are blurring. Ideas such as a “developer could do some of the testing” can lead to questions like “why do we need testers then?”. This sort of thing is uncomfortable for a lot of people.
  • When we’re talking about managing change, we’re not really talking about those people that listen to podcasts like this, as you’re likely to want to improve and react to change. We’re talking about those people that have no idea change is going on all around them, who don’t listen to the podcasts or go to meetups.
  • I find myself taking on far too much work and becoming burnt out, this can affect your motivation too. A technique to help with this is to make time to organise your work, even though this feels like work itself, it helps make sure you manage your time more effectively.
  • The Dunning-Kruger effect is always going on, each time you think you know enough, you find there are so many more things to learn. It can be hard to push through the downward cycle of motivation through feeling Imposter Syndrome.
  • Make sure your work-life balance is not just ok for you, but also the other people in your life.
  • There are many more reasons than just a lack of motivation for why “9 to 5” testers don’t come to meetups or learn about testing outside of work. People have families and commitments or some people like to have boundaries between their work and home life. Or people may not feel comfortable attending in person. The thing to focus on is whether or not people are willing to learn.
  • We should try and help those who can’t attend events by bringing the ideas to them in the workplace, through brown bag lunches or streaming talks. At the same time, we shouldn’t try to force people to pick up certain ideas but instead think about letting people find their own way to make use of these ideas. It’s useful to consider things like Intrinsic vs. Extrinsic Rewards.
  • It can be easier to change your current workplace than move to another workplace. You have reputation you can spend and you can try and find out why the workplace is resistant to a change you’d like to implement. However, not everyone is capable of speaking up about wanting change, there can be many reasons people might not feel comfortable or able to such as introversion, autism and deafness.
  • The challenge around motivation can be helped with a few simple things to keep in mind to help you along the right path:
    • Forge your own career path, don’t assume other people will do that for you. If you can take other people, great but don’t hold yourself back just because you feel you need to bring other people with you.
    • Use all of your tools are your disposal, find out the budget available, find out how much time you’ve got and find out how open your workplace is for going out to events. Make use of the skills available from the people around you.
    • Be true to yourself, have a sense of integrity and do things for the right reasons. If things go wrong you can at least know you don’t waste time second guessing your decisions.
    • Leave room to enjoy yourself outside of the office. If you want to work outside of the office, don’t feel you have to and don’t sacrifice other parts of your life in order to do so. Having extra happiness will enable you to be a better tester.


Neil’s blog post on 9 to 5 testers:

Sallyann Freudenberg’s CAST talk on Neurodiversity:

Meri Williams talk on stealing people lessons from AI, which touches on the motivators mentioned in Daniel Pink’s Drive:

Ep 61: It Takes Two

This week I talk to Maaret Pyhäjärvi about pair testing. But this is an interview with a twist: we did an hour of strong style pair testing (http://visible-quality.blogspot.co.uk/2015/10/a-course-on-testing-pairing-and-mobbing.html) first, and recorded our thoughts and experiences regarding that.

I cannot recommend pair testing enough! It’s hard work, but totally worth it – you have to think about communication a lot, including precision, and assumptions – and it really gets the job done.

Find Maaret on Twitter (https://twitter.com/maaretp) and at her blog (http://visible-quality.blogspot.co.uk/)

Ep 60: Go Soft or Go Home

This week I talk to Matt Bretten about Humans vs. Tools. Matt’s gonna be a voice you’ll be hearing more of moving forward. Stay tuned for more news!

This is an unfairly stacked deck, I admit. You can’t really take the position that tools are more important/better than the human side of testing. However, we did touch on myriad fascinating topics:

  • “A bad workman always blames his tools”
  • Psychology and it’s importance to software development
  • Writing code and using tools is the easy part, it’s harder to know when to make use of them and when not but also communicate and work together effectively
  • You could describe ideas, knowledge and skills as tools – for example I consider pairing a tool I find useful for certain situations
  • Boeing report – 80% of aircraft incidents caused by human error
  • The ELIZA effect

Ep 59: Hit Record (and play)

Quick news:

I am going to be at Leeds Testing Atelier on 20th September, which is a free one day testing conference, and I’m very excited to be there1., there’s going to be some amazing talks and some people I’m excited about meeting IRL at last.

Obviously I am also going to be at TestBash here in my adoptive city of Manchester, which is also very exciting! Plus more podcast interviews and other awesomeness.

This week, I want to talk about Selenium IDE.

Selenium has two parts: the script management/automated testing bit (Webdriver) and a Firefox add-on (IDE) that you can use to record browser actions and then play them back. It’s often denigrated by the testing community and for some very good reasons:

They’re brittle and flakey (but then again, so is every automation script that relies on find_element_by_id as far as I can tell). If nothing else, you’re only testing the UI, or through the UI2., and that can can cause issues (Mark Winteringham talks about this briefly in this blogpost here, and I saw him do a talk about this in Liverpool this month, it was very good, highly recommended, the video is on youtube3.). Some of these tests should be pushed down to the API/service level, not the UI level, and that isn’t really possible using the IDE add-on.

They get mis-sold. There are a lot of ‘record and play automation: so easy anyone can do it’ adverts and that annoys testers and leaves a bad taste in a lot of mouths (and rightly so). Testing is skilled work, and the idea that an automated record and play system can replace the work of a tester – even if that tester is solely an automation tester – is ridiculous. Writing them requires a human, and a tester at that. Someone who can bring all the skills of a tester; the curiosity, the need to gather information, things like that.

There are many reasons IDE is not a good choice for building an automation suite. But I’ve used IDE, and I found it useful, so I’m going to share my use case with you today.

I’ve literally just started Selenium, in bits and pieces, between other bits of work and podcasting, etc. and I had no idea where to start. I could find the header of the script from the documentation (where you call the webdriver and other bits you need from selenium), but after that I wasn’t sure. So I recorded a brief session around the site I wanted to get some automation on. The IDE has an option to export the scripts into code. So I exported the script into Python, which is the language I was planning to use, opened it in Sublime, and started to review the script.

The commands are slightly different, but it gave me a good starting point. From there I can make changes to make things robust. I wasn’t sure how to do something, but with the IDE commands in front of me, I could google those and then find out what they did, what they were called in webdriver, and if I could improve it in any way (IDE does implicit waits by default, for example, and I could change these to explicit waits.

In the world of Selenium based automation, it gave me a starting point. I converted the script to a ‘better’ webdriver script, and saved that out for repeated use. And once I got that down, it was a lot easier to start again, from scratch, to do another test. And yeah, it was hard and I still have no idea why people like coding, but I did it, and I felt I had a good grounding to do it and google about for help (read: copy and paste shit off stackoverflow as I am a true developer 😉 ). And now I’ve proven the concept, I’m doing more and it’s starting to form an automation strategy, which is incredibly exciting.

So IDE is a tool, and like any other it can be used or misused. For me, it’s a great starting point for Selenium, where I would probably still be bashing my head against a wall if I tried to build this from scratch. I have a lot to learn, but this was certainly a good springboard.


[1] http://leedstestingcommunity.co.uk/
[2] http://www.mwtestconsultancy.co.uk/cross-browser-checking-anti-pattern/
[3] https://www.youtube.com/watch?v=VGNxv9ilFbQ

Ep 58: Hot tub time machine

Or: The future of testing!

This super bonus episode is the second of my iterviews with Leigh Rathbone! We make several references to the previous episode (see here), so you may want to catch up on that episode first.

In this episode we discuss:

  • James Whitaker and the Hot Tub being the future of the internet of things
  • The changing landscape of testing and adding value
  • Crowdsourcing testing
  • Testing is dead, long live testing

Ep 57: Don’t go chasing waterfalls

This week is a guest episode! While I was in Liverpool for Liverpool Tester Gathering (http://www.meetup.com/Liverpool-Tester-Gathering/), I managed to grab some of Leigh Rathbone’s time for a couple of episodes! This is the first one and is about transitioning from Waterfall to Agile, with a focus on testers and test teams.

We cover the pros and cons of the different ways of working, and how to alleviate some of those cons. We reference the Agile Testing Quadrants:

The Agile Testing Quadrants, from The Agile Testing Book
The Agile Testing Quadrants, from The Agile Testing Book

And SMURFS: https://www.agileconnection.com/article/instead-mvps-maybe-we-should-be-releasing-smurfs

Ep 56: And our survey says

An interview with Rosie Hamilton!

This year Rosie did a large survey of software testers and then did the number crunching to produce a snapshot of testers in 2016. All the data and the scripts she used are up on GitHub for transparency, and for others to do their own analysis if they wish.

Rosie has now made a web app for people to explore the data she found without having to run scripts or learn R. She has just published a blog post about how she did this, with a link to the app.

I’ve been interested in the make up of testers; where we come from, when and how we decide to become testers for a while now, and I knew when I saw the survey that I wanted to invite Rosie on to the show to chat about it.

We talk about the survey process, from inception to analysis; what the survey taught her about testers, and about surveys, where testers come from, and the plans for the next iteration of the survey.

Summarised transcription:

The inception, the decision making, whys, whens etc. Did you have it all planned out from the beginning and have the analyses you wanted to do before you formulated the questions or what it more ad hoc than that?

  • It kind of happened accidentally. A lot of people had been asking me the same questions. I tried to answer from personal experience but couldn’t, I reached out to some friends on social media to get their views, but still wasn’t any close to answers.
  • Had a few doubts about asking people to fill out a form, mostly I wasn’t sure how I was going to analyse the results to find answers, but decided to go for it.
  • I spent about 4 weeks planning the survey. I had a list of questions I wanted answered and I worked backwards from there. I tested the survey on the testers I work with. The main concern which was raised while testing the survey was keeping it totally anonymous. Some of the questions got changed before it was unleashed on the community.
  • While the survey was open and collecting data I started planning how I was going to deal with the results. I started learning R which was a very steep learning curve at first.

Lessons learned, both from the survey and from doing the survey?

What I learned about surveys:

  • Open ended text boxes are a real pain to clean up and make sense of the results.
  • I should have asked more demographic questions which would have allowed the results to be split into more groups. From the start I didn’t want to divide the sample by gender as I thought gender was irrelevant but I wish I had asked which country or continent people were working in as it would have been nice to try find some patterns there.
  • The questions which were crystal clear with closed answers like yes/no true/false gave the best results.
  • The more data collected the better. You can never have too much data.
  • I learned that R is REALLY powerful. It lets you start writing the analysis before all the results are in. You can just keep feeding it data as the collection of responses grows.
  • I think being transparent from the start about why the data was being collected, what it was going to be used for and also making the results publicly available was what has made this study quite unique. I know there are other surveys about testing like the ‘state of testing’ but the raw data from that one is not in the public domain, only someone’s interpretation of the results is available.

What I learned about testers:

  • We are a really diverse group and we all have our own stories.
  • I spoke to one tester while the survey was going on, they said they had struggled to answer the what made you apply for your first testing job. They said they came to work one day and were told ‘you are a tester now, your old job doesn’t exist’.
  • I feel testing is still has a bit of stigma around being an unglamourous job. It is very hard to hire testers. I also feel that testing is not a really aspirational career choice given how few testers wanted to test while in Education.
  • One of the results which surprised me was that 2 out of 3 testers which started testing in the last two years didn’t study computing. I think students which study computing are mostly moving into development jobs. Which means junior testing jobs are being filled by people without computing backgrounds.
  • There was one person that completed the survey that when asked what the liked about testing answered ‘none of the above’. One in every four testers aren’t happy in their jobs so I think testing isn’t for everyone. I also think there are some really bad testing jobs out there. In the worst workplaces we have testers getting blamed for missing bugs, management not understanding what testers do and forcing testers to sign disclaimers stating there are no bugs.

Feedback from other people?

The feedback has been really positive. I’ve also realised even though the blog posts were written for a testing audience, other groups have been reading about the survey.

Future plans?

I have this crazy plan to turn all the data from this survey into a web application. I think this will be good because it will provide an interface for people to explore the data themselves. If I manage to create this web app that I’m dreaming about, I am certainly going to blog about it.

I definitely think I am going to survey testers again next summer, which should give me enough time to design a much better survey for 2017.

Ep 55: FOMO

Or, fear of missing out.

Quick news:

  • I’m at Liverpool Tester Gathering on Thursday 5th August. Come say hi if you’re there!
  • Next week I’m interviewing Rosie Hamilton!
  • The google form for telling me your origin story will be closed on Monday 1st August, so get going if you’ve not filled it in already!

This was not the episode I planned to record today. I couldn’t get that episode to flow properly, so instead I’m talking about FOMO. Now, this text is not as close to a transcript as normal, as I recorded this mostly on the fly, so it’s a bit rambly and not actually written down, but I’ve got the main points below.

On Tuesday, a group of us took part in the first #TuesdayNightTesting, which was a remote lean coffee evening. It was a lot of fun, and one of the questions was about getting testers in a slump into the community. And it got me thinking about the community, FOMO, and people that don’t want to be in the community. It’s been something I’ve been thinking about, and I’ve basically got a list of things I try to do to limit, control, and maximise my testing community involvement.

  1. Have a to-do list
  2. Use bulletjournal, use trello, use whatever tool you want but write or type out a to-do list. Even if you remember everything you need to do, visualising it will help you see how much space you have for new things.

  3. Find a niche
  4. I am a magpie of testing, I’ve spoken about this before; that I can find it hard to focus because everything looks so cool that I want to do it all. I think its easier for me to choose a niche for contributing, because it depends so much on what I enjoy, as opposed to what works well for my context, the problem I have, etc. But pick a contribution (blogs, twitter, podcast, speaking, code/tool development), and roll with it.

  5. Have a plan
  6. I am formulating a plan for the podcast. It has taken me a year to realise I need one, but I’m going to write down what the podcast is, what I want it to be, and how I’m going to continue it, so I know what I want to do. It doesn’t have to be detailed, but I think if you’re serious about doing something in a big way, you need a plan.

  7. Say no/Defeat FOMO
  8. Say no. You’re going to have to at some point so get used to it with small things. Or say ‘let me check’ and go check fully before saying yes or know. If seeing other people do and tweet and blog about things you’re missing out on is going to bother you, pull back a bit.

  9. Take a break
  10. Related to the above. Take a break. Self care is important, and self care whilst doing stuff and being part of tech is something I’m really interested in.

Ep 54: Origin Stories

A series of unconnected things:

  • I’ll be at the Liverpool Tester Meetup on 4th August! Come say hi!
  • I’ve got more interviews coming up and that is my plan for August – start reaching out for interviews.
  • 30 days of testing! This is a thing that Ministry of Test is doing, and though I got off to a strong start, I’ve started to flag a bit. I need to get back on course!
  • I’ve started working with selenium! I’ve picked python as I assumed a language I knew would be less of a learning curve, but that may change as we figure out what works for us.
  • I’ve been listening to a lot of creepy podcasts – The Magnus Archives, Archive 81. Highly recommended if you like creepy fiction!

I want to talk about getting into testing.

My session at Testbash will touch on getting into testing in less than ideal circumstances and learning testing in those circumstances, but today I just want to discuss getting into testing.

So many testers say they just ‘fell’ into testing, that they either didn’t see testing as their career, or that they didn’t even realise testing was a career, and I’m intrigued by it. I am a tester that did the same, and I’m sure there are other careers where people also just fall into it, but as a tester, it’s testing I’m interested in.

I’d be interested in collecting experiences of how people got into testing; not necessarily their background, but maybe the moment you realised this is what you wanted to do, or the moment you shifted to testing almost full time – whatever you want to think as the moment or series of moments you got into testing, I want to hear them.

I got into testing by saying yes. I was working as a customer support person in a digital agency. My co-worker, who was our only tester, had moved into release management but that meant there was less time for him to test (we were ostensibly a start up, so profit margins were slim). As the person who had bugs reported to them, and the person who had the most free time, relatively speaking, it seemed obvious that some of that work would come to me. I was still new to the job, and pitching in and helping out had never really gone wrong for me before so why not?

My testing was weak, relatively speaking. I mean, I have a science degree and can be pedantic about things, so you know, I wasn’t completely in the wild, but looking back I know I missed some things I would’ve caught now.

We were working off spreadsheets of test cases, which I actually found a useful starting off, because they gave me the foundations I needed to feel comfortable. There was nothing saying I couldn’t go off script, but we wanted to make sure we had everything down (incidentally, I think I only ever used the ones handed over to me, never ones I’d made myself. Seemed like a lot of faff for little reward).

I don’t remember the moment I became a tester, but I remember a moment I realised maybe I could do this.

I wasn’t happy with an implementation. It’s hard to describe without jumping into details but it was wrong. Firstly, I was proud of myself that I’d got enough knowledge of the system to know it was wrong. We pointed out the issue and got it fixed. However, the problem was twofold (maybe three). Firstly, we had offshore devs that had no real interaction or onboarding of the site. They were treated like code monkeys, and this led to a lack of investment and knowledge of the system. Two: the feature request had had no discussion, really. It came from the client, through the account manager, straight to said offshore dev, with no other input. So the feature hadn’t been technically planned, no real AC was on the ticket, nothing. Just ‘make this happen’.

That was the failure in my eyes – there was no guidance from the developer, and there wasn’t clear lines of communication on this stuff, so he didn’t ask for clarification. He was new to the system, so didn’t have the time or knowledge to really figure out what knock-on effects there were. So I started stepping in a little bit and trying to make sure we had a list of requirements and edge cases.

And I started to google. I wanted to see what other people did, how other companies handled this stuff (at this point our tester had left, leaving me as the go to tester person – did I mention the company was not so slowly going bankrupt?), and I found the testing community.

I think those two moments – realising I’d picked up a bug in the system and then finding the testing community (or the bits I inhabit anyway), were the moments I decided this is what I wanted to do.

And I think a lot of people get pulling into testing because ‘anyone can do it’, and for me it was realising that it goes deeper than that that brought me into the career properly.

I would absolutely love people to let me know how they got into testing. I can’t promise any fancy R scripts and graphs like Rosie Hamilton did, but I can promise at least a blog post on it!

Link to the form

Ep 53: You have no control who lives who dies who tells your story

Thiiings: I have 3 potential interviews coming up for the podcast and I am excited! More details when I get them. I am panicking about Testbash Manchester as I said I’d do a practice talk at the local Drupal User group in August and lols I’ve not even written the thing yet. Ministry of Test is doing a series of webinars, facilitated by Maaret Pyhäjärvi, which I’ve asked to take part in, so I need that topic as well.

Sooo, I want to talk about storytelling.

I’ve always been fascinated by stories. I was a reader as a child, I got lost in words more often than anything else. I read the fifth Harry Potter book the night before my Physics GCSE, staying up until the early hours because the alternative – not reading it – was unthinkable). I always thought it was some kind of magic, storytelling. Later, hearing stories became my obsession, through Ted talks, podcasts, and conference talks, and it’s still my obsession now.

Stories keep you engrossed, invite you on a journey. You’re not just telling someone something, you’re informing them, showing them, bringing them with you on something.

When I started hearing about testers as storytellers I sat up, and paid attention. I don’t have the imagination to write fiction – I’ve tried and I can’t hold a plot with two hands and a bucket, but writing has always brought me joy, and I am always trying to write and speak better, tell my own stories in a way that’s useful and engrossing.

And I realised, as I was reading about storytelling in testing and software development just how much space there is for storytelling in our work. I knew and had used personas before; and how this persona interacts with a product. What I hadn’t thought about is how writing bugs reports is storytelling.

Part of a tester’s job is to advocate for bugs. We report the bugs and in our reports are the whys and hows of raising them, and why we think they should be fixed. Sometimes this is nothing major – this doesn’t work at all, or doesn’t work in this environment. Sometimes it’s a bit more nuanced, “I would expect this here” “I shouldn’t see this because of this reason” and getting the details across fully and in a compelling way is important, because it will get the bug across to people who are in charge of fixing or scheduling the fix.

Sometimes it’s not a bug as much as it’s something that’s not been thought of, or something has been missed, and you need to explain why we need to include the change you’re asking for. Stories can help advocate for things.

So how can we make our stories compelling? First it needs to have a structure: a beginning, a middle, and end. This person went to here and did this and this happened. This is what we thought was going to happen, this is what should happen, this is what we need to happen, and this is why we think this should happen.

Tell the report to someone. This is one reason I’m inclined to talk to developers first when it comes to an awkward bug, or a bug that’s not obvious. I can talk through things and get a feel for what details are needed. If I can’t, or it doesn’t make sense to, I’ll write the bug out in bullet points, with an introduction and end to structure the bug. I make the title relevant (no puns in ticket titles, no matter how tempting. I save all my puns for you, listeners <3), and I make sure I include a desired resolution. Never leave a story on a cliffhanger, people will hate you. A pox on bug reports that are literally ‘the menu doesn’t work’.

Choose your evidence – annotate and refer to the images, videos, numbers you include, don’t just include them without explaining (this is something I am guilty of a lot). Otherwise it’s just decoration and something making the ticket bigger.

Exposition is allowed, and is preferred to a light touch when it comes to details. At the same time, we’re not Charles Dickens and none of us are being paid by the word (this was not actually true, but I like the idea of it). Choose your details wisely.

Write first, then edit before sending it to a developer. Choose what details make sense when you review the entire report.

When the bug comes back to you, detail the resolution – say x was done, or on reflection, y was done because of reasons a, b, c. Finish the story, close the loop. Start again.

We should tell clients more stories. Instead of saying ‘this will do x and y, say, this will allow you/your customers to do x and y, but not a, or b.” or, “we chose to implement this because of these advantages.”

And we should listen and help clients form their own stories. Point out plot holes, and suggest how to tighten the plot up. Offer different opinions, viewpoints, and expertise (you’d send a novel to a copy editor before printing, right?). Help them guide their clients or users or stakeholders through the journey of the site or the app, and make that journey part of a story. When something becomes part of your story it becomes something you care about, and engage with, which is important when it comes to developing successful software. Speak in a common language, and make sure the goal is the same on all sides.

Your user story statement should in fact tell people the story of the feature. As this person, this character, I want to do this thing to reach this goal. Break with the format if needed, but make sure the story elements are there.

Practice. I really like internal sprint walkthroughs. These happen prior to the sprint demo to the client, and it means that the team as a whole gets to look at the work we’ve done. We take each feature ticket, and demo how we meet that criteria. It’s practice for one; the lead dev can find the most sensible way to demo the entire sprint (to tell the story of the sprint, maybe?). It gives the team a chance to review progress as a whole, and make sure everything fits together well.

Hell, storytelling could be a good way of substituting metrics for useful data. Metrics don’t tell you much of anything, but you can at least supplement them with words. X% of stories were rejected with bugs, this is because of x, y, and z, is much better than x% of stories were rejected. Even better would be ‘we had issues with a few stories because of these reasons’, and then move forward, but that doesn’t fit into a graph nicely.

There’s a million things I haven’t done – I’ve not spoken about what happens when the story is taken out of your hands, or talking to people you don’t work with (interviewers, other testers, people who aren’t techy), but I wanted to focus on the sprint cycle.

The whole agile process it an exercise in storytelling, and I think we need to get better at telling them, and about helping other people develop them. Stories are fundamental to human nature – humans are rooted in narrative; we form lives and memories around stories, There’s no reason we can’t continue this in software development, and bring a bit more of the human into the software.

Further reading