Ep 88: There’s a hack for that

This week I talk to Jahmel (Jay) Harris. Jahmel is a Penetration tester/Security consultant at Digital Interruption. He also runs Manchester Grey Hats.

  • Things to consider before starting security testing
    • App permissions?
      • Information users need to give the app
      • Push notifications?
        • Fine usually, but be aware if anything sensitive if sent – shoulder surfing
  •  Wearables
    • New ways of interacting with devices
    • They are becoming more secure but issues at the start
    • With Android we found lots of ways to recover the data
    • Bluetooth LE and other radio protocols can be insecure.
  • Testing considerations iOS vs Android
    • Root vs non root
    • Jail break vs non jailbreak
  • Common vulnerabilities
    • WebViews
    • Sensitive data over HTTP
    • Javascript vulnerabilities – used to be able to get full shell in an app via advert in webview. Coffee shop or hotel wifi
    • How secure are these webview frameworks such as cordova
  • Vulnerable IPC (Inter-process communication)
    • Things like SQL injection or file traversal
    • Lack of protection/permissions
  • Logging
  • Auth
    • Fin tech (financial tech) app – could steal all money. They didn’t think about the auth on web services
  • Binary Checks
    • Is it worth checking for root detection/doing ssl pinning etc? It took someone over a year to bypass these controls on one of our client’s app. Then they need to look for vulns.
    • Obfuscation? Worthwhile? When I did the research into Android Wear, it took me weeks just to RE.
  • They stack. Easy to bypass one but hard to bypass all. Think about the risk of the app. Does it need that protection?
  • Tooling
    • Drozer
    • Needle
    • Frida
    • decompilers
  • Automation
    • Tooling isn’t quite there. There needs to be a big push by both devs and infosec. InfoSec can’t write good code but devs aren’t always aware of the latest threats.
    • Security shouldn’t be dev->pen test. Security needs to be considered at every stage. In requirements gathering etc
      https://www.digitalinterruption.com/secure-mobile-development (https://goo.gl/P1WYcV)- reduce the cost of pen testing

Ep 87: Players gonna play play play play play

Podcast news: Friend of the show, Neil Studd has a new podcast! It’s called Testers Island Discs and there’s an intro episode out tomorrow: https://twitter.com/TestersIsland?lang=en

Now, on with the show.

So this is all Richard Bradshaw’s fault.

He came and did a talk at the BBC back in…August, I think? And there was a slide in it about the phrase ‘I’ll have a play’ and how that isn’t what testers do, and it undersells our skills. He said (and I’m paraphrasing here as it was months ago) that we could write charter for that session and having a play becomes exploratory testing.

I kind of disagreed when I first heard that, because to me, a charter for Exploratory Testing is specific. There’s an aim, or something specific being tested: a certain area, or a certain user journey, or even a user journey as a user (a bad actor as security testing, or a visually impaired user or similar).

When I ‘have a play about’, I am doing just that – there’s no real charter. There are a couple of reasons I ‘play about’:

I’m getting familiar with a product
I’m not confident that I’ve tested everything but I’m not sure what I’m missing

The first one is pretty easy – I’m just seeing what I can see without getting too much help – what makes sense going in cold, what could be improved, are there any quick wins, things like that. I could definitely wrap that up in a charter and a time block and call that exploratory testing.

But the second one is much more nebulous. I don’t know what I’m looking for, other than a sense of being more comfortable with my testing. It usually happens that I’ve tested a story and not found any issues, but I’m not happy. I’ll go make a cup of tea, and head back to my desk and start looking around a bit wider. Sometimes I think up a scenario for the original story that I’d missed previously, sometimes I find completely unrelated issues, sometimes I get a sense that I understand the system a little more. It’s a lot hard to wrap that in a charter when I don’t know why I’m not happy. I can’t formulate the words for bits I’m not happy with sometimes.

Does this make any sense? Do other people have this?

But I at the same time, I’m no’t just clicking about, am I?

That’s what it feels like, but it’s not.

There’s context there – pre-existing domain knowledge, or client knowledge, or team knowledge. There’s knowledge of what apps or websites or programs are meant to do or how they’re meant to look. It’s knowing which browsers are relevant, and which are awkward to work with. I may not be able to put words to why I’m not happy, but if I let myself wander, a little defocused, and aimless, I might find something.

It’s playing around with all the knowledge of being a tester in the background.

And that was a weird thing to realise.

I had split my work mentally into structured testing that was in a testing mindset and just playing about which was freeform, nothing intense, more of a safety net, or a get to know the product thing. I think it’s because it’s fun as well – it’s fun to explore a system and check out all its nook and crannies, and I think I’d separated that out from structured, goal-driven testing?

But it’s not really separate. Playing about, clicking about, when you’re a tester, is testing. It might not be structured even as much as exploratory testing it, but it’s still testing. And you could probably wrap it up in a mission, maybe break it down into charters, and put a time limit on it. Then you’re doing exploratory testing. It’s like the difference between science and dicking around is taking notes. The difference between playing about and exploratory testing is writing a charter.

The next time I have a play about, I’m going to write a charter or a mission (even if it’s just ‘get more comfortable’), and take notes. This will also hopefully improve my notetaking skills, which is a bonus as my notes have gone to shit recently.

So, I hate to say it, but I guess I agree with Richard now. I really enjoy the phrase ‘playing about’ or ‘noodling about’ because it’s fun, and I think testing can be fun, but I also see that it means a lot. I might start switching it out ‘I’ll have an investigate’ or ‘I’ll take a look’, as that still suggests more structure and skill.

Ep 85: Sound Effects and Overdramatics

This week is the Manchester Testbash crew!

Matt, Claire, and I talk about out experiences of putting our first successful conference submissions together and how we’re preparing. We’re planning on doing a part two post Testbash to reflect on the days themselves.

You can find Claire on twitter and she writes for Ministry of Testing.

First, what we’re doing at Testbash:

Matt is giving a workshop(!) on APIs for beginners
Claire is giving a talk about her personal experience with imposter syndrome
I’m giving a talk on mental health and anxiety in testing

Tickets are still available!

Step one: Submission

Where can I start to gain confidence and experience before submitting?

  • Write blogs but don’t publish them
  • Take something you’ve read/watched/heard and create a small talk to a couple of people at work that you feel comfortable presenting to
  • Lightning talks/90 second talks
  • Attend meetups and just talk to other people – you will have something to share and you will be surprised at how different your experience can be.
  • Build up to bigger talks – present a talk to more of your company, or a local meetup (plenty of meetups that encourage new speakers).
  • If you get the opportunity – go to a peer conference
  • Outside encouragement – people will cheerlead you if you want to get into public speaking

What to submit

  • Technical talks
  • Overcoming a challenge / solving a problem
  • Personal experience report
  • Workshop

Where to submit

  • Consider whether you will have to pay to speak
  • MoT Open CFP
  • Other conferences which have a deadline for submissions
  • Smaller events like Leeds Testing Atelier
  • At home or abroad ?
  • Will your employer allow you to attend or will you need to book time off ?
  • Do you know anyone who has spoken at an event you are interested in ?
  • What was their experience like?
  • Can you submit the same talk to multiple conferences ?

How to put a submission together

  • What do I want to say
  • Why do I want to say it
  • What will other people get out of listening to me

Step two: oh god, what have I done (putting the talk/workshop together)

  • Scripted talk or more free-form ?
    • Notes
  • Avoid slides that you just read out
  • Using keywords or images as prompts

Step three: Practice AKA the countdown

  • Talk through it yourself – friends / family etc
  • What reads well on paper isn’t always very easy to say. Better to compromise your language so that it flows more naturally than tripping over pronunciations.
  • Practice at smaller events / internal company audience
  • Resources if you are new to speaking (i.e.: James Whittaker videos: one, two)

Episode 82: You’ve Got All These Great Answers

To All These Great Questions

WE’RE BACK and with a bumper episode of Let’s Talk About Tests, Baby.

Matt and I have both gone for interviews, so we decided to do an episode about it. We talk about applying to jobs, sussing out the language if job adverts, CVs and cover letters, and the dreaded interview. Matt also has experience on the other side of the table, so he brings that insight to the episode as well.

Mindmap
Interview MindMap

Things we mention:

BugFinders
uTest
http://www.testingeducation.org/BBST/
http://www.satisfice.com/info_rst.shtml
http://www.istqb.org/
http://www.askamanager.org/

Episode 81: It’s the small things that count

This week I talk to Andrew Morton about unit tests!

We cover the basics of units tests: what they are, and what they do. We also cover unit tests as documentation, pitfalls to avoid, and some tools you might see in the wild.

References and resources we mention:

https://github.com/testingchef/bad-units – A repo of bad unit tests
https://www.youtube.com/watch?v=q8I8-hVS5JI – Andrew’s Whiteboard Testing video on unit tests as documentation

Ep 80: Who lives, who dies, who tells your story

Manchester is my adoptive city, and I am even more in love with it after this week. Strange to feel pride and be heartbroken at the same time. I was hoping to get a bee tattoo but all the tattoo parlours have been overwhelmed by demand, so I may get one at a later date, and just make a donation to the fund.

Life goes on, and so do we.

I went to an event on Thursday about storytelling called What Makes You, You? I wanted to go to primarily get tips for my other podcast (Inner Pod), and help people come up with their own stories, but I actually found a lot that connected with me on a testing level.

So, testers being story tellers isn’t a new thing, people have spoken about it before (on this podcast even!), Huib Schoots is running a workshop on it at London Tester Gathering workshops etc., but the guy who did the talk/workshop thing (Andrew Thorp) really solidified the crossover between testers and storytelling.

Okay, so firstly, there were parts of this that were a bit wanky. Like the concept of storyselling (selling something using storytelling), but he did admit he usually gave this talk to corporate types who sell things for a living, not for creative types who are looking for a way to either sell themselves to get better at public speaking or interviewing. Overall I found it incredibly useful, and have some pointers in my arsenal when it comes to building a good story and helping people build theirs. If you’re in the area and interested, he’s doing the same talk again on 13th June.

The most common thing I hear when asking people to come on the show is something like ‘I’ve got nothing new to say/I’m not interesting enough’, which firstly is bullshit otherwise why would I be asking you to come on the show? But more importantly, Andrew had 3 principles for having a good story, and they are basically 3 principles for being a good tester.

Hoovering: Hoover up experiences. Not just your own, other people’s as well. Think about observational comedy and how those kinds of comedians can turn something small into something significant. They can see the importance in these things.

Testers should do the same. It’s only in learning about other’s experiences that I, as a web tester, can learn how people interact with the sites and apps they use. It’s as I learn about new technology that I can see how it’ll connect with my job going forward. It’s noticing the small things and the effect they may have that mean testers can catch cases that others may have missed.

Be interested: Being interested is great on many levels. Interested people tend to be interesting. Firstly, interested people tend to do active listening, which is listening where the listener fully concentrates, understands, responds and then remembers what is being said. It’s not reflective listening, where you repeat what the speaker said to drive home shared understanding, and it’s definitely not the false listening I find myself falling into occasionally, when I’m either jumping in to say something and slightly talking over people because I’m excited, or even worse, waiting for someone to stop speaking so you can speak. We all fall into the trap – we think faster than we can speak or listen – but we need to learn to listen.

This is fairly obvious in how it relates to testing, we should be listening to our co-workers, our stakeholders, our clients, everyone. Our job is just as much taking in other people’s stories as it is telling our own. In fact, one of my biggest issues is getting clients to tell us their stories. People know they need a website but often smaller businesses, or more corporate brands may not know what message, what story they are trying to sell.

Be willing to open up the bonnet. Curiosity! Curious people have things to say, and testers are curious people. Not just curious in their work but outside their work as well. Being curious not only facilitates the previous two points but also allows you to craft your own story. If you’re not curious how do you know what excites you, or angers you, or just leaves you apathetic. These, across contexts, will allow you to craft your own story.

I’m giving but a small sliver of my Testbash talk at Liverpool Tester Gathering on 15th June. Come along! https://www.meetup.com/Liverpool-Tester-Gathering/events/240216573/

More links
https://www.ted.com/talks/celeste_headlee_10_ways_to_have_a_better_conversation

Andrew’s free PDF on storytelling/storyselling: http://mojoyourbusiness.com/wp-content/uploads/2017/03/School-of-Mojo-Manifesto-21.03.17.pdf

Ep 79: It Takes Two (Rebroadcast)

Back in October last year, I spoke to Maaret Pyhäjärvi about pair testing. We did a session of pair testing then recorded our debrief/thoughts on the session. It is a great episode, and the shownotes are full of great resources, including the mindmap we used during our session.

However, the editing and sound quality was not amazing. Maaret was quite quiet compared to me, and it was difficult to listen to. I’ve been wanting to reedit and rebroadcast for a while, and finally circumstances came together to do this. The sound still isn’t perfect, but I think it’s a lot clearer, and the volume makes more sense. I’ve also made the outro a bit quieter so there isn’t suddenly a big jump in sound.

I hope you all enjoy the new improved episode 61!

Ep 78: May the Testbash be with you

So this week we finally discuss Testbash Brighton and the freshly announced Testbash Manchester!

First, Testbash Brighton. Testbash was awesome as always. Brighton moved to a new venue but all the normal Testbash awesomeness applied!

 

Conference Day Takeaways

  • Nice to see a focus on empathy
  • Empathy for developers
  • Empathy for customers – Tito customer feedback/Ethics in testing
  • Good balance of tech/non-tech
  • Range of topics! Starting as a new tester in agile, AI and testers, ethics, how testing has changed, API testing, Toolsmiths.
  • Not all speakers were testers: there was a professor + 2 Devs turned CEOs!
  • Testers creating better Turing tests kind of blew my mind despite being bloody obvious now I think about it
    • Testers as the ideal middleman between understanding how people work and how machines work.
  • Lots of juicy ideas to take into my new job that I’ve just started. Amy Philips and Del Dewar’s talks covering ideas on starting on a new project and what good leadership really means.

Open Space Takeaways

  • A nice, low pressure environment to share
    • Can be a discussion or a question, not necessarily a talk or workshop (though they can be done as well)
    • Dan Billing did a demo of Burp Suite and a whirlwind tour to Ticket Magpie (see episode 77)
  • Matt did a session on bringing testing into education (schools, uni, etc – see episode 75 for our talk on a testing syllabus)
  • I did a session on mental health with Mike Talks, which was so awesome
  • A talk popped up during the last break of the open space where we shared stories about people in our lives we were grateful for, and why, and it was a genuine love-in.

Testbash Takeover!

So if you check out the lineup for Testbash Manchester you should see a couple of familiar names! Matt is doing a workshop on API testing, aimed at people wanted to get into API testing who may not have done so before.

I am giving a talk called Anxiety Under Test. With as few spoilers as possible, I’m going to be talking about my anxiety, and how I’ve applied the strategies I’ve learned to help cope with my anxiety to testing, with a quick look at how you can check in with your own mental health and keep on top of it.

The rest of the lineup look A+ and we’re looking forward to an amazing few days!

If you can’t make the conference there will be the pre- and post-testbash meetups as always. These are free to attend so come along and hang out with a bunch of great people!