Automation is changing quality assurance testing, and QA professionals must adapt or lose relevance.
Quality Assurance (QA) testing is often done at the end of the software development process. On a recent episode of
, I spoke with Deborah Lewis, a Lead Quality Assurance Engineer at Red Ventures, about why this is a bad idea. We also talked about the importance of QA, how QA engineers can work effectively with other teams involved in the developer process (developers, product managers, etc.) and how technologies like automation are changing software testing.
Deborah has held roles as both a
. She’s also a co-worker of mine. Deborah and I both worked for CBS Interactive, which was acquired by Red Ventures in 2020 and is the parent company of TechRepublic, ZDNet, CNET and a host of other online properties.
What makes a good QA engineer?
Bill Detwiler: Before we get to talking about what we’re really here to talk about, which is why QA is so important to the software development process and why it’s often under appreciated in that process. I’d love for you to share with the audience, your story. How you got into QA, because you had this great story. And I think it really speaks to the roundabout ways that people often get into software development or technology and how skills that are learned maybe outside of the classic developer path can be not just beneficial, but really critical to building great software.
Deborah Lewis: Sure. Yeah, I think QA is really about being methodical, paying attention to details, having a deep understanding of the product and this can apply to anything. So, I have a background in publishing, I was a proofreader and copy editor for years, and it’s really the same skillset. Trying to improve the quality, that’s the goal always. And also trying to be an advocate for your customer, whether it’s the reader of the newspaper or the user of your Web App. Trying to make sure it’s as clear, understandable, and without error as possible. Perfection is an unreachable goal, but we try to do our best.
So, I worked in publishing for quite a long time, had moved to a website as a copy editor. We had less of a need for editorial content, but had no Quality Assurance person. And I was very lucky in that one of my coworkers and my boss recognized that I had already really been doing QA. Helping test the custom CMS tools that we had. And so they offered to shift me over to the QA team. And then you can learn what agile software development cycles are. You can learn what the product is, you can learn how your team works. But the skills to find the errors are sort of some people like to get into the nitty-gritty and some people don’t.
Bill Detwiler: And I can attest to that as someone who oddly enough has a little bit of an engineering background, but then got into editorial. I will admit to this, I’ll give myself props. I think I’m a decent interviewer, and I’m decent editor, and a fairly okay kind of writer. I’m a horrible copy editor. That is not me. The skill that you have, where you have an eye for transposition of letters. If my copy editor sent me back, one email, they sent me back a hundred about from and form because stupid spell check would not pick up on context.
So, I get what you’re saying. And I think that’s really important that you bring that out to talk about that kind of, not just the skill, because anyone can be taught to look for that. But it really is a part of, I don’t want to say your DNA, but it’s part of a mindset, it’s part of what you kind of enjoy doing. It’s something to be able to have an eye for detail and enjoy picking those things up because that’s what I rely on in my copy editors today. So, you’ve got that problem solving, you’ve got that want to improve quality. But you’ve also got a tenaciousness and sort of that, like you said, the passion to be persistent and to find those details, right?
Deborah Lewis: Exactly. Yes. Because, sometimes you have to be persistent about a change that needs to be made, but you also have to have some of these soft skills about how to present a change that needs to be made and why it needs to be made without coming across as criticizing someone’s work. Because we all want to produce the best quality product we can. So, we all have the same goal and you just have to stick to your guns sometimes when something really doesn’t work, but they don’t believe you, that it doesn’t work.
Bill Detwiler: How do you do that? It fascinates me. Well, actually it doesn’t because both writing editorial is very similar to creating software. We use words, we use video. To communicate something else, software use code. I mean, I quit writing code long time ago, because again, I realized my Fortran, my Pascal, a little C, I kind of got out of it and I realized, “Hey, this isn’t kind of my thing.” And found a different medium to communicate with. But one thing that all writers and it strikes me that engineers have in common is this passion for their work. The fact that they own their work. They see their work as theirs. And that’s really interesting that if you come in with the mentality of, “You did this wrong.” Immediately people can be defensive, they can put those walls up, they can react negatively.
Now, I’m not saying that’s the right way to be. As an author I can admit I’ve done that unfortunately, and then had to go back and apologize. As an editor too, I’ve done that with people and said, “You don’t have to be the other person (amendments 00:06:25).” So, how do you do that? I’d love to hear the actual kind of, because I think that could benefit people in hearing how they deliver that kind of feedback. So, if you’re in QA, or if you’re the coder, or if you’re someone, or the graphic designer, or the project, however it is, who’s receiving that information, how to take it. Can you maybe speak to, or share some of your experiences and what’s worked and what hasn’t worked when you’re sharing that kind of information?
Deborah Lewis: I mean, it’s going to depend on the person that you’re interacting with, because everyone’s different and sort of hears feedback differently. I think a good rule of thumb is to start with something positive, I really liked this, what if we did this to make it a little clearer for the end user? Something like that. So, that it’s not just, this doesn’t work, can we change it to this? It’s more of a conversation rather than a directive or a requirement. And also I need to be open to hearing them counter what I’ve just said, because maybe what they did really does work. And I just didn’t understand the intention behind it or the goal with it. So, it has to be a conversation rather than prescriptive, you did this wrong, we need to change it.
Bill Detwiler: I think that’s so important too. And again, more things should be conversational at realizing that everyone’s goal is the same. We don’t have to be confrontational to reach that goal. This isn’t about you proving your way is right or me proving my way is right. We can have a debate about that. Like, yes, I understand. Tell me why you did this. Why is this button shaped this way? Why does this feature take you here? And there might be, as you say something to that, that you didn’t think about or didn’t understand that they may have had a reason to.
But then again, they may not have thought about how the user or someone else might perceive because we all get blinders on. And again, very similar to the copy edit experience, which is I wrote that this way. Oh, that wasn’t my intent at all. Well, someone who reading it might take it that way. And so, you have to be very careful. So, I understand completely. So, with that kind of in mind and with your experiences, what kind of advice would you give to people maybe who aren’t part of a dev team, like you weren’t, but want to move their career in that direction?
Deborah Lewis: I would say find opportunities where you can step in and help. Find gaps, volunteer to help out, get to know the developers, have conversations with them, join Slack channels if you use Slack or whatever your chat app is. Join the channels where the developers and engineers are, and just read what they’re doing, and learn from that. Just finding ways to help figure out what the problem is, and then help figure out how to solve it. So, it’s like a job interview, if there’s a job posting for a specific role, you want to figure out what problem are they trying to solve and how are you with your specific skillset going to help them solve it?
Bill Detwiler: That’s very good advice for this or any job actually. So, let’s get down to the issue that we are kind of here to talk most about, which is the importance of QA in the software development process. So, what makes QA so important and critical to building good software?
QA should be part of the entire software development process
Deborah Lewis: I guess the quality. I mean, we’re all trying to build quality into the product. It is everyone’s responsibility, but as you mentioned, people can get blinders on when they’re doing their portion of the work. And this extra set of eyes, this fresh set of eyes can really pinpoint things that no one had thought about. In addition, if you involve QA in the entire cycle, you’re going to have a more efficient process. You’re going to reduce your risk, so you’re going to have fewer bugs get into production. You’re going to increase your products value and provide more value to customers. I mean, quality. Quality, quality, quality.
Bill Detwiler: And since you’ve been doing this for a long time, have you seen the risks to not having a robust QA process increase?
Deborah Lewis: That is an excellent question.
Bill Detwiler: I’m just thinking about, all of the things that we rely on our software and apps for. I mean, it was always critical. I’m not sort of saying that, but it seems now to me that there is a lot more exposure, especially with privacy and security. Or just with the ability of customers to go to other apps, other software, to go to other outlets. That it’s even more important now, not that it wasn’t important in the past. But the effects of perhaps allowing buggy code or a poorly performing app to go into the wild, maybe felt much more quickly than they were in the past and the consequences can be more severe.
Deborah Lewis: You make an excellent point one I hadn’t considered from that side. But for instance, an updated redesigned version of a native app. Users are always unhappy about change typically, but this was such a drastic change that the users hadn’t been prepared for. And so, there were lots of negative reviews in the App Store. So, that’s maybe not entirely specific to QA. I mean it was a fully rethought design of the app, but it’s an example of, if you had some other input, maybe you could have pivoted somewhere in that development process or considered preparing the users by announcing ahead of time, these changes are coming. This is what it’s going to change. That’s probably the best example I can think of.
Bill Detwiler: Yeah, no, that’s perfect. I love that example because I think it hits to that point, which is if QA, and good design, and incorporating user feedback should be part of that continuous cycle of CI/CD, really. I mean, if you’re talking about how you want to ensure that kind of you’re updating, you’re fixing bugs but when you’re making changes, you’re also evaluating the information coming into you so that it helps you develop those changes. So, no, I think that’s a perfect example. And I know you and I have talked about something a little bit, which is making QA part of the process from the very beginning. Like, it can’t be, hey, I wrote this, we’re ready to ship it out the door. You’ve got a week to look at it. You got any changes for it? That doesn’t really work, does it?
Deborah Lewis: It doesn’t because, if you haven’t built QA into the process and you leave it as this last step, you’re just going to ship a poor version of your product. Because not only do you have to leave time for QA to do his job, but you have to leave time for the developers to come back and fix the things that QA finds. So, it’s just more efficient. You include us from the beginning, allow us to take part in the discussions. Since we are so intimately familiar with the product, ideally as you’re designing and developing, we can step in and say, “Oh, but what about this? There’s that connection over there.” Because we see the whole picture, but we also know all these, like annoying quirks and details about it. So, I think we’re not just QA engineers, where we’re actually helping giving product feedback, giving technical feedback. We’re sort of the jack of all trades in a way.
Bill Detwiler: You probably, do you think that the QA engineers, you think they know the product almost better than anyone else, even maybe the end users, even the coders, even the project? I mean, maybe the product managers, but I get the sense from having tested internal tools, myself over my career, have filed troubled tickets, having filed a bug tickets myself. Having tested like you CMS systems before they go out as someone who’s a user. You get a much deeper understanding of all aspects of the application or the system, as opposed to even someone who just uses a small portion of it or has those blinders on and only focuses on one small thing. It seems to me that the QA people probably know those products that they work on better than almost anybody.
Deborah Lewis: I think you’re absolutely right. I mean, if we’re really doing our job well, in my opinion, we do understand it better than anyone else because, whatever system you used to track your software development. We use JIRA you’re seeing every ticket that goes across the board. You’re not necessarily working on every ticket, but I at least being sort of a proactive person, I watch the board and I look at every ticket on the board. So, I’m aware of everything that’s happening, even if it doesn’t directly come to me. So yes, I think we do understand from all perspectives, the product. Whereas most people have one track, or tunnel, or silo that they are working on.
Bill Detwiler: I think that’s really important because it gives you a chance to see and hopefully relate to the folks sort of writing the code, how everything works together, right. And how, hey, well if you’ve made this one change, that’s also going to change this over here, which might break that. I’ll give you an example. Just last week, I had a problem with my video and coding system, engineering fixed it, it broke something else. They had to fix that. And so I’m like, why is this not working? Why did it break this
And it’s nothing on them, because I do think that it’s almost impossible to find every small interconnection sometimes, especially with things that have quick fixes or very complicated systems. Sometimes you just need to roll things out and fix the small things later. Just don’t cause a security breach, don’t cause a privacy breach with PII, don’t do something really, don’t bring the site down, don’t-
Deborah Lewis: Don’t break the ads.
SEE: Resume refresh: Expert tips to make your CV stand out (free PDF) (TechRepublic)
QA must demonstrate its value to software development
Bill Detwiler: Whatever the core function of your app or your software is doing, don’t break that. But sometimes with the systems, especially legacy systems or the interconnectedness of systems, it’s difficult to map that out. And I think this is a great segue to my next question to you, which is how has QA changed over the years?
Deborah Lewis: Well, I think the industry as a whole… Well, so let me back up. Like in the publishing industry where competition and lower revenues caused them to look for ways where they could tighten their belts. Copy editing was one of the things that was first to go. And I think sort of those final checks and balances are often where we see cuts because you can and should have your developers doing their own testing products can help test. As you mentioned, if you haven’t broken the core functionality, you can roll it out and sort of have your users test a little bit and fix from there. So, I think we’re having to, shoot what’s the word. We’re having to prove this way…
Bill Detwiler: Justify, not justify. I don’t think that’s the right word. But it sounds like you’re saying that you have to step in and kind of say, hey, here’s the value that we’re adding to the process, right? I mean, I don’t want to put word in your mouth at all.
Deborah Lewis: Yeah. We need to demonstrate our value and what we bring to the table. That was the word I was looking for.
Bill Detwiler: Is that something that is… And you said that has changed. I think that we’re seeing that kind of now. What role does sort of the change in technology play? So, we talk a lot about manual QA versus kind of automation. And the way that systems have become so complicated that it almost requires some level of automation to ensure you’re doing the right kind of testing. Whether it’s QA or security testing or. So, we talk a lot about sort of this blending of DevSecOps together and automation being kind of key to that. So, talk maybe a little bit about what you’ve seen there in terms of basically automated tools coming in and being maybe good at some things, but still that manual. So, they don’t have to be maybe mutual exclusive.
So, I think you need those automated tools, but then sometimes you just need a real person to say, well, you know what? Because people are quirky things, right? We’re kind of odd creatures and you can program an automated tool to try every combination, but invariably there’s something. A lot of times it may be the person who created the algorithm didn’t think of, or there’s a different way to do things. So, talk maybe a little bit about that. Because, I do interviews with folks and I talked to developers about this when it comes to no-code and low-code tools as well. So, it’s very similar.
Deborah Lewis: Yes. I think automation is really valuable and it can maintain a certain level of quality. And it has regular checks to make sure that your pages aren’t 500 or whatever it is. And we have third party testing services that we can use and we do use where it gives us a broader coverage of devices, platforms, geolocation. Especially with devices, no one can maintain a device firm anymore. At least not for individual QA teams, because that would be astronomically expensive. So, we might have some basic native devices we test on, but these third parties services allow us to really have much broader coverage. And then your dedicated smaller manual QA team can do much more targeted strategic testing thinking through more from a product or customer user experience. It just allows us to be craftier about what we do.
Bill Detwiler: Well, and it sounds like I’m hearing you, it allows you to be more efficient?
Deborah Lewis: Yes.
Bill Detwiler: I mean, to make the best use of the humans, and to allow the technology to augment, and to offload some of that, either the expense as you’re talking about with the testing boards. I remember having a lab years and years ago and you could have multiple machines. You could see machines set across all different things. It’s just not practical now. To your point, exactly. And so I think that’s a really good point you make in that it’s up to QA teams to really evolve and to look for ways to maximize the effect of the different resources they have access to.
And humans are one resource, a third party automation, another resource. If you don’t do it right. Have you seen experiences where that just hasn’t been done very well? What happens, what’s the downside if you’re a QA team and you’re not sort of thinking about it this way. Or you’re an engineering or product management, and you’re not thinking about this way, what’s the risk that you run? because I think that’s important. Have you seen that happen?
Deborah Lewis: I don’t have personal experience with that. Let me think about this for a second, brain processing.
Bill Detwiler: I’m just wondering if it’s… To me, it seems like there would be. Again, I’m really curious because you’re much more close to this than I am, but to me it seems like that you run the risk of exactly what you were talking about earlier and going away of the dinosaur. And sort of you were talking about copy editor being, as the publishing industry was changing. You saw them as one of the first departments that got cut. Now I can say, I do have personal experience with that. I remember when that happened during my career. That was one of the first departments to go, and it was, “Well, now you’re going to do all your copy editing.”
And I know from my experience, that I understood the business decision behind it, but I also knew what it meant for the writers and the other editors to pick up that additional work who may be weren’t specifically suited to that. And there was a period of time, I think, on the web, in publishing where you saw a lot more typos than you used to. And readers would sometimes… I mean, very big publications, not just web only. But you started to see that have a negative effect on the perception of the quality of the org. And so my question is, and I apologize if it’s kind of leading, but I’m curious to see if that’s the same thing that you maybe have seen or heard of, or you think could happen in that QA kind of environment as well?
Deborah Lewis: No, I think you drew a really powerful parallel. And if you’re not doing QA right, then your product won’t be as valuable. Your customer base, your readers, your users are going to lose confidence in you. This is going to impact revenue, I believe. And on the other side of it, if you’re using resources, but not using them wisely, whether it’s hiring an automation team or hiring QA team, or signing a contract with a third party testing service. You’re spending money on something that you’re not getting a return on. So, again, this is going to affect your bottom line. So, it’s really using your resources to the best of their ability and being smart about it. And QA helps you be smart about what you’re doing.
How QA can build strong relationships with other software development teams
Bill Detwiler: And I’d love to kind of wrap up our conversation and I think this is a perfect kind of segue into that with, what advice would you give to folks looking for ways to improve QA within their organizations? And perhaps specifically for QA folks maybe who are looking to build those strong relationships with other people invested in the software development process like engineers, or like product managers?
Deborah Lewis: Sure. I mean, I always will say the earlier you get QA involved in the process, the better. So, that would be go toward how do you help improve the process? I think it’s really valuable to connect QA teams across the organization. They might be working on different products, but I’m sure they’re facing similar challenges or maybe have solved a problem that you are currently trying to solve. And it also fosters a sense of community and collaboration. So, I think that’s really helpful. And in terms of developing relationships with others on your team or in your organization. I think you always want to be trying to add value, in whatever shape that takes.
One thing for QA in particular. I think that people really appreciate is you need to write clear steps for how to reproduce the bug, just get a formula together and be clear every time. I also think it’s great to just always be curious, always listening, always learning. Join those Slack channels, learn what you can from other people, offer your feedback if you think you’ve got something helpful, maybe even if you don’t think it’s helpful. But join the conversation and this will raise visibility for you, for your team. It’ll help people understand what QA does and how we do it. And I think that’s it just participate.
SEE: Hiring Kit: Full Stack Developer (TechRepublic Premium)
Bill Detwiler: And I think that’s so important. And I love your example of being proactive of talking about how you monitor those Slack channels. Because it strikes me that in today’s world that you’re always having to… The days of being able to sit back and just sort of do nothing, and I won’t say do nothing, but maybe not constantly be talking about how you’re bringing value, and advocating for yourself, and advocating for the work and the value that you bring. It’s not possible in today’s kind of world, whatever you’re doing, whatever industry you’re in.
And this is on both a macro-level and a micro-level, the individual level and at the company-wide level. The department level to think about how we’re doing that. So, I love that analogy. Is it something that you think you’ve always had? I mean, is it something that you learned because as people listening to this think about, well that sounds great. How can I do that? I love those practical bits of advice, which is like clear tickets, be proactive, share the advice. Is it something that came naturally to you or did you have to learn that? And if so, how did that happen for you?
Deborah Lewis: For me personally, I think it came naturally. I’m an observer, I’m always observing what’s happening around me and sort of processing how that fits into the larger picture. I’m always asking if I can help. Well, always interested in helping whether that’s… So, I grew up with horses, so I was always asking where I could help, do you need this done? Do you need that done? When I was working in newspapers, I started out compiling the entertainment listings and then there was, I don’t recall how it came about, but I realized there was a need for a proofreader for the classifieds. So, I said, “Hey, can I do that? Hey, can I try this? Can I try that?” And I think just being a lifelong learner, just being interested, and curious, and wanting to learn. And if you take the steps to learn something, it will carry you forward into the next thing.