Imagine today is the first day at work at a new company as a tester. Excitement is always present when starting a new job. This opportunity feels even more thrilling, though. You're going to work at a company you're a fan of, testing their flagship application that you frequently use. It's almost a dream job for you.
You arrive early at the office, eager to learn and begin tackling whatever they throw your way. A co-worker introduces himself and mentions that she'll be spending a few hours with you this morning. She will show you around the office, where you'll be working, and assign you your first task.
"Great! I'm really excited to start as soon as possible!" you exclaim in an enthusiastic tone. You walk around the office and receive many introductions to the friendly people on your new team. After a brief tour, your co-worker leads you to your desk and workstation. You can't wait to start your testing duties soon.
You log in with the credentials you received from your welcome packet. Once logged in, you ask, "Alright, I'm logged in. Is everything set up for me to begin? How can I get started?"
Your co-worker opens a shared filesystem on the company's network and opens a test document. "Here you go," she says, pointing to the lengthy text on the screen. "This is what you need to get started."
You're slightly confused. You thought your co-worker would start by showing you the application, what processes the organization uses, and what apps you should use to start writing test cases. "Uh... what's this document?"
"This is our test script," she says. "This document contains a step-by-step guide on how to test the application. You don't need to read the whole thing in one go since you'll be going through each step. In no time, you'll have a solid grasp on what to do."
The reality of the situation begins to dawn on you. "Wait, you mean that all I need to do is follow this script?"
"That's right," she states coolly. "All you need is to go through the script, following each step as described. If you encounter a bug along the way, log it to our bug tracker. A developer will verify the report and fix it if necessary."
You feel that this can't be the only thing you're tasked to do. Maybe this is just what you have to do during your first week. Once you learn the ropes, your testing work will shift to other areas. So you ask about different testing approaches. "How's the automating testing situation? What about exploratory testing, can I poke around in the meantime to see if I uncover something?"
Your co-worker frowns. "Oh, no. We don't perform any other testing besides following this script in front of you. We stick to it and avoid anything else. We feel that it's enough to give us a good idea of the health of the application." She stands up and begins to leave your desk. "Alright, I'll let you get at it. I'll pass by later and see how things are going."
"But... but..." are the only words you can eke out as your co-worker leaves. You begin to question if you made a mistake accepting this job without checking their testing processes thoroughly.
Sometimes truth is stranger than fiction
How would you feel in that situation? I don't know about you, but if it were me, I'd want to find the nearest exit and run away as quickly as possible.
While this story was a fictional tale, sadly, it's not far from the truth for a few unlucky souls. Some companies require this kind of scripted testing and use it as their only means to test. Many testers have to deal with this in their day-to-day work. They have to follow a script, leaving behind any freedom to exercise their logic skills.
By "scripted testing," it means that they have a list of scenarios and test cases, each defined by a series of steps. Each step is a simple action to perform and altogether get to the desired result at the end. Essentially, you're just saying steps like "Open this application," "click that link," or "fill the form with this data." There's not a lot to this kind of testing.
There are different reasons as to why companies adopt this type of testing as their sole effort for quality. Some companies need this for business purposes, like a client or auditor needing to know precisely how you go through the testing process. Other companies might feel the need to have their testing process fully documented. Some organizations merely don't know any better, especially if there's no one leading their testing efforts.
The harm of only following a script
As you may have already thought, scripted testing is incredibly restrictive. All you do is follow a series of pre-determined steps with no deviation. It squashes the primary abilities of testers since there's no need to employ analytical or critical thinking. Besides preventing good testers from doing what they do best, scripted testing also has other issues.
These kinds of tests offer little value towards the main purpose of testing - improving the quality of the application under test. You're simply repeating the same steps over and over again. That repetition proves nothing, merely that the app hasn't changed much since the creation of the script.
Scripted testing also doesn't reflect the real usage of your application. The real world is not a series of steps - it's messy and unrestricted. Users of your software aren't constrained and won't do the same things every time. They won't do what the developers and product managers envisioned when building the app.
There's also the issue of human error coming into play. When you have to perform a series of steps manually, there's a good chance you'll miss something. Depending on the tests, it can lead to invalid bug reports and wasted time. The likelihood of human error increase when the tester's brain becomes numb to the repetition they perform day in and day out.
Even with all these issues, some companies flat out refuse to do any other kind of testing. Your company might be one of them. You or others could have pointed out these problems to "The Powers That Be" in your organization. Maybe you tried to persuade them to adopt other testing strategies. But it all ends up being ignored or outright rejected.
When you've exhausted all your options, what can you do?
Show and tell while following the script
If you're at your wits end on trying to convince someone that you need to do more than just scripted testing, you need to show them. The best way to convince anyone of anything is to show them how it improves what's in place. In this case, you need to show people the value of a balanced testing approach in your workplace.
Continue working with your current approach, following the script as they want. But in the meantime, start performing some exploratory testing as time permits. In between scripted test runs, poke around some areas that you think might be at high risk of having uncovered bugs.
When you discover a few bugs - and you will - take note. Write down your bug report, including details on how the issue surfaces. More important, however, is to point out where the script missed this bug. For instance, if there's a script for the area where the bug surfaced, mention what additional steps the script didn't have, which overlooked the bug.
You can strengthen your case if you know how to code. If you can write automated tests, start automating a few of the scripted scenarios. At the very least, automate the most time-consuming test cases or the ones that are more prone to human error. If you don't know how to code, choose a programming language to begin learning. It will serve you well in your career, even if you aren't able to do much coding at your current job.
When you have a few automated tests written, start measuring how much time these test cases take to complete. Use a timer and go through the test cases manually. Then run the automated test case and see how long it takes the computer to run it. Compare both times and keep a record of how each time stacks up. Unless there's a problem with the coding or framework you used to automate the test, your automated test run will take a lot less time.
After a while, you should have a list of bugs that the scripted tests wholly missed. If you were able to write automated tests, you should also have a rough idea of how much time you'll save with additional automation efforts.
With evidence in hand, you now have a strong case to bring to the higher-ups in your organization. This data should be more than enough to persuade your organization to think about adopting other approaches. It's hard to dismiss cold, hard facts.
If nothing changes after all that extra effort, there's not much you can do at this point, unfortunately. If the organization refuses to let go of only doing scripted testing for any reason, you'll either have to deal with it or seek another place to work.
Scripted testing still has its place
This article isn't a call for killing off scripted testing forever. Even with all the shortcomings that scripted testing has, it can serve your organization. It might have its place in the entire process of developing and testing an application.
One of the main benefits that you can extract from test scripts is documentation. Often, these test scripts are overly detailed and contain almost everything you want to know about how the application under test works. For people who don't know the ins and outs of the app, like new employees, these kinds of scripts can serve as a guide.
Scripted tests are also useful for ensuring that the "happy path" is still working as expected. When writing these scripts, it's often from the perspective that everything is working well. If you run these scripts from this perspective, you can still gain some value out of it.
You can also use scripted tests as the base for automating tests. Again, because these tests are detailed, it helps in the process of automation. Everything is there in black and white. All you need to do is translate those steps into whatever automated test framework you're using. It's easier than having to think of each action while writing the automated test case.
Although scripted testing has little value on its own, there are still some benefits to squeeze out of the process. Take advantage of having a script while balancing your approach to testing efforts. Don't discard it because of its limitations.
Conclusion - It's always better together
If you're in the unfortunate position where scripted testing is the only thing you can do, don't despair. You can still take a couple of steps to begin guiding your testing efforts towards a balanced plan.
Start by performing some exploratory testing in addition to your scripted tests. Put your strong analytical skills to use by covering more ground and examining areas of potential risk. This new exploration will uncover issues that your scripts will never find. Use these findings to expose the importance of this kind of testing.
Next, sprinkle in some automated testing to the process if you have the technical ability. If you don't, now is a great time to learn how to do it. Take some of the most important or most tedious scripted test cases and automate them any way you can. The purpose is to demonstrate that scripted testing can be automated and save lots of time. As a bonus, it will free you up for more exploratory testing.
Don't forget that you can still use scripted testing for other purposes. Use it as documentation to help new team members understand how things should work. Let it serve you as you bolster your test automation.
Combining different testing approaches will provide the most value for the organization's testing efforts. The sum of the whole is greater than each of the parts. Don't get locked into one approach. Mixing everything is a recipe for achieving successful testing efforts.
Have you successfully transformed your organization's testing efforts to a more balanced approach? Share your story in the comments section below!