A few days ago, the Ministry of Testing Twitter account posted this question:

This question got me thinking quite a bit about my struggles with automated testing. I've had my fair share throughout the years. Anyone who works in the field of software testing will inevitably experience moments where their efforts won't go as planned.

As I scrolled through what other testers commented, many of them caught my eye. The replies didn't grab my interest because they were surprising. Quite the opposite - the responses captured my attention because they were things I experienced first-hand. Based on the replies this tweet got, I know I'm not alone with these struggles.

Many responses hit close to home for me. I wanted to grab a few of them and expand a bit upon them. I'll talk about how I've dealt with them in hopes that it will help you navigate through the rough patches when facing the same issues.

Articles in this series

All five articles are also available in a single PDF for free. Just enter your email in the form below to have it sent straight to your inbox.

How Do You Know Test Automation Isn't Working For You Book Cover

Fix some of the most common automated testing issues with this collection of articles from Dev Tester.

Enter your email address to receive a free 37-page PDF with five articles to improve your organization's testing cycle. You will also receive a weekly email with the latest articles posted on Dev Tester.

I know how vital your email privacy is. You will never receive any spam from me, and you can unsubscribe at any time.

Putting out fires instead of advancing your efforts


The first tweet that grabbed my attention was by Angie Jones (@techgirl1908). Her reply covered a common problem that many testing teams face:

As I mentioned earlier, most of us have had times where our testing efforts don't go well. Sometimes it even works against us.

Have you ever dealt with issues like these in your test automation projects?

  • Some tests are so flaky that they alternate between passing and failing.
  • Your team runs the test suite just before lunch or before calling it a day because the tests take forever to run.
  • No one dares to even look at the test suite because it seems that just glancing at the code breaks something.

I've personally experienced some or all of these issues in a few projects. I've even experienced them multiple times during the lifetime of a single project.

It's incredibly frustrating having to spend most of your working hours fixing things instead of improving upon them. No one wants to do that kind of work. We want to be adding value to the test suite, the application we're testing, and the entire organization. When you're continually battling fires, day in and day out, it quickly sucks the life out of you.

I've noticed two main reasons why an automated test suite gets into this state of disarray. Let's discuss them, along with tactics you can use to improve your situation.

Too much change underneath the surface


The first problem leading testing teams to be in "firefighter mode" occurs when the application under test is in a constant state of change.

That's not to say that it's ideal for an application to remain constant, never changing. Everything needs to evolve and grow to avoid getting stagnant, and software is no exception. However, when change happens too quickly without giving others a chance to catch up, problems will arise. That's where the fires begin.

This constant pace of change happens during a few periods in a project's lifetime. It often occurs early on when a project has started. Many projects begin with a spike, used to explore a solution. While a spike's intended use is temporary, it's not uncommon to see the codebase stay active. If you have to begin test automation in such a project, expect changes to come fast and breakage to happen often.

Handling constant change early on in a project isn't much of a problem. Eventually, things will settle down and become more stable as time goes on. But be aware when a spike goes too long. If this happens, it's a red flag that the project isn't well-defined. Stakeholders might still be in the process of figuring things out, leading developers to create tidal waves of change that testers will have to battle.

In some cases, these constant changes don't come from spikes or stakeholders, though. There are times when the teams assigned to a project are less than ideal for the task at hand. It could be that there are too many junior team members with little supervision or just inappropriately managed conflict brewing between the team. No matter the reason, a poorly-assembled or dysfunctional team leads to projects going off the rails.

If your testing team is having problems keeping a stable test suite due to constant changes, collaboration is the key. The testing team needs to collaborate closely with the developers and the stakeholders to ensure the efforts of everyone is aligned.

Better yet, don't split your teams to handle different tasks. The responsibility of having a healthy project needs to be shared. When everyone is aware of what their teammates are working towards, there will be less of a worry of fires popping up.

Continuing to move forward instead of dealing with the problems


The second reason that has testers dealing with constant issues is that the team continues to move forward instead of stopping. They spend little time dealing with the problems and focus more on building their tests.

For instance, let's say there's a team of five testers working on test automation for a project. The test suite has a plethora of issues causing it to fail regularly. The team then assigns one tester to fix the problem while the remaining four testers continue writing tests.

You may think that this is okay because at least there's someone actively working on fixing the problem. The issue is that you're going to create new problems at a faster pace than dealing with the old ones. Worse yet, if the person assigned to fixing the issues is not as skilled as the rest of the team, the gap widens. It's relatively common that these tasks are assigned to junior team members, simply because the senior members of the team don't want to work on the problems.

It's like a car that has a slow leak in one tire. You can continue to drive the vehicle, moving it forward. But the tire will deflate as time goes on. Sooner or later, your tire becomes completely flat, and you'll have no choice but to stop. And since you didn't deal with the leak early on, you also damaged other parts that were working fine like the rim, brake lines, and fenders. What was a quick and cheap fix became a long, expensive process.

The solution for this is simple - stop what you're doing and fix the problem. The team should pause new work and focus its efforts on making sure existing tests are stable. The team shouldn't be forced to focus on two things at once. Fix the problems first, so future tasks have your full attention. To use the previous analogy, stop driving your car when you know you have a flat tire.

Summary: Take a time out and regroup as a team


When dealing with an environment where you feel like you're fighting - and losing - an uphill battle, the best thing you can do is put things on hold for a bit. It's not what most teams are willing to do, especially in this day and age of "always move forward at any costs" and "Go, go go!"

However, taking a step back is essential if you want your work to be useful for the long haul. Testing isn't a one-and-done deal - it's an evolving process throughout the lifetime of your project.

When problems begin to flare up, take a breather. Get together with your entire team - developers, testers, and stakeholders alike - and talk things through. Regroup as a single unit and work on solutions that will serve everyone's needs. Get the fires under control before they spread out of your control.

Additionally, make sure that once your team deals with the issues, always stay on top of things. It's much easier to combat problems when they arise compared to handling them at a later time. The snowball effect can be hazardous to your testing efforts. Don't let your test suite decay into chaos again.

Have you noticed other reasons that cause testing teams to be in a constant state of fixing things? How have you dealt with them? Share your story in the comments below.

You can join the Ministry of Testing's community at https://www.ministryoftesting.com/, and you can read Angie Jones's blog at https://angiejones.tech/

Photo credit: Jay Heike on Unsplash