Inspired by a tweet by the Ministry of Testing last week, I began writing a series discussing areas of test automation where it doesn't work quite as you expected.

There were plenty of responses in the tweet that I wanted to cover. The main reason is that many testers out there - myself included - have dealt with these problems at some point in our careers.

This article covers the next reply to the tweet that I called out to me. It has to do with a missing piece of the test automation puzzle that some organizations forget more often than not.

Articles in this series

All five articles are also available in a single PDF for free. Just enter your email in the form below to have it sent straight to your inbox.

How Do You Know Test Automation Isn't Working For You Book Cover

Fix some of the most common automated testing issues with this collection of articles from Dev Tester.

Enter your email address to receive a free 37-page PDF with five articles to improve your organization's testing cycle. You will also receive a weekly email with the latest articles posted on Dev Tester.

I know how vital your email privacy is. You will never receive any spam from me, and you can unsubscribe at any time.

Automation without the automation


Paul Merrill (@dpaulmerrill) had an interesting reply to the tweet:

What this reply refers to is when there's a test suite that contains test cases, but someone needs to "push a button," so to speak. Test execution doesn't get triggered by an event like someone pushing code to the application's repository. Instead, tests are executed manually by someone.

We can also expand this reply by including needing someone to pay attention to the test results. Instead of the team receiving an email or Slack notification, for instance, someone needs to spend time checking the results of the test run and let others know about it.

As you might expect, having someone babysit a test suite isn't anyone's best use of their time. It also significantly reduces the effectiveness of having an automated test suite in the first place.

I've seen test suites in different organizations under varying states of automation. Some teams don't have any tests running automatically at all. Some other teams only have portions of the test suite automated. An example is the application's unit tests running automatically, but the end-to-end tests need manual intervention.

Why does this repeatedly happen on some teams?

Common reasons for teams skipping the automation parts


Unfortunately, it seems more common for teams to rely on someone running the tests instead of handing this work off to an automated system. After all, robots are better suited for this type of repetitive, mind-numbing work.

There are a few reasons that appear over and over again for teams to skip automating their test suite.

The team doesn't have a continuous integration system in place


A continuous integration (CI) system helps automate various phases of the software development process. For instance, one of the primary uses of a CI tool is to detect changes in an application's codebase and trigger a test run. A well-established CI system is an invaluable tool in any organization working on software. Even with its usefulness, some teams don't have such a system in place.

Often, the reason teams don't set up a proper CI system is that there's a common thought that it's more trouble to manage than it's worth. In some cases, it might be right. The typical situation where this happens is when the team decides to run their own system (like self-hosting a tool like Jenkins) and don't have enough resources to keep it running.

However, these days, many CI services make setting up your project a breeze. CircleCI and Travis CI are examples of popular CI services that don't require much time to get up set up. Many software development platforms like Heroku and GitLab are making continuous integration a core part of their offerings. There's no shortage of options out there.

There's no excuse not to have a CI system in place nowadays. The tools are getting more straightforward to use and more powerful with what they provide. The benefits far outweigh any costs.

Expectations that developers and testers should be the ones running the tests and raising alerts


Surprisingly enough, some teams rely on humans for automated tests. These teams try to enforce testing practices by making developers and testers run the tests themselves and notify of any issues.

That's not to say that this practice should cease to exist. On the contrary, this is an essential practice that all teams should have. But when the expectation of having a passing test suite lies on the team instead of an automated system, this practice isn't as useful.

As with most test-related activities, the key is balance. Balancing the approach of having the team run tests before committing changes and having an automated system checking later in the pipeline is an ideal plan of attack. This way, the team can flag issues as early as possible, and the automated system can take care of alerting the team is something slipped through the cracks.

Developers and testers should have a responsibility to check their work, doing their best to detect issues early in the pipeline. But this should never be your only line of defense.

Lack of experience in the team


Sometimes, teams don't have enough experience setting up and maintaining a robust testing environment. It's particularly evident on teams where there are few - if any - experienced testers on the team.

I've noticed these less-experienced teams tend to focus on writing tests and don't pay much attention to ensuring the tests run in a practical manner. It's no fault of their own. If there's no one guiding testing efforts, it's easy for the automation part getting overlooked, especially if they've never had any experience with test automation.

Education is the answer to this problem. Organizations need to give their team the tools and opportunities to learn and grow. There are lots of resources available to learn any area of testing to fill in the gaps of knowledge you or your team might have. Ask your company about continuing your education to make your work more valuable to them and yourself.

Classes or self-education shouldn't be the only way to go about this. Mentoring and coaching are some of the best ways to have people learn new skills. If you're one of the senior members on a team of junior testers and developers, take the lead and teach along the way.

Summary: Cover your bases and automate all that you can


It's crazy to think that some teams forget the "automation" portion of automation testing, but it happens frequently. The first thing to do is get to the bottom of why automation isn't happening at your organization. There are many reasons for it, as outlined in this article.

If it's due to a lack of continuous integration, look into setting a system up. There might be resistance by others, thinking that it's not worth the time. But plenty of services out there handle most of the heavy lifting for you. You don't have to spend a ton of time having a reliable CI system running your tests for you.

If there's an expectation that you and your teammates need to keep an eye out for the tests, change that expectation by balancing your approach. Keep the practice of having developers and testers run tests, but offload more work to an automated system. You don't need to be a chaperone to your test suite.

When the team consists of less-experienced testers and developers, take the time and space needed for everyone to learn. Formal classes, online courses, and pair programming with senior members of the team bring tons of value in the long run for the entire organization.

Without an automated test environment in place, teams miss out on potentially the main benefit of this kind of testing. If you're on a team where people have to get directly involved with test automation efforts, there's no excuse to keep it that way forever. You can take steps to fix it right now. Implementing a true automation strategy makes your organization better for it.

Have you ever worked somewhere that said they had automated tests, but they weren't automated at all? Share your story with how you managed it!

Photo credit: Aleksandar Cvetanovic on Unsplash