I've recently been thinking a lot about all of the projects I've worked on throughout my career as a developer focused on testing. I've been working on web applications for over 15 years, and while technology has moved forward since the days when I began, a lot of the underlying architecture for most of the work I've done has been the same.
When you work on the same tech stack in different projects, you'll tend to find similar libraries and frameworks in use. For example, you'll often find RSpec in use for Ruby on Rails applications, or Jest for JavaScript applications. Some teams follow standard practices with these tools, and anyone familiar with them can jump in quickly.
This behavior also extends to teams that are either new to test automation or don't dedicate too many resources to maintaining their existing test suite. These teams often make mistakes with their automated testing that show up again and again throughout different organizations.
This article goes through three of the most common mistakes I've seen across multiple teams when building and maintaining an end-to-end test suite for their application.
1) Testers write too many tests
I don't know about you, but the first time I discovered automating end-to-end tests, I felt like I unlocked a magical skill that would save me an incredible amount of time throughout the workweek. "You mean I can write some code and it'll do my testing work for me? Sign me up!"
Whenever someone learns the wonders of test automation, their first inclination often is to test absolutely everything that they can. It's a noble goal, but it's also one that leads to wasted time and tons of frustration down the road. If you add long-running end-to-end tests to the mix, that problem can quickly spiral out of control before you know it.
It's great to write end-to-end tests to verify that a particular workflow behaves as expected from start to finish. The problem happens when you want to verify all of your workflows. These tests are often slow, and if you write too many of them, it'll slow the entire team down to a crawl. Eventually, you'll end up with a test suite that no one wants to run because it takes forever to complete.
Reliability and maintainability are also essential factors to a healthy codebase. The more end-to-end tests you have, the higher the risk of your test code becoming brittle and unmaintainable. Since these tests cover lots of ground, it's not unusual for them to break due to any minor change somewhere in the system. That means you'll have to update your tests more often. In the end, you'll have to spend more time making sure your test suite is working and easy to update.
The solution to minimizing these risks is eliminating tests that provide little benefit while keeping your test suite lean. Most teams want to automate as much as possible, but end-to-end testing isn't meant for complete coverage of your application. Focus on writing test cases for what matters the most. Using end-to-end tests to automate your app's most critical sections allows you to reap the most benefit out of your time by preventing slow, unstable, and messy test suites.
2) Testers won't ask other team members for help
Failure to communicate between teams is one of the leading causes that stop projects dead in their tracks. You can't expect a team to put in quality work when no one talks to each other. Unfortunately, it seems to happen quite frequently with testers. This problem isn't one-sided, though. Either the testing team doesn't talk with others outside of the group, or testers don't receive any information from the rest of the organization.
Any good testing team knows that quality is a whole team effort. While a QA team can have the sole responsibility of writing automated end-to-end tests, it doesn't mean they can or should do everything on their own. Testers can't live in a silo. They need outside assistance to perform at their best. Testers have plenty of opportunities to bring in other team members to help with quality across the organization.
The first group you can call in for support are the developers on your team. They'll have all the technical knowledge behind what you're testing and can help make your life easier by making the app more testable. For instance, they can help improve how you can identify page elements on the website for your tests or do reviews to improve your code.
Another group you can tap for help are the folks responsible for DevOps. With end-to-end tests, you'll most likely need separate environments and services for testing purposes. These folks can help set up these systems to make it easier for you to execute your tests without disrupting the work of others. DevOps can also improve the efficiency of your tests by observing how the system under test performs during your test runs and improving any bottlenecks along the way.
It doesn't stop at with the technical support either. Non-technical members of the team have invaluable information that can guide your testing efforts in different ways. Product managers have better insight into the application's current usage, which can help you shape your test plan. Customer support can shed insight on areas where real-world users have often stumbled upon bugs. These are just a few examples you can find throughout any company.
You can't do everything by yourself, and you shouldn't be expected to, either. Don't isolate yourself or be shy, and ask around for help. It'll ensure your end-to-end tests are both useful and exactly what you need to increase the quality of everyone's work.
3) Testers don't fully understand how the application under test works
As testers, it's easy to place our focus on testing alone. After all, it's the work that you're responsible for completing. No one expects you to start doing software development or answer customer support requests along with your work. However, having a one-track mind when it comes to quality often leads to subpar results because you're missing the details that help build a stable test foundation.
This issue often manifests itself in the choice of tools you and your team make at the beginning of your work on the test suite. Without having a firm grasp on how the application under test does its magic, you're at risk of choosing the wrong tools for the job. You can probably get your tests working with an inadequate tool, but chances are it will hurt the test suite's stability and long-term maintainability.
For instance, using a heavy, full-featured end-to-end testing framework might be excessive for an app with few interconnecting pieces, leaving you with a slow and bloated test suite. Likewise, using a simple integration testing tool for an app with a complex backend architecture won't provide the coverage you need, missing significant gaps that an end-to-end framework can handle with ease.
I recently covered this topic in more detail, which will help you avoid the pitfalls of choosing the wrong tools when starting with test automation.
Dennis Martinez
Not understanding how the application under test works also ties into the previous point about talking with other team members. You can waste tons of time trying to automate some tests because you're unaware of different services or potential technical limitations that can hinder your testing efforts.
For example, an application can have an asynchronous job processing system running in the background to process certain information. If you didn't talk with the development team, you might have been unaware of how the system processes the data. You may have spent countless hours wondering why your automated tests for that section didn't work properly without knowing this.
You don't need to be an expert on the complete technology stack that runs the applications you're testing. But having some knowledge around how the system works will improve your testing efforts. You'll know which tools are more appropriate, and you'll learn how to tackle different areas according to how the system works.
Summary
When you work on different end-to-end test automation projects, you might spot a couple of mistakes that happen more than once. These issues most often pop up with teams that are just starting with test automation or treat it as an afterthought.
The first issue is when testers want to automate all the things and write too many tests. Knowing the power of automation can become addictive. It's easy to go overboard with this power and try to cover every nook and cranny in your application. While it often comes from good intentions, it leads to buggy and sluggish test suites.
Another issue that pops up is testers isolating themselves from the rest of the team. They don't take time to get insight from others, missing out on beneficial details that can shape their test plans. Everyone working on the same project, from developers to system administrators to project managers, has insight that you won't get in your day-to-day assignments.
One of the most prevalent problems happens when testers focus too much on automation and too little on the system that they're testing. They know how the application works on a functional level, but they're unaware of what lies beneath the surface. Because of this lack of knowledge, testers waste time using inefficient tools and running into roadblocks that others could have cleared.
Thankfully, solving these issues doesn't take a lot of effort if you catch them early. Keep your end-to-end tests lean and focused on handling what's essential and leave the rest to other forms of testing. Ask for help from the rest of the team and use their feedback to improve what you test and how you're testing. Finally, take the time to understand the applications you're working with - a little time spent up-front will save you lots of time down the road.
Even when using the same libraries and frameworks, every team works in their way. You'll encounter different problems from time to time. As long as you become aware of the obstacles ahead and can work towards resolving the issues, you'll continue boosting your skills and continue on your path of being the best tester you can be.
Have you encountered one or more of these common mistakes in your testing efforts? Are there other mistakes you've seen more than once in your testing work? Leave a comment below and share your experiences!
