With the increase of complexity of web applications these days, along with the added demands of frequent deploys, testing is one of the most crucial parts for successful product delivery. As testers, we've found ourselves needing to look for better ways to handle these increased requests and improve our test coverage across the applications we test.
One of the more popular forms of automated testing nowadays is API testing. Many modern applications use APIs to communicate with various components and modules, both internally and externally. Given the increased reliance on separate systems, it's crucial to ensure these systems work interdependently and as expected.
Other forms of testing, such as manual testing and end-to-end testing, typically handle the duties of making sure each piece of the puzzle for your project works together. However, end-to-end test suites and checking that everything works manually have their drawbacks, the main one being that these tests are time-consuming to execute.
On the other hand, API testing provides tons of value in a much shorter timeframe compared to manual or end-to-end testing. You can create a test plan that covers a large footprint of your application's main functionality while being easier to maintain in the long run. API tests also have additional benefits like starting the testing process quicker since APIs are often the first thing developers build and are less prone to change compared to constructing the application UI.
If you're looking to dive into API testing to reap the long-term benefits with less of a short-term hassle, here are a few areas to consider to ensure you get the most out of your tests during the product's life cycle.
Check the existing documentation and keep it up to date
Before undertaking any API testing for your project, the first thing you should do is take some time to seek existing documentation and become familiar with how things are supposed to function. You'll need details like available endpoints, request and response formats, expected payload structures, and so on. Knowing how your APIs work is essential for building a reliable, maintainable test suite.
Depending on your team, you might have formal documentation, like OpenAPI or Swagger documents. Some groups have auto-generated documentation through inline comments with Java's Javadoc or Python's Docstrings. Others might have informal documentation like a shared Google document or Excel spreadsheet. The format of the material doesn't matter, as long as it clearly tells you how to expect the API should work.
However, before grabbing the available documentation and running towards writing your first automated script, ensure the documentation is up to date. In every team I've worked with, their documentation always has outdated sections. Sometimes, it's entirely out of date with the existing implementation. It's not uncommon to find documentation written years ago by someone who no longer works at the company. Outdated information about an API can waste days or even weeks of your time by misleading your efforts from the start.
If your team finds that the available documentation needs some improvement, spend the next day or two updating any information to help with the team's testing efforts. If someone in QA knows the ins and outs of the API, the testing team can handle this. More likely, you'll have to bring in a developer to help flesh out the missing pieces.
So far, we're assuming you have some documentation to begin testing. But in the real world, lots of teams don't have any documentation whatsoever. If you find yourself in this scenario, QA and development need to meet and come up with some documentation of the API, at the very least. Having easily accessible information about the system will benefit everyone now and in the future, so take the time to set it up as soon as possible.
Plan out what you should test first
The most common mistake any testing team can make when kick-starting their testing efforts is to jump in with no real plan in place. Not knowing what needs to be tested first - and more importantly, what doesn't need testing now - can kill your efforts even before you make your first API request. It's really important to understand what to work on and what to defer to a future date. You can't test everything, so you need to choose where to spend your time when starting.
You and your team will likely have a clear idea of which areas need the most attention for testing. There's a good chance your application has some trouble spots you know about, so it's a safe bet to start with those. Try to keep your initial batch of tests short, especially if you're automating these tests since you'll likely need time to figure out how to organize your work effectively.
If you're not clear about what your team should test first, look for signs of these in the application under test:
- Areas that often break: Almost every software project has sections that create the most bugs, like an embodiment of the Pareto principle for testing - 80% of the bugs come from 20% of the codebase. Focusing your testing efforts on these sections with a higher defect rate can increase your application's quality quicker than any other area you tackle.
- Areas that are difficult to test: Some API endpoints and flows need more work than others to validate. For example, you might have an endpoint that requires no fewer than ten parameters and header combinations to test different responses, or you need to access more than two endpoints to get the information you need. These areas are prone to human error and will benefit from early testing and documentation, and perhaps a touch of automation.
- Areas with high business value that need to be rock-solid: Depending on your organization, you'll probably have some critical areas for running the business. These areas need lots of attention and are safe places to tackle first. For instance, an incorrectly-configured payment processing setup or a broken onboarding process can negatively affect your company.
The key here is that any testing process should be incremental. Take the most critical areas into consideration first, develop a test plan for those areas, and implement it into your testing workflow. Carrying out your plan in smaller, manageable chunks helps you demonstrate the value of your efforts quicker and establishes the foundation for a sound test environment.
Decide what to validate and look for inconsistencies
In most modern applications, you'll find many similarities on what to test for APIs. Presumably, you'll interact with a REST or GraphQL API using XML or JSON. The typical assertions made for these environments include verifying different request formats, the response body, the returned status codes, any changes in the resource's state, etc. Unless the application has specific needs, there's not much variation in what you need to validate between different APIs.
You can check all of these items for your tests. However, you won't usually need to perform the same assertions for every situation. To get the most value out of the testing process, decide upfront the most crucial responses to validate in your API. You can minimize the risk and improve the long-term support of your test scenarios by selectively choosing what you need to test and what you can ignore.
For example, some applications don't have any specific use for the response headers or some status codes returned by the API. You can validate these in your tests, but these are low-value assertions that only add more cruft to your process, making them more brittle and time-consuming to maintain. On the other hand, if your application relies on specific status codes or headers to ensure the application doesn't break, you should always assert the API is returning the correct information.
Another area that's often overlooked but highly important in APIs is consistency between endpoints. Some development teams don't place much care in standardizing how the API consumes and returns data. Although inconsistency is not a bug in the strictest sense of the word, it does increase the risk for future bugs due to confusing testers and developers.
One example is with a RESTful API I've worked with recently. The API had no standardized way to return errors. For some endpoints, the API would return errors in an errors
key; in others, it would use error
or message
. Status codes were also all over the place. It led to the frontend developers consuming the API to continually deal with bugs because of the confusion with different types of error responses. It also made it impossible for them to write functions to handle these errors consistently across the UI, making their code a jumbled mess.
Look for potential security gaps along the way
APIs can be potent tools when it comes to data. A developer can craft an API endpoint that returns tons of valuable information for another application to consume. The developer can also allow control of what data to return. But like all software development, APIs require special care to ensure that they're producing the right data for what's needed at the time - no more, no less.
It's a complicated process from a development standpoint since it requires a thorough understanding of how the API will get used. More often than not, developers won't have the time or desire to comprehend all possible use cases, so they err on the side of making things easier for everyone using the API. They relax restrictions and include more data than necessary, leading to some significant security risks.
Security concerns typically don't fall upon QA, but testers are in an excellent position to become a safety net thanks to our ability to observe what developers might miss. Unfortunately, most API testers only perform the basics - they check the necessary status codes and response structure and call it a day. Doing the minimum is a missed opportunity to improve the application's quality by making it more secure.
You might think you don't have what it takes to perform this kind of work, but you don't need to be a security guru. You already have the ability to help spot these insecure areas effectively. Here are some things you can do while performing testing to help make your API more secure:
- Be on the lookout for data leaks: As mentioned above, some APIs return too much information in their response body. In some cases, an API response can contain personally identifiable information that shouldn't be exposed to the public, such as email addresses, credit card details, or social security numbers. While testing public endpoints, pay attention to the response body and alert the development team of any sensitive data you spot.
- Test for restricted access: If your API has endpoints or resources that return information that should be limited to a user or a group, you should verify that non-authorized users can access these. For example, if an endpoint uses some form of authentication like a token, try accessing it without the token. Another example is if the API uses a parameter to fetch a resource belonging to an authenticated user, check if you can access it logged in as another user.
- Check boundaries to prevent abuse: Some APIs have endpoints that allow a user to return a specific number of objects. You can do some boundary value testing, which can expose security holes, like entering a negative number, a very high number, or a non-numeric value. Another area to check is if the API has rate limiting in place by repeatedly accessing the API. Rate limiting prevents abuse and potential denial of service attacks that can bring your entire system down.
These pointers are just the tip of the iceberg when it comes to API security. If you're interested in learning more about improving the security of your APIs, the OWASP API Security Project covers your needs.
Give yourself time to explore
Automation is an excellent choice for your API testing strategy. APIs tend to be some of the most stable areas of an application and cover a lot of ground, so you'll encounter fewer issues than end-to-end testing while still having decent test coverage. It's a big reason why many teams prefer API testing since it provides plenty of value.
However, some teams place too much confidence in automating API tests. They might think that having enough automated test coverage on the API level is sufficient. Remember that not everything can be automated in testing, and API test automation is no different. You still have plenty of scenarios where non-automated API testing is useful.
As with all other forms of testing, the most effective strategy is to balance different types of testing into the mix. With API testing, that means taking the time to do some manual, exploratory testing. Use automation to handle regressions or scenarios that can be data-driven. Then, use exploratory testing to uncover what your automation doesn't know or isn't built to handle. It doesn't have to be a time-consuming process, either. You can fire off a few different requests to the API using a tool like Postman, changing a few parameters to see how the API responds.
Some good areas to explore are changing required request headers and payloads. If your API requires a Content-Type
header, what happens if you send a non-supported MIME type or nothing at all? Will the API accept different languages and Unicode characters? What about sending extra parameters in the payload? By spending just a few minutes doing some random checks around the API, you'll be surprised at what bugs surface through this kind of testing, which automation can't do.
Summary
API testing is becoming more critical in the world of QA, thanks to the prevalence of APIs in modern applications. It's also becoming one of the most preferred types of testing with teams worldwide next to end-to-end testing, given the high value and amount of test coverage it can provide. For those looking to boost their application's quality through API testing, you have a few places to consider to make it easier on your team.
First, you should become familiar with your APIs through available documentation, ensuring it's as up to date as possible. If your team never took the time to document the APIs in use, now's the time to do it. You can't test if you're not sure what exists in the system, so make sure you take the time to have things written down.
With your knowledge about the API, you might want to dive straight into testing, but that's a mistake. Instead, make a plan on what you should test first so you can remain focused on providing value for your organization. It would help if you also decided what to validate when it comes to requests and responses. You'll probably already have a few ideas on where to start.
Finally, don't get bogged down with the basics or by trying to automate every single aspect of your API. During testing, keep an eye out on potential security issues like personally identifiable information and endpoints that return data from another user. And always remember that automation isn't a silver bullet - a well-balanced approach that includes exploratory testing will help you dig up more bugs than automated tests can.
Just like automation doesn't solve all your testing needs, API testing doesn't replace other forms of testing. You'll still need unit testing and end-to-end testing, among others, to get the most out of your efforts. However, you can get far with test coverage on a well-structured API test suite while minimizing the reliance on the application UI. It's an excellent strategy to employ for your projects.
What other tactics do you use while testing your APIs? Share your tips and tricks with other testers in the comments section!