Have you ever heard of the Tacoma Narrows Bridge? It was the third-longest suspension bridge in the world! But four Months after it first opened in 1940, it collapsed! In an attempt to keep costs down, cheap girders had been used. This decision ultimately doomed the bridge…
You might think a missing test plan won’t result in a serious software fail. However, a 2017 study found that software failures cost the U.S. economy $1.7 trillion in financial losses that could’ve been avoided with proper testing.
Your Dev team invests a lot of time and resources into feature releases and software changes that go to production, so you need to make sure that they operate properly, without regression or user letdowns. You need to test them. Try to find bugs. Bottom line is, you need to make sure that whatever your users do, the system will respond as designed and will not break.
Whether you’re a junior QA, a product analyst, or a developer who was given a mission to write a test plan, and you don't really know what to do, or how to make the best of it—this is the place for you.
Test plan who?
The “classic” definition of a test plan is a document that outlines the strategy, objectives, resources, schedule, and success criteria for testing a specific new feature or piece of software. BUT the more common, and generally more useful, test plan is a detailed document that includes all test cases and how to execute them, in order to discover defects, errors, or any other gaps that might cause the software to not act as intended.
The first thing you’ll need when writing a test plan is a PRD or a characterization document that describes All the requirements of the feature, and it should be well detailed. You will need this in order to gain a deep understanding of the feature. If this document does not exist, make one with the help of the person that created and designed the feature. It is very important to have a detailed document that describes and goes through the feature’s requirements and capabilities.
1) Test plan structure
Start by creating a test cases structure, which will be filled with the test cases you write. Here is a classic structure:
Divide the feature into categories and create tabs/sections accordingly. This will make things simpler for you, particularly if the feature is complex or the PRD is long. Here is an example of tabs split by categories in a recent test plan for a big feature we developed (“Magic Event Builder”):
Create a “What you'll need” tab/ticket that includes all of the tools you will need to execute the plan. Work with the Dev team to think about what you might need in order to test the feature (for example: environments, users, useful sources links).
2) From PRD to test cases
After you have the test plan structure, you can start to fill it up with test cases. The easiest way to come up with test cases is to go to the PRD and break it up into pieces, trying to divide and isolate each feature’s requirements. Each requirement will give you at least one test case. Then look at the big picture, learn the main flows and translate them into test cases that combine several requirements together.
Write each test case under its relevant component.
3) Brainstorming - Catch them all
By now you should have nice coverage of the feature, but nice is not enough — you want to catch them all (all the bugs, of course). For that, you will need to be a bit more creative.
Think about user behavior in your software and try to think from the user’s perspective. This will help you approach the feature from a different angle, and to add more test cases accordingly.
After all that, think about different types of tests:
System testing: Test the entire integrated system against its requirements.
Load and stress testing: Test your software performance as the workload increases or goes beyond normal conditions.
Compatibility testing: Test your software against various types of hardware, operating systems, and environments.
API testing: Test the API created for the application in multiple scenarios.
Integration testing: Test multiple software modules or features as a group.
Install testing: Test the install/uninstall process your customers will go through.
Environment testing: Test the software in different environments.
4) Test review
This is a crucial part of the recipe and cannot be underestimated. Now send your test plan for review, to the Dev team, Design team, Product team and to anyone who took part in developing the feature. When sending your test plan, ask for feedback from your coworkers and ask them if they think you covered everything there is to cover with the feature. Since you can’t always be perfect, this process will make sure you do not miss anything important. After getting feedback, augment and change the test plan accordingly.
For the final ingredient of the recipe, prioritize. Prioritize your test cases.
Review your list with the Product team and give a score to each test case (1 to 5), reflecting its importance to delivery. Why? Time is money :). You might not be able to implement the whole test plan, and you should start testing with the crucial cases.
Testing isn’t just another thing to check off of your list. It’s an important procedure and can potentially save you a lot of time and resources. Writing a test plan needs to be carefully thought through and planned. Do yourself a favor and develop the habit of retrospective each test plan to make sure that you improve from time to time, and share your feedback to develop the rest of your team's test-driven mindset.