Squeezing Juice out of Over-ripe Tests
Problem and Solution Overview
This recipe just states how to execute a solution. Read our Legacy Newsletter Edition: Make Tests Independent (30 April) to understand the specific problem we are solving and the solution approach.
Preparation: Revert your change
Put your change in another branch or a stash. You’re going to first improve the tests, then re-do the change.
Part 1: Identify the common code
Let’s call the tests that are failing together the ripe tests. The ripe tests are failing together because they all execute some common code – the juice. You change is directly or indirectly altering the behavior of the juice. Identify the common code chunks. There might be several flavors of juice, if your change has widespread indirect impacts.
Part 2: Test the common code directly
- Do Extract Method on the juice part of each ripe test.
- Eliminate any identical duplicate extracted methods. Keep any that differ even a little bit.
- Create a new test class & make a test that executes each one of your extractions. Don’t bother adding assertions; just do what was there before. These are your juice tests.
- Make sure all tests pass.
Part 3: Identify Exit Conditions
Pick one ripe test that uses common code to prepare for testing something else. For that test, perform the following steps.
- Comment out the call to the juice (the method you extracted above).
- See what fails: what needs to be set up? This is one exit criterion from the juice.
- Immediately after the method call, write a (failing) assertion that asserts the criterion.
- Before the assertion, write code that will set up the system as needed. Use the most direct mechanism you have and do the minimum necessary (setting a value is better than calling a function that sets values).
- Make sure the ripe test now gets past both the assertion and the problem you identified in step 2.
- Uncomment the call to the juice and make sure the test still passes.
- Check it in.
- Repeat from step 1 until the ripe test passes with the juice commented out.
- Identify the juice test executes the same juice method as this ripe test did. Cut the block of assertions from the top of the ripe test and paste them into the end of the juice test. Now the juice test verifies as much behavior of the juice as the ripe test used to.
- Delete the commented-out method call.
- Check everything in.
Do this until you run out of time. Even if you don’t get through all of them, you will significantly improve your test independence.
Part 4: Make your change
Now make your change exactly as you normally would. You will still get false test failures on any test you didn’t have time to clean. That’s OK. Just change the expected or whatever approach you would have done. We aren’t trying to make the tests perfect in one step; we are just trying to make them better.
Bonus: Prevent future problems
Sometimes you will have a little time to improve test independence, but won’t have just made a change that finds all the duplicate tests. How do you find the best juice and the tests that are ripe with it?
- Execute your test suite under code coverage, but ignore uncovered code. Instead, look for hot spots: sections of code that are executed by the greatest number of tests.
- Look at your top 3 hot spots and pick the one that is most likely to be changed by near-term stories. That is your juice and the tests that ran it are your ripe tests.
- Proceed with Part 2 from the procedure above.