Track flaky specs automatically using this simple tweak in RSpec builds
GoCardless’ Product Development team recently had a summer Hackathon – during which, we developed a simple tweak for our RSpec builds to track flaky specs (or, intermittently failing tests).
There’s a lot said about flaky specs because they often lead to wasted time and effort. At one point or another, we have probably all been about to merge or deploy changes and a required build check stops us – only because a flaky spec acted-up!
When this happens, there’s no option but to retrigger the build to see it go green to unblock the change. We try and remind ourselves to write up a ticket and come back to this flaky spec but this isn’t always possible when you have a number of other things you are working on.
So, we needed an automated way to keep track of flaky specs so that they are visible and we are reminded about fixing them.