About this tool
Most teams can say how many automated tests they have. Fewer can say whether those tests are a signal anyone trusts. This scorecard asks twelve questions about what actually happens today, two for each of six areas: strategy and scope, framework and code quality, CI/CD integration, test data and environments, reliability and flakiness, and reporting and ownership. Each answer is worth 0 to 3 points. Each area is scored out of 100 from its two answers, and the overall score is the average of the six areas, so a team with a beautiful framework that only runs on one laptop can’t score well. The overall score maps to one of four levels: Ad hoc (below 35), Emerging (35 to 59), Established (60 to 84) and Optimised (85 and above). The weakest areas below 80, up to three of them, come back as concrete next steps with links to the lessons and tools that cover them. The scoring is a fixed rubric, written out in the FAQ below. It is a conversation starter for a retro or a planning session, not a benchmark against other companies: there is no dataset of other teams behind it, and none is claimed.
How to use it
- Answer each question with the option that describes what your team does today. Aspirations and plans score as the option you are at now.
- Your score updates as soon as all twelve questions are answered. Answers are saved in this browser, so you can come back to them.
- Read the "Work on next" list: the weakest areas first, each with one concrete step and links to go deeper.
- Use "Copy share link" to send the result to your team. The link carries the answers, so whoever opens it sees the same scores. "Copy summary" gives plain text for Slack, a ticket or a retro doc.
What the six areas measure
- Strategy and scope: whether automation targets risk on purpose, and whether most checks sit at the cheapest level that can catch the bug (unit and API before UI).
- Framework and code quality: whether test code is structured (page or component objects, fixtures, config) and reviewed like production code, and whether locators survive markup changes.
- CI/CD integration: whether tests run on every change without anyone remembering to run them, and how long a pull request waits for the answer.
- Test data and environments: whether tests own their data and can run in parallel and in any order, against an environment that is the same each time.
- Reliability and flakiness: whether a red build means a real problem. Fixed sleeps and re-running until green are the usual culprits.
- Reporting and ownership: whether a failure can be diagnosed from the CI report, and whether the whole team, not just QA, acts on it.
How to use the result with a team
Ask two or three people (a tester, a developer, whoever runs the pipeline) to fill it in separately, then compare share links. Where the answers differ is usually more useful than the score itself: if one person picks "on every pull request" and another picks "nightly", the team doesn’t agree on what the pipeline does.
Pick one area from "Work on next" per quarter. Moving one area from 33 to 67 is worth more than spreading effort across all six. Re-take the scorecard at the end of the quarter with the same people.
Test Automation Readiness Scorecard FAQ
How is the score calculated?
Each answer is worth 0, 1, 2 or 3 points, in the order the options are listed. Each area has two questions, so it scores out of 6, shown as a percentage. The overall score is the average of the six area percentages, rounded. Levels: Ad hoc below 35, Emerging 35–59, Established 60–84, Optimised 85 and above.
Is this an industry benchmark?
No. The scorecard is a fixed rubric based on widely accepted test automation practice, not a comparison with other companies. There is no dataset of other teams behind it, and SDETRole doesn’t collect your answers to build one.
Where are my answers stored?
In your browser’s local storage, and in the share link’s fragment (the part after #) when you copy it. Browsers don’t send the fragment to servers, so SDETRole never receives your answers.
If I open a colleague’s link, will it overwrite my answers?
No. A shared result is shown without being saved. If you change an answer, you start your own copy from their answers, and that copy is saved in your browser.
We have no automation yet. Is it still useful?
Yes. Answer honestly and you will score low in most areas; the "Work on next" list then tells you where to start, which is usually strategy (what to automate first) and CI (running it automatically from day one).