Full end-to-end coverage is expensive to build and even more expensive to maintain — every extra scenario is another thing that can flake, another few minutes on every pull request, another reason engineers start ignoring red CI. A smoke suite that runs in minutes catches the failures that actually block launches: login broken, payment form dead, navigation regressed. That's a different goal from "test everything," and it changes how the suite gets designed from the first test onward.
Ten scenarios, not a hundred
We pick flows that map directly to revenue or retention: sign-up, login, the core dashboard action the product exists to support, a settings change, and payment or booking if the product has one. Each test is independent and seeds its own data where possible, so a failure in test three never cascades into false failures in tests four through ten.
The number matters less than the principle. Ten focused scenarios that the team trusts completely beat sixty that nobody has time to triage. If a new critical path gets added to the product — a new checkout step, a new onboarding flow — it earns a slot in the smoke suite; if an old one stops being used, its test gets deleted rather than left to bit-rot into a maintenance tax nobody remembers agreeing to.
Run on every pull request
Smoke tests gate merges to main. They run headed in CI with trace artifacts captured on failure, so reviewers see screenshots and a step-by-step trace without needing to reproduce the failure locally. That's the difference between a five-minute look at a CI artifact and a twenty-minute detour pulling a branch and re-running it by hand.
Gating merges on this suite only works if the suite is fast and reliable enough that engineers don't start treating it as noise. We keep the target under five minutes wall-clock for the full smoke pack, parallelized across workers, specifically so it never becomes the step people wait on and resent — the moment a check starts feeling like a tax instead of a safety net, teams find ways around it.
Stable selectors over clever XPath
We prefer data-testid attributes or accessible roles agreed with engineering, not CSS classes or positional XPath that break the moment a designer reorders a div. Selectors tied to implementation details turn every unrelated refactor into a wave of test failures that have nothing to do with the actual behavior being tested.
Flaky tests get fixed or deleted, not retried into submission. A test that fails one run in five and gets waved through with a re-run button trains the team to stop reading CI results at all — a red pipeline that everyone ignores is strictly worse than no test, because it costs the same maintenance effort while providing none of the confidence.
Pair with API checks for payments
Browser tests confirm what the user sees: the payment form accepts a card, the confirmation screen renders, the booking appears in the list. They are not the right tool for confirming what happens after that — webhook delivery, subscription state changes, retry logic on a failed charge. Those get separate API-level tests against Stripe's test mode or a mocked provider, running in staging where side effects are cheap to inspect and safe to break.
Splitting the two also keeps the smoke suite fast. A UI test that waits on a real webhook round-trip is a UI test that's slow and occasionally flaky for reasons that have nothing to do with the interface — exactly the kind of test that erodes trust in the whole suite.
A wider nightly suite, optional
Deeper regression packs — edge cases, less-common browser and device combinations, longer user journeys — can run overnight against a staging environment where a slower suite doesn't block anyone's afternoon. Release day itself relies on the smoke suite plus a short window of manual exploratory testing on the specific devices and browsers automated coverage doesn't reach well, like older Safari versions or in-app webviews.
This split — fast and narrow on every PR, broader and slower overnight — is what lets the team ship several times a day without either skipping tests or waiting on them.
What we skip on purpose
A smoke suite is deliberately narrow, and naming what it doesn't cover matters as much as naming what it does — otherwise "we have tests" quietly gets read as "we have full coverage," which sets the wrong expectation for everyone downstream of the pipeline.
- Pixel-perfect visual regression. Screenshot-diffing every component against a golden image produces a wall of false positives from font rendering and anti-aliasing differences between CI and local machines, for a category of bug — a few pixels off — that rarely blocks a release the way a broken login does.
- A full cross-browser matrix in CI. Running the whole smoke suite against every browser and OS combination on every pull request multiplies CI time for diminishing returns; that coverage belongs in the nightly pack or in periodic manual passes, not gating every merge.
- 100% code-coverage targets. A coverage number tells you what lines executed, not whether the behavior that matters was actually verified. Chasing a percentage encourages tests written to touch code rather than tests written to catch the failures that would actually hurt a launch.
Skipping these isn't a shortcut — it's what keeps the smoke suite fast enough to run on every single pull request without anyone resenting it, which is the property that makes it useful in the first place.
Good QA is not maximum coverage. It is the right checks at the right stage of the pipeline, run consistently enough that the team actually trusts a green build.