QA How-To
Smoke Testing vs Sanity Testing
Understand smoke versus sanity testing through clear definitions, real release examples, scope rules, automation guidance, and practical checklists.
1,905 words | Article schema | FAQ schema | Breadcrumb schema
Overview
Smoke and sanity testing are both fast confidence checks, which is why teams often use the terms loosely. The useful distinction is the question each check answers. Smoke testing asks whether a new build is stable enough for deeper testing at all. Sanity testing asks whether a specific change or fix behaves plausibly enough to continue with targeted or broader validation. One is broad and shallow, the other narrow and focused.
Terminology varies between organizations, so definitions matter less than an agreed trigger, scope, owner, and result. This guide establishes a practical model and applies it to an online banking release. You will see what belongs in each suite, how to avoid common overlap, when to automate, how to report failure, and how both checks connect to regression testing. The goal is a faster, clearer release decision rather than winning a vocabulary debate.
Define Smoke Testing by Its Decision
Smoke testing is a short, broad pass across critical capabilities after a build or deployment. It verifies that the application starts, major services respond, essential integrations connect, and the most important user journeys can begin or complete at a basic level. If smoke fails, the build is rejected or investigation begins before the team spends hours on detailed cases. The gate protects scarce testing time and exposes deployment mistakes while they are still fresh.
For online banking, a smoke set might verify the login page loads, a valid customer can authenticate, account balances appear, an internal transfer can be submitted in the test environment, transaction history opens, and logout ends the session. It does not test every transfer boundary, beneficiary rule, currency, permission, or visual detail. The suite samples the product's vital signs and deployment wiring.
Define Sanity Testing by Its Change Focus
Sanity testing is a narrow check around a small change, fix, or configuration adjustment. It confirms the changed behavior and a few close dependencies before more expensive validation. After fixing a defect where scheduled transfers execute one day early, sanity might create a transfer in the affected time zone, verify its calculated execution date, edit it, and confirm the scheduler records the intended date.
Sanity is not merely retesting one exact case. Good sanity coverage adds the nearest credible side effects, such as immediate transfers remaining unchanged and the scheduled item appearing correctly in the list. It still stays focused. Testing every payment method, account type, locale, and historical report belongs to regression or a planned feature pass. Write the chosen boundary down so unexpected scope growth is discussed instead of absorbed silently.
Compare Trigger, Breadth, and Outcome
A smoke test is usually triggered by a fresh build, deployment, environment refresh, major configuration change, or service restoration. Its breadth spans several critical capabilities, while depth is limited. The outcome is build acceptance for continued testing or an early stop. A sanity test is triggered by a specific small change or resolved defect. Its breadth is limited to that area, while depth around the change is greater. The outcome is confidence to continue focused regression, accept the fix, or return it for more work.
Both should be short, repeatable, and selected by risk. Neither proves that the entire application is release ready. A passing smoke suite says the build is testable, not correct in every detail. A passing sanity check says the targeted behavior appears coherent, not that unrelated workflows are safe. Reports should preserve these boundaries so stakeholders do not interpret a green fast check as comprehensive certification.
- Smoke trigger: new build or deployment.
- Smoke shape: broad across vital paths, shallow within each.
- Sanity trigger: localized change or fix.
- Sanity shape: narrow area, deeper around immediate effects.
- Shared goal: reject obvious problems before expensive testing.
Build a Risk-Based Smoke Suite
List the capabilities whose failure makes the product unusable, untestable, unsafe, or commercially nonviable. Include platform health, authentication, primary transaction, essential reads and writes, and a small number of critical integrations. Choose one stable representative path for each. Test both frontend availability and a meaningful business result, because a homepage returning 200 says little about a broken database connection. Include at least one write when write capability is essential to deeper testing.
Keep the suite intentionally small and fast enough to run after every relevant deployment. Give each check owned data and clear cleanup. Avoid unstable third-party paths unless they are truly critical, then distinguish a vendor outage from a broken build. Review smoke membership when customer journeys or architecture changes. A case does not belong merely because it is easy to automate or has existed for years.
Derive a Sanity Set From the Change
Start with the ticket, acceptance criteria, code diff, changed services, and original defect evidence. Identify the direct behavior, adjacent branches, shared state, and one negative path. For the scheduled transfer fix, direct behavior is the execution date. Adjacent risks include immediate transfer dates, editing schedules, cancellation, time-zone conversion, and displayed confirmation. Select only those touched by the same rule or data.
Talk with the developer about implementation reach. A one-line shared date-library change can justify wider sanity than a localized copy update. Conversely, a large diff of generated files may not increase behavioral scope. Write a brief charter naming what will be checked and what will wait for regression. This makes sanity an evidence-based selection rather than whichever cases the tester remembers first.
Walk Through a Banking Release Example
Suppose build 8.14.0 includes a scheduled-transfer fix plus routine dependency updates. Smoke begins after deployment. It checks service health, valid login, balances, one immediate internal transfer, transaction history, and logout. If login fails for all users, testing stops even though the scheduled-transfer code might be correct. The build cannot support meaningful validation. The report should identify whether authentication, deployment configuration, or test data caused the stop.
After smoke passes, sanity targets the fix. Create a transfer for tomorrow in the customer's time zone, confirm the stored UTC value maps back correctly, edit the date, and verify execution status. Check an immediate transfer to ensure it does not inherit scheduling. If sanity fails, report the focused evidence and reject the fix while other teams may still investigate unrelated release areas. A broader regression later covers beneficiaries, limits, recurring schedules, notifications, statements, roles, and supported platforms.
Know What Does Not Belong
Detailed boundary matrices, all browser combinations, rare roles, extensive accessibility evaluation, deep security probes, and full data migration checks generally do not belong in a routine smoke suite. They may be critical release tests, but adding them to the first gate delays the decision it is designed to make. Keep separate suites with appropriate triggers rather than making smoke mean every important test.
Sanity should not expand into unbounded exploratory testing across the product. Follow credible connections from the change, record surprising observations, and create a new charter when investigation reveals wider impact. It also should not skip direct retesting. A few neighboring checks cannot compensate for failing to reproduce the original defect on the fixed build. Scope discipline is what makes both techniques fast.
Automate Stable Checks and Preserve Diagnosis
Smoke tests are strong automation candidates because they run frequently, use stable critical paths, and provide rapid deployment feedback. Place service health and API checks before slower browser journeys. Run them against the deployed artifact and publish artifacts that reveal whether failure came from application code, configuration, data, or dependency availability. A fast red result with no logs wastes the time automation was meant to save.
Some sanity checks can be automated by selecting existing tests mapped to changed components or tags. New fixes may need a temporary manual check before their automated regression is added. Do not delay feedback just to build automation, and do not leave the defect uncovered afterward. Tests must be independent and data-safe, especially when deployment pipelines run concurrently. Rerunning a flaky smoke test until green undermines the gate.
Report Results Without Overclaiming
A smoke report should identify build, environment, start and end time, selected critical checks, pass or fail, blocked cases, defects, and the build-acceptance decision. If a check fails because test data or an external sandbox is unavailable, report that the build's critical capability remains unverified. Environment problems are not product defects, but they still prevent a confident green result. Preserve the first failure evidence before anyone retries or repairs the environment.
A sanity report should name the change, direct retest, nearby checks, evidence, open observations, and recommendation for that change. Use precise statements such as `Scheduled transfer date fix passed in UTC-5 and UTC+5:30; recurring transfers and DST transition are reserved for regression.` This is more useful than `sanity passed` because it tells decision makers exactly what confidence was earned and what risk remains.
Agree on Team Vocabulary and Exit Rules
Some teams call a build verification test smoke, others use sanity for the same gate, and vendors may define the terms differently. Create a short internal definition with examples, triggers, maximum duration, owners, environment, entry conditions, and actions after failure. Name suites after their decision when ambiguity persists, such as `deployment acceptance` and `transfer-fix confidence`. Put those names in pipeline output so stakeholders interpret the result consistently.
Review suite health using duration, flake rate, escaped build-blocking defects, and time to diagnosis. Remove redundant checks and replace fragile UI setup with faster APIs when the user interface is not the risk. The best distinction is operational: everyone knows why the run started, what a pass permits next, and what happens when it fails. Consistent behavior matters more than textbook terminology.
Frequently Asked Questions
What is the main difference between smoke and sanity testing?
Smoke testing broadly samples critical product paths to decide whether a build is testable. Sanity testing focuses on a specific fix or small change and its immediate neighbors to decide whether targeted validation can continue.
Which comes first, smoke testing or sanity testing?
For a new build containing a fix, smoke normally comes first because the environment and core product must work before focused validation is meaningful. Sanity then evaluates the change. Team workflows may combine them when scope is very small.
Are smoke and sanity testing part of regression testing?
They can use cases from a regression repository, but they serve narrower decisions. Smoke is a critical baseline selection, while sanity is a change-focused selection. Full regression provides wider evidence according to release risk.
Can smoke testing be automated?
Yes. Stable critical checks are excellent automation candidates and can gate deployments quickly. Combine fast health and API checks with a small number of integrated UI journeys, and retain diagnostic artifacts on failure.
Who performs sanity testing?
A QA engineer, developer, or cross-functional delivery team can perform it, depending on ownership and tooling. The important points are knowledge of the change, controlled data, agreed scope, and a visible result.
How long should smoke and sanity testing take?
There is no universal duration, but both should deliver rapid feedback compared with full regression. Set a team target based on deployment cadence, risk, and environment speed, then remove low-value or unstable checks that prevent meeting it.