Smoke vs sanity vs regression (2026)

TL;DR

Smoke tests answer whether the build is testable, sanity tests answer whether a focused change behaves plausibly, and regression tests answer whether existing behavior still works across a wider risk surface. Run them in that order when all three are needed, but tailor scope to the change and release context.

Key Takeaways

• Smoke testing checks whether a build is stable enough for deeper testing.
• Sanity testing checks a narrow changed area and its immediate effects after a focused change or fix.
• Regression testing samples broader existing behavior to find unintended change impact.
• The categories can share test cases, but their purpose, scope, and execution trigger differ.
• Use risk, dependency reach, and feedback cost to select a suite rather than relying on fixed labels alone.
• Keep every suite small enough for its feedback deadline and review it when product risk changes.

Smoke vs sanity vs regression describes three test selections, not three completely different testing techniques. Smoke testing checks whether a build is stable enough to test. Sanity testing checks a focused changed area and nearby behavior. Regression testing checks a wider set of existing capabilities for unintended impact.

The confusion comes from overlapping test cases and inconsistent team vocabulary. A successful login may be part of a smoke suite on every deployment and part of a regression suite before release. What changes is the question, trigger, breadth, and stopping decision. This guide gives working rules, realistic examples, and an implementation model you can adapt without arguing over labels.

TL;DR

All three can be manual or automated. Automation is common for repeatable checks, but the defining feature is the suite's purpose and scope.

1. Smoke vs Sanity vs Regression: Precise Definitions

Smoke testing is a build confidence check. It covers critical capabilities lightly enough to provide a fast signal. A web product smoke suite might verify that the application loads, a user can authenticate, a core record can be created, and a health-critical dependency responds. It does not prove that each feature is correct. It answers whether further testing is worthwhile.

Sanity testing is focused confirmation after a bounded change. If a team fixes tax calculation for one region, sanity coverage verifies the corrected examples, nearby boundaries, error handling, and the immediate checkout total. The scope is deeper than smoke in the changed area but much narrower than full regression. A failed sanity check suggests that the requested change is not coherent enough for broader evaluation.

Regression testing looks for unintended effects on behavior that previously worked. Its selection follows the change's dependency and risk surface, not only the edited screen. A shared price calculation change may require coverage for carts, subscriptions, refunds, invoices, reports, and API consumers. Regression can range from a focused impacted suite to a broad release suite.

These definitions are purpose-first. Calling every quick automated run "smoke" weakens the term. Calling a complete release suite "sanity" because it runs after bug fixes creates confusion. Establish definitions in the test strategy, then name suites according to the decisions their results support.

2. Smoke vs Sanity vs Regression Comparison

The following comparison highlights tendencies, not universal rules. Execution time depends on product size and delivery frequency, so avoid prescribing a fixed number of minutes.

Sequence also distinguishes them. After deployment, smoke normally runs first because deeper tests have little value on a fundamentally broken build. Sanity may run next for the feature under change. Regression then expands based on impact and release need. A small continuous delivery change might combine the first two signals or run impacted regression continuously.

The same test can appear in multiple logical views. Keep one implementation and attach metadata or manifest entries instead of copying code into three directories. Duplication causes fixes to drift and inflates maintenance without increasing coverage.

3. Where Each Suite Fits in a Delivery Workflow

For a new deployment to a shared test environment, begin with environment and smoke checks. Confirm the expected version, configuration, major dependencies, authentication, and one critical business path. If the build fails at this level, stop expensive suites, capture diagnostics, and either repair or redeploy. Continuing can flood the team with derivative failures.

After smoke passes, test the reason for the build. For a focused defect fix, run sanity checks against the reported reproduction, boundary cases, negative paths, and the immediate integration. Sanity is not a ceremonial click through the happy path. It should challenge the logic that changed while remaining bounded.

Select regression from an impact map. Review changed modules, shared libraries, data schema, configuration, permissions, consumers, and critical revenue or safety paths. A UI copy change may require a tiny regression selection. A dependency upgrade or authorization middleware change can justify broad coverage even if the code diff appears small.

A practical pipeline often has multiple regression tiers. Fast affected checks run on each pull request. A broader stable suite runs after merge or on a schedule. Pre-release testing covers critical cross-system and environment risks. The labels matter less than explicit triggers, time budgets, ownership, and release decisions.

Emergency fixes need the same logic under tighter time. Run minimum smoke on the target, focused sanity on the incident, and regression on plausible blast radius. Document any consciously deferred evidence and monitor the affected outcome after release. Urgency should sharpen selection, not eliminate reasoning.

4. How to Design an Effective Smoke Suite

A smoke suite should fail for conditions that make the build unsuitable for its next activity. Select a small number of representative checks across startup, reachability, authentication, critical navigation, essential read and write operations, and indispensable dependencies. Include deployment identity so a green result cannot accidentally validate the previous version.

For an ecommerce application, smoke might verify:

1. The deployed version and required services report healthy.
2. A test user can sign in and sign out.
3. Product search returns a known seeded item.
4. The item can be added to a cart.
5. Checkout can reach a safe payment simulation and create an order.
6. The order appears in account history.

Keep assertions shallow but meaningful. "HTTP 200" alone is weak if an error page also returns 200. Verify a business marker, state change, or response contract. Keep test data isolated and reusable without collisions. If a check creates data, generate a unique identifier and clean up safely or use a disposable environment.

Do not turn smoke into a miniature full regression suite. Each added scenario delays the build acceptance signal and increases unrelated failure probability. If a check does not affect whether deeper testing can begin, it likely belongs elsewhere.

Track reliability aggressively. A smoke failure should prompt action, so flaky smoke tests cause disproportionate harm. Capture version, environment, response details, timestamps, and correlation identifiers. The owner should be able to distinguish product failure, deployment failure, dependency failure, and test failure quickly.

5. How to Design Focused Sanity Testing

Sanity starts from a change hypothesis: what was changed, what behavior should now differ, and what nearby behavior could be disturbed? Read the acceptance examples, code or design impact when available, original defect evidence, and developer notes. Then define a narrow test charter.

Suppose an API fix allows apostrophes in customer surnames. A credible sanity set includes creating "O'Neil," updating an existing surname, retrieving it without encoding loss, searching for it, and confirming that validation for genuinely invalid characters still works. It may also check display in the immediate profile UI. It does not automatically require every customer management report, import, and notification unless analysis shows those paths share the changed transformation.

Sanity should include positive, negative, boundary, and state-aware examples within its scope. Retesting only the exact reported values can confirm the patch while missing an overbroad fix. If a minimum length defect changed a comparison operator, test below, at, and above the boundary.

Focused exploratory testing works well here. Use a short charter such as "Explore saved-address editing after normalization changes, emphasizing international characters, cancellation, retry, and concurrent updates." Automation confirms stable examples, while exploration looks for surprising interactions.

Record what was intentionally outside scope. If the changed utility also feeds batch import but that integration is deferred to regression, say so. Clear boundaries prevent "sanity passed" from being interpreted as whole-product confidence.

6. Build a Risk-Based Regression Testing Strategy

Regression scope should follow plausible impact. Start with the changed component, then trace dependencies outward: callers, consumers, shared data, asynchronous events, caches, permissions, reports, notifications, mobile clients, and operational processes. Combine this technical view with business criticality and usage.

Use tiers to balance feedback and confidence:

• Tier 1: deterministic tests for changed code and directly dependent components, suitable for pull requests.
• Tier 2: broader service, contract, and critical journey coverage after merge or deployment.
• Tier 3: release-oriented cross-system, compatibility, data migration, resilience, and focused exploratory coverage.
• Scheduled suites: valuable cases that are too costly or environment-dependent for every change.

Historical defects help but should not dominate. A previously failing case is a useful regression candidate when the impact is material and recurrence plausible. However, keeping every old test forever creates a museum rather than a strategy. Periodically remove redundant, obsolete, or low-signal cases.

Selection based only on code files can miss configuration, schema, feature-flag, and dependency risk. Add explicit triggers for shared infrastructure and sensitive domains. Authorization, payment, identity, data migration, and public contracts often justify broader regression.

For API-heavy products, protocol-focused guides such as GraphQL API testing can help move exhaustive input and contract checks below the UI. Browser regression can then concentrate on critical user integration rather than repeating every service rule.

7. Implement Runnable Suite Selection

One implementation can support multiple suite views without duplicating tests. The following current Node.js example uses the built-in test runner and a small environment-based selection helper.

Create suite.js:

export const selectedSuite = process.env.TEST_SUITE ?? "regression";

const levels = {
  smoke: new Set(["smoke"]),
  sanity: new Set(["smoke", "sanity"]),
  regression: new Set(["smoke", "sanity", "regression"])
};

export function shouldRun(tags) {
  const allowed = levels[selectedSuite];
  if (!allowed) {
    throw new Error("Unknown TEST_SUITE: " + selectedSuite);
  }
  return tags.some((tag) => allowed.has(tag));
}

Create checkout.test.js:

import test from "node:test";
import assert from "node:assert/strict";
import { shouldRun } from "./suite.js";

function total(items) {
  return items.reduce((sum, price) => sum + price, 0);
}

test("cart calculates a basic total", {
  skip: !shouldRun(["smoke"])
}, () => {
  assert.equal(total([10, 5]), 15);
});

test("fixed decimal scenario remains correct", {
  skip: !shouldRun(["sanity"])
}, () => {
  assert.equal(total([10.25, 5.5]), 15.75);
});

test("empty cart has zero total", {
  skip: !shouldRun(["regression"])
}, () => {
  assert.equal(total([]), 0);
});

With "type": "module" in package.json, run TEST_SUITE=smoke node --test, TEST_SUITE=sanity node --test, or TEST_SUITE=regression node --test. On Windows shells, set the environment variable using the shell's normal syntax.

This simple hierarchy is illustrative. In a real repository, many sanity selections should be change-specific rather than one global set. Store ownership and risk metadata, report skipped tests clearly, and run a broad safety suite at an agreed cadence so a faulty dependency map cannot hide failures indefinitely.

8. Decision Rules for Smoke vs Sanity vs Regression

Ask four questions before selecting a suite. First, is this a new or changed deployment whose basic testability is unknown? If yes, run smoke. Second, is there a focused feature or fix that needs direct confirmation? If yes, run sanity for that area. Third, can the change affect existing behavior beyond the direct acceptance examples? Almost every meaningful change can, so select regression proportional to the blast radius. Fourth, what unique risks require exploration or nonfunctional testing?

Examples make the rules concrete:

• New test environment deployment with no application change: environment checks plus smoke, then the intended environment validation.
• One CSS alignment fix: focused visual sanity, accessibility observation if structure changed, and minimal critical regression.
• Shared date library upgrade: smoke plus broad regression for scheduling, billing, reporting, time zones, and serialization.
• Payment incident hotfix: production-safe smoke, deep sanity for the failure and recovery, impacted checkout and refund regression, plus monitoring.
• New feature behind a disabled flag: smoke for default behavior, sanity within controlled flag exposure, regression for shared components and flag-off behavior.

There is no rule that full regression must follow every sanity run. Small teams and continuous delivery systems often use an impacted regression selection per change, with broader suites scheduled or tied to release risk. Conversely, a one-line security configuration change may deserve more regression than a large isolated content update.

Write the selection and rationale into the test result. "Sanity passed" is ambiguous. "Address normalization sanity passed for create, update, retrieval, search, invalid input, and cancellation on build 842; batch import remains in nightly regression" supports an actual decision.

9. Maintain Fast, Trusted Test Suites

Assign every suite a feedback objective and time budget based on the delivery workflow. Smoke must complete soon enough to protect downstream capacity. Sanity must be easy to target by feature or change. Regression tiers must return before the decision they inform. Do not publish arbitrary universal durations; measure your system and optimize its constraint.

Review failures by cause: product defect, environment, data, dependency, or test defect. A high test-defect rate reduces trust. Fix uncontrolled clocks, shared accounts, order dependence, brittle locators, unsafe cleanup, and hidden network reliance. Retry can gather diagnostic evidence, but a passed retry should remain visible rather than rewriting history as green.

Review membership quarterly or after major architecture changes. Promote a test to smoke only if its failure makes further testing unsafe or wasteful. Add a case to regression when it covers material behavior and a plausible recurrence. Remove duplicated assertions and obsolete features. Parameterize stable equivalence classes rather than copying near-identical cases.

Use production incidents and exploratory findings to improve selection. A missed cross-account authorization defect may lead to API-level authorization regression across resource types, not dozens of slow UI copies. The AI-assisted exploratory testing guide can help generate additional charters, but human testers must validate relevance and expected behavior.

Publish suite health next to product results. Contributors should see duration, skipped cases, quarantines, failure cause, and owner. A green badge with silently excluded tests is not confidence.

10. Reporting Results and Release Confidence

Report the question each suite answered. A smoke report states whether the named build and environment are stable enough for the next activity. A sanity report states which changed behavior and boundaries were checked. A regression report states the selected risk surface, passes, failures, exclusions, and residual risk.

Include build identity, commit or artifact, configuration, environment, data source, start and end time, suite version, and failure evidence. For manual checks, record concise steps and observations. For automation, link logs and artifacts while protecting credentials and customer data.

Avoid converting pass percentage into a release verdict without context. Ninety-nine percent can hide one failed payment or authorization check. A lower percentage can result from many duplicated low-impact failures. List failed critical capabilities and explain whether a safe workaround, containment, or retest exists.

When a suite is blocked, distinguish "not tested" from "failed." If the environment is unavailable, the team lacks evidence. It should not report the product as defective or passed. The release owner then decides whether to restore the environment, use alternate evidence, or accept explicit uncertainty.

A good summary is decision-ready: "Build 842 passed smoke. Address normalization sanity passed six scoped behaviors. Impacted regression passed profile, checkout address, and notification contracts. Batch import was not run because the environment was unavailable; release remains blocked by that required evidence."

Interview Questions and Answers

Q: What is the difference between smoke, sanity, and regression testing?

Smoke checks build stability across essential paths, sanity checks a narrow changed area, and regression checks broader existing behavior for unintended impact. They differ mainly in purpose, scope, trigger, and the decision made from failure.

Q: Which testing should run first?

Smoke normally runs first on a new deployment because deeper testing is wasteful if the build is fundamentally broken. Focused sanity and risk-based regression follow when their questions apply.

Q: Can the same test case belong to smoke and regression?

Yes. A critical login test can support quick build acceptance and broader release regression. Keep one implementation and expose it through suite metadata to avoid duplication.

Q: Is sanity testing always manual?

No. Stable focused checks are good automation candidates, while short exploratory charters add value around a change. The category is defined by focused purpose, not execution method.

Q: Do you run full regression after every bug fix?

Not automatically. I run smoke if there is a new deployment, sanity for the fix, and regression selected from the change's blast radius and business risk. Broad regression is warranted for shared or sensitive components.

Q: How do you choose a smoke suite?

I select a small set of reliable checks whose failure means deeper testing should stop. They cover deployment identity, essential dependencies, authentication, and representative critical read and write paths.

Q: What is retesting compared with regression?

Retesting directly confirms that a reported defect or failed case now behaves correctly. Regression looks beyond that fix for unintended impact on previously working behavior. Focused sanity often includes retesting plus nearby checks.

Common Mistakes

• Calling every short suite smoke without defining its build acceptance decision.
• Treating sanity as an unstructured happy-path check.
• Running full regression for every tiny change without considering feedback cost.
• Selecting regression only from visible UI changes and missing shared services or data.
• Duplicating the same test implementation across suite folders.
• Continuing deep testing after smoke shows the environment is fundamentally invalid.
• Reporting blocked tests as product failures or successful passes.
• Allowing flaky smoke checks to train the team to ignore failures.
• Using pass percentage without identifying failed critical behavior.
• Confusing retesting of a fix with broader regression coverage.

Conclusion

Smoke vs sanity vs regression is a choice among confidence questions. Smoke decides whether the build deserves deeper testing. Sanity determines whether a focused change is coherent. Regression investigates unintended effects across the relevant existing product surface.

Define the purpose, trigger, scope, owner, and failure action for each suite. Reuse implementations through metadata, keep the early signal fast, and let risk determine regression breadth. The next time a build arrives, state the question you need answered before selecting the label.

Interview Questions and Answers

Frequently Asked Questions

Related Guides

• Smoke Testing vs Sanity Testing
• Smoke testing: A Complete Guide for QA (2026)
• Sanity testing: A Complete Guide for QA (2026)
• Stress vs soak vs spike testing (2026)
• Automating regression triage with AI (2026)
• Building a regression selection agent (2026)