Writing acceptance criteria with AI (2026)

TL;DR

Writing acceptance criteria with AI is safest when the model receives a curated requirements evidence pack and produces structured candidate scenarios plus questions. Humans must resolve policy, remove invented assumptions, verify traceability, and approve each atomic, observable criterion.

Key Takeaways

• Use AI to transform approved evidence into candidate criteria, not to invent missing product policy.
• Provide goals, actors, rules, examples, constraints, dependencies, and explicit unknowns in a versioned evidence pack.
• Separate business rules, happy paths, alternatives, boundaries, failures, permissions, accessibility, and observability.
• Require atomic, observable criteria with stable IDs and a clear source or unresolved-question reference.
• Generate structured output, then validate syntax, traceability, contradictions, testability, and stakeholder intent.
• Use decision tables and boundary analysis before converting rules into Given, When, Then scenarios.
• Keep product owners accountable for meaning and QA accountable for testability and risk coverage.

Writing acceptance criteria with AI can accelerate refinement, expose missing rules, and turn scattered product notes into reviewable scenarios. The model should draft from supplied evidence, identify ambiguity, and propose coverage. It should not decide business policy, silently convert assumptions into requirements, or replace agreement among product, engineering, design, security, and QA.

The reliable 2026 workflow is evidence pack -> rule model -> candidate criteria -> deterministic validation -> stakeholder review -> versioned approval -> tests. This guide gives QA and SDET engineers a practical method, a current structured-output example, review checklists, traceability rules, and interview-ready explanations.

TL;DR

Every generated criterion needs evidence or an explicit unresolved question. If the model cannot support a rule, it must not present that rule as approved behavior.

1. What Writing Acceptance Criteria With AI Should Achieve

Writing acceptance criteria with AI should produce a better refinement conversation, not merely more sentences. Good criteria define the observable conditions under which a product increment is acceptable. They establish scope, examples, failure behavior, boundaries, permissions, and externally visible outcomes without prescribing unnecessary implementation.

AI is useful at transformation tasks. It can normalize vocabulary, split compound statements, identify missing actors, turn decision rules into examples, compare criteria for contradictions, suggest boundary conditions, and format candidate scenarios. It can also highlight that the source never says what happens on duplicate submission, timeout, partial success, unauthorized access, or empty input.

The danger is plausible invention. If a story says "members can cancel an order," a model may assume a 30-minute window, refund policy, notification, and administrator override. Those might be reasonable product questions, but they are not acceptance criteria until an accountable owner answers them. Require the model to classify output as supported criterion, derived candidate, assumption, or open question.

Define team ownership. Product owns business meaning and priority. Design owns interaction and content intent. Engineering owns feasibility and technical constraints. Security and legal own relevant controls. QA challenges ambiguity, observability, risk, and testability. AI assists all of them but owns none of the decisions.

Acceptance criteria differ from entry and exit gates for a testing phase. The entry and exit criteria guide explains those process controls. Here, the focus is the behavior and quality conditions for one product capability.

2. Build a Requirements Evidence Pack

A strong result begins with a bounded evidence pack. Do not paste an entire backlog and ask the model to "write complete criteria." Select current, authoritative artifacts and label their status. Include the story or problem statement, product goal, actors, permissions, business rules, examples, UI states, API contract, data definitions, nonfunctional constraints, dependencies, analytics needs, rollout conditions, known exclusions, and unresolved questions.

Use stable source IDs such as STORY-42, RULE-REFUND-3, DESIGN-12, and API-ORDER-v2. Criteria can then cite their evidence. Mark sources as approved, draft, superseded, or disputed. If two sources conflict, preserve the conflict for review rather than asking the model to choose silently.

A compact evidence template is:

feature_id: ORDER-CANCEL-42
goal: Allow eligible customers to cancel an unshipped order without support.
actors:
  - signed-in customer who owns the order
approved_rules:
  - id: RULE-1
    text: Orders in PROCESSING state can be cancelled.
  - id: RULE-2
    text: SHIPPED and DELIVERED orders cannot be cancelled.
  - id: RULE-3
    text: A successful cancellation records actor and UTC timestamp.
interfaces:
  - id: API-1
    text: POST /orders/{orderId}/cancellation returns the current order status.
unknowns:
  - Should a cancellation send email?
out_of_scope:
  - partial item cancellation

Remove secrets and unnecessary personal data. Replace production examples with synthetic values. Keep the evidence pack in version control or an auditable requirements system, and hash the exact version used for generation.

For larger initiatives, connect source IDs and approved criteria through a requirement traceability matrix so change impact remains visible after the refinement meeting.

3. Choose a Criterion Form That Fits the Rule

Given, When, Then is useful but not mandatory for every requirement. Choose the form that makes behavior clearest. For interaction scenarios, BDD syntax expresses initial state, action, and outcome. For a simple invariant, a declarative bullet can be shorter. For many interacting conditions, build a decision table first. For numeric or temporal limits, write boundary examples. For APIs, include status, body, state change, and idempotency.

Avoid ceremonial BDD. Given the app is open, When I use the app, Then it works has structure but no contract. Include only relevant preconditions, one action or event, and observable outcomes. Background setup should not hide important permissions or state.

Ask the model to preserve domain language from the evidence pack. Terms such as cancelled, voided, refunded, and deleted may have different meanings. Maintain a glossary and reject synonyms that change policy.

When logic branches across several inputs, use the decision table testing guide before asking for scenarios. Generate one scenario per meaningful rule row, then use boundary analysis for numeric fields. This reduces duplicate criteria and exposes impossible or uncovered combinations.

4. Prompt for Evidence, Coverage, and Uncertainty

A good prompt assigns a narrow job, provides the evidence pack, defines output fields, states prohibited behavior, and includes a review rubric. It should say that unknowns remain unknown, contradictions are reported, and every criterion cites source IDs.

Use instructions like these:

• Draft candidate acceptance criteria only from approved sources.
• Do not invent time limits, roles, notifications, error messages, or policies.
• Put missing decisions in open_questions.
• Give each criterion a stable candidate ID.
• Keep each criterion atomic and observable.
• Include happy path, alternative, boundary, failure, permission, concurrency, accessibility, and audit risks only when supported or clearly marked as proposals.
• Separate required behavior from suggested test ideas.
• Cite the source IDs supporting each criterion.
• Report contradictions and duplicates.

Do not ask for a fixed large number such as "write 30 criteria." Quotas encourage repetition and invented edge cases. Ask for necessary coverage and allow the model to return gaps.

Use a two-pass process. First, extract actors, states, rules, examples, contradictions, and questions. Review that rule model. Second, generate criteria from the approved rule model. This is easier to debug than one prompt that jumps from raw notes to finished BDD.

Treat retrieved requirements as untrusted content if the workflow uses RAG. A document saying "ignore earlier instructions" is part of the product corpus, not a new system instruction. Keep generation without mutation tools and require explicit approval before writing back to the backlog.

5. Generate Structured Candidate Criteria With a Current API

Structured output prevents formatting drift and makes candidate criteria easier to validate. The current OpenAI Python SDK supports parsing a Responses API result into Pydantic models for compatible models. This script uses a local YAML evidence file as user content, returns criteria plus unresolved questions, and never writes to a backlog.

# generate_criteria.py
import os
import sys
from typing import Literal

from openai import OpenAI
from pydantic import BaseModel, Field

class Criterion(BaseModel):
    id: str = Field(pattern=r"^AC-[0-9]+
quot;)
    kind: Literal["happy_path", "alternative", "boundary", "failure", "permission", "audit"]
    given: list[str]
    when: str
    then: list[str]
    source_ids: list[str]
    proposed: bool

class CriteriaDraft(BaseModel):
    feature_id: str
    criteria: list[Criterion]
    open_questions: list[str]
    contradictions: list[str]
    excluded_items: list[str]

def generate(evidence: str) -> CriteriaDraft:
    client = OpenAI()
    response = client.responses.parse(
        model=os.environ["OPENAI_MODEL"],
        instructions=(
            "Draft candidate acceptance criteria from approved evidence only. "
            "Do not invent product policy. Keep criteria atomic and observable. "
            "Set proposed=true when a criterion needs a product decision. "
            "Cite supporting source IDs and list unresolved questions separately."
        ),
        input=[{
            "role": "user",
            "content": "Requirements evidence pack:\n" + evidence,
        }],
        text_format=CriteriaDraft,
    )
    if response.output_parsed is None:
        raise RuntimeError("No parsed criteria draft returned")
    return response.output_parsed

if __name__ == "__main__":
    evidence_text = open(sys.argv[1], encoding="utf-8").read()
    draft = generate(evidence_text)
    print(draft.model_dump_json(indent=2))

Set OPENAI_API_KEY and OPENAI_MODEL, install current openai and pydantic, then run python generate_criteria.py evidence.yaml. Schema conformance does not prove that citations exist or that behavior matches stakeholder intent. The next validation layer must check those facts.

6. Validate AI-Generated Acceptance Criteria Deterministically

Validate the artifact before human review so people spend time on meaning rather than broken format. Parse the structured output, enforce unique IDs, verify every cited source exists, reject empty Given or Then lists as your standard requires, flag duplicated steps, and ensure proposed criteria cannot enter the approved set.

The following test checks core invariants on a saved draft:

# test_criteria_draft.py
import json
from pathlib import Path

APPROVED_SOURCES = {"RULE-1", "RULE-2", "RULE-3", "API-1"}

def test_generated_criteria_contract() -> None:
    draft = json.loads(Path("criteria-draft.json").read_text(encoding="utf-8"))
    criteria = draft["criteria"]
    ids = [item["id"] for item in criteria]

    assert len(ids) == len(set(ids)), "Criterion IDs must be unique"
    assert all(item["when"].strip() for item in criteria)
    assert all(item["then"] for item in criteria)

    for item in criteria:
        assert set(item["source_ids"]) <= APPROVED_SOURCES
        assert not item["proposed"], f"{item['id']} still needs a product decision"

Add semantic lint rules cautiously. A style checker can flag vague words such as quickly, user-friendly, appropriate, normally, and should work. It cannot decide the correct replacement. A contradiction checker can identify that one criterion allows cancellation in SHIPPED while another forbids it, but a stakeholder must resolve which source wins.

Validate observability. Then the database is updated correctly is not useful unless the observable state or authorized query is defined. Then the customer sees an error needs state, timing, persistence, and recovery details appropriate to the risk. Verify each criterion describes one behavior and does not bundle unrelated analytics, email, and UI outcomes unless they form one atomic transaction.

7. Review for Completeness Without Inventing Scope

Completeness is not the number of scenarios. It is coverage of supported business behavior and relevant risks. Review the rule model across actors, permissions, states, transitions, inputs, outputs, failures, boundaries, concurrency, accessibility, audit, privacy, and recovery. Mark unsupported items as questions or out of scope.

For the cancellation example, useful questions include:

• What happens if state changes from processing to shipped during the request?
• Is a repeated cancellation idempotent?
• What response should an unauthorized user receive?
• Does a successful API response require the audit record to commit atomically?
• How does the UI represent an in-progress request?
• What happens when the service times out after cancellation commits?
• Is email notification required or intentionally excluded?

These are not automatically criteria. They are refinement prompts. Once product and engineering answer them, update the evidence pack and regenerate or edit the affected candidates.

Use example mapping: connect rules, examples, and questions. A rule without examples may hide interpretation. An example without a rule may encode accidental behavior. A question without an owner can remain unresolved until implementation, when it becomes expensive.

Check nonfunctional acceptance only where the team can measure it under defined conditions. Replace "loads fast" with a specified user journey, environment, data volume, percentile, observation point, and approved threshold. AI must not invent the threshold. It can produce the measurement template and ask for the missing number.

8. Convert Approved Criteria Into Tests and Traceability

Acceptance criteria are not a complete test suite. They state acceptance boundaries, while QA still designs exploratory, integration, compatibility, resilience, security, performance, and observability tests. One criterion may map to several tests, and one test may support several criteria.

Assign stable IDs only when candidates are approved. Map each criterion to sources, design, implementation, automated tests, exploratory charters, defects, and release evidence. Preserve version history. If RULE-2 changes, the trace should identify affected criteria and tests without asking a model to guess from prose alone.

Generate test ideas in a separate artifact. This prevents examples intended to explain behavior from becoming an exhaustive test claim. For every approved criterion, ask for positive, negative, boundary, state-transition, concurrency, permission, recovery, and accessibility ideas as relevant. QA reviews and prioritizes by risk.

A criterion such as "Given an owned PROCESSING order, When the customer cancels it, Then status is CANCELLED and one audit record contains actor and UTC timestamp" may produce API contract, UI, database integration, concurrency, idempotency, and authorization tests. The criterion remains concise while the test design expands.

Keep automation assertions aligned with observable outcomes. Avoid coupling to internal implementation unless that implementation is part of the contract. A UI scenario can verify a status and control state, while an API or integration test verifies the audit transaction.

The writing a test plan guide shows how approved criteria feed broader scope, risks, environments, data, responsibilities, and exit decisions.

9. Handle Change, Collaboration, and Governance

Requirements change. Store the evidence pack version, generation instructions, model configuration, structured draft, review comments, approved criteria, and human approvers. When sources change, compare at the rule and criterion level. Do not regenerate the whole feature and replace approved text without a reviewed diff.

Classify changes:

Use AI to summarize the diff and propose impacted IDs, but verify against deterministic source links. The model may miss an indirect dependency or overstate a wording change.

In refinement, display source, candidate criterion, and open question together. This keeps discussion grounded. Record decisions immediately with owner and date. A statement made in a meeting should not become durable truth only because an AI summary sounded confident.

Establish data governance. Requirements may include customer commitments, security architecture, or confidential roadmap information. Use approved tools and retention settings, minimize supplied content, and keep access aligned with the source system. AI output should inherit the source classification until reviewed.

Measure workflow quality through clarification lead time, escaped ambiguity defects, criteria churn, trace coverage, review overrides, and unsupported assertion rate. Do not optimize for criteria generated per minute.

10. Scale Writing Acceptance Criteria With AI in CI and Backlog Workflows

Writing acceptance criteria with AI at scale needs gates, not automatic publishing. Trigger a draft when an evidence pack changes, validate the schema and citations, attach a diff to the work item, and request named reviewers. Only approved criteria should receive durable IDs or feed downstream test generation.

A safe workflow is:

1. export a bounded, classified evidence pack,
2. run rule extraction and report contradictions,
3. stop if required decisions are unresolved,
4. generate a structured candidate draft,
5. run deterministic lint and trace checks,
6. perform product, QA, engineering, design, and security review as relevant,
7. approve and version criteria,
8. generate separate test ideas,
9. update trace links,
10. monitor requirement and implementation changes.

Do not let a failed model call block urgent manual refinement. The workflow should degrade to human authoring with the same template and review gates. Do not auto-close questions merely because later generation omitted them. Persist unresolved items until a named owner resolves or explicitly defers them.

Use a small evaluation set of real stories with human-approved rules, known ambiguities, and deliberately conflicting sources. Measure unsupported policy additions, missed contradictions, trace precision, atomicity, observability, and reviewer edit distance. Re-run it when prompts, model, schemas, or evidence preprocessing change.

The goal is a requirements quality system where AI shortens mechanical drafting and broadens challenge questions. Human accountability, durable evidence, and test engineering discipline remain the control plane.

Interview Questions and Answers

Q: What role should AI play in acceptance criteria?

AI should organize approved evidence, draft atomic scenarios, identify gaps, and suggest coverage. It should not choose business policy or turn assumptions into approved requirements. Human stakeholders remain responsible for meaning, feasibility, risk, and approval.

Q: How do you prevent hallucinated requirements?

I use a bounded evidence pack with source IDs and require citations for every criterion. Unsupported ideas go into proposed items or open questions. Deterministic validation rejects unknown sources, and reviewers resolve policy before approval.

Q: Is Given, When, Then always the best format?

No. It works well for observable interactions and state transitions. Decision tables are clearer for condition combinations, boundary examples for thresholds, and declarative rules for simple invariants. I choose the representation that exposes the rule most precisely.

Q: How do you know a criterion is testable?

It identifies relevant preconditions, one event or action, and observable outcomes. Terms are defined, boundaries and state are clear, and the result can be checked without subjective interpretation. Any missing policy appears as a question, not hidden assumption.

Q: Are acceptance criteria the same as test cases?

No. Criteria define the conditions for accepting the feature. Test cases add data, steps, technical layers, negative paths, environments, and risk coverage. One criterion can require several tests across UI, API, integration, security, and accessibility.

Q: How would you handle conflicting requirement sources?

I would preserve both source IDs, flag the contradiction, identify the accountable decision maker, and block approval of affected criteria. The model should not choose based on wording or recency unless governance explicitly defines precedence.

Q: What metrics would you use for an AI criteria workflow?

I would measure unsupported additions, missed contradictions, citation accuracy, atomicity, observability, reviewer edits, criteria churn, and escaped ambiguity defects. Generation volume is not a quality metric.

Q: How should acceptance criteria evolve after approval?

Change them through versioned review linked to updated source rules. Preserve IDs when meaning is stable, supersede them when behavior changes, update trace links, and rerun impacted tests. Never silently regenerate and overwrite approved criteria.

Common Mistakes

• Giving the model only a one-line story: Supply actors, rules, examples, states, constraints, and unknowns.
• Treating plausible assumptions as requirements: Route them to questions with owners.
• Demanding a fixed number of criteria: Quotas create repetition and invented scope.
• Using Given, When, Then ceremonially: Keep scenarios atomic, relevant, and observable.
• Mixing acceptance criteria and test cases: Maintain separate artifacts and trace links.
• Skipping source citations: Unsupported behavior becomes hard to detect.
• Ignoring negative and permission paths: Happy-path prose is not acceptance coverage.
• Approving vague adjectives: Replace them with measurable conditions supplied by stakeholders.
• Auto-writing generated text to the backlog: Require validation, diff, and named approval.
• Regenerating after change without preserving history: Version rules, criteria, decisions, and tests.

Conclusion

Writing acceptance criteria with AI works when the model is a disciplined drafting and review assistant grounded in approved evidence. Structured candidates, source citations, explicit questions, deterministic validation, stakeholder approval, and traceability keep speed from becoming invented scope.

Start with one real feature and build its evidence pack before writing a prompt. Ask the AI to extract rules and questions first, then draft criteria only after the team resolves ambiguity. That sequence produces criteria people can agree on, test, maintain, and trust.

Interview Questions and Answers

Frequently Asked Questions

Related Guides

• AI powered visual testing with vision models (2026)
• AI test data generation with Faker and LLMs (2026)
• Automating regression triage with AI (2026)
• Generating API tests from OpenAPI with AI (2026)
• Generating Selenium tests with AI (2026)
• Generating test cases from a PRD with AI (2026)