pytest vs Robot Framework for Test Automation (2026)

TL;DR

Use pytest for code-centric Python suites and Robot Framework for acceptance workflows that benefit from named keywords and HTML logs. Run the same representative test through both before standardizing.

Key Takeaways

• Choose pytest when Python code, fixture isolation, and separately collected parameter cases fit your team.
• Choose Robot Framework when readable workflow keywords and built-in HTML logs help reviewers.
• Keep application behavior identical when comparing runners; a local cart example isolates the test-layer difference.
• Reset mutable state per test, and per Robot template row when rows share one test case.
• Register pytest markers, standardize Robot tags, and verify filtered CI counts.
• Evaluate browser and API libraries separately from the test runner itself.

Pytest vs Robot Framework is a choice about how your team writes and maintains automated checks. Choose pytest when engineers want Python expressions, fixtures, and direct access to application code. Choose Robot Framework when readable keyword-driven test cases and built-in HTML execution logs matter more. Both can test APIs, browsers, and services; neither supplies a browser or API client by itself.

This guide puts the same cart behavior under both runners. You will see the files, commands, failure messages, and CI artifacts involved, then use those observations to choose a default for your team. The example stays local, so you can run every check without credentials, a browser, or a network service.

TL;DR

For a Python-heavy SDET team, start with pytest. Its plain assert, fixtures, parametrization, and access to Python libraries keep complex logic close to code. For a team that reviews acceptance checks with analysts or domain specialists, Robot Framework's named keywords and report.html can make intent and execution history easier to inspect. Robot still needs Python or another library language underneath when the application requires custom operations.

Robot keywords can call Python, and pytest tests can use readable names. For browser automation, compare the driver separately in the Robot Framework versus Playwright guide.

What You Will Build

• A tiny cart module whose prices are integer cents, avoiding floating-point ambiguity.
• A pytest suite with a fresh cart fixture, a marker, a parametrized discount test, and a negative case.
• A Robot suite that calls the same module through a Python keyword library, resets state per test, and uses a template for discount rows.
• Two CI-friendly commands that emit results into distinct directories.

The point is a fair comparison: identical application behavior and expected totals, with only the test layer changing. Keep the files in the locations shown. Run commands from the project root so imports and relative paths resolve consistently.

Prerequisites

Use a supported Python installation with python and pip available in a terminal. Create a clean directory, then install current compatible releases of both tools into one virtual environment. Do not copy an arbitrary version pin from an article; record the resolved versions from your environment and pin those versions in your own lockfile before adding this to CI.

mkdir framework-comparison
cd framework-comparison
python -m venv .venv
. .venv/bin/activate
python -m pip install pytest robotframework
python -m pytest --version
python -m robot --version

On Windows PowerShell, activate with .venv\Scripts\Activate.ps1. Both version commands must print an installed version. The pytest beginner tutorial covers collection and command-line usage.

Step 1: Create the shared application

Create cart.py in the project root. Prices use cents as integers. Reject zero or negative item prices and discount percentages outside zero through 100 so both test suites have a meaningful failure path.

# cart.py
from dataclasses import dataclass, field


@dataclass
class Cart:
    prices: list[int] = field(default_factory=list)

    def add(self, price_cents: int) -> None:
        if price_cents <= 0:
            raise ValueError("price must be positive")
        self.prices.append(price_cents)

    def total(self) -> int:
        return sum(self.prices)

    def discounted_total(self, percent: int) -> int:
        if not 0 <= percent <= 100:
            raise ValueError("discount must be between 0 and 100")
        return self.total() * (100 - percent) // 100

default_factory gives each cart its own list. // 100 rounds down to a whole cent. In production, specify the rounding policy and test its boundaries.

Verify the module before involving either framework:

python -c "from cart import Cart; c = Cart(); c.add(999); print(c.total(), c.discounted_total(10))"

Expect 999 899. If the import fails, make sure the shell is in framework-comparison and cart.py is in that directory. This separates application errors from runner configuration errors. Keep the module unchanged for both suites.

Step 2: Write the first pytest checks

Make tests/python and create a function-scoped fixture in tests/python/conftest.py. Pytest discovers fixtures in conftest.py for tests beneath that directory, and the default scope creates a new cart for each test invocation. The test file can request the fixture by parameter name without constructing or importing a second cart.

# tests/python/conftest.py
import pytest
from cart import Cart


@pytest.fixture
def cart() -> Cart:
    return Cart()

# tests/python/test_cart.py
import pytest
from cart import Cart


@pytest.mark.smoke
def test_two_items_sum(cart: Cart) -> None:
    cart.add(1200)
    cart.add(300)
    assert cart.total() == 1500


def test_discount_rounds_down(cart: Cart) -> None:
    cart.add(999)
    assert cart.discounted_total(10) == 899

The assert expressions are normal Python. Pytest shows compared values on failure. The smoke marker may warn until Step 5 registers it. A session-scoped fixture would retain the mutable list unless reset.

Verify collection and execution:

python -m pytest -q tests/python/test_cart.py

Expect 2 passed, possibly accompanied by the marker warning. A missing cart fixture means conftest.py is in the wrong directory or has a different function name. This example uses pytest's ordinary fixture mechanism; the pytest fixtures guide shows how the same lifecycle idea extends to browser contexts.

Step 3: Write equivalent Robot Framework checks

Robot reads keywords from test libraries. Put CartLibrary.py in the project root and have it delegate to the same Cart methods. Type annotations let Robot convert the numeric cells into Python integers for these keyword arguments. Exposing narrow, domain-specific keywords keeps the .robot file readable without hiding the actual cart behavior.

# CartLibrary.py
from cart import Cart


class CartLibrary:
    def __init__(self) -> None:
        self.cart = Cart()

    def new_cart(self) -> None:
        self.cart = Cart()

    def add_item(self, price_cents: int) -> None:
        self.cart.add(price_cents)

    def cart_total(self) -> int:
        return self.cart.total()

    def discount_total(self, percent: int) -> int:
        return self.cart.discounted_total(percent)

Create tests/cart.robot. Robot treats two or more spaces as a separator in this format. Its built-in Should Be Equal As Integers compares the returned number with the expected cell as integers. Test Setup invokes New Cart before each test, making the reset visible in the suite definition.

*** Settings ***
Library       ../CartLibrary.py
Test Setup    New Cart

*** Test Cases ***
Two Items Sum
    [Tags]    smoke
    Add Item    1200
    Add Item    300
    ${total}=    Cart Total
    Should Be Equal As Integers    ${total}    1500

Discount Rounds Down
    Add Item    999
    ${total}=    Discount Total    10
    Should Be Equal As Integers    ${total}    899

Run the suite from the project root:

python -m robot --outputdir results/robot tests/cart.robot

Expect two passing tests and paths for output.xml, log.html, and report.html under results/robot. Open the HTML log to inspect the called keywords and arguments. If the library import fails, check the ../CartLibrary.py path relative to tests/cart.robot, and run from the root so cart is importable. The Robot Framework tutorial explains suite and resource-file organization after this small example.

Step 4: Compare data-driven test cases

Append this function to tests/python/test_cart.py. Each parameter set becomes a separately collected pytest item. That means a failure at 25 percent identifies that case, while the other percentages can still pass independently. cart remains function-scoped for every invocation, including each parameter set.

@pytest.mark.parametrize(
    ("percent", "expected"),
    [(0, 1000), (10, 900), (25, 750)],
)
def test_discount_cases(cart: Cart, percent: int, expected: int) -> None:
    cart.add(1000)
    assert cart.discounted_total(percent) == expected

Append these sections to tests/cart.robot after the existing tests. The template passes each row to Discount Is. Its keyword calls New Cart for every row because multiple rows belong to one Robot test, and Test Setup runs only once for that test. Without this reset, later rows would see the previous row's item and fail for the wrong reason.

Discount Cases
    [Template]    Discount Is
    0     1000
    10    900
    25    750

*** Keywords ***
Discount Is
    [Arguments]    ${percent}    ${expected}
    New Cart
    Add Item    1000
    ${actual}=    Discount Total    ${percent}
    Should Be Equal As Integers    ${actual}    ${expected}

Check how each runner presents the cases:

python -m pytest -q tests/python/test_cart.py
python -m robot --outputdir results/robot tests/cart.robot

Expect five pytest items and three Robot tests. Robot's three discount iterations appear inside one templated test in its log; pytest's three parameter values are separate test results. If independent test-level counts, retry selection, or ownership per data row matter, model Robot data as separate named tests or otherwise design reporting around the template's aggregation. For a different approach to readable scenarios, compare pytest-bdd versus Behave.

Step 5: Select a fast smoke subset

Markers and tags solve the same scheduling problem using different syntax. Register pytest's marker to make misspellings visible, then ask each runner for only the test tagged smoke. Add pyproject.toml in the root; if your project already has one, add this table to it rather than replacing existing settings.

# pyproject.toml
[tool.pytest.ini_options]
markers = ["smoke: fast cart sanity checks"]
addopts = "--strict-markers"

python -m pytest -q -m smoke tests/python
python -m robot --include smoke --outputdir results/smoke tests/cart.robot

Both commands should select Two Items Sum and leave the discount tests out. --strict-markers makes a typo such as @pytest.mark.somke a collection error, which prevents a test from silently dropping out of a filtered CI job. Robot tags are free-form, so adopt a documented spelling convention and review it in code. Robot's --include option filters by tag; pytest's -m option evaluates marker expressions. Do not assume their more advanced expression grammars are interchangeable when copying a CI matrix between tools.

Keep the smoke job small enough to run on every change. Verify its selected count in CI; tagging every test defeats the filter.

Step 6: Check invalid input and failure diagnostics

Append a negative test to the pytest file. It verifies both the exception type and the message from the shared module. The context manager fails if no exception is raised, so a silent acceptance of a zero price cannot pass.

def test_zero_price_is_rejected(cart: Cart) -> None:
    with pytest.raises(ValueError, match="price must be positive"):
        cart.add(0)

Create a separate Robot suite file for the equivalent case. This avoids inserting a test after the *** Keywords *** section of tests/cart.robot, where Robot would interpret it as a keyword definition. Robot's built-in expectation keyword matches the exception message exposed by Add Item.

*** Settings ***
Library       ../CartLibrary.py
Test Setup    New Cart

*** Test Cases ***
Zero Price Is Rejected
    Run Keyword And Expect Error    ValueError: price must be positive    Add Item    0

Save that block as tests/cart_errors.robot. Verify the behavior with both runners:

python -m pytest -q tests/python/test_cart.py
python -m robot --outputdir results/robot-errors tests/cart_errors.robot

Expect six pytest items and one passing Robot negative test. As a diagnostic experiment, temporarily change the expected error text to an impossible value, rerun the individual test, read the mismatch, then restore the text. Pytest shows the failing assertion or exception expectation in the terminal; Robot records the failed keyword and surrounding steps in log.html. Avoid logging real secrets as keyword arguments because verbose logs can preserve them in CI artifacts.

Step 7: Produce comparable CI artifacts

A terminal pass count is useful locally, but CI needs machine-readable results. Pytest can write JUnit XML using its built-in option. Robot writes its native XML plus HTML log and report by default. Keep each runner's files separate so one job cannot overwrite the other runner's output or make a missing suite look successful.

mkdir -p artifacts
python -m pytest -q --junitxml=artifacts/pytest.xml tests/python
python -m robot --outputdir artifacts/robot tests/cart.robot tests/cart_errors.robot

Verify that artifacts/pytest.xml, artifacts/robot/output.xml, artifacts/robot/log.html, and artifacts/robot/report.html exist. On Windows PowerShell, use New-Item -ItemType Directory -Force artifacts instead of mkdir -p artifacts. Upload those files as CI artifacts even when the test command fails; otherwise the most useful failure evidence disappears with the job workspace. The runner exit status should still fail the job when a test fails.

Measure duration on your own suite. Pytest can use pytest-xdist for worker processes; Robot commonly uses Pabot for parallel execution. Configure either explicitly and isolate shared accounts, download paths, and database rows before comparing CI time.

Step 8: Decide how much abstraction to maintain

Inspect the files you now own. In pytest, tests call Cart directly and one fixture constructs it. In Robot, the suite reads like a business procedure, but CartLibrary.py is another interface to design, document, and change when application behavior changes. That wrapper earns its keep when many tests reuse stable business operations such as "Create customer" or "Approve invoice." It becomes costly if every low-level Python method gets a one-line keyword with a nearly identical name.

For API testing, pytest can call a Python HTTP client from tests or fixtures. Robot can use a library such as RequestsLibrary, or a custom Python library when the protocol or authentication flow needs more control. For browser work, pair pytest with an appropriate browser package or pair Robot with Browser Library or SeleniumLibrary. Keep the underlying driver, selector policy, authentication setup, and cleanup in the evaluation; comparing only the runner's syntax gives an incomplete answer.

Pilot one workflow with a happy path, failure path, setup, data, and CI report. Ask maintainers to change a field, add a case, and debug a dependency. Record which tasks need specialist help. See the hybrid test automation framework guide if you keep both tools.

Verify that both suites remain discoverable after organizing them:

python -m pytest --collect-only -q tests/python
python -m robot --dryrun --outputdir results/dryrun tests/cart.robot tests/cart_errors.robot

Expect six collected pytest items and four Robot tests without missing keywords. A dry run checks Robot syntax and keyword resolution; it does not validate cart behavior.

Pytest vs Robot Framework: Ownership and Maintenance

A suite's long-term cost is driven by how often behavior changes and who is expected to repair the checks. Pytest keeps branching, loops, data structures, and helper logic in Python. That is useful for contract checks with many payload variations or integration tests that inspect internal objects. Reviewers need to understand Python, however, and a dense fixture chain can hide setup just as thoroughly as a deep keyword chain.

Robot puts the test narrative in a table-like format. A domain reviewer can often understand Add Item and Cart Total without reading CartLibrary.py, but somebody still needs to implement and maintain those keywords. Large suites can acquire ambiguous keywords with overlapping names, resource imports, and accidental global state. Keep keyword names tied to domain actions, limit nesting, and make each failure point visible in the log. If a test demands substantial algorithmic logic, put that logic in a tested Python library and call a focused keyword.

Both frameworks can coexist in one repository. That is reasonable when unit and API checks are already in pytest while a small acceptance suite benefits from Robot's readable workflows. It also means two dependency trees, runner commands, conventions, and report formats. Assign an owner for each suite and a reason for the split. Do not duplicate the same cart assertions indefinitely just because a migration pilot used both; decide which runner owns each production check after the comparison.

Which Should You Choose: Pytest vs Robot Framework

Choose pytest when the team primarily writes Python, wants direct calls into application code, needs many isolated parameter cases, or relies on fixtures and plugins across unit, API, and integration layers. Its fit is especially strong where engineers inspect assertion diffs in pull requests and maintain the tests beside the code they exercise. If your team is new to it, start with one test file and a fixture; the pytest versus unittest comparison can help explain a migration from built-in unittest.

Choose Robot Framework when acceptance flows need to be read and discussed by people who do not normally write Python, or when the built-in HTML execution log is a core review artifact. Plan for a capable library maintainer and a controlled keyword vocabulary. Robot is not a no-code route around test engineering; its readability depends on how well the underlying libraries model the business domain.

If the choice remains close, make a two-week pilot with the same application flow and reviewers. Track how long a real requirement change takes, whether failure triage reaches the cause, and how many files a routine edit touches. Those observations are more useful than generic claims that one runner is always faster or more maintainable. Keep the winner as the default, and document clear exceptions for the other tool.

Interview Questions and Answers

A hiring discussion often tests whether you understand the trade-off beneath syntax. Explain where data isolation lives, what a parameter row means to reporting, and which additional library drives a browser or API. The interviewQnA entries below provide concise model answers you can practice aloud; adapt them to projects you have actually maintained. For a wider set of framework prompts, review the Robot Framework interview questions and top pytest interview questions.

Common Mistakes

• Comparing different test layers: A pytest unit test and a Robot browser journey do not measure runner speed fairly. Hold the application work constant, as the cart example does.
• Assuming Robot has every integration built in: Browser and API interactions usually need additional libraries. Check those dependencies and their lifecycle before committing to a stack.
• Sharing mutable state: A session-scoped pytest fixture or a Robot library instance can leak data across checks. Use fresh state or an explicit reset, and verify parallel workers separately.
• Counting template rows as independent Robot tests: One templated test can contain several iterations. Design report granularity around what the runner actually records.
• Leaving custom pytest markers unregistered: Typos in markers can produce misleading filtered jobs. Register markers and enable strict validation.
• Hiding business logic in test syntax: Complex calculations belong in a tested helper or library. Keep the test focused on inputs, actions, and observable outcomes.
• Publishing sensitive execution logs: Robot HTML logs can include keyword arguments. Use safe test data and configure secret handling before uploading artifacts.

Troubleshooting

Problem: ModuleNotFoundError: cart -> Run python -m pytest or python -m robot from the project root used in this guide. Confirm cart.py and CartLibrary.py are there and the active Python environment is the one that installed the runners.

Problem: Robot cannot import CartLibrary.py -> Check that tests/cart.robot and tests/cart_errors.robot refer to ../CartLibrary.py. The path is resolved from the suite file, not from an arbitrary terminal directory.

Problem: Discount template rows fail after the first row -> Put New Cart inside the Discount Is keyword. Test Setup runs once per test case, while all rows in this template execute within the same test case.

Problem: Pytest reports an unknown smoke marker -> Add the [tool.pytest.ini_options] table from Step 5 to the root pyproject.toml, then rerun collection. Keep the marker spelling identical in config and decorator.

Problem: CI shows passing tests but no reports -> Verify the upload path matches the runner output path. Pytest writes artifacts/pytest.xml only when the XML option is supplied; Robot's HTML files live under the directory given to --outputdir.

Problem: Results differ between local and CI runs -> Print python -m pytest --version and python -m robot --version in the job, pin the working dependency set, and compare working directories and environment variables. Debug the first differing precondition before editing assertions.

Where To Go Next

Turn the pilot into a repository convention: one default runner, a naming rule, a fixture or keyword ownership policy, and an artifact upload rule. If you choose pytest for browser checks, explore Python Playwright fixtures with pytest. If you choose Robot for acceptance work, expand the Robot Framework tutorial into resource files only when shared keywords genuinely reduce repeated workflow code. Keep the cart example as a training exercise, not as duplicate production coverage.

Conclusion

Pytest vs Robot Framework comes down to the maintenance interface your team needs. Pytest exposes Python directly and gives strong fixture and parameter mechanisms; Robot expresses workflows as keywords and produces detailed HTML artifacts by default. Both are credible when the surrounding libraries, test data, and CI practices are sound. Run the shared example, ask maintainers to change and debug it, then standardize the runner that makes those routine jobs clearer for your team.

Interview Questions and Answers

Frequently Asked Questions

Related Guides

• GitHub Actions vs Jenkins for Test Automation (2026)
• TestRail vs Zephyr vs Xray for Test Management (2026)
• Building an MCP server for test automation (2026)
• How to Build a hybrid test automation framework (2026)
• How to Structure a scalable test automation framework (2026)
• Java Concurrency for Test Automation Complete Guide (2026)