Record Videos for Failed Selenium Grid Sessions

TL;DR

Attach a selenium/video sidecar to each Grid browser container, name each recording by session, and use a pytest hook to retain the recording only when the test fails. Always quit the driver and wait for the encoder to finalize the file before CI uploads it.

Key Takeaways

• Run one video sidecar per browser container and share its virtual display.
• Use automatic video naming so parallel sessions never overwrite one another.
• Record every session, then delete passed-test videos because the final outcome is unknown until teardown.
• Capture the WebDriver session ID before quit so each pytest result maps to one video.
• Wait for video finalization before moving, deleting, or uploading an artifact.
• Upload failure videos with logs, screenshots, and test metadata under the same artifact key.

A practical selenium grid video recording failed sessions workflow records each browser session, identifies its final test outcome, and removes the video when the test passes. You cannot know that outcome when the session starts, so failure-only recording is normally implemented as failure-only retention, not as a recorder that starts after the failure.

This tutorial adds that evidence pipeline to the Selenium Grid cloud scaling complete guide. You will run a Grid with an isolated Chrome node and video sidecar, execute passing and failing pytest cases, and prove that only the failed session remains.

The design is intentionally small enough to run locally but uses the same boundaries you need in CI: unique artifact keys, deterministic cleanup, encoder finalization, and restricted retention. Pin compatible image tags in your own repository and validate newer tags before promotion.

What You Will Build

You will build a complete failure-video pipeline:

• A Selenium Hub and one Chrome node on a private Compose network.
• A video sidecar connected to the Chrome container's virtual display.
• A bind-mounted artifact directory that persists recordings outside the recorder container.
• Two pytest tests, one passing and one intentionally failing.
• A pytest fixture that saves the WebDriver session ID before quit().
• A session-end cleanup hook that removes videos for passed tests.
• A manifest that maps retained videos to failed pytest node IDs.
• A CI pattern that uploads only failure evidence.

The important lifecycle is:

create driver -> record session -> run assertions -> capture outcome
-> quit driver -> finalize video -> delete pass or retain failure -> upload

Do not start recording only inside an exception handler. By then, the interaction that caused the failure has already happened.

Prerequisites

Install Docker Engine 26 or newer with Docker Compose v2, Python 3.11 or newer, and curl. Allow at least 2 CPU cores, 4 GB of memory, and enough disk for short MP4 files. Chrome and FFmpeg consume real resources even when the test is idle.

Check the tools:

docker version
docker compose version
python3 --version
curl --version

Create the lab:

mkdir selenium-grid-failure-video
cd selenium-grid-failure-video
mkdir -p tests artifacts/videos artifacts/results
python3 -m venv .venv
. .venv/bin/activate
python -m pip install 'selenium>=4.39,<5' 'pytest>=8.3,<10'

This tutorial pins Selenium Grid and Chrome node 4.44.0-20260505 with video image ffmpeg-8.1-20260505. These dated tags form a compatible release set and make the example reproducible. Review a newer docker-selenium release in a branch before changing every image together. Never mix Hub and node releases.

Use this lab only against systems you are authorized to test. Videos can contain credentials, personal data, and internal application state.

Step 1: Start a Grid with a Video Sidecar

Create compose.yaml:

name: selenium-grid-failure-video

services:
  hub:
    image: selenium/hub:4.44.0-20260505
    ports:
      - '4444:4444'

  chrome:
    image: selenium/node-chrome:4.44.0-20260505
    shm_size: 2gb
    environment:
      SE_EVENT_BUS_HOST: hub
      SE_EVENT_BUS_PUBLISH_PORT: '4442'
      SE_EVENT_BUS_SUBSCRIBE_PORT: '4443'
      SE_NODE_MAX_SESSIONS: '1'
      SE_NODE_OVERRIDE_MAX_SESSIONS: 'false'
    depends_on:
      - hub

  chrome-video:
    image: selenium/video:ffmpeg-8.1-20260505
    environment:
      DISPLAY_CONTAINER_NAME: chrome
      SE_VIDEO_FILE_NAME: auto
    volumes:
      - ./artifacts/videos:/videos
    depends_on:
      - chrome

Start the pinned stack:

docker compose config --quiet
docker compose up -d
docker compose ps

DISPLAY_CONTAINER_NAME tells the recorder which container exposes the X display. SE_VIDEO_FILE_NAME=auto avoids a shared fixed filename and lets the docker-selenium recorder derive a session-specific name. One maximum session keeps this first lab easy to reason about.

Do not place two browser containers behind one recorder. Run a recorder beside each browser node, or use a supported dynamic-node recording design. Otherwise the video can show the wrong display or mix unrelated activity.

Verify the step: wait for readiness and confirm one available slot:

until curl -fsS http://localhost:4444/status | grep -q '"ready": true'; do sleep 2; done
curl -fsS http://localhost:4444/status
docker compose logs --tail=30 chrome-video

The status response should report a ready Grid. The recorder log should show FFmpeg initialization without repeated connection failures.

Step 2: Create a Session-Naming Helper

Create tests/conftest.py with imports, shared state, and a safe artifact name function:

import json
import re
import time
from pathlib import Path

import pytest
from selenium import webdriver

VIDEO_DIR = Path('artifacts/videos')
RESULT_DIR = Path('artifacts/results')
OUTCOMES = {}
SESSIONS = {}

def safe_name(nodeid: str) -> str:
    value = re.sub(r'[^A-Za-z0-9._-]+', '_', nodeid)
    return value.strip('._')[:160] or 'unnamed-test'

The pytest node ID is useful to humans but unsafe as a raw path. It can contain slashes, brackets, spaces, and parameter values. Normalize it before using it in a retained filename. Keep the session ID separately because that is the bridge to the recorder output.

Do not send secrets or full test data in the se:name capability. Grid UIs, logs, and video metadata may expose it. A short normalized test identifier is sufficient.

Verify the step: import the helper and inspect one value:

python -c "from tests.conftest import safe_name; print(safe_name('tests/test_checkout.py::test_card[visa]'))"

If Python cannot import tests.conftest, add an empty tests/__init__.py and retry. Expect a filesystem-safe name containing test_checkout.py_test_card_visa.

Step 3: Create and Close Remote WebDriver Correctly

Append this fixture to tests/conftest.py:

@pytest.fixture
def driver(request):
    options = webdriver.ChromeOptions()
    options.set_capability('se:name', safe_name(request.node.nodeid))

    remote = webdriver.Remote(
        command_executor='http://localhost:4444',
        options=options,
    )
    SESSIONS[request.node.nodeid] = remote.session_id
    yield remote
    remote.quit()

Save session_id immediately after session creation. After quit(), a client should not depend on driver state. The yield fixture guarantees normal teardown after a passed or assertion-failed test. pytest still executes fixture teardown when the test body raises.

If browser construction itself fails, there is no valid session and usually no useful session video. Treat that as Grid provisioning evidence instead: preserve Hub logs, node logs, health output, and the original exception.

The se:name capability makes the session recognizable in Grid. Do not add Chrome's headless argument to this recorded workflow. The docker-selenium video recorder captures the node's graphical display and does not support recording a headless browser. The browser can still run without a desktop window on the CI host because its display exists inside the node container.

Verify the step: ask pytest to collect fixtures without running a test:

pytest --fixtures tests | grep -A2 '^driver'

You should see the local driver fixture. Run python -m py_compile tests/conftest.py to catch syntax errors before opening a browser.

Step 4: Capture the Final pytest Outcome

Append the result hook:

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()
    if report.when == 'call':
        OUTCOMES[item.nodeid] = report.outcome
    elif report.when == 'setup' and report.failed:
        OUTCOMES[item.nodeid] = 'failed'
    elif report.when == 'teardown' and report.failed:
        OUTCOMES[item.nodeid] = 'failed'

pytest emits separate reports for setup, call, and teardown. A failure-video policy must treat failure in any phase as a failed test. The hook stores the call outcome and then lets setup or teardown failures override it.

A skipped test is neither a pass nor a product failure by default. This tutorial removes skipped-session videos. If skips diagnose unstable infrastructure in your suite, retain them explicitly and mark the reason in the manifest. Decide that policy before CI, not after disk fills.

If you use pytest-xdist, process-local dictionaries are not a cross-worker database. Write one small result record per worker or use pytest's report serialization hooks, then perform retention after all workers finish. The single-process implementation here is runnable and deliberately avoids pretending to be xdist-safe.

Verify the step: run a collection-only command and compile the file:

python -m py_compile tests/conftest.py
pytest --collect-only -q tests

Compilation should exit with code 0. Collection may report no tests until you complete Step 6.

Step 5: Retain Selenium Grid Video Recording Failed Sessions

Append these functions and the session-end hook:

def wait_for_video(session_id: str, test_name: str, timeout: int = 30):
    candidates = [
        VIDEO_DIR / f'{test_name}_{session_id}.mp4',
        VIDEO_DIR / f'{session_id}.mp4',
    ]
    deadline = time.monotonic() + timeout
    previous = None
    stable = 0

    while time.monotonic() < deadline:
        existing = next((path for path in candidates if path.exists()), None)
        if existing:
            size = existing.stat().st_size
            stable = stable + 1 if size == previous and size > 0 else 0
            previous = size
            if stable >= 2:
                return existing
        time.sleep(1)
    return None

def pytest_sessionfinish(session, exitstatus):
    RESULT_DIR.mkdir(parents=True, exist_ok=True)
    manifest = []

    for nodeid, session_id in SESSIONS.items():
        name = safe_name(nodeid)
        video = wait_for_video(session_id, name)
        result = OUTCOMES.get(nodeid, 'failed')

        if video is None:
            manifest.append({
                'test': nodeid, 'outcome': result, 'video': None,
                'error': 'video not finalized before timeout',
            })
            continue

        if result == 'failed':
            retained = VIDEO_DIR / f'FAILED_{name}_{session_id}.mp4'
            video.replace(retained)
            manifest.append({
                'test': nodeid, 'outcome': result,
                'sessionId': session_id, 'video': str(retained),
            })
        else:
            video.unlink()

    (RESULT_DIR / 'video-manifest.json').write_text(
        json.dumps(manifest, indent=2), encoding='utf-8'
    )

Waiting for a stable nonzero size matters. FFmpeg writes the file while the browser runs and finalizes the MP4 after the session ends. Uploading too early can produce an unreadable or truncated file. The fallback outcome is failed, which favors evidence preservation if a hook never produced a call report.

With SE_VIDEO_FILE_NAME=auto, the pinned recorder derives the sanitized se:name value and appends the session ID. The session-ID-only candidate provides a defensive fallback when metadata is unavailable. If you later change recorder versions or set SE_VIDEO_FILE_NAME_SUFFIX=false, update the candidate mapping and rerun the deliberate failure test.

Verify the step: compile once more:

python -m py_compile tests/conftest.py

Also confirm artifacts/videos is writable by the recorder. On Linux, ls -ld artifacts/videos should show permissions compatible with the container user. Prefer correcting ownership over making the directory world-writable.

Step 6: Run One Passing and One Failing Test

Create tests/test_video_policy.py:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait

URL = 'https://www.selenium.dev/selenium/web/web-form.html'

def submit_form(driver, value):
    driver.get(URL)
    text = WebDriverWait(driver, 10).until(
        lambda current: current.find_element(By.NAME, 'my-text')
    )
    text.send_keys(value)
    driver.find_element(By.CSS_SELECTOR, 'button').click()
    return driver.find_element(By.ID, 'message').text

def test_passing_session_video_is_removed(driver):
    assert submit_form(driver, 'pass') == 'Received!'

def test_failed_session_video_is_retained(driver):
    assert submit_form(driver, 'failure demo') == 'Not the real message'

Run both tests. The nonzero pytest exit is expected because the second assertion is deliberately wrong:

pytest -q tests/test_video_policy.py || test $? -eq 1
find artifacts -maxdepth 2 -type f -print

The first test proves cleanup. The second proves retention. If the form in your real application loads asynchronously, apply Selenium explicit wait examples instead of adding fixed sleeps. Do not copy this intentional failure into a normal regression suite. Keep it as an infrastructure smoke test or mark it so only the artifact-pipeline job runs it.

Verify the step: artifacts/videos should contain one file whose name starts with FAILED_. artifacts/results/video-manifest.json should contain one failed record with its session ID and video path. Play the MP4 and confirm it shows form entry, submission, and the final page. If two videos remain, inspect the recorded outcome and filename mapping before adding broader glob deletion.

Step 7: Publish Failure Evidence in CI

Run the same lifecycle in CI: start Compose, wait for Grid, run pytest, collect evidence, then tear down. Keep pytest's exit code while allowing artifact upload to run. A generic shell sequence is:

set +e
pytest -q tests
TEST_EXIT=$?
set -e

find artifacts/videos -type f -name 'FAILED_*.mp4' -print
# Invoke your CI provider's artifact upload step here.

docker compose logs --no-color > artifacts/results/grid.log
docker compose down --remove-orphans
exit "$TEST_EXIT"

Provider upload syntax changes independently of Selenium, so use your CI system's current official artifact action. Follow the CI test evidence artifact guide to keep reports, logs, screenshots, and videos discoverable under one run. Configure it to upload artifacts/videos/FAILED_*.mp4, video-manifest.json, the pytest report, and Grid logs. Set a short retention period that satisfies debugging and compliance needs.

Gather logs before docker compose down. Always perform teardown with a job-level finalizer so cancelled or failed runs do not leak containers. If artifact upload fails, report that separately from the product test failure. Missing evidence should be visible, but it should not rewrite the original assertion.

Verify the step: inspect the CI run for one downloadable MP4 and the manifest. Confirm the job still finishes failed because of pytest, not green because the upload step succeeded. Download and play the video from a clean machine to prove that the uploaded object is complete.

Step 8: Scale the Pattern Without Mixing Sessions

A single static node is a learning configuration. For parallel execution, keep the isolation rule: one node display, one recorder, and one active session unless you have explicitly validated a supported multi-session video layout. Give each worker its own artifact namespace.

For elastic hosts, follow the Selenium Grid dynamic node Docker tutorial. For pod-level scheduling and storage, use the Selenium Grid Kubernetes autoscaling step-by-step guide. Both designs still need a durable test-to-session-to-artifact mapping.

Avoid deleting by modification time or by the newest file. Parallel sessions finish out of order, retries can create multiple sessions, and delayed encoding can reorder file visibility. Use a unique session ID and include retry attempt in the human-readable test name.

Verify the step: run two sequential sessions first and prove two distinct automatic filenames. Then test your actual parallel runner with unique namespaces. For every failed result, assert exactly one manifest entry and a playable video. For every passed result, assert no matching video survives cleanup.

Troubleshooting

Problem: the recorder creates no MP4 -> Remove any --headless Chrome argument, confirm DISPLAY_CONTAINER_NAME exactly matches the Compose browser service, inspect recorder logs, and verify that node and recorder share a Docker network. Confirm the browser actually created a session.

Problem: the MP4 exists but is zero bytes or will not play -> Wait for driver.quit() and encoder finalization. Do not stop the recorder before retention runs. Increase the finalization timeout for long or resource-constrained jobs and inspect FFmpeg errors.

Problem: videos overwrite one another -> Set SE_VIDEO_FILE_NAME=auto, use one recorder per node display, and never share a constant output name across concurrent sessions. Validate the filename scheme under your pinned image.

Problem: passed videos remain -> Check that the call outcome is stored, the session ID was captured before quit, and the recorder's actual filename matches a candidate. Inspect the manifest before changing cleanup. Never use an unbounded *.mp4 deletion in a shared artifact directory.

Problem: failed video is missing in CI -> Preserve the pytest exit code but run cleanup and upload in an always-executed finalizer. Check directory ownership, artifact glob expansion, upload retention, and whether Compose was stopped before FFmpeg finished.

Problem: the video shows the wrong test -> Enforce one active browser session per recorded display or switch to isolated dynamic nodes. Correlate the session ID in the manifest with Grid logs and test metadata.

Where To Go Next

Return to the complete Selenium Grid cloud scaling guide to place video retention alongside capacity, security, observability, and cost controls. Video explains the visible browser sequence, but it does not replace network, console, or server telemetry.

Add trace and capacity correlation with the Selenium Grid OpenTelemetry monitoring setup. When a video appears frozen, a trace can show whether the session waited in the queue, the WebDriver command stalled, or the application response was slow.

Choose the next implementation based on infrastructure:

• Use dynamic Docker nodes for Selenium Grid when one host should create disposable nodes on demand.
• Use Kubernetes autoscaling for Selenium Grid when pods, sidecars, persistent storage, and cluster scheduling are already operational standards.

Whichever path you choose, keep video optional for low-risk suites. Encoding every session costs CPU, storage, and upload time even when passed videos are later deleted. Measure that overhead on representative tests.

Interview Questions and Answers

Q: How do you record video only for failed Selenium tests?

Record the complete session, capture the test outcome, and delete the video after a pass. A recorder cannot retroactively capture interactions before an assertion fails. Failure-only retention is the reliable interpretation.

Q: Why use one video recorder per browser node?

The recorder captures a virtual display, not an abstract Grid session. One recorder per isolated display prevents mixed or incorrect footage and gives a clean artifact boundary.

Q: How do you correlate a pytest result with a Grid recording?

Capture driver.session_id immediately after Remote WebDriver creation and store it against pytest's node ID. Configure automatic recording names, then write the retained session ID and path into a manifest.

Q: Why must CI wait after driver quit?

The encoder needs time to flush buffers and finalize the MP4 container. Uploading or stopping the sidecar too early can leave a truncated file even though the test itself finished.

Q: What changes for parallel tests?

Use isolated browser displays and unique artifact namespaces. Do not choose a video by timestamp or latest-file ordering, and make result storage safe across runner processes.

Q: Is video enough to debug Selenium Grid failures?

No. Video shows visible browser behavior but not queue delay, WebDriver transport errors, console exceptions, network responses, or server faults. Correlate it with Grid logs, screenshots, browser logs, and telemetry.

Best Practices

• Pin compatible Hub, node, browser, and recorder images.
• Record from session start, then enforce failure-only retention after the final outcome.
• Capture the session ID before calling quit().
• Use automatic unique filenames and a manifest, not timestamps.
• Wait for a stable nonzero file before rename or upload.
• Keep one recorder per isolated browser display.
• Preserve setup and teardown failures as failed outcomes.
• Upload logs and result metadata beside each failure video.
• Encrypt artifact storage and use the shortest useful retention period.
• Redact test data and never put credentials in se:name.
• Monitor encoder CPU, storage, and upload overhead.
• Test the evidence pipeline with a deliberate controlled failure.

Conclusion

A dependable selenium grid video recording failed sessions setup records before the problem occurs and makes the retention decision afterward. The durable correlation key is the WebDriver session ID, while pytest owns the authoritative pass or fail result.

Start with one node and one recorder. Prove that a pass leaves no MP4 and a failure leaves one playable, mapped artifact. Then carry the same isolation, finalization, security, and manifest rules into Docker elasticity or Kubernetes autoscaling.

Interview Questions and Answers

Frequently Asked Questions

Related Guides

• Monitor Selenium Grid with OpenTelemetry: Selenium Grid OpenTelemetry Monitoring Setup
• Autoscale Selenium Grid on Kubernetes Step by Step
• Configure Selenium Grid Dynamic Docker Nodes
• Docker for QA Tutorial for Beginners (2026)
• Docker for Selenium Grid: Step by Step (2026)
• Git for QA Tutorial for Beginners (2026)