Intercept Network Requests with Selenium BiDi in Python

TL;DR

Enable BiDi on Selenium browser options, add a `before_request` handler through `driver.network`, and resolve each intercepted request with `continue_request()` or `fail_request()`. Filter narrowly, verify the browser-visible result with a bounded wait, and remove the handler in teardown.

Key Takeaways

• Enable BiDi before creating the browser session and confirm that webSocketUrl was negotiated.
• Register a before_request handler before the action that produces the traffic.
• Resolve every intercepted request exactly once by continuing or failing it.
• Filter by parsed host and path before changing traffic so unrelated resources remain untouched.
• Move callback observations to the test thread and use bounded waits for assertions.
• Remove handlers in finally blocks and keep interception state local to one driver.
• Test a visible application outcome, not merely the fact that a callback ran.

The selenium bidi intercept network requests python workflow lets you inspect a real browser request and decide whether it should continue or fail. Selenium drives the page as usual, while WebDriver BiDi provides the event channel needed to pause the request before it leaves the browser.

This tutorial builds a pytest suite that observes document traffic, blocks one test-owned image, and proves the resulting application behavior. If you need the larger event model first, read the complete Selenium BiDi automation guide.

Use interception for browser-specific evidence and resilience scenarios. Keep broad status, schema, and business-rule coverage in API tests because those tests are faster and isolate server behavior more clearly.

What You Will Build

You will create a small Python project that can:

• Start Chrome with a negotiated WebDriver BiDi connection.
• Serve a deterministic local page and image from a test-owned HTTP server.
• Observe requests without changing the page.
• Fail one exact image request and assert the UI fallback.
• Package filtering and cleanup in a reusable context manager.
• Produce useful failure evidence without logging secrets or request bodies.

The local server makes the examples repeatable. It also avoids blocking an analytics script, ad, or production endpoint by accident. The same handler pattern works against an approved test environment after you replace the allowlist and assertions.

Prerequisites

Use Python 3.11 or later, Selenium 4.46.0, pytest 8.3.5, and a current stable Chrome installation. The examples use Chrome because it offers a straightforward local baseline. Validate Firefox or Grid separately before adding it to your required matrix.

Create the project and install pinned packages:

mkdir selenium-bidi-network
cd selenium-bidi-network
python -m venv .venv
source .venv/bin/activate
python -m pip install selenium==4.46.0 pytest==8.3.5

On Windows PowerShell, activate with .venv\Scripts\Activate.ps1. Selenium Manager normally locates or downloads a compatible driver. Managed CI environments should pin the browser image and record the Selenium client, browser, driver, and Grid versions in diagnostics.

Create a file named test_network_interception.py. Every following step extends that file.

Verification: Run python --version, python -m pip show selenium, and python -m pytest --version. Confirm Python is at least 3.11 and Selenium reports 4.46.0.

Step 1: Enable a Selenium BiDi Session in Python

Request BiDi while constructing the browser options. Session capabilities are negotiated once, so changing the option after webdriver.Chrome() cannot add the WebSocket connection.

import pytest
from selenium import webdriver


@pytest.fixture
def driver():
    options = webdriver.ChromeOptions()
    options.enable_bidi = True
    browser = webdriver.Chrome(options=options)
    browser.set_page_load_timeout(10)

    assert browser.capabilities.get("webSocketUrl")
    yield browser
    browser.quit()


def test_bidi_session_is_available(driver):
    assert isinstance(driver.capabilities["webSocketUrl"], str)

The fixture makes one driver per test and always quits it. The capability assertion fails near setup if the local driver or remote node did not negotiate BiDi. That is much clearer than letting a later request-handler call fail.

Do not print the complete WebSocket URL in shared build logs. It is useful for a disposable local diagnosis, but build output only needs browser versions and whether the capability exists. For headless CI, add options.add_argument("--headless=new") before creating the driver.

Verification: Run python -m pytest -q -k bidi_session. One test should pass. If it fails, upgrade the Selenium client and browser together, then inspect the capabilities returned by the Grid node.

Step 2: Create a Deterministic Test Page

Add a local HTTP server fixture. The page loads one image and reports either loaded or failed in a status element. This gives the interception test a visible business outcome.

import base64
import threading
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

PNG = base64.b64decode(
    "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0l"
    "EQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII="
)


class DemoHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == "/":
            body = b"""<!doctype html>
              <title>BiDi request test</title>
              <p id='status'>waiting</p>
              <img src='/hero.png'
                   onload=\"status.textContent='loaded'\"
                   onerror=\"status.textContent='failed'\">"""
            self.send_response(200)
            self.send_header("Content-Type", "text/html; charset=utf-8")
        elif self.path == "/hero.png":
            body = PNG
            self.send_response(200)
            self.send_header("Content-Type", "image/png")
        else:
            body = b"not found"
            self.send_response(404)
            self.send_header("Content-Type", "text/plain")

        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)

    def log_message(self, format, *args):
        pass


@pytest.fixture(scope="module")
def site_url():
    server = ThreadingHTTPServer(("127.0.0.1", 0), DemoHandler)
    thread = threading.Thread(target=server.serve_forever, daemon=True)
    thread.start()
    host, port = server.server_address
    yield f"http://{host}:{port}"
    server.shutdown()
    server.server_close()
    thread.join(timeout=2)

Binding to port zero asks the operating system for an available port. ThreadingHTTPServer prevents the browser's parallel resource requests from blocking one another. The server is module-scoped, but the browser and its handlers remain test-scoped.

The server is intentionally tiny. For production framework tests, prefer a maintained test application or a fixture supplied by the service team rather than embedding a large fake server in UI tests.

Verification: Add def test_page_loads(driver, site_url): driver.get(site_url); assert driver.title == "BiDi request test". Run it and confirm the page opens without an external network dependency.

Step 3: Observe and Continue Network Requests

Now install a before_request handler. Adding this handler creates an interception point, so every callback invocation must be resolved. Use finally inside the callback to guarantee that passive observation cannot freeze navigation.

from selenium.webdriver.support.ui import WebDriverWait


def test_observes_document_and_image(driver, site_url):
    observed = []

    def observe(request):
        try:
            observed.append((request.method, request.url))
        finally:
            request.continue_request()

    handler_id = driver.network.add_request_handler(
        "before_request", observe
    )
    try:
        driver.get(site_url)
        WebDriverWait(driver, 5).until(
            lambda current: current.find_element("id", "status").text
            == "loaded"
        )
        WebDriverWait(driver, 5).until(
            lambda _: any(url.endswith("/hero.png") for _, url in observed)
        )

        assert any(method == "GET" and url == site_url + "/"
                   for method, url in observed)
    finally:
        driver.network.remove_request_handler(
            "before_request", handler_id
        )

The callback only copies safe fields into a list. Assertions occur on the pytest thread, where failures are reported normally. Browsers can request icons or internal resources, so never assume the first event is the page you care about. Match the exact expected URL.

WebDriverWait also establishes a boundary for asynchronous delivery. A fixed sleep can be too short on CI and needlessly slow locally.

Verification: Run python -m pytest -q -k observes. The status must become loaded, and the collected events must include both the document and image. Temporarily remove continue_request(), observe the bounded page-load failure, then restore it.

Step 4: Filter the Exact Request You Own

A substring such as if "hero" in request.url is unsafe. Parse the URL and compare its scheme, hostname, port, and path. This prevents a lookalike domain or query value from matching your rule.

from urllib.parse import urlsplit


def matches_image(request_url, site_url):
    actual = urlsplit(request_url)
    allowed = urlsplit(site_url)
    return (
        actual.scheme == allowed.scheme
        and actual.hostname == allowed.hostname
        and actual.port == allowed.port
        and actual.path == "/hero.png"
        and actual.query == ""
    )


def test_filter_matches_only_owned_image(site_url):
    assert matches_image(site_url + "/hero.png", site_url)
    assert not matches_image(site_url + "/hero.png?copy=1", site_url)
    assert not matches_image(site_url + "/other.png", site_url)
    assert not matches_image("https://example.com/hero.png", site_url)

Make the policy a pure function. You can unit test it without launching a browser, and a reviewer can see exactly which traffic is eligible for mutation. Expand the predicate deliberately when your test needs query parameters or several paths.

Never record full authorization headers, cookies, query strings, or request bodies in general-purpose diagnostics. Store the method, sanitized origin, path, and decision. If sensitive payload inspection is essential, redact before the data enters pytest output or a CI artifact.

Verification: Run python -m pytest -q -k filter. All four assertions should pass. Add a lookalike hostname such as 127.0.0.1.example.com and confirm it does not match.

Step 5: Selenium BiDi Intercept Network Requests Python Failure Test

Fail the exact image request and continue every other request. A transport failure differs from an HTTP 404 or 500 because no application response is returned. This test models a broken connection to one resource.

def test_fails_one_image_request(driver, site_url):
    decisions = []

    def block_owned_image(request):
        if matches_image(request.url, site_url):
            decisions.append(("failed", request.url))
            request.fail_request()
        else:
            decisions.append(("continued", request.url))
            request.continue_request()

    handler_id = driver.network.add_request_handler(
        "before_request", block_owned_image
    )
    try:
        driver.get(site_url)
        status = WebDriverWait(driver, 5).until(
            lambda current: (
                current.find_element("id", "status").text
                if current.find_element("id", "status").text
                == "failed" else False
            )
        )

        assert status == "failed"
        assert decisions.count(("failed", site_url + "/hero.png")) == 1
    finally:
        driver.network.remove_request_handler(
            "before_request", handler_id
        )

Each branch resolves the paused request exactly once. Do not place a second unconditional continue_request() in finally, because the blocked branch would then try to resolve the same request twice. The decision list proves that the intended request reached the handler, while the DOM assertion proves that the application reacted.

This is stronger than asserting only len(decisions) > 0. A callback can run while the user-visible fallback remains broken. Your acceptance criterion should describe the UI response to lost traffic.

Verification: Run python -m pytest -q -k fails_one. The page status should become failed, and exactly one image decision should be recorded. Change the filter path to /missing.png and confirm the test fails because the image loads.

Step 6: Package Interception and Cleanup

Repeated setup invites leaked handlers. Wrap registration and removal in a context manager so the handler lifetime is visible around the browser action.

from contextlib import contextmanager


@contextmanager
def request_handler(driver, callback):
    handler_id = driver.network.add_request_handler(
        "before_request", callback
    )
    try:
        yield
    finally:
        driver.network.remove_request_handler(
            "before_request", handler_id
        )


def test_context_manager_cleans_up(driver, site_url):
    failed = []

    def callback(request):
        if matches_image(request.url, site_url):
            failed.append(request.url)
            request.fail_request()
        else:
            request.continue_request()

    with request_handler(driver, callback):
        driver.get(site_url)
        WebDriverWait(driver, 5).until(
            lambda current: current.find_element("id", "status").text
            == "failed"
        )

    driver.get(site_url)
    WebDriverWait(driver, 5).until(
        lambda current: current.find_element("id", "status").text
        == "loaded"
    )
    assert failed == [site_url + "/hero.png"]

The second navigation is a cleanup test. It proves that traffic returns to normal after leaving the context. Keep the context narrow, ideally around one action and its assertions. Do not install a global mutation handler for the entire test session.

If a callback itself raises before resolving a request, navigation can hang. Keep callbacks small, use explicit branches, and catch only exceptions you can handle safely. A framework wrapper may record a sanitized error and fail the unresolved request, but it should never silently continue after a policy bug.

Verification: Run this test by itself, then run the complete file. Both navigations must finish, first with failed and then with loaded.

Step 7: Make the Suite Reliable on CI and Grid

Run one driver and one handler state object per test worker. Request IDs and callbacks belong to a single BiDi session. Sharing a list or handler across parallel drivers produces misleading counts and teardown races.

Use returned capabilities as your first remote check:

def bidi_diagnostics(driver):
    capabilities = driver.capabilities
    return {
        "browserName": capabilities.get("browserName"),
        "browserVersion": capabilities.get("browserVersion"),
        "bidiAvailable": bool(capabilities.get("webSocketUrl")),
    }

Log this sanitized dictionary on failure, not the WebSocket address. A remote provider must negotiate webSocketUrl and route the WebSocket for the session. A normal WebDriver session can succeed even when the event channel is unavailable.

Keep interception tests focused. One test can verify a failed image, another can verify an API transport failure, and an ordinary UI test should run without mutation. If you need latency or offline behavior, follow the Selenium BiDi network conditions tutorial because delay, throughput, and offline states require different assertions and cleanup.

Verification: Run python -m pytest -q locally and in one CI browser before enabling a matrix. Confirm the suite finishes without hanging and that the cleanup test still loads the image on its second navigation.

Step 8: Choose the Right Selenium BiDi Intercept Network Requests Python Scenario

Before adding another handler, name the failure you need to reproduce. Interception is a mechanism, not the assertion. A request can be observed, continued with a change, failed before a response exists, or allowed to receive a real HTTP error. Those cases exercise different browser and application paths.

The examples in this tutorial deliberately cover observation and transport failure. They do not pretend that fail_request() returns a chosen status code. If the requirement says, "show the not-found placeholder after a 404," configure your test server or stub service to return 404. If it says, "recover when the image host is unreachable," fail the request before a response and assert the transport-error fallback.

Request modification deserves the same precision. Selenium Python continue_request() accepts supported overrides such as method, URL, headers, cookies, or body, but the values must follow the WebDriver BiDi protocol shapes expected by the current Selenium binding. Do not copy a CDP header dictionary into a BiDi test and assume it is portable. Start with the smallest override, keep the destination on a test-owned allowlist, and verify the change at a local echo endpoint before using it in a larger journey.

Treat observation and interception as separate test intentions even when they use the same handler API. An observation test records a safe projection and immediately continues traffic. A mutation test makes one controlled decision and proves both that the decision occurred and that the user-visible behavior changed. This separation prevents a diagnostic listener from quietly becoming a global fault injector.

Finally, decide whether the browser is necessary. If the acceptance criterion is only "the endpoint returns 500 for invalid input," write an API test. Choose Selenium BiDi when browser request construction, browser transport behavior, or the rendered fallback is part of the requirement. This keeps the UI suite smaller while preserving the evidence that only a real browser can provide.

Verification: Review every interception test name and assertion. Each name should identify observation, transport failure, request modification, HTTP response, or network conditions. Confirm the handler decision and the visible assertion match the same row in the table.

Troubleshooting

driver.network is unavailable -> Confirm Selenium is the pinned current release and options.enable_bidi = True was set before session creation. Inspect the returned capability for webSocketUrl. Upgrade the client, browser, driver, and Grid node as a compatible set.

Navigation hangs after the handler runs -> One callback path did not call continue_request() or fail_request(). Trace a sanitized decision for every request and ensure each branch resolves it exactly once.

The target is never intercepted -> Register the handler before navigation or the click that triggers traffic. Compare parsed URL components, and inspect observed safe URLs for redirects, a different port, or an unexpected path.

The wrong resources fail -> Replace substring or regular-expression matching with an exact allowlist of scheme, host, port, and path. Run filter unit tests with lookalike URLs before launching the browser.

Assertions inside the callback disappear or behave inconsistently -> Do not use pytest assertions on the callback thread. Store a small typed observation, wait for it with a timeout, and assert on the test thread.

The test passes locally but fails on Grid -> Verify that the provider supports and proxies WebDriver BiDi for the selected browser. Start with one session, log sanitized capabilities, and confirm the same pinned versions before adding parallel workers.

Where To Go Next

Return to the Selenium BiDi automation complete guide to connect request control with authentication, script events, and cross-browser planning. Then extend this test in small, separate slices:

• Use the Java Selenium BiDi authentication handler tutorial when a protected environment presents an HTTP authentication challenge rather than a DOM login form.
• Add the Selenium BiDi JavaScript error capture examples to correlate a failed resource with an uncaught frontend exception.
• Use the Selenium BiDi network conditions tutorial for controlled latency, offline, and recovery scenarios.
• Apply Selenium explicit wait best practices conceptually in Python too: wait for observable state and keep every wait bounded.

Keep each protocol feature test-scoped. Combining several listeners can be useful for one diagnosis, but separate acceptance tests provide clearer failure ownership.

Interview Questions and Answers

Q: Why does a Selenium BiDi request handler need to continue unmatched traffic?

A before_request handler creates an interception point. Unmatched traffic remains paused unless the callback resolves it. Continue normal requests and fail only the exact request required by the scenario.

Q: Why should URL filtering compare parsed components?

Substring matching can accept lookalike hosts, query strings, or unrelated paths. Parsing lets the test require the intended scheme, host, port, and path. This protects third-party traffic and makes the mutation policy reviewable.

Q: What is the difference between fail_request() and returning HTTP 500?

fail_request() simulates a transport-level failure, so the browser receives no HTTP response. HTTP 500 is a valid response from a server. Applications can handle these cases differently, so tests should model and name them separately.

Q: Why avoid assertions inside an event callback?

The callback can execute outside the pytest test thread. An assertion failure there may not be reported as the test's primary failure and can leave a request paused. Capture data in the callback, then wait and assert on the test thread.

Q: How do you prove interception cleanup worked?

Remove the handler in a finally block or context manager. Navigate again after cleanup and assert normal behavior. This catches leaked handlers that would otherwise affect a later test.

Q: When should you prefer an API test over Selenium BiDi?

Use API tests for broad response status, schema, authorization, and business-rule coverage. Use BiDi when the requirement concerns what a real browser sends, blocks, receives, or displays during a user journey.

Best Practices

• Register the handler before the action that creates the request.
• Continue or fail every intercepted request exactly once.
• Allowlist test-owned origins and paths before mutating traffic.
• Keep callbacks short and perform assertions on the test thread.
• Use bounded waits for both event evidence and visible application state.
• Remove handlers in finally, even when navigation or assertions fail.
• Keep driver, callback, decisions, and handler ID local to one test.
• Sanitize URLs and never log cookies, authorization values, or sensitive bodies.
• Pin a verified Selenium and browser combination in CI.
• Test transport failure, HTTP errors, latency, and offline mode as distinct risks.

Conclusion

To implement selenium bidi intercept network requests python tests safely, enable BiDi during session creation, register a before_request handler, filter an exact test-owned URL, and resolve every request with continue_request() or fail_request(). Verify both protocol evidence and the visible application outcome.

Start with the local deterministic example, then move the allowlist to your approved test environment. Keep teardown explicit and each mutation narrow, and your interception tests will remain useful in local runs, CI, and a supported Grid.

Interview Questions and Answers

Frequently Asked Questions

Related Guides

• Selenium DevTools in Selenium 4 in Python (2026)
• Selenium BiDi network in Python (2026)
• How to Use Selenium BiDi network in Java (2026)
• How to Use Selenium DevTools in Selenium 4 in Java (2026)
• Selenium network interception CDP in Python (2026)
• Add a Selenium BiDi Authentication Handler in Java