Resource library

QA How-To

Selenium DevTools in Selenium 4 in Python (2026)

Learn selenium DevTools in Selenium 4 python with runnable examples for CDP commands, performance logs, response bodies, headers, blocking, and debugging.

24 min read | 2,615 words

TL;DR

In Python, driver.execute_cdp_cmd sends Chromium CDP commands, while Chrome performance logs expose many network events as JSON messages. Enable the required domains before navigation, filter events precisely, use explicit deadlines instead of sleep, and keep a normal WebDriver assertion for the visible result.

Key Takeaways

  • Use execute_cdp_cmd for direct Chromium commands and enable performance logging when the test needs CDP network events.
  • Configure logging capabilities before creating the driver because they are session capabilities.
  • Enable Network and set headers or blocked URL patterns before navigation.
  • Poll and parse performance log entries by CDP method instead of treating the log as plain text.
  • Retrieve only selected response bodies and expect some cached, redirected, or expired bodies to be unavailable.
  • Keep CDP helpers Chromium-specific and prefer supported WebDriver BiDi features when cross-browser coverage is the goal.

selenium DevTools in Selenium 4 python lets a test send Chrome DevTools Protocol (CDP) commands through driver.execute_cdp_cmd() and observe many CDP events through Chrome's performance log. This combination supports practical tasks such as capturing request metadata, checking response status, retrieving a selected response body, adding headers, blocking URLs, and applying browser conditions.

Python's public surface differs from Selenium Java's typed DevTools listeners. A synchronous Python test can use command names and dictionary arguments directly, then parse performance log messages. Selenium also exposes newer bidirectional connections and WebDriver BiDi modules, but direct CDP remains useful for Chromium-specific coverage in 2026.

The examples use Selenium 4.45.0 as a stable 2026 baseline. CDP itself changes with Chromium, so keep command payloads small, refer to the protocol version supported by your browser, and isolate browser-specific code from page objects.

TL;DR

import json
from selenium import webdriver

options = webdriver.ChromeOptions()
options.set_capability("goog:loggingPrefs", {"performance": "ALL"})
driver = webdriver.Chrome(options=options)

try:
    driver.execute_cdp_cmd("Network.enable", {})
    driver.get("https://www.selenium.dev/")

    for entry in driver.get_log("performance"):
        message = json.loads(entry["message"])["message"]
        if message["method"] == "Network.responseReceived":
            response = message["params"]["response"]
            print(response["status"], response["url"])
finally:
    driver.quit()
Python mechanism Direction Typical use
execute_cdp_cmd(name, args) Test to Chromium Enable domains, set headers, block URLs, request a body
Chrome performance log Chromium to test, buffered Read request, response, and loading events
WebDriver commands Request and response User interactions and visible assertions
WebDriver BiDi APIs Bidirectional Standards-based event workflows as support matures

1. What selenium DevTools in Selenium 4 python Means

There is no need to open the visible Chrome DevTools panel. Selenium talks to the browser's protocol endpoint programmatically. execute_cdp_cmd() accepts a fully qualified command such as Network.enable and a Python dictionary containing that command's parameters. It returns a dictionary when the protocol command has a result.

Events work differently in a conventional synchronous Python script. ChromeDriver can publish DevTools performance events into the performance log. Each log entry contains a JSON string, and the nested message object has a CDP method plus params. The test drains that buffer with driver.get_log("performance"). Calling get_log consumes the available entries, so a framework should parse and retain only the evidence it needs.

Use this layer when browser integration is the test subject. Examples include proving that a lazy-loaded widget called an endpoint, validating a fallback after a CDN request is blocked, or correlating a broken screen with a 500 response. Use normal WebDriver for clicks and UI results. Use a direct API client for broad payload validation. If you are selecting an automation stack, Selenium vs Cypress comparison explains the larger tradeoffs.

2. Install Selenium and Create a CDP-Ready Driver

Create a virtual environment and pin Selenium through your normal dependency workflow. The exact browser and Selenium version may move, but a lock file keeps CI reproducible. Selenium Manager normally resolves the local driver when webdriver.Chrome() starts.

python -m venv .venv
source .venv/bin/activate
python -m pip install "selenium==4.45.0"

Performance logging is a Chrome capability, so set it before driver creation. CDP commands themselves do not require performance logging. You need the capability only when consuming event messages through get_log.

from selenium import webdriver

def create_chrome() -> webdriver.Chrome:
    options = webdriver.ChromeOptions()
    options.add_argument("--window-size=1440,1000")
    options.set_capability(
        "goog:loggingPrefs",
        {"performance": "ALL", "browser": "ALL"},
    )
    return webdriver.Chrome(options=options)

Do not call set_capability after webdriver.Chrome() exists. Capabilities describe the session that ChromeDriver creates. For Grid, confirm that the remote node preserves the Chrome logging preference and permits CDP access. Some cloud vendors provide their own network capture interface, which may be more stable than depending on a provider's CDP tunnel.

3. Run selenium DevTools in Selenium 4 python Commands

execute_cdp_cmd is synchronous: Selenium sends one CDP command and returns its decoded result or raises an error. Enable a domain before commands or events that depend on it. The simplest health check enables Network and asks for the browser's user agent through the Runtime domain.

from selenium import webdriver

driver = webdriver.Chrome()
try:
    driver.execute_cdp_cmd("Network.enable", {})
    result = driver.execute_cdp_cmd(
        "Runtime.evaluate",
        {"expression": "navigator.userAgent", "returnByValue": True},
    )
    user_agent = result["result"]["value"]
    print(user_agent)
finally:
    driver.quit()

Command names and parameter keys come from CDP, not from Python method names. Misspelling Network.enable or sending a parameter that the active protocol does not accept produces a runtime error rather than an editor-visible type error. Keep these calls behind named helpers, validate critical dictionary keys, and cover the helper with a small integration test against the CI browser image.

Avoid using Runtime.evaluate for normal DOM interaction. Selenium locators, waits, and element methods express user behavior more clearly. JavaScript or CDP evaluation is justified when the value is browser instrumentation that WebDriver does not expose.

4. Parse Chrome Performance Logs Into Network Events

A performance entry has outer logging metadata and an inner CDP message. Centralize the decoding so every test does not repeat nested json.loads calls. Ignore malformed or non-CDP records defensively, then filter by method.

import json
from collections.abc import Iterator
from typing import Any
from selenium import webdriver

def cdp_messages(driver: webdriver.Chrome) -> Iterator[dict[str, Any]]:
    for entry in driver.get_log("performance"):
        try:
            yield json.loads(entry["message"])["message"]
        except (KeyError, TypeError, json.JSONDecodeError):
            continue

options = webdriver.ChromeOptions()
options.set_capability("goog:loggingPrefs", {"performance": "ALL"})
driver = webdriver.Chrome(options=options)
try:
    driver.execute_cdp_cmd("Network.enable", {})
    driver.get("https://www.selenium.dev/")

    responses = []
    for message in cdp_messages(driver):
        if message.get("method") != "Network.responseReceived":
            continue
        response = message["params"]["response"]
        responses.append((int(response["status"]), response["url"]))

    assert any(status == 200 for status, _ in responses)
finally:
    driver.quit()

The buffer can be noisy. Filter by URL, resource type, initiator, or request ID, and avoid writing the entire record to CI. Authorization and cookie data may be present. A network event is diagnostic evidence, not automatically a test assertion. Define which request matters and why its method, status, or payload proves the requirement.

5. Wait for a Specific API Response With a Deadline

Single-page applications often send requests after initial navigation. A one-time log drain can run too early. Poll get_log until the selected event arrives or a monotonic deadline expires. A short polling interval is acceptable here because the API exposes a buffered log, but time.sleep should not be the condition itself.

import json
import time
from selenium import webdriver

def wait_for_response(
    driver: webdriver.Chrome, url_fragment: str, timeout: float = 10.0
) -> dict:
    deadline = time.monotonic() + timeout
    while time.monotonic() < deadline:
        for entry in driver.get_log("performance"):
            message = json.loads(entry["message"])["message"]
            if message.get("method") != "Network.responseReceived":
                continue
            response = message["params"]["response"]
            if url_fragment in response["url"]:
                return message["params"]
        time.sleep(0.1)
    raise TimeoutError(f"No response matched {url_fragment!r} within {timeout}s")

options = webdriver.ChromeOptions()
options.set_capability("goog:loggingPrefs", {"performance": "ALL"})
driver = webdriver.Chrome(options=options)
try:
    driver.execute_cdp_cmd("Network.enable", {})
    driver.get("https://app.example.test/orders")
    params = wait_for_response(driver, "/api/orders")
    assert int(params["response"]["status"]) == 200
finally:
    driver.quit()

Replace the illustrative host with your controlled environment. Drain stale logs before the triggering click when the page already made a similar request. If several calls share a URL, also match the HTTP method or correlate a request ID. Continue with a visible assertion so the test covers browser rendering, not only transport.

6. Capture a Selected JSON Response Body

Network.responseReceived provides requestId. Pass it to Network.getResponseBody soon after the matching event. The result contains body and base64Encoded. Text JSON is normally returned directly, but robust code handles Base64. Retrieval can fail when a response came from certain caches, was redirected, is too old, or belongs to a target that closed.

import base64
import json

def response_body(driver, request_id: str) -> bytes:
    result = driver.execute_cdp_cmd(
        "Network.getResponseBody", {"requestId": request_id}
    )
    body = result["body"]
    if result.get("base64Encoded"):
        return base64.b64decode(body)
    return body.encode("utf-8")

params = wait_for_response(driver, "/api/orders")
raw = response_body(driver, params["requestId"])
payload = json.loads(raw.decode("utf-8"))
assert isinstance(payload.get("orders"), list)

This snippet assumes the driver and wait_for_response helper from the previous section. A production helper should add a maximum accepted body size and a clear error that includes a redacted URL. Do not snapshot entire customer payloads. Assert only fields needed for the browser scenario and move broad contract coverage to API pagination testing techniques or OpenAPI schema testing.

7. Set Extra Headers, Block URLs, and Emulate Conditions

CDP commands can prepare controlled browser conditions before navigation. Extra headers are useful for a test-run identifier or a feature gate in a nonproduction environment. Blocked URLs let the suite verify graceful degradation. Network emulation can model deterministic latency and offline state, but it is not a replacement for load testing or real-device performance testing.

from selenium import webdriver

driver = webdriver.Chrome()
try:
    driver.execute_cdp_cmd("Network.enable", {})
    driver.execute_cdp_cmd(
        "Network.setExtraHTTPHeaders",
        {"headers": {"X-Test-Run": "recommendations-fallback"}},
    )
    driver.execute_cdp_cmd(
        "Network.setBlockedURLs",
        {"urls": ["*://cdn.example.test/recommendations/*"]},
    )
    driver.execute_cdp_cmd(
        "Network.emulateNetworkConditions",
        {
            "offline": False,
            "latency": 150,
            "downloadThroughput": 100_000,
            "uploadThroughput": 50_000,
            "connectionType": "cellular3g",
        },
    )
    driver.get("https://shop.example.test/product/42")
    assert "Recommendations unavailable" in driver.page_source
finally:
    driver.quit()

The throughput values are bytes per second and are illustrative test inputs, not claims about real networks. Name the chosen profile in test output. Use a fresh browser session afterward so emulation cannot contaminate unrelated tests. Never add a header that bypasses authorization in production. If the condition is supported through a stable WebDriver capability or BiDi API, consider that more portable interface first.

8. Build Reusable Pytest Fixtures Without Leaking State

A fixture can own Chrome capabilities, DevTools enablement, and teardown. A separate NetworkProbe can parse logs. Keep the driver function-scoped unless session reuse has a measured benefit and rigorous cleanup. Browser protocol settings, cookies, cache, listeners, and log buffers are all state that can cross test boundaries.

import pytest
from selenium import webdriver

@pytest.fixture
def chrome():
    options = webdriver.ChromeOptions()
    options.set_capability("goog:loggingPrefs", {"performance": "ALL"})
    driver = webdriver.Chrome(options=options)
    driver.execute_cdp_cmd("Network.enable", {})
    yield driver
    driver.quit()

def test_catalog_loads_products(chrome):
    chrome.get("https://shop.example.test/catalog")
    params = wait_for_response(chrome, "/api/products")
    assert int(params["response"]["status"]) == 200
    assert chrome.find_elements("css selector", "[data-testid='product-card']")

For parallel pytest workers, each worker must create its own browser. Do not put captured events in a module-global list. If a failure hook attaches logs, redact them and cap their size. A compact record containing timestamp, method, status, resource type, and sanitized URL is usually more useful than megabytes of raw events.

Keep the probe outside page objects. Page objects should model the catalog or checkout UI, while protocol fixtures model test infrastructure. This distinction makes it easier to run the same business test on Firefox without the CDP assertion.

9. Compare Direct CDP With WebDriver BiDi

Selenium's Python bindings are adding standards-based WebDriver BiDi modules, and driver.bidi_connection() also appears in Selenium's CDP examples for asynchronous access. The naming can be confusing: a bidirectional connection helper is not automatically the same as using standardized WebDriver BiDi commands. Read the imported module and command namespace.

Criterion Direct CDP with execute_cdp_cmd WebDriver BiDi
Browser reach Chromium-specific Designed for cross-browser support
Python style Synchronous command plus dictionaries Module APIs and event handlers, often async-aware
Protocol stability Tracks Chromium versions Standards-oriented
Feature depth Broad Chromium surface Expanding Selenium and browser coverage
Best decision Existing Chromium-only need New portable capability when supported

Do not migrate solely because one API sounds newer. List the features the suite actually needs, target browsers, Grid or vendor limitations, and diagnostic requirements. Prefer BiDi when it meets those needs reliably. Retain a thin CDP adapter for gaps. Do not expose raw protocol objects to hundreds of tests, because that turns a future migration into a rewrite.

10. Troubleshoot Missing Events and CDP Errors

If driver.get_log("performance") is unavailable, confirm that goog:loggingPrefs was set on ChromeOptions before session creation and that the current environment supports Chrome performance logs. If the list is empty, enable Network before the action and verify the page really issued the expected request. Remember that get_log drains entries. An earlier helper may have consumed them.

For unknown command or invalid parameter errors, record the Selenium version, browser version, command name, and sanitized arguments. CDP methods can change as Chromium moves. Do not blindly copy a payload from a different protocol version. Prefer well-established commands with minimal parameters, and validate your helper against the pinned CI image.

When Network.getResponseBody fails, retrieve the body sooner and ensure the request ID came from the same active target. Cached, redirected, streaming, and very large responses need special handling. When the browser scenario only needs a status code, do not request the body at all. For broad negative testing, a direct API suite is clearer than forcing every validation through DevTools.

11. Correlate Browser Console and Network Evidence

A failed request may be only half the story. The application can catch a 500 response and render the right fallback, or it can throw a JavaScript error while processing a valid response. Configure the Chrome browser log with the same pre-session capability pattern, then collect sanitized console messages alongside the selected network event. Keep the channels separate because a console timestamp and a CDP event timestamp may use different representations. Correlate them by the user action, route, endpoint, and a test-run header when your nonproduction service echoes one safely.

options = webdriver.ChromeOptions()
options.set_capability(
    "goog:loggingPrefs",
    {"performance": "ALL", "browser": "ALL"},
)
driver = webdriver.Chrome(options=options)

# Trigger the application behavior first.
severe_messages = [
    entry for entry in driver.get_log("browser")
    if entry.get("level") == "SEVERE"
]
for entry in severe_messages:
    print(entry["message"])

Do not automatically fail every test for every severe console entry. Browser extensions, third-party widgets, and known environmental messages can create noise. Define an allowlist narrowly, review it, and fail on errors owned by the product or the scenario. Capture the original message once, not repeated polling duplicates. Never place access tokens or unredacted payloads into the correlation ID.

12. Control Memory, Security, and Diagnostic Volume

Performance logging can generate thousands of records on a modern application. Drain the buffer during a long scenario, immediately discard unrelated methods, and retain compact dictionaries rather than entire raw messages. If the test needs only one response, stop collecting once the match is found. A high-volume observability test should be separated from the normal functional suite and measured for runtime and artifact cost.

Redact request headers, query parameters, cookies, and response bodies before they reach console output or CI attachments. Prefer an allowlist of fields to keep over a denylist of secrets to remove. A new authentication header or personal-data field can bypass an outdated denylist. Apply artifact access, encryption, and retention policies to browser captures just as you would to server logs.

CDP instrumentation can also change the system it observes. Blocking resources, adding headers, disabling cache, and throttling network traffic alter browser state. Name those conditions in the test, start a fresh session for unrelated coverage, and avoid combining several modifications unless their interaction is the requirement. When a diagnostic helper has no assertion value in a passing run, enable it only on demand or keep its retained output minimal. This discipline makes failures easier to read and prevents protocol tooling from becoming the source of suite instability.

Keep one protocol smoke test that opens a known page, enables Network, and proves that a selected response event can be decoded. Run it when the browser image or Selenium dependency changes. That small test separates protocol plumbing failures from application failures and gives maintainers a clear upgrade signal. It should use controlled, nonsensitive traffic and finish quickly.

Interview Questions and Answers

Q: How do you execute a CDP command with Selenium Python?

Call driver.execute_cdp_cmd() with the qualified command name and an arguments dictionary. For example, driver.execute_cdp_cmd("Network.enable", {}) enables the Network domain. The method returns a dictionary for commands that produce a result.

Q: How can a synchronous Python test receive CDP network events?

For Chrome, enable the performance log capability before driver creation. Read entries with driver.get_log("performance"), decode the nested JSON message, and filter on methods such as Network.responseReceived. The log is buffered and drained when read.

Q: Why should a network assertion use a monotonic deadline?

A single read may occur before a later SPA request. Polling until a monotonic deadline makes the condition explicit and avoids problems if the system clock changes. The helper returns immediately when the selected event arrives and raises a diagnostic timeout otherwise.

Q: What is risky about response-body capture?

The body can be large, sensitive, Base64 encoded, or unavailable after caching or eviction. I retrieve only a selected endpoint, cap and redact diagnostics, and use API tests for comprehensive payload validation.

Q: Is execute_cdp_cmd cross-browser?

No. It is a Chromium CDP interface. I keep it in Chrome-specific helpers and use WebDriver or supported WebDriver BiDi APIs for cross-browser requirements.

Q: How do you avoid event leakage in pytest?

I use a function-scoped driver fixture, drain only its logs, store events on an instance owned by the test, and always quit the browser. I avoid module-global buffers and remove any modified condition by starting a clean session.

Q: When is a direct API test better than Selenium network inspection?

It is better when the requirement concerns many payload fields, schemas, error combinations, or fast data-driven coverage. DevTools is valuable when the browser's actual integration, request, or recovery behavior is the subject.

Common Mistakes

  • Setting goog:loggingPrefs after the driver is created. Session capabilities cannot be retrofitted that way.
  • Treating performance entries as flat JSON. The CDP method is inside the decoded outer entry's message object.
  • Reading the performance log in two helpers and expecting both to see the same records. Reads drain the current buffer.
  • Navigating before enabling Network or preparing blocked URLs and headers. Early traffic then uses the original state.
  • Matching only a broad URL fragment when background polling calls the same route. Include method, resource type, or another identifier.
  • Logging raw headers and bodies. Redact secrets and minimize retained evidence.
  • Using CDP JavaScript evaluation for ordinary element interaction. Prefer WebDriver locators and actions.
  • Assuming throttling values reproduce a real mobile network. They are controlled browser inputs, not physical-network proof.

Conclusion

A maintainable selenium DevTools in Selenium 4 python workflow separates commands, events, and visible assertions. Use execute_cdp_cmd for focused Chromium controls, enable performance logging before session creation for buffered events, filter by a precise request, and wait with a bounded deadline.

Begin with one browser-integration risk, such as detecting a failed catalog request and asserting the fallback UI. Put the CDP details in a helper, keep sensitive data out of artifacts, and choose WebDriver BiDi when it provides the same capability across the browsers you support.

Interview Questions and Answers

What does execute_cdp_cmd do in Selenium Python?

It sends a named Chrome DevTools Protocol command with a dictionary of parameters and returns the decoded result. It is useful for Chromium capabilities not covered by standard WebDriver. I isolate calls because CDP is browser-specific and evolves with Chromium.

How do Chrome performance logs expose CDP events?

ChromeDriver writes supported DevTools events into a buffered performance log when the capability is enabled at session creation. Each entry contains a nested JSON message with method and params. Reading get_log drains the available entries.

How would you wait for one API response in a Python UI test?

I would enable Network before the trigger, poll decoded performance messages until a monotonic deadline, and match a precise URL plus method or request characteristic. I would raise a descriptive timeout and separately assert the visible UI result.

How do you retrieve and decode a CDP response body?

I take requestId from Network.responseReceived and send Network.getResponseBody. I inspect base64Encoded, decode when necessary, and parse the expected content type. I retrieve only selected bodies and redact diagnostics.

What test isolation issues can CDP create?

Blocked URLs, headers, throttling, cookies, cache, and drained log buffers all belong to session state. I give each parallel test its own driver and event storage, use function-scoped fixtures, and start a clean session for unrelated conditions.

When would you choose WebDriver BiDi over direct CDP?

I choose BiDi when Selenium and the target browsers support the required capability reliably because it is standards-oriented. I keep direct CDP for Chromium-only depth or current gaps. The decision is per capability, not based only on novelty.

Why should page objects not parse performance logs?

Page objects should represent user-facing screens and interactions. Protocol parsing is infrastructure with browser-specific lifecycle and redaction concerns. Keeping it in a probe makes page objects portable and protocol upgrades localized.

Frequently Asked Questions

How do I use Chrome DevTools commands in Selenium Python?

Call driver.execute_cdp_cmd with a CDP method name and arguments dictionary. Enable the related domain first when required, and execute setup commands before navigation or the action under test.

How do I capture network logs in Selenium Python?

Set goog:loggingPrefs performance to ALL on ChromeOptions before creating the driver. Read driver.get_log('performance'), decode each entry's nested JSON message, and filter for Network methods.

Can Selenium Python read an API response body from Chrome?

Yes. Capture the requestId from Network.responseReceived and call Network.getResponseBody through execute_cdp_cmd. Retrieve it promptly and handle the base64Encoded flag.

Why is the Selenium performance log empty?

The performance logging capability may not have been set before driver creation, the Network domain may not have been enabled, or another helper may have drained the buffer. Remote providers can also restrict this log type.

Does execute_cdp_cmd work in Firefox?

No, it sends Chromium CDP commands. Use standard WebDriver or a supported WebDriver BiDi capability for Firefox and cross-browser automation.

How can I block requests with Selenium Python?

Enable Network, then call Network.setBlockedURLs with a urls list before loading the page. Use the technique to test a named fallback condition and start a fresh session afterward.

Should Selenium UI tests validate complete API payloads?

Usually not. Inspect only fields needed to prove browser integration, then use a direct API suite for detailed schemas, negative cases, and data-driven coverage.

Related Guides