Web Scraping & API Glossary

When you call browser.newPage() on a browser connected via connectOverCDP(), Playwright throws "Cannot read properties o...

Playwright's getByRole() locator uses the browser's accessibility tree to find elements by their semantic role. When you...

This error means a JavaScript evaluation—like page.evaluate() or page.evaluateHandle()—was interrupted because the page ...

The message "execution context was destroyed, most likely because of a navigation" means Playwright started evaluating J...

"Frame was detached" means the iframe you were interacting with was removed from the DOM or replaced while your script h...

page.goto: net::ERRABORTED means the browser started a navigation but the request was cancelled before a response was re...

net::ERRCONNECTIONREFUSED means the browser attempted a TCP connection to the target host and port, and the operating sy...

net::ERRINTERNETDISCONNECTED means the browser has no active network path to reach the target — either the machine itsel...

net::ERRINVALIDAUTH_CREDENTIALS is thrown when a server challenges the browser with HTTP Basic Authentication and the cr...

When Playwright's request fixture (or APIRequestContext) is used alongside page in the same test, their cookies and sess...

This happens when the authentication setup step saves session state to a file, but the test project that runs your tests...

Exact text assertions break on whitespace normalization, added punctuation, translated strings, or copy edits that don't...

Clicking a button that is disabled or in a loading state produces no action — the click goes through Playwright's action...

Cannot read properties of undefined in a Playwright test is a JavaScript runtime error, not a browser or network error. ...

npx playwright install --with-deps chromium fails on Ubuntu 26.04 with something like Unsupported platform: ubuntu26.04-...

Playwright browser launches fail in CI when the container image is missing the Linux shared libraries that Chromium requ...

This timeout fires when the element exists in the DOM but Playwright cannot confirm it is visible, enabled, and not obsc...

CSS class selectors in Playwright tests couple test code to implementation details of the component's styling. When a de...

This error means a test() call was executed outside of a spec file — most commonly because a helper module or config fil...

Download tests hang when the event listener is attached after the download has already started. The browser fires the do...

"Element is not attached to the DOM" surfaces when a Playwright action targets an element that was in the DOM when queri...

net::ERRNAMENOT_RESOLVED means the browser's DNS resolver could not find an IP address for the hostname in the URL. In P...

The "Executable doesn't exist" error means the @playwright/test package is installed but the corresponding browser binar...

"Execution context was destroyed" appears when Playwright has a JavaScript handle or evaluation in flight and the page n...

File upload failures in Playwright almost always come down to three root causes: targeting the wrong element (a styled <...

Geolocation and browser permissions fail to apply in Playwright tests when they are set at the wrong stage of the browse...

Headless-only failures are among the most confusing Playwright issues because the test is syntactically correct and pass...

Playwright on Windows fails to launch headlessshell.exe with an error about missing DLL files — often vcruntime140.dll, ...

Playwright's role-based locators (getByRole, getByLabel) match elements using the computed accessible name, which may co...

locator.isVisible() is a point-in-time query that returns the current visibility state immediately without any retrying....

Leaked state between tests happens when one test leaves side effects that affect subsequent tests in the same worker: co...

locator.count() returns a number immediately — it does not assert, does not retry, and does not fail if the count is wro...

Mobile emulation in Playwright fails to trigger the mobile layout when viewport and device settings are applied after co...

networkidle as a waitUntil condition tells Playwright to wait until there are no network connections for at least 500ms....

"No frame for selector" and similar iframe-related errors happen when test code tries to locate elements inside an ifram...

Index-based locators with .nth() are fragile because they rely on the DOM rendering elements in a specific, stable order...

getByText() matches any element in the document that contains the specified text, including headings, list items, table ...

page.waitForTimeout() (a hard sleep) is the most common anti-pattern in Playwright test suites. It introduces fixed dela...

Parallel Playwright workers interact with a shared database simultaneously, causing data collisions when tests assume ex...

Popup and new-tab tests become flaky when page.waitForEvent('popup') is called after the action that opens the popup. Ju...

A "Protocol error: invalid argument" originates from the Chrome DevTools Protocol (CDP) layer and means Playwright sent ...

Random browser crashes in CI — where tests pass locally but fail unpredictably in CI runners — are almost always resourc...

Playwright retries are designed to improve stability for genuinely intermittent conditions — network latency, slow CI ma...

Route mocking fails silently when the URL pattern doesn't match what the browser actually requests, or when page.route()...

Shadow DOM components encapsulate their internal DOM in a shadow root that is intentionally hidden from the main documen...

Playwright's --shard=N/M splits test files across M runners, but the distribution is based on file count — not test coun...

A strict mode violation in Playwright fires when a locator resolves to more than one element at action time. The error m...

When this error appears alongside a crash — rather than an orderly context.close() call — it means the browser process e...

TimeoutError is thrown when Playwright cannot complete an action or find an actionable element within the configured tim...

waitForResponse misses requests for exactly the same reason waitForEvent('download') and waitForEvent('popup') do — the ...

Weak URL assertions allow tests to pass on the wrong page, producing false positives that hide navigation bugs. expect(p...

Tests that depend on WebSocket connections or Server-Sent Events (SSE) are inherently more sensitive to CI network insta...

Memory-related worker crashes in Playwright parallel runs manifest as browser processes being killed mid-test, producing...

A Playwright strict mode violation means the locator you used resolved to more than one element, but the operation requi...

The error "Target page, context or browser has been closed" surfaces when Playwright tries to interact with a page, fram...

Playwright runs headless by default — the browser launches without a visible UI, which is faster and what CI environment...

npm package '@playwright/mcp@latest' is not installed in the npx cache shows up when an MCP client (Claude Desktop, Clau...

-32603 is the generic JSON-RPC "Internal error" code, and when the Playwright MCP server returns it with the message "ta...

DevTools windows can be treated as regular pages by enabling the handleDevToolsAsPage option when launching or connectin...

BackendNodeId is exposed in the a11y snapshot. Each node in the snapshot includes a backendNodeId that lets you map acce...

Use the capability to retrieve detailed initiator data from CDP when available, and filter out goog: data from events by...

This feature allows opening a page in a tab or a window. newPage() can now be called with window options to choose where...

Overview Landmarks such as header, nav, main, aside, and footer provide semantic regions that assist screen readers and ...

Answer To fix Puppeteer not finding the Chrome binary on Windows, make sure the detector checks the common install locat...

When you want to bypass, not fix net::ERRCERTDATE_INVALID occurs when a TLS certificate has expired or its validity peri...

The CDP message ID generator can be configured by passing a custom idGenerator to the Connection constructor. This enabl...

To stop the xdg-open popup in Puppeteer, configure a Chrome policy URLAllowlist and use a Chrome binary that reads that ...

Summary This change adds a public getter to CdpBrowserContext to expose the internal Connection object. It returns the p...

How to expose the url property for links If you need the full URL of a link in Puppeteer, use the url property that was ...

Summary Duplicate header values should normally be merged into a single header value separated by a comma and a space. T...

Fetch.enable wasn't found is raised when trying to enable the Fetch domain for a worker. The fix is to ignore this error...

Why this happens net::ERRCERTDATE_INVALID means the TLS certificate on the target site has expired or its validity perio...

net::ERRCERTSYMANTEC_LEGACY means your browser (Chrome, Edge, or any Chromium-based browser—including Puppeteer and Play...

Puppeteer now dispatches each CDP message in its own JavaScript task by scheduling dispatch with setTimeout. This ensure...

To open DevTools for a page in Puppeteer, use the new Page.openDevTools() method. It calls the DevTools interface for th...

Use the ignoreCache option with Page.reload to reload while ignoring the browser cache. ``js await page.reload({ ignoreC...

Solution Use the Emulation.setUserAgentOverride command via a CDP session to override the user agent instead of relying ...

The deprecation note indicates that the HTTPRequest.postData API is deprecated in Puppeteer. This means you should avoid...

Fixes Puppeteer not waiting for all targets when connecting by only awaiting child targets for tab targets. When connect...

To align with the protocol behavior, create the Response when the responseStarted event fires, rather than after the res...

Summary The pageerror event may emit not only Error objects but also values of unknown type. Treat the payload as unknow...

browser.close() and browser.disconnect() both end your current control flow, but they affect the browser lifecycle diffe...

The test server was removed from the release-please workflow to simplify the release process and remove an unnecessary e...

If you run into TS18028 private identifiers errors when compiling Puppeteer types with TypeScript, set the TypeScript ta...

To fix the TS18028 error, set the TypeScript target to ES2015 or higher. The error occurs because private identifiers (#...

Summary The startup hang in headful mode on Ubuntu was caused by the Firefox Backup Service during startup. A practical ...

Firefox currently mutates the headers object returned by request.headers() in a way that does not reflect in the respons...

This was fixed to prevent accidental mutations of the underlying headers. HttpRequest.headers() no longer allows mutatin...

Summary Firefox addon pages navigated via moz-extension:// are treated as webextension contexts. Puppeteer currently doe...

Blocking at the scraper level is usually triggered by the shape of individual requests, not overall traffic volume — a m...

Scraped data fails silently. A selector that grabs the wrong element, a currency symbol left in a number field, or a dat...

Pagination is where scrapers silently lose data: a job that "works" but only returns page one, or a loop that never term...

On a JavaScript-heavy site, the HTML your scraper fetches with a plain request and the DOM a visitor actually sees are d...

Crawling answers "what pages exist." Scraping answers "what's in them." A crawler walks links to build a URL list; a scr...

Scraping itself — an HTTP request to a public URL — isn't illegal. What creates legal exposure is what you do with the d...

The right scraping tool depends on one question: does the data you need exist in the raw HTML, or does it only appear af...

Ethical scraping is about what happens to the data after extraction, not just how politely you fetch it. A scraper can r...

A 429 is the server telling your scraper "too many requests, slow down" — not "you're banned." It's a rate-limit signal,...

Answer The best format depends on how you plan to use the data. CSV is simple and works well for tabular data and quick ...

Web scraping is the process of pulling specific fields out of a web page — a price, a title, a review count — and turnin...

Your first crawl captures a snapshot of a documentation site, product guide, or internal wiki — but source content chang...

Crawling and scraping get conflated because most real pipelines do both — but they optimize for different things, and tr...

Crawl frequency is a tradeoff between data freshness and load — on the target site and on your own infrastructure. A fix...

Blocking at the crawl level is usually a pattern problem, not a per-request problem: a crawler that hits hundreds of pag...

Answer Crawl a documentation site by scoping the crawl to the docs subtree with a URL filter, deciding between sitemap.x...

On a single-page app, the links that lead to other pages often don't exist in the initial HTML at all — they're generate...

Crawling's legal exposure lives mostly at the access-control layer: did you bypass something meant to keep you out. A cr...

Before you commit to building a crawler in-house, you hit a fork: write and maintain your own, or pay a service to do it...

Crawling tools split on how they handle scale and scheduling, not on whether they can fetch a page. A tool built for one...

A crawler's job is discovery, so what it collects per page is metadata about the page — not the specific fields a scrape...

Crawl budget is the number of pages you can realistically fetch from a site in a given window — bounded by the target's ...

robots.txt is a plain-text file at a site's root that declares which paths crawlers may or may not request, per user age...

Web crawling is discovery: starting from one or a few seed URLs, following links to find every other page on a site, and...