Glossary

Web Scraping & API Glossary

Comprehensive glossary of web scraping, crawling, and API terms. Learn the essential concepts and terminology used in web data extraction.

How can DevTools windows be treated as a page in Puppeteer?

Puppeteer

DevTools windows can be treated as regular pages by enabling the handleDevToolsAsPage option when launching or connectin...

How can I expose BackendNodeId in the a11y snapshot?

Puppeteer

BackendNodeId is exposed in the a11y snapshot. Each node in the snapshot includes a backendNodeId that lets you map acce...

How can I get detailed initiator data from CDP in Puppeteer?

Puppeteer

Use the capability to retrieve detailed initiator data from CDP when available, and filter out goog: data from events by...

How can I open a page in a tab or a window using Puppeteer?

Puppeteer

This feature allows opening a page in a tab or a window. newPage() can now be called with window options to choose where...

How can landmarks improve accessibility testing in Puppeteer?

Puppeteer

Overview Landmarks such as header, nav, main, aside, and footer provide semantic regions that assist screen readers and ...

How to configure CDP message ID generator in Puppeteer?

Puppeteer

The CDP message ID generator can be configured by passing a custom idGenerator to the Connection constructor. This enabl...

How to disable xdg-open popup in Puppeteer?

Puppeteer

To stop the xdg-open popup in Puppeteer, configure a Chrome policy URLAllowlist and use a Chrome binary that reads that ...

How to expose the Connection from CdpBrowserContext in Puppeteer

Puppeteer

Summary This change adds a public getter to CdpBrowserContext to expose the internal Connection object. It returns the p...

How to expose the url property for links in Puppeteer

Puppeteer

How to expose the url property for links If you need the full URL of a link in Puppeteer, use the url property that was ...

How to fix duplicate response headers in Puppeteer

Puppeteer

Summary Duplicate header values should normally be merged into a single header value separated by a comma and a space. T...

How to fix Fetch.enable wasn't found error for workers in Puppeteer

Puppeteer

Fetch.enable wasn't found is raised when trying to enable the Fetch domain for a worker. The fix is to ignore this error...

How to fix Puppeteer ExtensionTransport tasks and session management

Puppeteer

Puppeteer now dispatches each CDP message in its own JavaScript task by scheduling dispatch with setTimeout. This ensure...

How to open DevTools for a Page in Puppeteer?

Puppeteer

To open DevTools for a page in Puppeteer, use the new Page.openDevTools() method. It calls the DevTools interface for th...

How can I reload a Puppeteer page while ignoring the cache?

Puppeteer

Use the ignoreCache option with Page.reload to reload while ignoring the browser cache. ``js await page.reload({ ignoreC...

How to use Puppeteer to override user agent with emulation setUserAgentOverride instead of network interception?

Puppeteer

Solution Use the Emulation.setUserAgentOverride command via a CDP session to override the user agent instead of relying ...

What does it mean that HTTPRequest.postData is deprecated in Puppeteer?

Puppeteer

The deprecation note indicates that the HTTPRequest.postData API is deprecated in Puppeteer. This means you should avoid...

What fixes Puppeteer not waiting for all targets when connecting?

Puppeteer

Fixes Puppeteer not waiting for all targets when connecting by only awaiting child targets for tab targets. When connect...

What is the correct event to create a Response in Puppeteer WebDriver?

Puppeteer

To align with the protocol behavior, create the Response when the responseStarted event fires, rather than after the res...

What is the correct type for the pageerror event in Puppeteer?

Puppeteer

Summary The pageerror event may emit not only Error objects but also values of unknown type. Treat the payload as unknow...

What is the difference between browser.close and browser.disconnect

Puppeteer

browser.close() and browser.disconnect() both end your current control flow, but they affect the browser lifecycle diffe...

What is the reason for removing the test server from release-please in Puppeteer

Puppeteer

The test server was removed from the release-please workflow to simplify the release process and remove an unnecessary e...

What target should I set for TypeScript to build Puppeteer types with tsc?

Puppeteer

If you run into TS18028 private identifiers errors when compiling Puppeteer types with TypeScript, set the TypeScript ta...

What TypeScript target should I use to compile Puppeteer-core types?

Puppeteer

To fix the TS18028 error, set the TypeScript target to ES2015 or higher. The error occurs because private identifiers (#...

Why are headers not updated when using Puppeteer page.setRequestInterception(true) on Firefox?

Puppeteer

Firefox currently mutates the headers object returned by request.headers() in a way that does not reflect in the respons...

Why does HttpRequest.headers() not allow mutating data in Puppeteer?

Puppeteer

This was fixed to prevent accidental mutations of the underlying headers. HttpRequest.headers() no longer allows mutatin...

Why is page.goto never awaited for Firefox addons pages in Puppeteer?

Puppeteer

Summary Firefox addon pages navigated via moz-extension:// are treated as webextension contexts. Puppeteer currently doe...