Skip to content

Add dynamic content info #25419

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: production
Choose a base branch
from
Open

Add dynamic content info #25419

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
eddc615
add dynamic content info
simonabadoiu Sep 25, 2025
883dfb6
add dynamic content info to content endpoint
simonabadoiu Sep 25, 2025
b78e12b
add dynamic content info to all ednpoints
simonabadoiu Sep 25, 2025
75df855
add dynamic content info to puppeteer
simonabadoiu Sep 25, 2025
9034d65
remove dynamic content info to puppeteer
simonabadoiu Sep 25, 2025
569a6b8
add backticks
simonabadoiu Sep 25, 2025
2942e1f
add new paragraph
simonabadoiu Sep 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions src/content/docs/browser-rendering/rest-api/content-endpoint.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,8 @@ Many more options exist, like setting HTTP headers using `setExtraHTTPHeaders`,
file="setting-custom-user-agent"
product="browser-rendering"
/>

<Render
file="single-page-application"
product="browser-rendering"
/>
6 changes: 6 additions & 0 deletions src/content/docs/browser-rendering/rest-api/json-endpoint.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -354,3 +354,9 @@ In this example, Browser Rendering first calls Anthropic's Claude Sonnet 4 model
file="setting-custom-user-agent"
product="browser-rendering"
/>

<Render
file="single-page-application"
product="browser-rendering"
/>

6 changes: 6 additions & 0 deletions src/content/docs/browser-rendering/rest-api/links-endpoint.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,3 +233,9 @@ curl -X POST 'https://api.cloudflare.com/client/v4/accounts/<accountId>/browser-
file="setting-custom-user-agent"
product="browser-rendering"
/>

<Render
file="single-page-application"
product="browser-rendering"
/>

6 changes: 6 additions & 0 deletions src/content/docs/browser-rendering/rest-api/markdown-endpoint.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,9 @@ curl -X 'POST' 'https://api.cloudflare.com/client/v4/accounts/<accountId>/browse
1. **Content extraction:** Convert a blog post or article into Markdown format for storage or further processing.
2. **Static site generation:** Retrieve structured Markdown content for use in static site generators like Jekyll or Hugo.
3. **Automated summarization:** Extract key content from web pages while ignoring CSS, scripts, or unnecessary elements.

<Render
file="single-page-application"
product="browser-rendering"
/>

6 changes: 6 additions & 0 deletions src/content/docs/browser-rendering/rest-api/pdf-endpoint.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,9 @@ curl -X POST https://api.cloudflare.com/client/v4/accounts/<acccountID>/browser-
file="setting-custom-user-agent"
product="browser-rendering"
/>

<Render
file="single-page-application"
product="browser-rendering"
/>

6 changes: 6 additions & 0 deletions src/content/docs/browser-rendering/rest-api/scrape-endpoint.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,9 @@ Many more options exist, like setting HTTP credentials using `authenticate`, set
file="setting-custom-user-agent"
product="browser-rendering"
/>

<Render
file="single-page-application"
product="browser-rendering"
/>

6 changes: 6 additions & 0 deletions src/content/docs/browser-rendering/rest-api/screenshot-endpoint.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,3 +164,9 @@ Many more options exist, like setting HTTP credentials using `authenticate`, set
file="setting-custom-user-agent"
product="browser-rendering"
/>

<Render
file="single-page-application"
product="browser-rendering"
/>

5 changes: 5 additions & 0 deletions src/content/docs/browser-rendering/rest-api/snapshot.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,3 +108,8 @@ curl -X POST 'https://api.cloudflare.com/client/v4/accounts/<accountId>/browser-
/>

Many more options exist, like setting HTTP credentials using `authenticate`, setting `cookies`, and using `gotoOptions` to control page load behaviour - check the endpoint [reference](/api/resources/browser_rendering/subresources/snapshot/) for all available parameters.

<Render
file="single-page-application"
product="browser-rendering"
/>
11 changes: 11 additions & 0 deletions src/content/partials/browser-rendering/single-page-application.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
### Single Page Applications (SPAs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Single Page Applications (SPAs)
## Scrape a Single Page Applications (SPAs)


When scraping a Single Page Application (SPA) with dynamic content, you must ensure the page has fully loaded. To do this, you have two main options:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When scraping a Single Page Application (SPA) with dynamic content, you must ensure the page has fully loaded. To do this, you have two main options:
To scrape a Single Page Application (SPA) with dynamic content, make sure the page has fully loaded using one of these methods:


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Recommended: Use `waitForSelector` to wait for a specific element to appear on the page.
- Use `goToOptions` with `networkidle0` or `networkidle2`. Between the `networkidle` options, `networkidle2` is more efficient because it waits until there are less than two ongoing network connections, whereas `networkidle0` waits until all network connections are idle and all resources (including asynchronous JavaScript) have loaded.

:::note
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:::note

Use `waitForSelector` to wait for a specific element to appear on the page. This is often the most reliable and efficient method.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use `waitForSelector` to wait for a specific element to appear on the page. This is often the most reliable and efficient method.


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

Use `goToOptions` with `"networkidle0"` or `"networkidle2"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use `goToOptions` with `"networkidle0"` or `"networkidle2"`

- `"networkidle0"` waits for all network connections to be idle, meaning all resources (including asynchronous JavaScript) have been loaded
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `"networkidle0"` waits for all network connections to be idle, meaning all resources (including asynchronous JavaScript) have been loaded

- `"networkidle2"` is a more efficient alternative that waits until there are only two or fewer ongoing network connections
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `"networkidle2"` is a more efficient alternative that waits until there are only two or fewer ongoing network connections

:::
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:::

Loading