What can i do when i have operation TimeOut ?

[karimWorldSpace]

Hi community,

I'm working on a personal project where I need to retrieve the title and HTML content of a webpage (a simple task).

Sometimes, the URL I visit has protections like cookies, but the HTML content is already fully loaded, so I don’t actually care about the cookie. All the information I need is in the HTML.

Here’s my problem:

1. When I try to evaluate the title of the page, I often get a timeout error.
2. To handle this, I retry the process after the first failure, but I can’t do more than that ?.
3. What’s confusing is that if I manually check the title in the browser’s console, it works perfectly. However, when I try to retrieve the title programmatically in PHP, it doesn’t work.
4. Does anyone know why this might happen or how I can fix it?

Thanks for your help!

Here is my code

` $urls = $urlScrapedByKeyWordRepository->findBy(['isUsedForGeneration' => false]);
shuffle($urls);
$urls = array_slice($urls, 0, 2);

    if ($urls) {
        /** @var UrlScrapedByKeyword[] $urls */
        foreach ($urls as $key => $url) {
            $urlScrapped = ltrim($url->getUrl(), './');
           // $urlScrapped =  $urlScrapped;

            $browser = $this->createBrowser();
            $page = $browser->createPage();
            $html = false;

            try {
                $page->navigate($urlScrapped, ['strict'])->waitForNavigation(Page::INTERACTIVE_TIME, 6000);
                $page->evaluate("console.log('document.title')");

               // -> here where my code crash so i catch the error below
                $pageTitle = $page->evaluate('document.title')->waitForResponse()->getReturnValue();

                if ($pageTitle == 'Before you continue')
                {
                    $this->AcceptGoogleCookies($page);
                    $pageTitle = $page->evaluate('document.title')->waitForResponse()->getReturnValue();
                } 

                echo($pageTitle.' from normal way');
                $pageContent = $page->getHtml(2500);
                sleep(1);
                if ($pageContent) echo('content OK');

                if ($pageTitle == 'Before you continue') $this->AcceptGoogleCookies($page);
            } catch (OperationTimedOut $e) {
                // Here in the console of the navigator, i can see this operation work correctly
               $page->evaluate("console.log(document.title)");

                // !!----catch the error and retry to evaluate title but again crash ----!!
                $pageTitle = $page->evaluate('document.title')->getReturnValue(); 

                if ($pageTitle == 'Before your continue') $this->AcceptGoogleCookies($page);

                echo $pageTitle.' from error';
                $pageContent = $page->getHtml(2500);
                sleep(1);
                if ($pageContent) echo('content OK from error');
            } catch (NavigationExpired $e) {
                echo "Erreur de NavigationExpired lors de l'évaluation du titre : $pageTitle</br>";
            }
        }`

[divinity76]

There is a bug in chrome-php/chrome (or maybe chrome-php/wrench)
That can cause anything to timeout, occasionally.

Even stupidly trivial stuff like
Page->evaluate("document.querySelectorAll("video")?.[0]?.src?.trim()")->getReturnValue()

Can timeout. That's an instant operation, it should take no time at all, but sometimes the return value never arrive.

I have been unable to find the source. Last time I went looking for it, I found chrome-php/wrench#24 but it didn't help.

I use this workaround:

private function evaluate(string $script): mixed
    {
        // there are random timeouts in the page evaluation.. probably an issue in the chrome-php/chrome library...
        try {
            return $this->page->evaluate($script)->getReturnValue();
        } catch (\HeadlessChromium\Exception\OperationTimedOut $e) {
            try {
                return $this->page->evaluate($script)->getReturnValue();
            } catch (\HeadlessChromium\Exception\OperationTimedOut $e) {
                return $this->page->evaluate($script)->getReturnValue();
            }
        }
    }

I would like to hear from other people who experience it 🤔

• it seems to happen most often on Windows, but it also happens on Linux.
• happens more often on laptops running on battery, less often on computers plugged in to the socket.

[prime-5506]

Thanks for sharing the issue and the code, . You're facing OperationTimedOut exceptions when trying to evaluate document.title with a headless browser (likely based on Chromium), while it works manually in the browser. This typically means there's a mismatch between what your script thinks the page state is vs. what the browser is actually doing.

Here’s a breakdown of why this might happen and how to fix or work around it:

---

✅ Understanding the Problem

1. Page not ready when evaluate() is called: Even after waitForNavigation, dynamic pages (especially behind JavaScript or cookie walls like Google’s) may delay setting document.title.

2. Protection mechanisms (like Google's):

Pages like Before you continue are often JavaScript-heavy consent pages.

These interfere with your browser automation because the actual content is hidden behind a consent interaction.

3. Incorrect use of waitForResponse():

You're calling evaluate()->waitForResponse()->getReturnValue() — but evaluate() often returns immediately unless you're awaiting a JS promise.

---

🔧 Suggestions for Fixing

1. Replace waitForNavigation(Page::INTERACTIVE_TIME, 6000)

This might be firing too early. Try:

$page->navigate($urlScrapped)->waitForLoad(Page::LOAD);

This waits until the page load event has fully fired. If the page is JS-heavy, INTERACTIVE_TIME might be too early.

---

2. Wait for document.title dynamically

Instead of immediately calling evaluate('document.title'), use a loop to poll the title until it changes from 'Before you continue' or is not empty.

$maxTries = 5;
$waitedTitle = null;
for ($i = 0; $i < $maxTries; $i++) {
try {
$waitedTitle = $page->evaluate('document.title')->getReturnValue();
if (!empty($waitedTitle) && $waitedTitle !== 'Before you continue') {
break;
}
} catch (OperationTimedOut $e) {
// You could optionally retry here
}
sleep(1); // wait between retries
}

---

3. Use waitForFunction() to block until document.title is correct

Some headless libraries support this:

$page->waitForFunction('document.title && document.title !== "Before you continue"', 5000);

If your library supports something like waitForFunction(), it’s more reliable than catching timeouts.

---

4. Fallback evaluation in catch block

In your catch (OperationTimedOut) block, you retry with:

$pageTitle = $page->evaluate('document.title')->getReturnValue();

But this may also fail. Wrap this retry in its own try {} block to catch nested errors.

---

5. Log full HTML for debugging

To inspect what the browser sees when it fails:

file_put_contents("debug.html", $page->getHtml());

Then open debug.html in your own browser — this helps diagnose if you're seeing a consent page, captcha, or blank.

---

🧪 Optional Debug Tip

Add this to log page console output if supported:

$browser->on('console', function ($message) {
echo "Console: " . $message->text() . "\n";
});

---

✅ Summary of Fixes

Use waitForLoad() instead of waitForNavigation(INTERACTIVE_TIME)

Poll document.title with retries/sleep

Wrap retry logic inside its own try block

Log page HTML to diagnose consent/captcha walls

If possible, implement waitForFunction('document.title !== "Before you continue"')

Your welcome..🛡😊

[hopcv58]

Is there any update on this? I faced this problem for a long time, and started to think about returning to puppeteer for consistency

[divinity76]

I suggest looking very closely at the networking code. I think messages from the browser to php get lost in transmission, rarely, causing these timeouts.

[divinity76]

Going by the reproduce code in #560
Where a usleep(1) triggers it, there's probably a race condition involved.

(Btw #560 is incorrectly closed by the timeout bot for inactivity. It's not a solved issue.)

[roben]

I also have these issues running Linux. I try to crawl a web page with an infinite loading content list. My injected javacript does it's thing on the web page and php-chrome waits in a loop for the result variable to be populated. After about 2 minutes (randomly more or less), the timeout occurs. It is not recoverable by trying again, I have to reconnect to the browser.

Strangely enough, I initially forgot a sleep in this loop so it checked reeaally often and quickly, so I thought that to be the issue. But even after adding a sleep(1), it still times out after roughly the same duration. So it also seems to depend on the duration of the connection and not the check frequency (alone).

[roben]

I fixed it for now by filtering / chunking the result set to produce smaller responses. Not very satisfying, though...