Add an example for how to use chromedp.NewRemoteAllocator. #19
Conversation
|
Had to account for https://bugs.chromium.org/p/chromium/issues/detail?id=813540. This should now be the final version. |
remote/main.go
Outdated
| // | ||
| // Chromium will return the error "Host header is specified and is not an IP address or localhost" if the | ||
| // DevTools are connected to via a hostname other than localhost. | ||
| func replaceResolvedHostnameIfNeeded(url *url.URL) error { |
There was a problem hiding this comment.
chromedp already does this; remove this function
remote/main.go
Outdated
| } | ||
|
|
||
| resp, err := http.Get(chromiumRemoteUrl.String()) | ||
|
|
There was a problem hiding this comment.
remove all the unnecessary spacing, for example before the error checks
remote/main.go
Outdated
| log.Fatal(err) | ||
| } | ||
|
|
||
| ctxt, cancel := chromedp.NewRemoteAllocator(context.Background(), wsUrl) |
There was a problem hiding this comment.
we generally use actx for allocator contexts, and ctx for regular contexts.
it would also be better to create both contexts in the same block.
remote/main.go
Outdated
| var body string | ||
| if err := chromedp.Run(ctxt, | ||
| chromedp.Navigate(url), | ||
| chromedp.Sleep(time.Second), // wait for things to settle |
There was a problem hiding this comment.
don't use a sleep. you can check existing examples like text/ for reference.
remote/main.go
Outdated
| } | ||
|
|
||
| for _, target := range targets { | ||
| if target.Url == "about:blank" { // we assume this is the default page and that it is always around |
There was a problem hiding this comment.
sorry, but this is wrong - RemoteAllocator should be used with the browser's websocket url, not a page's.
remote/main.go
Outdated
| return "", err | ||
| } | ||
|
|
||
| bytes, err := ioutil.ReadAll(resp.Body) |
There was a problem hiding this comment.
you can avoid this ReadAll via json.NewDecoder(resp.Body).Decode(...)
There was a problem hiding this comment.
Removed because no longer needed.
|
Thanks for the review and sorry for the mess. I hadn't realized that chrome prints the devtools endpoint when it starts 🤦♂️ FWIW In my project I start a crawler and Glad I can offer you a more canonical example now. |
From the godocs of
chromedp.NewRemoteAllocatorI was not able to work out how it is supposed to be used. I ended up doing it like this. If this is how it is supposed to be used, it might make a good addition for the examples repo.