Adding extension causes extra unwanted blank page to open and extension's opened page becomes a 'ghost' page and is completely unreachable!

[ivarsj10s]

On JavaScript adding an extension which opens 'welcome' page such as MetaMask => extension opens 1 new page and you can close this page and the default page.

On Playwright Python adding an extension which opens 'welcome' page such as MetaMask => extension opens 1 extra blank page (2 blank pages / total) and a new 'extension' 'ghost' page which is unnaccessible from Playwright Python and you cannot close it.

I briefly mentioned this issue on #681 where the developer who got assigned fixed my main issue in the title but skimmed over this issue. The developer's given fix only fixed browser.close() raising errors => into => not raising errors, but not this issue. This issue is still 'at large' (i.e. especially of a criminal or dangerous animal) at liberty; escaped or not yet captured.)

PROBLEM: this bug causes an unaccessible 'extension' page to just hang there and I cannot close it using code. It's not inside browser.pages.

Illustrative image sets with descriptive captions:

1. Playwright JS
   https://imgur.com/a/X13eydu

2. Playwright Python
   https://imgur.com/a/mkINpWl

EDIT: adding source as by confusing from @pavelfeldman 's reply

import playwright
from playwright.sync_api import sync_playwright

EXT_1_DIR = "C:/Users/PC/Desktop/PW_PY/metamask_unpck"
USER_DIR = "C:/Users/PC/AppData/Local/Google/Chrome/User Data/Profile 2"

ARGS = [
    "--disable-extensions-except={}".format(EXT_1_DIR),
    "--load-extension={}".format(EXT_1_DIR)
]


with sync_playwright() as p:
    browser = p.chromium.launch_persistent_context(USER_DIR, headless=False, args=ARGS)
    page = browser.new_page()
    _ = input("Wait till Metamask loads? ")
    print("len_pages()", len(browser.pages))
    for pg in browser.pages:
        pg.close()

    _ = input("Close browser? ")
    browser.close()

EDIT: update to @pavelfeldman 's input.

new source

import playwright
from playwright.sync_api import sync_playwright

EXT_1_DIR = "C:/Users/PC/Desktop/PW_PY/metamask_unpck"
USER_DIR = "C:/Users/PC/AppData/Local/Google/Chrome/User Data/Profile 2"

ARGS = [
    "--disable-extensions-except={}".format(EXT_1_DIR),
    "--load-extension={}".format(EXT_1_DIR)
]


with sync_playwright() as p:
	# 1) launch MetaMask
    browser = p.chromium.launch_persistent_context(USER_DIR, headless=False, args=ARGS)
    _ = input("Wait till Metamask loads? ")
   
   	# 2) print len() pages and bg_pagesshould be 2, but is 1 (bug)
    print("len_pages()", len(browser.pages))
    print("len_background_pages()", len(browser.background_pages))

    _ = input("Close pages? ")
    # 2.1) pages don't get closed, only blank page is detected and closed, 2nd page does not get detected,
    # or, closed
    for pg in browser.pages:
        pg.close()


    # 3) close the browser, old bug error on closing was fixed by your dev - all good- thanks him
    _ = input("Close browser? ")
    browser.close()

As we see, the page doesn't fall into browser.background_pages it's just not there. Also, note, for Playwright JS it falls into normal pages.

Illustrative screenshot:
https://imgur.com/a/Yh1BIQr

[pavelfeldman]

I don't think I get it. Your Python script has an extra page = browser.new_page() that creates the page that you refer to the one extra page. So it looks like working as intended to me.

As for the bg page, it lands in a separate collection: https://playwright.dev/python/docs/next/api/class-browsercontext#browser_contextbackground_pages

[ivarsj10s]

I don't think I get it. Your Python script has an extra page = browser.new_page() that creates the page that you refer to the one extra page. So it looks like working as intended to me.

As for the bg page, it lands in a separate collection: https://playwright.dev/docs/next/api/class-browsercontext#browsercontextbackgroundpages

It doesn't land in browser.background_pages as well. Also, note, it lands in regular pages for JS PlayWright. The new_page() my lousy coding - apologies on that; but it doesn't change the issue. The issue is still the same, new (opened by extension) page is not accessible in pages, and, as you mentioned to check - it's not in background_pages as well.

Good spot, the two tabs is my mistake. MetaMask tab being unreachable tho' - still the issue there. I'm pretty certain it's a bug. Unless you can provide me another way to access this page (that I don't know of) - then I could use to work around this issue.

[mxschmitt]

Probably you are affected by this bug: microsoft/playwright#2676

Something like that works for me, I've added the wait_for_event call to wait until we got a background page:

from playwright.sync_api import sync_playwright

# these need to be replaced to your ones; Metamask unpacked can be downloaded here
# https://github.com/MetaMask/metamask-extension/releases/tag/v9.5.0
EXT_1_DIR = "C:/Users/PC/Desktop/PW_PY/metamask_unpck"
USER_DIR = "C:/Users/PC/AppData/Local/Google/Chrome/User Data/Profile 2"

ARGS = [
    "--disable-extensions-except={}".format(EXT_1_DIR),
    "--load-extension={}".format(EXT_1_DIR),
]

with sync_playwright() as p:
    context = p.chromium.launch_persistent_context(USER_DIR, headless=False, args=ARGS)
    context.wait_for_event("backgroundpage")
    print("len_pages()", len(context.pages), len(context.background_pages))
    for pg in context.pages:
        pg.close()

    _ = input("Close browser? ")
    context.close()

[ivarsj10s]

@mxschmitt I tried your snippet, len works; but I'm unable to do anything on this background page. I cannot close it using context.background_pages[0].close() ; I also can't interact with it, say context.background_pages[0].click("button.button") will just timeouterror instead of clicking the button in the center.

also, the content() shows the old non-interpreted source

me being unable to close it using code is the biggest issue for me. context.background_pages[0].close() won't work, absolutely no effect and the broken tab keeps being visually open.

illustrative images:
https://imgur.com/a/msqEFw8

[mxschmitt]

Hi, I will get back to you, probably in a few days and probably add some tests from upstream around it to ensure this works. Sorry for the inconveniences!

[ivarsj10s]

Hi! Yeah that's fine. I'll monitor this thread and check with what you come up. Thanks!

[mxschmitt]

I have the assumption you don't want to interact with the background page, more you want to interact with the extension itself. I tried the following which worked for me:

with sync_playwright() as p:
    context = p.chromium.launch_persistent_context(USER_DIR, headless=False, args=ARGS)
    page = context.wait_for_event("page")
    print(page.title())
    page.click("text=Get Started")
    page.click("text=Import wallet")
    page.click("text=I Agree")
    context.close()

Let me know about your use-case, thanks!

[ivarsj10s]

@mxschmitt as per my exact use case; 1) I wanna log in to the extension's opened page, 2) then close it.

Your solution works, but I don't get it. Why does context.wait_for_event("page") make this page accessible in context.pages? I come from the old timers' Selenium background and there (as far as I know) wait for something just waits for something. Here this wait makes page accessible inside context.pages; it's not accessible in context.pages even if I wait much longer with say time.sleep(5);

What's the internal logic behind this behavior? Especially if it's not a bug; I just don't get how / why it works in this exact way?

[ivarsj10s]

@mxschmitt on your prev. question. I don't know the web /slash/ extension making internals good enough, I don't know in-an-actual-sense what a background page is. So apologies if my answer to your prev. question is a bit naive and not tech-savvy.

[mxschmitt]

(I'm no Chromium extension expert)

From my understanding background pages are non-user facing scripts which extensions can use to hook into the Chromium extension API etc. In your case, you want to automate the user-facing part of the extension, so this is a regular page. Coming back to the question why wait_for_event("page"). This is because the page in your extension get's created asynchronously and takes some time to initialize. You could also use a while loop and check e.g. every 100ms if len(context.pages) == 2 but we emit the pages event when new pages get created, so it's probably better. The reason because time.sleep(5) is not working is probably because it halts the whole execution and we internally use greenlet to schedule new incoming events which would add it to context.pages.

[ivarsj10s]

@mxschmitt thanks for your input; but while loop blocks i.e. at least there's no new pages added if I'm using a while loop to use as sleep.

My code.

def datetime_sleep(duration=1):
    start = datetime.datetime.now()
    delta = datetime.timedelta(seconds=duration)
    while True:
        new_delta = datetime.datetime.now() - start
        if new_delta > delta:
            break

and then I use this one instead of time.sleep and same effect. no new page is added; and I cannot access it using context.pages without using your 'the solution' you gave a couple posts up.

[mxschmitt]

Okay, sorry for the delay. The issue is that time.sleep blocks the whole thread and does not allow other asynchronous operations to process. To bypass that you should use page.wait_for_timeout(5000), I will add that to the known issues section.

[ivarsj10s]

Thanks; please leave this as 'triaging' for longer for now. I need more time given to me to practically try out the your adviced on using page.wait_for_timeout(<ms>) for the 'explicit wait' functionality. I just don't have the timeblock yet. When I reach a timeblock to go through this and through your reply on the other ticket, I'll try them out and comment

[mxschmitt]

Closing since the issue seemed stale as part of the triage process. Please create a new issue if the issue is still persistent.