Fix Playwright flake

[david-crespo]

Did a few things, and together they seem to have worked. There are no more reports of flaky tests in the Playwright run log.

Mostly the flake I was seeing was around row actions menu items being unstable (popping in and out of visibility) or being detached from the DOM. When I tested manually I found that there were several reasons the menu could just close for no reason while you're trying to click on it.

• Use zustand for toast state instead of React context to reduce unnecessary renders
  • This was because a toast closing would cause a render that would cause the menu to close
• Don't poll instances, instance, and instance disks on a 5s interval — these would cause the actions menu to close too
• Get rid of the actionTimeout: 10000 in the Playwright config. This applied to stuff like clicks, not really sure why I put there to begin with. This one probably doesn't matter, though, because nothing should really take that long.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
console-ui-storybook	✅ Ready (Inspect)	Visit Preview	Jul 27, 2022 at 3:31AM (UTC)

[david-crespo]

The tests pass but are still flaking a bit. I really don't get what's going on.

Running 28 tests using 2 workers, shard 2 of 2
°················×····°···×·±×±

  1) [webkit] › pages/__tests__/instance/networking.e2e.ts:7:1 › Instance networking tab ===========

    Timeout of 30000ms exceeded.

    page.click: Target closed
    =========================== logs ===========================
    waiting for selector "role=menuitem[name="Edit"]"
      selector resolved to visible <div tabindex="-1" role="menuitem" data-valuetext="Ed…>Edit</div>
    attempting click action
      waiting for element to be visible, enabled and stable
    element was detached from the DOM, retrying
    waiting for selector "role=menuitem[name="Edit"]"
    ============================================================

      58 |     .locator('role=button[name="Row actions"]')
      59 |     .click()
    > 60 |   await page.click('role=menuitem[name="Edit"]')
         |              ^
      61 |   await page.fill('role=textbox[name="Name"]', 'nic-3')
      62 |   await page.click('role=button[name="Save changes"]')
      63 |   await expectNotVisible(page, ['role=cell[name="nic-2"]'])

        at /home/runner/work/console/console/app/pages/__tests__/instance/networking.e2e.ts:60:14

    Pending operations:
      - page.click at app/pages/__tests__/instance/networking.e2e.ts:60:14


    Retry #1 ---------------------------------------------------------------------------------------

    Timeout of 30000ms exceeded.

    page.click: Target closed
    =========================== logs ===========================
    waiting for selector "role=menuitem[name="Edit"]"
      selector resolved to visible <div tabindex="-1" role="menuitem" data-valuetext="Ed…>Edit</div>
    attempting click action
      waiting for element to be visible, enabled and stable
    element was detached from the DOM, retrying
    waiting for selector "role=menuitem[name="Edit"]"
    ============================================================

      58 |     .locator('role=button[name="Row actions"]')
      59 |     .click()
    > 60 |   await page.click('role=menuitem[name="Edit"]')
         |              ^
      61 |   await page.fill('role=textbox[name="Name"]', 'nic-3')
      62 |   await page.click('role=button[name="Save changes"]')
      63 |   await expectNotVisible(page, ['role=cell[name="nic-2"]'])

        at /home/runner/work/console/console/app/pages/__tests__/instance/networking.e2e.ts:60:14

    Pending operations:
      - page.click at app/pages/__tests__/instance/networking.e2e.ts:60:14


    attachment #1: trace (application/zip) ---------------------------------------------------------
    test-results/pages-__tests__-instance-networking.e2e.ts-Instance-networking-tab-webkit-retry1/trace.zip
    Usage:

        npx playwright show-trace test-results/pages-__tests__-instance-networking.e2e.ts-Instance-networking-tab-webkit-retry1/trace.zip

    ------------------------------------------------------------------------------------------------

  2) [webkit] › pages/project/networking/VpcPage/VpcPage.e2e.ts:109:3 › VpcPage › can update firewall rule 

    Error: expect(received).toHaveCount(expected) // deep equality

    Expected: 4
    Received: 1
    Call log:
      - expect.toHaveCount with timeout 5000ms
      - waiting for selector "tbody >> tr"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"


      112 |
      113 |     const rows = await page.locator('tbody >> tr')
    > 114 |     await expect(rows).toHaveCount(4)
          |                        ^
      115 |
      116 |     // allow-icmp is the one we're doing to change
      117 |     const oldNameCell = page.locator('td >> text="allow-icmp"')

        at /home/runner/work/console/console/app/pages/project/networking/VpcPage/VpcPage.e2e.ts:114:24

  Slow test file: [webkit] › pages/__tests__/instance/networking.e2e.ts (1m)
  Slow test file: [webkit] › pages/project/networking/VpcPage/VpcPage.e2e.ts (34s)
  Slow test file: [firefox] › pages/project/networking/VpcPage/VpcPage.e2e.ts (21s)
  Slow test file: [firefox] › pages/__tests__/project-selector.e2e.ts (16s)
  Slow test file: [firefox] › pages/__tests__/ssh-keys.e2e.ts (16s)
  Consider splitting slow test files to speed up parallel execution

  2 flaky
    [webkit] › pages/__tests__/instance/networking.e2e.ts:7:1 › Instance networking tab ============
    [webkit] › pages/project/networking/VpcPage/VpcPage.e2e.ts:109:3 › VpcPage › can update firewall rule 
  2 skipped
  24 passed (2m)
Done in 135.92s.