Updating Web Platform Tests can cause ERR_STREAM_PREMATURE_CLOSE

[austinkelleher]

Issue

When updating WPT resources, I received a ERR_STREAM_PREMATURE_CLOSE error. I also tested updating streams and I did not see the error. I'm not sure whether this is due to the size of data being downloaded.

Node version (tried multiple): v19, v16, v14
Hardware:

System Version: macOS 12.2
Model Name: MacBook Pro
Model Identifier: MacBookPro18,2
Chip: Apple M1 Max
Total Number of Cores: 10 (8 performance and 2 efficiency)
Memory: 32 GB

➜  git node wpt resources
---------------------- Checking updates for resources... -----------------------
Last local update for resources is fbee64516446
✔  Last update in upstream is d7b14d101038
⠦ Querying files for resources/webidl2FetchError: Invalid response body while trying to fetch https://api.github.com/graphql: Premature close
    at consumeBody (file:///Users/austinkelleher/Documents/open-source/node-core-utils/node_modules/node-fetch/src/body.js:234:60)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Response.text (file:///Users/austinkelleher/Documents/open-source/node-core-utils/node_modules/node-fetch/src/body.js:158:18)
    at async Request.json (file:///Users/austinkelleher/Documents/open-source/node-core-utils/lib/request.js:49:18)
    at async Request.query (file:///Users/austinkelleher/Documents/open-source/node-core-utils/lib/request.js:107:20)
    at async Request.gql (file:///Users/austinkelleher/Documents/open-source/node-core-utils/lib/request.js:67:22)
    at async GitHubTree.getFiles (file:///Users/austinkelleher/Documents/open-source/node-core-utils/lib/github/tree.js:106:18)
    at async Promise.all (index 2)
    at async GitHubTree.getFiles (file:///Users/austinkelleher/Documents/open-source/node-core-utils/lib/github/tree.js:117:24)
    at async ResourcesUpdater.getAssetList (file:///Users/austinkelleher/Documents/open-source/node-core-utils/lib/wpt/index.js:61:34) {
  type: 'system',
  errno: 'ERR_STREAM_PREMATURE_CLOSE',
  code: 'ERR_STREAM_PREMATURE_CLOSE',
  erroredSysCall: undefined
}

Suggested Fix

Switching to undici from node-fetch fixed the issue 😬 ! I'll open a PR.

[targos]

Switching to undici from node-fetch fixed the issue 😬 ! I'll open a PR.

#595

[austinkelleher]

@targos Thanks for the info. Very helpful.

Yes, it's true that there is a minimum required version for unidici's fetch to be imported. Is it important that this project supports running on older versions of Node? I see that one of the GitHub actions is running against Node 12 and Node 14: https://github.com/nodejs/node-core-utils/blob/main/.github/workflows/nodejs.yml#L29

If it is important that this project is able to run on older versions of Node, we'll need to formulate a different fix for the above issue. Currently, it seems that the WPT resources folder fails to update. I was able to successfully update the resources folder using undici.

Perhaps we could check the version of Node.js and lazily load undici's fetch?

[targos]

I think it's important that this tool can run on the Node.js versions that are most likely used by our collaborators.

I'd be +1 on removing support for Node.js 12. Not sure about 14.

[panva]

Every single WPT update i'm fighting this error with a while loop. Took 15 minutes of looping now to update WPTs.

[targos]

We can probably drop support for 14 at this point.

[austinkelleher]

@targos @panva I'm happy to push forward on the original suggestion to switch to undici + drop support for Node 14. I had this working locally back when I opened the issue.