About implicit encoding and late errors

[sffc]

The encoding of the byte payload is of increased impact for { type: "text" } than it is for other import types.

Evidence: If a JSON file contains invalid UTF-8, the HTML module loader throws an exception. Side note: HTML would ideally read the encoding from the Content-Type header, but it doesn't seem to do this.

HTML Demo: https://gist.github.com/sffc/6d62c6f0518485347bda688c4a2f20a4

I also tested this with Node.js, which also throws an exception.

$ nvm run 25 node.mjs
Running node v25.2.1 (npm v11.6.2)
<anonymous_script>:1
{

SyntaxError: /Users/sffc/mac_projects/sandbox/simplerserver/utf16le.json: Expected property name or '}' in JSON at position 1 (line 1 column 2)

The error is triggered virtually 100% of the time with utf16le. Triggering it with well-formed latin1 is possible but requires a specific byte sequence.

With this proposal, { type: "text" }, there won't be an opportunity to throw a SyntaxError, which causes implicitly garbled text to enter the scope of a JavaScript program, which is unfortunate to introduce in a new language feature.

[eemeli]

Do I understand right that your concern is that in browsers and other implementations that follow the HTML spec, which explicitly and only supports UTF-8, the JS string representation would be incorrect, rather than an error?

Is there some part of this that you think needs to be reflected in the ECMA-262 spec?

[sffc]

I am concerned that hosts, both HTML and native engines, seem to gloss over the encoding problem by assuming that the bytes are UTF-8, when in reality UTF-8 is not universal, and this proposal hides the problem by producing mojibake strings instead of syntax errors (syntax errors which I acknowledge happen only some of the time).

HTML is especially egregious by explicitly ignoring the Content-Type header. I'm told that WinterTC intends to follow the HTML spec, which only increases the impact of this issue. As TC39, we should maybe not stick JS developers with this problem until it is more fully backed in the downstream module loading specifications.

[annevk]

I don't think it throws for all cases of invalid UTF-8. E.g., if you ended up with "[invalid bytes here]" it would probably work fine and return a JSON string?