'decodeURIComponent' fails to throw URIError for invalid UTF-8 sequences

[ayuan0828]

Description

When decoding the invalid UTF-8 sequence %ED%BF%BF, decodeURIComponent outputs replacement characters ��� instead of throwing a URIError, violating ECMA-262 specifications.

Version

a29e8f5

Testcase

print(decodeURIComponent('%ED%BF%BF'));

Output

���

Expected output

Throw 'URIError'

[phoddie]

I'm able to reproduce this. It does catch some invalid sequences, but clearly not all. Will investigate further. Thank you.

[phoddie]

The implementation of Decode did not implement Step 6:

If Octets does not contain a valid UTF-8 encoding of a Unicode code point, throw a URIError exception.

This omission allowed surrogate pairs and overlong encodings to be accepted. This has been fixed, consistent with RFC 3629.

The change, which will be integrated in a few days, is to add these lines to xsGlobal.c after line 502:

				if (sequence != gxUTF8Sequences) {
					if ((sequence[-1].lmask >= (txU4)d) ||		// overlong
						((d >= 0xD800) & (d <= 0xDFFF)))		// surrogate pair
						mxURIError("invalid URI");
				}

Note that both before and after this change, XS passes all test262 tests for decodeURIComponent() and decodeURI(). Perhaps test262 should be amended to cover these cases.

[phoddie]

Thank you for opening an issue for this on test262.