http request for non existent URL occurs error in onWritable() of httpserver.js

[kitazaki]

I updated Node-RED MCU Edition from Moddable SDK 5.3.3 to 5.4.1, I have an issue on http-in and http-response node.

Environment (OK)
Moddable SDK 5.3.3
with Node-RED 3.1.15 and 4.0.8
Device: esp32/xiao_esp32s3_sense

Environment (NG)
Moddable SDK 5.4.1
with Node-RED 3.1.15 and 4.0.8
Device: esp32/xiao_esp32s3_sense

I tried camera server with Node-RED MCU Edition using mcu-camera node with http-in and http-response nodes.

https://github.com/kitazaki/node-red-contrib-mcu-camera

If I accessed camera URL and photo image was displayed in browser, but I have an error with xsbug CONSOLE error message.

xsbug CONSOLE

node-red-mcu/nodes/network/httpserver/httpserver.js (140) # Break: TypeError: cannot coerce undefined to object (in onWritable)!

I found the reason why this error is occured from the activity of the browser requests favicon.ico (/favicon.ico) otherwise the requested URL (/camera).

The response of HTTP 404 from the device with no error in Moddable SDK 5.3.3, but with error in Moddable SDK 5.4.1.
Is there any work around or the fix code in httpserver.js.

[phoddie]

Thanks for the report. I hadn't seen your camera node before. 😎

I expect this needs to be fixed in the Moddable SDK. If you could, please share the exported flow for this (flows.json). That will make debugging easier.

[kitazaki]

@phoddie

Thank you for your comment and sorry for no sample flow.
I attach flows.json which is simple to output hello text for HTTP request URL /hello.

flows.json

A browser requests URLs /hello and /favicon.ico.

If you use curl command,
curl http://192.168.3.156/hello
and
curl http://192.168.3.156/favicon.ico

1st request is successed with no error.
2nd request is failed with error.

[phoddie]

Thanks for the quick help! I'm able to reproduce the problem with the information you've provided.

[phoddie]

The problem is a little subtle and relates to changes made to support the camera server.

The Node-RED HTTP server always registers an onWritable callback to send the response body, even when it does not have a response body. So, the server calls onWritable to get the body. However, the payload is undefined and so bad things happen in the onWritable callback, as you see. The Node-RED HTTP server should handle that case. Here's a small change to do that for httpserver.js:

	onWritable(count) {
		const payload = this.payload;
		if (!payload)
			return void this.write();
		const end = Math.min(payload.position + count, payload.byteLength);

The 419 HTTP server did not call onWritable in this case until recently. That's why this is breaking now.

And... the 419 HTTP server doesn't expect write to be called with no arguments in this case. That's an oversight that also needs to be fixed. In $MODDABLE)/examples/io/listener/httpserver/httpserver.js apply this:

	write(data) {
		if ("sendResponseBody" !== this.#state)
			throw new Error("bad state");

		if (!data) {
			if ("number" === typeof this.#remaining)
				throw new Error("bad data");

I don't have time to test this thoroughly now, unfortunately. If you are able to give it a try, please let me know how it goes. This area is subtle enough that it is easy to make mistakes. 🤦‍♂️

[kitazaki]

@phoddie

Thank you for your comment on fix codes.
This works fine for me.

[phoddie]

I committed the change above along with another change to set the Content-Length header to 0 for 404 responses. That solves the problem without requiring a change to the Moddable SDK 419 http server (though I will commit that too, since it is correct). Setting Content-Length allows the client to keep the connection open for re-use which can be more efficient if there are multiple requests to the server.