Skip to content

Add CameraFeed support for Web #106784

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

shiena
Copy link
Contributor

@shiena shiena commented May 24, 2025

fixed: godotengine/godot-proposals#12493

Current Limitation:
The platform/web/js/libs/library_godot_camera.js library includes certain functionalities that are inherently asynchronous. We are currently investigating how to execute these functions synchronously, as this is a requirement for our project, but haven't yet found a solution.
I was able to build with JSPI and call asynchronous functions synchronously in Chrome. However, JSPI only works in Chrome and Edge 137.
https://webassembly.org/features/
https://learn.microsoft.com/en-us/microsoft-edge/web-platform/release-notes/137#javascript-promise-integration-jspi-in-webassembly

Supported browsers:

  • Chrome v137.0.7151.69
  • Firefox ESR v128.11.0esr
  • Safari v18.5(20621.2.5.11.8)

@shiena shiena force-pushed the feature/support-web-camera branch 3 times, most recently from 140bdef to f939c20 Compare May 24, 2025 23:16
@shiena
Copy link
Contributor Author

shiena commented May 25, 2025

I've tried enabling Asyncify as follows, which I believe should fix it, but the build is still failing.

diff --git a/platform/web/SCsub b/platform/web/SCsub
index cf872e65ff..5bb105e1cd 100644
--- a/platform/web/SCsub
+++ b/platform/web/SCsub
@@ -106,6 +106,8 @@ else:
     sys_env.Append(LIBS=["idbfs.js"])
     build = sys_env.add_program(build_targets, web_files + ["web_runtime.cpp"])
 
+sys_env.Append(LINKFLAGS=["-sASYNCIFY=1"])
+
 sys_env.Depends(build[0], sys_env["JS_LIBS"])
 sys_env.Depends(build[0], sys_env["JS_PRE"])
 sys_env.Depends(build[0], sys_env["JS_POST"])
cache:INFO: generating system asset: symbol_lists/8eab473934c10451394fc82e8912c6b3712c9ff9.json... (this will be cached in "C:\Users\shien\dev\emsdk\upstream\emscripten\cache\symbol_lists\8eab473934c10451394fc82e8912c6b3712c9ff9.json" for subsequent builds)
cache:INFO:  - ok
unexpected expression type
UNREACHABLE executed at C:\b\s\w\ir\cache\builder\emscripten-releases\binaryen\src\passes\Asyncify.cpp:1142!
em++: error: 'C:/Users/shien/dev/emsdk/upstream\bin\wasm-opt --strip-target-features --post-emscripten -Os --low-memory-unused --asyncify --pass-arg=asyncify-propagate-addlist [email protected]_*,env.__asyncjs__*,*.fd_sync,*.emscripten_promise_await,*.emscripten_idb_load,*.emscripten_idb_store,*.emscripten_idb_delete,*.emscripten_idb_exists,*.emscripten_idb_clear,*.emscripten_idb_load_blob,*.emscripten_idb_store_blob,*.emscripten_sleep,*.emscripten_wget_data,*.emscripten_scan_registers,*.emscripten_lazy_load_code,*._load_secondary_module,*.emscripten_fiber_swap,*.SDL_Delay --zero-filled-memory --pass-arg=directize-initial-contents-immutable --no-stack-ir bin\godot.web.template_debug.wasm32.nothreads.wasm -o bin\godot.web.template_debug.wasm32.nothreads.wasm -g --mvp-features --enable-bulk-memory --enable-exception-handling --enable-multivalue --enable-mutable-globals --enable-reference-types --enable-sign-ext --enable-simd' failed (returned 3221226505)
scons: *** [bin\godot.web.template_debug.wasm32.nothreads.js] Error 1

@Calinou
Copy link
Member

Calinou commented May 25, 2025

Asyncify makes binaries a lot larger, so we need to be able to use it selectively only for functions that need it.

cc @adamscott

@shiena shiena force-pushed the feature/support-web-camera branch 3 times, most recently from a6e548e to 9354b30 Compare May 26, 2025 19:48
@AThousandShips AThousandShips added this to the 4.x milestone May 27, 2025
@shiena shiena force-pushed the feature/support-web-camera branch 5 times, most recently from be89cda to 706e8d0 Compare June 2, 2025 04:48
@shiena shiena marked this pull request as ready for review June 2, 2025 05:19
@shiena shiena requested review from a team as code owners June 2, 2025 05:19
@shiena
Copy link
Contributor Author

shiena commented Jun 2, 2025

Ready for review.
The demo site requires Chrome 137+ due to its use of JSPI.
I wanted to use ASYNCIFY=1 in SCsub but ran into errors, so I'm currently using ASYNCIFY=2 for JSPI.

Demo site: https://shiena.github.io/godot-camerafeed-demo/
Demo source code: https://github.com/shiena/godot-camerafeed-demo

@shiena shiena force-pushed the feature/support-web-camera branch from 706e8d0 to 6854b08 Compare June 2, 2025 05:31
@shiena shiena requested a review from a team as a code owner June 2, 2025 05:31
Copy link
Collaborator

@Faless Faless left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The demo site requires Chrome 137+ due to its use of JSPI.
I wanted to use ASYNCIFY=1 in SCsub but ran into errors, so I'm currently using ASYNCIFY=2 for JSPI.

This means it can't be merged, we at least want support the latest version of Chrome, Firefox and iOS (and we also try to support the latest version of Firefox ESR, but that has been on and off in the past).

That said, it should be possible to implement the API by relying on callbacks instead of requiring asyncify.

Comment on lines 37 to 43
EM_ASYNC_JS(void, godot_js_camera_get_cameras, (void *context, CameraLibrary_OnGetCamerasCallback p_callback_ptr), {
await GodotCamera.api.getCameras(context, p_callback_ptr);
});

EM_ASYNC_JS(void, godot_js_camera_get_capabilities, (void *context, const char *p_device_id_ptr, CameraLibrary_OnGetCapabilitiesCallback p_callback_ptr), {
await GodotCamera.api.getCameraCapabilities(p_device_id_ptr, context, p_callback_ptr);
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't use the EM_*_JS functions, as they break dynamic linking.

@shiena
Copy link
Contributor Author

shiena commented Jun 2, 2025

That said, it should be possible to implement the API by relying on callbacks instead of requiring asyncify.

I'm trying to get a list of CameraFeed objects using CameraServer.feeds() after setting CameraServer.monitoring_feeds = true in GDScript. Once I have a CameraFeed, I then want to retrieve its available formats using CameraFeed.formats.

However, both of these methods return a JavaScript Promise, which makes using callbacks inherently slow.
If there's an alternative approach, I'd be happy to try it out immediately.

@shiena shiena force-pushed the feature/support-web-camera branch 3 times, most recently from 18f57bb to f5fd45b Compare June 4, 2025 18:07
@shiena
Copy link
Contributor Author

shiena commented Jun 4, 2025

@Faless
Removed Asyncify dependency and confirmed functionality in the following browsers.

  • Chrome v137.0.7151.69
  • Firefox ESR v128.11.0esr
  • Safari v18.5(20621.2.5.11.8)

However, please note the following limitations:

@shiena shiena force-pushed the feature/support-web-camera branch from f5fd45b to ed064b2 Compare June 4, 2025 18:32
@adamscott
Copy link
Member

Can you share an MRP to test on? I would like to test your PR, but it would be faster for me to have a MRP to test on.

@shiena
Copy link
Contributor Author

shiena commented Jul 1, 2025

@adamscott
This is for all platforms, but I'm testing functionality with this project.
https://github.com/shiena/godot-camerafeed-demo

@shiena
Copy link
Contributor Author

shiena commented Jul 3, 2025

I'll make this a draft until it's merged to apply the signal from #108165.

@shiena shiena marked this pull request as draft July 3, 2025 15:51
@shiena shiena force-pushed the feature/support-web-camera branch from a518a0b to 983af1d Compare July 3, 2025 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Suppport CameraFeed for Web
5 participants