Skip to content

Add Pyodide support and CI jobs for Zarr #1903

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 81 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
9f5a110
Add CI job to test out-of-tree Pyodide builds
agriyakhetarpal May 22, 2024
29282fc
Add `[msgpack]` dependency for `numcodecs`
agriyakhetarpal May 23, 2024
d465742
Bump to Pyodide 0.26.0, update comments
agriyakhetarpal May 27, 2024
cdf0bb2
Try to run tests without async
agriyakhetarpal May 27, 2024
dfe0321
Move shared file to rootdir, outside v2 and v3
agriyakhetarpal May 27, 2024
b100ec9
Move `fasteners` import inside ThreadSynchronizer
agriyakhetarpal May 27, 2024
b0dddca
Make the tests directory importable, fix `_shared`
agriyakhetarpal May 27, 2024
d227728
Import list of greetings from `numcodecs`
agriyakhetarpal May 27, 2024
fdb2bef
Skip some tests that use threading
agriyakhetarpal May 27, 2024
621077a
Skip some tests that use `fcntl`
agriyakhetarpal May 27, 2024
7ae9a97
Skip tests that require `dbm`
agriyakhetarpal May 27, 2024
22eb6da
Move `IS_WASM` logic to internal `zarr` API
agriyakhetarpal May 27, 2024
6836947
Skip a few tests trying to import `multiprocessing`
agriyakhetarpal May 27, 2024
fe3bf27
Skip tests that use async and threading code
agriyakhetarpal May 27, 2024
08997ec
Improve `asyncio_tests_wrapper`, fix test imports
agriyakhetarpal May 27, 2024
9bfc860
Skip entire `test_codecs.py` file
agriyakhetarpal May 27, 2024
9bcb350
Skip yet another test that requires threads
agriyakhetarpal May 27, 2024
9985abb
xfail test where array's fill values are different
agriyakhetarpal May 27, 2024
7ea12ef
xfail test because Emscripten FS
agriyakhetarpal May 27, 2024
a6565de
Skip last test that tries to run threads
agriyakhetarpal May 27, 2024
85f621c
Another test that tries to run threads
agriyakhetarpal May 27, 2024
1a64255
xfail another array's differing `fill_values` test
agriyakhetarpal May 27, 2024
c8cb38b
Skip entire sync file under WASM, no threading
agriyakhetarpal May 27, 2024
eb36d40
Restore pytest config options, remove when needed
agriyakhetarpal May 28, 2024
e3bf365
Merge main
agriyakhetarpal May 26, 2025
b3a5b8a
Bump Emscripten, Pyodide xbuildenv, Node.js versions
agriyakhetarpal May 26, 2025
42d2792
Running on `ubuntu-latest` should be fine
agriyakhetarpal May 26, 2025
27068e2
Don't persist credentials with git clone
agriyakhetarpal May 26, 2025
aff9b18
Don't pin the version of `pyodide-build`
agriyakhetarpal May 26, 2025
bb2c136
Use same xbuildenv for building and testing
agriyakhetarpal May 26, 2025
710195a
Temporarily build numcodecs for WASM as well
agriyakhetarpal May 26, 2025
07e5bc9
Use Pyodide 0.28.0a2 for now
agriyakhetarpal May 26, 2025
403fbb0
Use `fetch-depth: 0` to bring correct versions
agriyakhetarpal May 27, 2025
24dbc77
Skip `test_multiprocessing` for WASM
agriyakhetarpal May 27, 2025
ecea615
Skip all sync tests
agriyakhetarpal May 27, 2025
d919bd7
xfail `test_array_roundtrip` for now
agriyakhetarpal May 27, 2025
0bb7d47
Skip `test_group_members_performance[memory]` for now
agriyakhetarpal May 27, 2025
fb59eba
Set concurrency and max workers as 1
agriyakhetarpal May 27, 2025
4c6bed6
Update `zarr.config` tests to match
agriyakhetarpal May 27, 2025
6754131
Move WASM check to resolve circular import
agriyakhetarpal May 27, 2025
f426ed7
Mark Blosc `test_typesize` as a known failure case
agriyakhetarpal May 27, 2025
405d247
Skip `async.concurrency` config override test case
agriyakhetarpal May 27, 2025
e93073a
Mark some indexing tests as flaky on WASM
agriyakhetarpal May 27, 2025
d862953
Oops, fix a config test
agriyakhetarpal May 27, 2025
cbd1d4d
Fix another config test
agriyakhetarpal May 27, 2025
3e8bdef
Hook into Pyodide WebLoop
agriyakhetarpal May 27, 2025
4ee492e
Move `zarr.constants` to `zarr._constants`
agriyakhetarpal May 27, 2025
d94970e
Bump to Pyodide 0.28.0a3
agriyakhetarpal May 27, 2025
cd3424c
Fix typo
agriyakhetarpal May 27, 2025
5044a22
`asyncio_mode = "auto"` works now, clean it up
agriyakhetarpal May 28, 2025
e131867
Restore `test_group_members_performance`
agriyakhetarpal May 28, 2025
c7c22dc
Disable SIMD when building numcodecs
agriyakhetarpal May 28, 2025
e62c933
Oops, disable AVX2, SSE2 at the right place
agriyakhetarpal May 28, 2025
d3bcf56
Debug improper numcodecs version
agriyakhetarpal May 28, 2025
e4b7379
Debug numcodecs version again
agriyakhetarpal May 28, 2025
35cecc1
Fetch tags manually for now
agriyakhetarpal May 28, 2025
ceb70c7
Force Zarr to install
agriyakhetarpal May 28, 2025
1ea992f
Install `numcodecs` with `crc32c`
agriyakhetarpal May 28, 2025
f51ddd1
Install the rest of the missing dependencies
agriyakhetarpal May 28, 2025
0ec47b9
Escape wheel filename correctly
agriyakhetarpal May 28, 2025
e1617c0
Remove extra install line
agriyakhetarpal May 28, 2025
23e34bf
Fix misquoted end
agriyakhetarpal May 28, 2025
22e6795
Remove lenience for performance test, skip it instead
agriyakhetarpal May 29, 2025
da8bfc7
Undo async.concurrency to 1, improve performance
agriyakhetarpal May 29, 2025
81f5df3
Add `slow_wasm` marker, skip orthogonal indexing tests
agriyakhetarpal May 29, 2025
bad9920
Move WebLoop patch to `conftest.py`
agriyakhetarpal May 29, 2025
2217455
Mark more indexing tests as slow in WASM
agriyakhetarpal May 29, 2025
86f8785
Fix condition for slow WASM tests
agriyakhetarpal May 29, 2025
3230892
Clearer skip message for slow WASM tests
agriyakhetarpal May 29, 2025
07b5645
Merge main
agriyakhetarpal May 29, 2025
ef70cbd
Add release note for Pyodide/WASM support.
agriyakhetarpal May 29, 2025
9212c0e
Ignore WASM code paths Codecov doesn't know about
agriyakhetarpal May 29, 2025
86323eb
Bring back numcodecs version check
agriyakhetarpal May 29, 2025
9c6af32
Revert "Bring back numcodecs version check"
agriyakhetarpal May 29, 2025
090c62b
Add guidance highlighting JSPI requirement
agriyakhetarpal May 30, 2025
3f2d41d
Merge branch 'main' into emscripten
agriyakhetarpal May 30, 2025
b89f682
Drop `shutdown_asyncgens` fixture
agriyakhetarpal Jun 3, 2025
700aae4
Clarify docs for JSPI availability and usage
agriyakhetarpal Jun 3, 2025
93680d2
Merge branch 'main' into emscripten
agriyakhetarpal Jun 3, 2025
401311e
Fix lint error
agriyakhetarpal Jun 3, 2025
dd25a36
Update release note with JSPI info
agriyakhetarpal Jun 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Import list of greetings from numcodecs
  • Loading branch information
agriyakhetarpal committed May 27, 2024
commit d227728f23ee33dd9264490281f67f6123434f2f
9 changes: 8 additions & 1 deletion tests/v2/test_core.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,14 @@
Zlib,
)
from numcodecs.compat import ensure_bytes, ensure_ndarray
from numcodecs.tests.common import greetings

try:
from numcodecs.tests.common import greetings
except ModuleNotFoundError:
greetings = ['¡Hola mundo!', 'Hej Världen!', 'Servus Woid!', 'Hei maailma!',
'Xin chào thế giới', 'Njatjeta Botë!', 'Γεια σου κόσμε!',
'こんにちは世界', '世界,你好!', 'Helló, világ!', 'Zdravo svete!',
'เฮลโลเวิลด์']
from numpy.testing import assert_array_almost_equal, assert_array_equal

import zarr.v2
Expand Down