Skip to content

V2 PCodec decompression error #3176

Open
@slowjazz

Description

@slowjazz

Zarr version

3.0.8

Numcodecs version

0.15.1

Python Version

3.12.7

Operating System

Mac

Installation

uv pip

Description

I write an array with Pcodec and dtype of np.float32, using zarr_format=2
Using zarr-python 3 will throw if I try to read it.

Steps to reproduce

Example:


# /// script
# dependencies = [
#   "zarr>=3,<4",
#   "pcodec==0.3.4",
#   "numcodecs==0.15.1"
# ]
# ///
#

import zarr
import numpy as np
import sys
from numcodecs import PCodec

print(sys.version)
print(zarr.__version__)

# write zarr
compressor = PCodec(level=12)
z = zarr.open(
    "test.zarr", mode="w", shape=(1000, 1000), chunks=(10, 10), compressor=compressor, zarr_format=2
)
z[:] = np.ones((1000, 1000))

# open zarr
z = zarr.open('test.zarr', zarr_format=2)
z[:50] # throws
zarr.print_debug_info()

uv has a problem with uv run v3-example.py, presumably because of some non-python panic, but the stack trace after installing the reqs is:

3.12.7 (main, Oct  1 2024, 02:05:46) [Clang 15.0.0 (clang-1500.3.9.4)]
0.1.dev2553+gbbdefac
Traceback (most recent call last):
  File "/Users/kli/workplace/zarr-python/v3-example.py", line 19, in <module>
    z = zarr.open('test.zarr', zarr_format=2)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/_compat.py", line 43, in inner_f
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/api/synchronous.py", line 201, in open
    obj = sync(
          ^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/sync.py", line 163, in sync
    raise return_result
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/sync.py", line 119, in _runner
    return await coro
           ^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/api/asynchronous.py", line 355, in open
    return AsyncArray(store_path=store_path, metadata=_metadata_dict)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/array.py", line 280, in __init__
    metadata_parsed = parse_array_metadata(metadata)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/array.py", line 161, in parse_array_metadata
    return ArrayV2Metadata.from_dict(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/metadata/v2.py", line 196, in from_dict
    return cls(**_data)
           ^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/metadata/v2.py", line 93, in __init__
    compressor_parsed = parse_compressor(compressor)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/zarr/core/metadata/v2.py", line 301, in parse_compressor
    return numcodecs.get_codec(data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kli/workplace/zarr-python/.venv/lib/python3.12/site-packages/numcodecs/registry.py", line 54, in get_codec
    raise UnknownCodecError(f"{codec_id!r}")
numcodecs.errors.UnknownCodecError: codec not available: ''pcodec''

Additional output

3.12.7 (main, Oct  1 2024, 02:05:46) [Clang 15.0.0 (clang-1500.3.9.4)]
3.0.8
platform: macOS-15.4.1-arm64-arm-64bit
python: 3.12.7
zarr: 3.0.8

**Required dependencies:**
packaging: 25.0
numpy: 2.3.1
numcodecs: 0.15.1
typing_extensions: 4.14.0
donfig: 0.8.1.post1

**Optional dependencies:**
numcodecs: 0.15.1

**Not Installed:**
botocore
cupy-cuda12x
fsspec
s3fs
gcsfs
universal-pathlib
rich
obstore

I've also found that creating the array with zarr version 2.18.7 results in the same outcome.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugPotential issues with the zarr-python library

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions