diff --git a/changes/2720.doc.md b/changes/2720.doc.md deleted file mode 100644 index a48b700ca1..0000000000 --- a/changes/2720.doc.md +++ /dev/null @@ -1 +0,0 @@ -Document removal of `zarr.storage.init_group` in v3 migration guide, with replacement using `zarr.open_group`/`zarr.create_group`. diff --git a/changes/3492.doc.md b/changes/3492.doc.md deleted file mode 100644 index 66719c7b50..0000000000 --- a/changes/3492.doc.md +++ /dev/null @@ -1 +0,0 @@ -Document the `threading.max_workers` configuration option in the performance guide. diff --git a/changes/3546.misc.md b/changes/3546.misc.md deleted file mode 100644 index 77fa0acb5f..0000000000 --- a/changes/3546.misc.md +++ /dev/null @@ -1 +0,0 @@ -Upgrade to spec0 compat (python 3.14 max, python 3.12 min). \ No newline at end of file diff --git a/changes/3611.bugfix.md b/changes/3611.bugfix.md deleted file mode 100644 index 8b889d79fc..0000000000 --- a/changes/3611.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -Fix `SyncError` raised when assigning a `zarr.Array` as the value in a `__setitem__` call (e.g. `dst[:] = src` where `src` is a zarr array). The source array is now converted to a NumPy array before entering the async codec pipeline. diff --git a/changes/3679.feature.md b/changes/3679.feature.md deleted file mode 100644 index cdf35382e4..0000000000 --- a/changes/3679.feature.md +++ /dev/null @@ -1,3 +0,0 @@ -Adds a new in-memory storage backend called `ManagedMemoryStore`. Instances of `ManagedMemoryStore` -function similarly to `MemoryStore`, but instances of `ManagedMemoryStore` can be constructed from -a URL like `memory://store`. \ No newline at end of file diff --git a/changes/3748.feature.md b/changes/3748.feature.md deleted file mode 100644 index 5fd94fe0a3..0000000000 --- a/changes/3748.feature.md +++ /dev/null @@ -1 +0,0 @@ -Added `array.read_missing_chunks` configuration option. When set to `False`, reading missing chunks raises a `ChunkNotFoundError` instead of filling them with the array's fill value. diff --git a/changes/3781.feature.md b/changes/3781.feature.md deleted file mode 100644 index 191ca5ed4a..0000000000 --- a/changes/3781.feature.md +++ /dev/null @@ -1 +0,0 @@ -Added `Struct` class (subclass of `Structured`) implementing the zarr-extensions `struct` dtype spec. Uses object-style field format and dict fill values. Legacy `Structured` remains available for backward compatibility. diff --git a/changes/3793.misc.md b/changes/3793.misc.md deleted file mode 100644 index 1b872dbcc2..0000000000 --- a/changes/3793.misc.md +++ /dev/null @@ -1 +0,0 @@ -Skip the read-before-write step when a write covers an entire shard, eliminating an unnecessary `get` on full-shard overwrites with the sharding codec. diff --git a/changes/3797.bugfix.md b/changes/3797.bugfix.md deleted file mode 100644 index c683213c5d..0000000000 --- a/changes/3797.bugfix.md +++ /dev/null @@ -1,2 +0,0 @@ -Fix an issue that prevents the correct parsing of special NumPy ``uint32`` dtypes resulting e.g. -from bit wise operations on ``uint32`` arrays on Windows. diff --git a/changes/3800.misc.md b/changes/3800.misc.md deleted file mode 100644 index 65497524b3..0000000000 --- a/changes/3800.misc.md +++ /dev/null @@ -1 +0,0 @@ -Rename the `DimensionNames` type to `DimensionNamesLike` to reflect that it models the wide input type that is narrowed to the `dimension_names` attribute on V3 array metadata, matching the `XLike` naming convention used by other input-narrowing types. diff --git a/changes/3802.feature.md b/changes/3802.feature.md deleted file mode 100644 index c57a7a98da..0000000000 --- a/changes/3802.feature.md +++ /dev/null @@ -1,16 +0,0 @@ -Add support for rectilinear (variable-sized) chunk grids. This feature is experimental and -must be explicitly enabled via ``zarr.config.set({'array.rectilinear_chunks': True})``. - -Rectilinear chunks can be used through: - -- **Creating arrays**: Pass nested sequences (e.g., ``[[10, 20, 30], [50, 50]]``) to ``chunks`` - in ``zarr.create_array``, ``zarr.from_array``, ``zarr.zeros``, ``zarr.ones``, ``zarr.full``, - ``zarr.open``, and related functions, or to ``chunk_shape`` in ``zarr.create``. -- **Opening existing arrays**: Arrays stored with the ``rectilinear`` chunk grid are read - transparently via ``zarr.open`` and ``zarr.open_array``. -- **Rectilinear sharding**: Shard boundaries can be rectilinear while inner chunks remain regular. - -**Breaking change**: The ``validate`` method on ``BaseCodec`` and ``CodecPipeline`` now receives -a ``ChunkGridMetadata`` instance instead of a ``ChunkGrid`` instance for the ``chunk_grid`` -parameter. Third-party codecs that override ``validate`` and inspect the chunk grid will need to -update their type annotations. No known downstream packages were using this parameter. diff --git a/changes/3828.misc.md b/changes/3828.misc.md deleted file mode 100644 index 8704ecceb6..0000000000 --- a/changes/3828.misc.md +++ /dev/null @@ -1,2 +0,0 @@ -`CodecPipeline.read` and `CodecPipeline.read_batch` now return a tuple of typeddict objects -that each carry information about the request for a chunk from storage. \ No newline at end of file diff --git a/changes/3830.misc.md b/changes/3830.misc.md deleted file mode 100644 index f622038f7e..0000000000 --- a/changes/3830.misc.md +++ /dev/null @@ -1 +0,0 @@ -Optimize the performance of indexing operations when using an array-like indexer on a single dimension. diff --git a/changes/3833.misc.md b/changes/3833.misc.md deleted file mode 100644 index 1f3c87b482..0000000000 --- a/changes/3833.misc.md +++ /dev/null @@ -1 +0,0 @@ -Remove the warning that is emitted when any Numcodecs codec is instantiated. diff --git a/changes/3836.doc.md b/changes/3836.doc.md deleted file mode 100644 index a3138a9a45..0000000000 --- a/changes/3836.doc.md +++ /dev/null @@ -1,2 +0,0 @@ -Corrects the type annotation reported for the `batch_info` parameter in the `CodecPipeline.write` -method docstring. \ No newline at end of file diff --git a/changes/3837.misc.md b/changes/3837.misc.md deleted file mode 100644 index 1a2c6f0e4f..0000000000 --- a/changes/3837.misc.md +++ /dev/null @@ -1 +0,0 @@ -Add the static github actions tool `zizmor` to our CI and pre-commit checks. \ No newline at end of file diff --git a/changes/3845.doc.md b/changes/3845.doc.md deleted file mode 100644 index c478596103..0000000000 --- a/changes/3845.doc.md +++ /dev/null @@ -1 +0,0 @@ -Remove result="ansi" from code blocks in the user guide that were causing empty output cells in the rendered documentation. diff --git a/changes/3846.bugfix.md b/changes/3846.bugfix.md deleted file mode 100644 index bfda1b1693..0000000000 --- a/changes/3846.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -Fix `ZipStore.list()`, `list_dir()`, and `exists()` to auto-open the zip file when called before `open()`, consistent with the existing behavior of `get()` and `set()`. diff --git a/changes/3863.bugfix.md b/changes/3863.bugfix.md deleted file mode 100644 index 2902bdf443..0000000000 --- a/changes/3863.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -Fix handling of `NaT` default fill values for `datetime64` and `timedelta64` data types. Equality checks now use `numpy.isnat` so that the default fill value compares correctly against `NaT`. diff --git a/changes/3874.feature.md b/changes/3874.feature.md deleted file mode 100644 index 592c5b330d..0000000000 --- a/changes/3874.feature.md +++ /dev/null @@ -1 +0,0 @@ -Add `cast_value` and `scale_offset` codecs. \ No newline at end of file diff --git a/changes/3897.misc.md b/changes/3897.misc.md deleted file mode 100644 index 53b2bd9b52..0000000000 --- a/changes/3897.misc.md +++ /dev/null @@ -1,2 +0,0 @@ -Bump the minimum version of `typing-extensions` to 4.13 to support the `extra_items` -keyword argument on `TypedDict` (PEP 728). \ No newline at end of file diff --git a/changes/3900.removal.md b/changes/3900.removal.md deleted file mode 100644 index d38de0ce2e..0000000000 --- a/changes/3900.removal.md +++ /dev/null @@ -1 +0,0 @@ -Remove deprecated `zarr.convenience` and `zarr.creation` modules. \ No newline at end of file diff --git a/changes/3901.removal.md b/changes/3901.removal.md deleted file mode 100644 index d1f08d80eb..0000000000 --- a/changes/3901.removal.md +++ /dev/null @@ -1 +0,0 @@ -Remove the deprecated `zarr_version` parameter from several functions and methods. That parameter is replaced with `zarr_format`. \ No newline at end of file diff --git a/changes/3902.removal.md b/changes/3902.removal.md deleted file mode 100644 index ddfc9813e2..0000000000 --- a/changes/3902.removal.md +++ /dev/null @@ -1 +0,0 @@ -Remove deprecated `Group` methods `array`, `require_dataset`, and `create_dataset`. \ No newline at end of file diff --git a/changes/3903.removal.md b/changes/3903.removal.md deleted file mode 100644 index 7dc0a83177..0000000000 --- a/changes/3903.removal.md +++ /dev/null @@ -1 +0,0 @@ -Remove deprecated `AsyncArray.create` and `Array.create` methods. \ No newline at end of file diff --git a/changes/3920.bugfix.md b/changes/3920.bugfix.md deleted file mode 100644 index 289d12970c..0000000000 --- a/changes/3920.bugfix.md +++ /dev/null @@ -1 +0,0 @@ -Use the unit associated with the `Datetime64` data type when creating the default `Nat` scalar value. \ No newline at end of file diff --git a/docs/release-notes.md b/docs/release-notes.md index e79d359e3f..7e3511a0a7 100644 --- a/docs/release-notes.md +++ b/docs/release-notes.md @@ -2,6 +2,63 @@ +## 3.2.0 (2026-04-30) + +### Features + +- Adds a new in-memory storage backend called `ManagedMemoryStore`. Instances of `ManagedMemoryStore` + function similarly to `MemoryStore`, but instances of `ManagedMemoryStore` can be constructed from + a URL like `memory://store`. ([#3679](https://github.com/zarr-developers/zarr-python/issues/3679)) +- Added `array.read_missing_chunks` configuration option. When set to `False`, reading missing chunks raises a `ChunkNotFoundError` instead of filling them with the array's fill value. ([#3748](https://github.com/zarr-developers/zarr-python/issues/3748)) +- Added `Struct` class (subclass of `Structured`) implementing the zarr-extensions `struct` dtype spec. Uses object-style field format and dict fill values. Legacy `Structured` remains available for backward compatibility. ([#3781](https://github.com/zarr-developers/zarr-python/issues/3781)) +- Add support for rectilinear (variable-sized) chunk grids. This feature is experimental and + must be explicitly enabled via `zarr.config.set({'array.rectilinear_chunks': True})`. + + Rectilinear chunks can be used through: + + - **Creating arrays**: Pass nested sequences (e.g., `[[10, 20, 30], [50, 50]]`) to `chunks` + in `zarr.create_array`, `zarr.from_array`, `zarr.zeros`, `zarr.ones`, `zarr.full`, + `zarr.open`, and related functions, or to `chunk_shape` in `zarr.create`. + - **Opening existing arrays**: Arrays stored with the `rectilinear` chunk grid are read + transparently via `zarr.open` and `zarr.open_array`. + - **Rectilinear sharding**: Shard boundaries can be rectilinear while inner chunks remain regular. + + **Breaking change**: The `validate` method on `BaseCodec` and `CodecPipeline` now receives + a `ChunkGridMetadata` instance instead of a `ChunkGrid` instance for the `chunk_grid` + parameter. Third-party codecs that override `validate` and inspect the chunk grid will need to + update their type annotations. No known downstream packages were using this parameter. ([#3802](https://github.com/zarr-developers/zarr-python/issues/3802)) + +- Add `cast_value` and `scale_offset` codecs. ([#3874](https://github.com/zarr-developers/zarr-python/issues/3874)) + +### Bugfixes + +- Fix `SyncError` raised when assigning a `zarr.Array` as the value in a `__setitem__` call (e.g. `dst[:] = src` where `src` is a zarr array). The source array is now converted to a NumPy array before entering the async codec pipeline. ([#3611](https://github.com/zarr-developers/zarr-python/issues/3611)) +- Fix an issue that prevents the correct parsing of special NumPy `uint32` dtypes resulting e.g. + from bit wise operations on `uint32` arrays on Windows. ([#3797](https://github.com/zarr-developers/zarr-python/issues/3797)) +- Fix `ZipStore.list()`, `list_dir()`, and `exists()` to auto-open the zip file when called before `open()`, consistent with the existing behavior of `get()` and `set()`. ([#3846](https://github.com/zarr-developers/zarr-python/issues/3846)) +- Fix handling of `NaT` default fill values for `datetime64` and `timedelta64` data types. Equality checks now use `numpy.isnat` so that the default fill value compares correctly against `NaT`. ([#3863](https://github.com/zarr-developers/zarr-python/issues/3863)) +- Use the unit associated with the `Datetime64` data type when creating the default `Nat` scalar value. ([#3920](https://github.com/zarr-developers/zarr-python/issues/3920)) + +### Improved Documentation + +- Document removal of `zarr.storage.init_group` in v3 migration guide, with replacement using `zarr.open_group`/`zarr.create_group`. ([#2720](https://github.com/zarr-developers/zarr-python/issues/2720)) +- Document the `threading.max_workers` configuration option in the performance guide. ([#3492](https://github.com/zarr-developers/zarr-python/issues/3492)) +- Corrects the type annotation reported for the `batch_info` parameter in the `CodecPipeline.write` + method docstring. ([#3836](https://github.com/zarr-developers/zarr-python/issues/3836)) +- Remove result="ansi" from code blocks in the user guide that were causing empty output cells in the rendered documentation. ([#3845](https://github.com/zarr-developers/zarr-python/issues/3845)) + +### Deprecations and Removals + +- Remove deprecated `zarr.convenience` and `zarr.creation` modules. ([#3900](https://github.com/zarr-developers/zarr-python/issues/3900)) +- Remove the deprecated `zarr_version` parameter from several functions and methods. That parameter is replaced with `zarr_format`. ([#3901](https://github.com/zarr-developers/zarr-python/issues/3901)) +- Remove deprecated `Group` methods `array`, `require_dataset`, and `create_dataset`. ([#3902](https://github.com/zarr-developers/zarr-python/issues/3902)) +- Remove deprecated `AsyncArray.create` and `Array.create` methods. ([#3903](https://github.com/zarr-developers/zarr-python/issues/3903)) + +### Misc + +- [#3546](https://github.com/zarr-developers/zarr-python/issues/3546), [#3793](https://github.com/zarr-developers/zarr-python/issues/3793), [#3800](https://github.com/zarr-developers/zarr-python/issues/3800), [#3828](https://github.com/zarr-developers/zarr-python/issues/3828), [#3830](https://github.com/zarr-developers/zarr-python/issues/3830), [#3833](https://github.com/zarr-developers/zarr-python/issues/3833), [#3837](https://github.com/zarr-developers/zarr-python/issues/3837), [#3897](https://github.com/zarr-developers/zarr-python/issues/3897) + + ## 3.1.6 (2026-03-19) ### Features @@ -17,7 +74,7 @@ - Correct the target bytes number for auto-chunking when auto-sharding. ([#3603](https://github.com/zarr-developers/zarr-python/issues/3603)) - Fixed a bug in the sharding codec that prevented nested shard reads in certain cases. ([#3655](https://github.com/zarr-developers/zarr-python/issues/3655)) - Fix obstore `_transform_list_dir` implementation to correctly relativize paths (removing `lstrip` usage). ([#3657](https://github.com/zarr-developers/zarr-python/issues/3657)) -- Raise error when trying to encode :class:`numpy.dtypes.StringDType` with `na_object` set. ([#3695](https://github.com/zarr-developers/zarr-python/issues/3695)) +- Raise error when trying to encode `numpy.dtypes.StringDType` with `na_object` set. ([#3695](https://github.com/zarr-developers/zarr-python/issues/3695)) - `CacheStore`, `LoggingStore` and `LatencyStore` now support with_read_only. ([#3700](https://github.com/zarr-developers/zarr-python/issues/3700)) - Skip chunk coordinate enumeration in resize when the array is only growing, avoiding unbounded memory usage for large arrays. ([#3702](https://github.com/zarr-developers/zarr-python/issues/3702)) - Fix a performance bug in morton curve generation. ([#3705](https://github.com/zarr-developers/zarr-python/issues/3705))