PersistentDataset not usable anymore (v1.5.1) ?

Since the modifications to the PersistentDataset made in version 1.5.1, the persistent dataset does not allow to save and load metadata. This mean we cannot invert back the initial transforms or even know the original filename (and probably a lot more).

To Reproduce
Steps to reproduce the behavior:

Install MONAI version 1.5.1
Load and transform an image through the PersistentDataset
Re-load the image from cache
Try to invert and save the image

Expected behavior
The original image and outputted image should be the same

Code used

data = ["./image.nrrd"]
transforms = Compose([LoadImage(ensure_channel_first=True), Rotate90()])
ds=PersistentDataset(data, transform=transforms, cache_dir="./cache")
reloaded_data = ds[0]
reloaded_data = ds[0]
inverter = Invert(transforms)
inverted = inverter(reloaded_data)
saver = SaveImage("./out", output_ext=".nrrd")
saver(inverted)

Let me know if there is a way to bypass this behavior, thank you.

#8566 fixed the vulnerabilities associated with the pickle module and torch.load(..., weights_only=False) by defensively serialising PyTorch tensors only. It looks like MetaTensors didn't make the cut.

MONAI/monai/data/dataset.py

Lines 211 to 213 in 9c6d819

    
                   Cached data is expected to be tensors, primitives, or dictionaries keying to these values. Numpy arrays will 
        
                   be converted to tensors, however any other object type returned by transforms will not be loadable since 
        
                   `torch.load` will be used with `weights_only=True` to prevent loading of potentially malicious objects.

Objects are saved to cache after being converted to PyTorch tensors with the default argument of track_meta=False

MONAI/monai/data/dataset.py

Line 401 in 9c6d819

obj=convert_to_tensor(_item_transformed, convert_numeric=False),

and loaded in weights-only mode

MONAI/monai/data/dataset.py

Line 380 in 9c6d819

return torch.load(hashfile, weights_only=True)

I've tried converting with track_meta=True but that sets this off

MONAI/monai/data/dataset.py

Lines 384 to 387 in 9c6d819

    
           except (UnpicklingError, RuntimeError) as e:  # corrupt or unloadable cached files are recomputed 
        
               if "Invalid magic number; corrupt file" in str(e) or isinstance(e, UnpicklingError): 
        
                   warnings.warn(f"Corrupt cache file detected: {hashfile}. Deleting and recomputing.") 
        
                   hashfile.unlink()

and recomputes the tensor every time, effectively bypassing the main benefit of caching.

While I appreciate the severity of the security issues addressed by #8566 is high, I feel like the breaking nature of this change wasn't properly advertised. The pull request marked itself as a non-breaking change, its pre-merge check even identified this problem, and we get this notice at the end of a class docstring:

MONAI/monai/data/dataset.py

Line 214 in 9c6d819

Legacy cache files may not be loadable and may need to be recomputed.

This warranted a deprecation path in 1.5.x, or bumping the version to 1.6.0 at the very least per semantic versioning conventions. This is a breaking change that fundamentally alters PersistentDataset behaviour for anyone using transforms that produce MetaTensors i.e. most non-trivial preprocessing pipelines.

fix: pin MONAI version

@SebGoll

Environment:

MacOS 15.6.1 (24G90)
sys.version == '3.12.11 (main, Jul 11 2025, 22:26:01) [Clang 20.1.4 ]'
uv -v: uv 0.7.21 (Homebrew 2025-07-14)
uv add monai[all]==1.5.1

After some more digging it seems like MetaTensors are supported, but NumPy arrays aren't.

@SebGoll I've recreated your specific example using this sample .nrrd file:

from monai.data.dataset import PersistentDataset
from monai.transforms import LoadImage, Rotate90, Compose, Invert, SaveImage
from pathlib import Path

img = Path("./BallBinary30x30x30.nrrd")
transforms = Compose(
    [
        LoadImage(ensure_channel_first=True),
        Rotate90(),
    ]
)
ds = PersistentDataset([img], cache_dir=".", transform=transforms)
_ = ds[0]
rotated_from_cache = ds[0]
inverter = Invert(transforms)
inverted = inverter(rotated_from_cache)
saver = SaveImage("./out", output_ext=img.suffix)
saver(inverted)

Running this example as is yields:

Corrupt cache file detected: 384d57fe2ef3e1e8844bd384282a9808.pt. Deleting and recomputing.
2025-10-22 19:43:47,591 INFO image_writer.py:197 - writing: out/BallBinary30x30x30/BallBinary30x30x30_trans.nrrd

I was able to get the cache file read by:

modifying the MONAI source to add track_meta=True at this line

MONAI/monai/data/dataset.py

Line 401 in 9c6d819

obj=convert_to_tensor(_item_transformed, convert_numeric=False),
registering these NumPy types and TraceKeys as safe globals before using PersistentDataset:

import monai.utils
import numpy as np
import torch

torch.serialization.add_safe_globals([
    np._core.multiarray._reconstruct,
    np.ndarray,
    np.dtype,
    np.dtypes.Int64DType,
    np.dtypes.Float64DType,
    monai.utils.enums.TraceKeys,
])

I think the broader issue remains: this is a breaking change to core functionality that was introduced in a patch release without deprecation warnings or migration guidance. Users upgrading from 1.5.0 to 1.5.1 will find their caches invalidated with no clear path forward beyond manually clearing and rebuilding them.

Note: I also observed differences between the input and output .nrrd files when comparing them directly (54KB vs 108KB), but I think that's due to the input file using shorts, but LoadImage converting to float32 and Rotate90 introducing minor floating-point precision effects in the spatial metadata.

@SebGoll

Hello @SebGoll and @iyassou,

I had noticed similar issues with PersistentDataset no longer supporting MetaTensor objects, so I have submitted this PR with my solution. PersistentDataset now accepts track_meta and weights_only directly, allowing for MetaTensors to be cached and read with track_meta=True and weights_only=False. The default arguments preserve the current behavior of the library, so it does not resolve the backwards compatibility issue you had mentioned.

Fix #8599: Add track_meta and weights_only arguments to PersistentDat…

Fix Project-MONAI#8599: Add track_meta and weights_only arguments to …

Navigation Menu

Repository navigation

PersistentDataset not usable anymore (v1.5.1) ? #8599

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	Cached data is expected to be tensors, primitives, or dictionaries keying to these values. Numpy arrays will
	be converted to tensors, however any other object type returned by transforms will not be loadable since
	`torch.load` will be used with `weights_only=True` to prevent loading of potentially malicious objects.

	except (UnpicklingError, RuntimeError) as e: # corrupt or unloadable cached files are recomputed
	if "Invalid magic number; corrupt file" in str(e) or isinstance(e, UnpicklingError):
	warnings.warn(f"Corrupt cache file detected: {hashfile}. Deleting and recomputing.")
	hashfile.unlink()

PersistentDataset not usable anymore (v1.5.1) ? #8599

Description

Activity

iyassou commented on Oct 22, 2025

iyassou commented on Oct 22, 2025

mccle commented on Nov 12, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions