torchwrench.serialization package

torchwrench.serialization.as_builtin(x: Counter, **kwargs) → dict[Any, int][source]
torchwrench.serialization.as_builtin(x: date, **kwargs) → str
torchwrench.serialization.as_builtin(x: Enum, **kwargs) → str
torchwrench.serialization.as_builtin(x: Path, **kwargs) → str
torchwrench.serialization.as_builtin(x: Pattern, **kwargs) → str
torchwrench.serialization.as_builtin(x: Namespace, **kwargs) → dict[str, Any]
torchwrench.serialization.as_builtin(x: Mapping[K, V], **kwargs) → dict[K, V]
torchwrench.serialization.as_builtin(x: DataclassInstance, **kwargs) → dict[str, Any]
torchwrench.serialization.as_builtin(x: NamedTupleInstance, **kwargs) → dict[str, Any]
torchwrench.serialization.as_builtin(x: T_BuiltinScalar, **kwargs) → T_BuiltinScalar
torchwrench.serialization.as_builtin(x: Any, **kwargs) → Any

Convert an object to a sanitized python builtin equivalent recursively.

This function can be used to sanitize data before saving to a JSON, YAML or CSV file.

Additional objects to convert can be added dynamically with pythonwrench.register_as_builtin_fn function decorator.

Here is the list of default objects converted to built-in: - tuple -> list - collections.Counter -> dict - datetime.date -> str - argparse.Namespace -> dict - re.Pattern -> str - pathlib.Path -> str - enum.Enum -> str - Mapping -> dict - Iterable -> list - Dataclass -> dict - NamedTuple -> dict

Note: By default, tuple objects are converted to list.

Args:

x: Object to convert to built-in equivalent.

torchwrench.serialization.dump_csv(data: Iterable[Mapping[str, Any]] | Mapping[str, Iterable[Any]] | Iterable, fpath: str | Path | None = None, *, overwrite: bool = True, to_builtins: bool = False, make_parents: bool = True, backend: 'csv' | 'pandas' | 'auto' = 'auto', header: bool | 'auto' = 'auto', **csv_backend_kwds) → str[source]

Dump content to csv format.

torchwrench.serialization.dump_json(data: Any, file: str | Path | None | TextIOBase = None, /, *, overwrite: bool = True, make_parents: bool = True, to_builtins: bool = False, indent: int | None = 4, ensure_ascii: bool = False, **json_dumps_kwds) → str[source]

Dump content to JSON format into a string and/or file.

Args:

data: Data to dump to JSON. file: Optional filepath to save dumped data. Not used if None. defaults to None. overwrite: If True, overwrite target filepath. defaults to True. make_parents: Build intermediate directories to filepath. defaults to True. to_builtins: If True, converts data to builtin equivalent before saving. defaults to False. indent: JSON indentation size in spaces. defaults to 4. ensure_ascii: Ensure only ASCII characters. defaults to False. **json_dump_kwds: Other args passed to json.dumps.

Returns:

Dumped content as string.

torchwrench.serialization.dump_pickle(data: Any, file: str | Path | PathLike | BinaryIO | None = None, /, *, overwrite: bool = True, make_parents: bool = True, to_builtins: bool = False, **pkl_dumps_kwds) → bytes[source]

Dump content to PICKLE format into bytes and/or file.

Args:

data: Data to dump to PICKLE. file: Optional filepath to save dumped data. Not used if None. defaults to None. overwrite: If True, overwrite target filepath. defaults to True. make_parents: Build intermediate directories to filepath. defaults to True. to_builtins: If True, converts data to builtin equivalent before saving. defaults to False. **pkl_dumps_kwds: Other args passed to pickle.dumps.

Returns:

Dumped content as bytes.

torchwrench.serialization.dump_to(obj: Any, fpath: str | Path | PathLike | None | BinaryIO = None, *args, saving_backend: 'csv' | 'json' | 'jsonl' | 'h5py' | 'numpy' | 'pickle' | 'safetensors' | 'torch' | 'torchaudio' | 'yaml' | None = 'torch', **kwargs) → str | bytes[source]

Save to file using the correct backend.

torchwrench.serialization.dump_torch(obj: object, f: str | PathLike | BinaryIO | IO[bytes] | None = None, pickle_module: Any = <module 'pickle' from '/home/docs/.asdf/installs/python/3.12.10/lib/python3.12/pickle.py'>, pickle_protocol: int = 2, _use_new_zipfile_serialization: bool = True, _disable_byteorder_record: bool = False, *, overwrite: bool = True, make_parents: bool = True) → bytes[source]
torchwrench.serialization.load_csv(fpath: str | Path | TextIOBase, /, *, orient: 'list' = 'list', header: bool = True, comment_start: str | None = None, strip_content: bool = False, backend: 'csv' | 'pandas' | 'auto' = 'auto', delimiter: str | None = None, **csv_backend_kwds) → list[dict[str, Any]][source]
torchwrench.serialization.load_csv(fpath: str | Path | TextIOBase, /, *, orient: 'dict', header: bool = True, comment_start: str | None = None, strip_content: bool = False, backend: 'csv' | 'pandas' | 'auto' = 'auto', delimiter: str | None = None, **csv_backend_kwds) → dict[str, list[Any]]
torchwrench.serialization.load_csv(fpath: str | Path | TextIOBase, /, *, orient: 'dataframe', header: bool = True, comment_start: str | None = None, strip_content: bool = False, backend: 'csv' | 'pandas' | 'auto' = 'auto', delimiter: str | None = None, **csv_backend_kwds) → DataFrame

Load CSV file using CSV or pandas backend.

torchwrench.serialization.load_from(fpath: TextIO | BinaryIO, *args, saving_backend: 'csv' | 'json' | 'jsonl' | 'h5py' | 'numpy' | 'pickle' | 'safetensors' | 'torch' | 'torchaudio' | 'yaml' = 'torch', **kwargs) → Any[source]
torchwrench.serialization.load_from(fpath: str | Path | PathLike, *args, saving_backend: 'csv' | 'json' | 'jsonl' | 'h5py' | 'numpy' | 'pickle' | 'safetensors' | 'torch' | 'torchaudio' | 'yaml' | None = 'torch', **kwargs) → Any

Load from file using the correct backend.

torchwrench.serialization.load_pickle(file: str | Path | BinaryIO, /, **pkl_loads_kwds) → Any[source]

Load content from PICKLE file.

Args:

file: Filepath file path. **pkl_loads_kwds: Other args passed to pickle.loads.

torchwrench.serialization.load_torch(f: str | PathLike | BinaryIO | IO[bytes], map_location: Callable[[Storage, str], Storage] | device | str | dict[str, str] | None = None, pickle_module: Any = None, *, weights_only: bool = Ellipsis, mmap: bool | None = None, **pickle_load_args: Any) → Any[source]
torchwrench.serialization.read_from(fpath: str | Path | PathLike | TextIO | BinaryIO, *args, saving_backend: 'csv' | 'json' | 'jsonl' | 'h5py' | 'numpy' | 'pickle' | 'safetensors' | 'torch' | 'torchaudio' | 'yaml' | None = 'torch', **kwargs) → Any[source]

Load from file using the correct backend.

torchwrench.serialization.save_to(obj: Any, fpath: str | Path | PathLike | None | BinaryIO = None, *args, saving_backend: 'csv' | 'json' | 'jsonl' | 'h5py' | 'numpy' | 'pickle' | 'safetensors' | 'torch' | 'torchaudio' | 'yaml' | None = 'torch', **kwargs) → str | bytes[source]

Save to file using the correct backend.

Submodules