nncore.io

IO

nncore.io.io.load(name_or_file, format=None, **kwargs)[source]

Load data from files.

Parameters:

name_or_file (list | str | file object) – Paths to the files or file objects.
format (str, optional) – Format of the file. If not specified, the file format will be inferred from the file extension. Currently supported formats include json/jsonl, yaml/yml, pickle/pkl, hdf5/h5, npy/npz, xml, txt, and sh. Default: None.

Returns:

The loaded data.

Return type:

any

nncore.io.io.dump(obj, name_or_file, format=None, overwrite=True, **kwargs)[source]

Dump data to a file.

Parameters:

obj (any) – The object to be dumped.
name_or_file (str | file object) – Path to the file or a file object.
format (str, optional) – Format of the file. If not specified, the file format will be inferred from the file extension. Currently supported formats include json/jsonl, yaml/yml, pickle/pkl, hdf5/h5, npy/npz, xml, txt, and sh. Default: None.
overwrite (bool, optional) – Whether to overwrite it if the file exists. Default: True.

nncore.io.io.loads(string, format='pickle', **kwargs)[source]

Load data from strings.

Parameters:

string (list | str | btyearray) – Strings of the data.
format (str, optional) – Format of the string. Currently supported formats include json/jsonl, yaml/yml, pickle/pkl and xml. Default: 'pickle'.

Returns:

The loaded data.

Return type:

any

nncore.io.io.dumps(obj, format='pickle', **kwargs)[source]

Dump data to a string.

Parameters:

obj (any) – The object to be dumped.
format (str, optional) – Format of the string. Currently supported formats include json/jsonl, yaml/yml, pickle/pkl and xml. Default: 'pickle'.

Returns:

The dumped string.

Return type:

str

nncore.io.io.list_from_file(filename, encoding='utf-8', offset=0, separator=',', max_length=-1)[source]

Load a text file and parse the content as a list of tuples or str.

Parameters:

filename (str) – Path to the file to be loaded.
encoding (str, optional) – The encoding of the file.
offset (int, optional) – The offset of line numbers. Default: 0.
separator (str | None, optional) – The separator to use for parsing tuples. If not specified, each line would be treated as a str. Default: ','.
max_length (int, optional) – The maximum number of lines to be loaded. -1 means all the lines from the file will be loaded. Default: -1.

Returns:

The loaded str list.

Return type:

list[str]

nncore.io.io.open(file=None, mode='r', format=None, as_decorator=None, **kwargs)[source]

Open a file and return a file object. This method can be used as a function or a decorator. When used as a decorator, the function to be decorated should receive the handler using an argument named f. File and mode can be overrided during calls using the arguments file and mode. Argument file and format should not be None at the same time.

Parameters:

file (str | None, optional) – Path to the file to be loaded.
mode (str, optional) – The loading mode to use. Default: 'r'.
format (str, optional) – Format of the file. If not specified, the file format will be inferred from the file extension. Currently supported formats include jsonl and hdf5/h5. Default: None.
as_decorator (bool | None, optional) – Whether this method is used as a decorator. Please explicitly assign a bool value to this argument when using this method in a Python Shell. If not specified, the method will try to determine it automatically.

Returns:

The opened file object.

Return type:

file object

Handlers

class nncore.io.handlers.FileHandler[source]: Base class for file handlers. The inherited classes can optionally override load_from_file, dump_to_file, load_from_str, and dump_to_str methods to support loading or dumping data.

class nncore.io.handlers.HDF5Handler[source]: Handler for HDF5 files.

class nncore.io.handlers.JSONHandler[source]: Handler for JSON files.

class nncore.io.handlers.JSONLHandler[source]: Handler for JSON Lines files.

class nncore.io.handlers.NumPyHandler[source]: Handler for NumPy files.

class nncore.io.handlers.NumPyzHandler[source]: Handler for compressed NumPy files.

class nncore.io.handlers.PickleHandler[source]: Handler for Pickle files.

class nncore.io.handlers.TXTHandler[source]: Handler for plain text files.

class nncore.io.handlers.XMLHandler[source]: Handler for XML files.

class nncore.io.handlers.YAMLHandler[source]: Handler for YAML files.