nncore.utils

Binder

nncore.utils.binder.bind_getter(*vars)[source]

A syntactic sugar for automatically binding getters for classes. This method is expected to be used as a decorator.

Parameters:

*vars – strings indicating the member variables to be binded with getters. The name of member variables are expected to start with an underline (e.g. ‘_name’ or ‘_depth’).

Example

>>> @bind_getter('name', 'depth')
>>> class Backbone:
...
...     _name = 'ResNet'
...     _depth = 50
...
>>> # Equals to:
>>> class Backbone:
...
...     _name = 'ResNet'
...     _depth = 50
...
...     @property
...     def name(self):
...         return copy.deepcopy(self._name)
...
...     @property
...     def depth(self):
...         return copy.deepcopy(self._depth)
nncore.utils.binder.bind_method(key, methods)[source]

A syntactic sugar for automatically binding methods of classes with their attributes. This method is expected to be used as a decorator.

Parameters:

  • key (str) – The key of the attribute to be binded.

  • methods (list[str]) – The list of method names to be binded.

Example

>>> @bind_method('_cfg', ['get', 'pop'])
>>> class Config:
...
...     _cfg = dict()
...
>>> # Equals to:
>>> class Config:
...
...     _cfg = dict()
...
...     def get(self, *args):
...         return self._cfg.get(*args)
...
...     def pop(self, *args):
...         return self._cfg.pop(*args)

Config

class nncore.utils.config.CfgNode(*args, **kwargs)[source]

An extended OrderedDict class with several practical methods.

This class is an extension of the built-in type OrderedDict. The interface is the same as a dict object and also allows access config values as attributes. The input to the init method can be either a single dict or several named parameters.

copy() → a shallow copy of od[source]
update([E, ]**F) → None.  Update D from dict/iterable E and F.[source]

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

class nncore.utils.config.Config(*args, filename=None, freeze=False, **kwargs)[source]

A facility for better CfgNode objects.

This class inherits from CfgNode and it can be initialized from a config file. Users can use the static method Config.from_file to create a Config object.

static from_file(filename, freeze=False)[source]

Build a Config object from a file.

Parameters:

  • filename (str) – Path to the config file. Currently supported formats include py, json, and yaml/yml.

  • freeze (bool, optional) – Whether to freeze the config after initialization. Default: False.

Returns:

The constructed config object.

Return type:

Config

Data

nncore.utils.data.swap_element(matrix, i, j, dim=0)[source]

Swap two elements of an array or a tensor.

Parameters:

  • matrix (np.ndarray | torch.Tensor) – The array or tensor to be swapped.

  • i (int | tuple) – Index of the first element.

  • j (int | tuple) – Index of the second element.

  • dim (int, optional) – The dimension to swap. Default: 0.

Returns:

The swapped array or tensor.

Return type:

np.ndarray | torch.Tensor

nncore.utils.data.is_seq_of(seq, item_type, seq_type=(<class 'list'>, <class 'tuple'>))[source]

Check whether it is a sequence of some type.

Parameters:

  • seq (Sequence) – The sequence to be checked.

  • item_type (tuple[type] | type) – Expected item type.

  • seq_type (tuple[type] | type, optional) – Expected sequence type. Default: (list, tuple).

Returns:

Whether the sequence is valid.

Return type:

bool

nncore.utils.data.is_list_of(seq, item_type)[source]

Check whether it is a list of some type.

A partial method of is_seq_of.

nncore.utils.data.is_tuple_of(seq, item_type)[source]

Check whether it is a tuple of some type.

A partial method of is_seq_of.

nncore.utils.data.slice(seq, length, type='list')[source]

Slice a sequence into several sub sequences by length.

Parameters:

  • seq (list | tuple) – The sequence to be sliced.

  • length (list[int] | int) – The expected length or list of lengths.

  • type (str, optional) – The type of returned object. Expected values include 'list' and 'tuple'. Default: 'list'.

Returns:

The sliced sequences.

Return type:

list[list]

nncore.utils.data.concat(seq)[source]

Concatenate a sequence of sequences.

Parameters:

seq (list | tuple) – The sequence to be concatenated.

Returns:

The concatenated sequence.

Return type:

list | tuple

nncore.utils.data.interleave(seq)[source]

Interleave a sequence of sequences.

Parameters:

seq (list | tuple) – The sequence to be interleaved.

Returns:

The interleaved sequence.

Return type:

list | tuple

nncore.utils.data.flatten(seq)[source]

Flatten a sequence of sequences and items.

Parameters:

seq (list | tuple) – The sequence to be flattened.

Returns:

The flattened sequence.

Return type:

list | tuple

nncore.utils.data.to_dict_of_list(in_list)[source]

Convert a list of dicts to a dict of lists.

Parameters:

in_list (list) – The list of dicts to be converted.

Returns:

The converted dict of lists.

Return type:

dict

nncore.utils.data.to_list_of_dict(in_dict)[source]

Convert a dict of lists to a list of dicts.

Parameters:

in_dict (dict) – the dict of lists to be converted.

Returns:

The converted list of dicts.

Return type:

list

Env

nncore.utils.env.collect_env_info(modules=['numpy', 'PIL', 'cv2'])[source]

Collect information about the environment.

This method will try and collect all the information about the entire environment, including platform, Python version, CUDA version, PyTorch version, etc., and return a str describing the environment.

Parameters:

modules (list[str], optional) – The list of module names to be checked. Default: ['numpy', 'PIL', 'cv2'].

Returns:

The environment information.

Return type:

str

Logger

nncore.utils.logger.get_logger(logger_or_name=None, fmt='[%(asctime)s %(levelname)s]: %(message)s', datefmt='%Y-%m-%d %H:%M:%S', log_level=20, log_file=None)[source]

Initialize and get a logger by name.

If the logger has not been initialized, this method will initialize the logger by adding one or two handlers, otherwise the initialized logger will be directly returned. During initialization, a StreamHandler will always be added. If log_file is specified and the process rank is 0, a FileHandler will also be added.

Parameters:

  • logger_or_name (logging.Logger | str | None, optional) – The logger or name of the logger. Default: None.

  • fmt (str, optional) – Logging format of the logger. The format must end with '%(message)s' to make sure that the colors can be rendered properly. Default: '[%(asctime)s %(levelname)s]: %(message)s'.

  • datefmt (str, optional) – Date format of the logger. Default: '%Y-%m-%d %H:%M:%S'.

  • log_level (str | int, optional) – Log level of the logger. Note that only the main process (rank 0) is affected, and other processes will set the level to ERROR thus be silent at most of the time. Default: logging.INFO.

  • log_file (str | None, optional) – Path to the log file. If specified, a FileHandler will be added to the logger of the main process. Default: None.

Returns:

The expected logger.

Return type:

logging.Logger

nncore.utils.logger.log_or_print(msg, logger_or_name=None, log_level=20, **kwargs)[source]

Print a message with a logger. If logger is a valid logging.Logger or a name of the logger, then it would be used. Otherwise this method will use the normal print function instead.

Parameters:

  • msg (str) – The message to be logged.

  • logger_or_name (logging.Logger | str | None, optional) – The logger or name of the logger to use. Default: None.

  • log_level (int, optional) – Log level of the logger. Default: logging.INFO.

Path

nncore.utils.path.expand_user(path)[source]

Expand user in a path.

Parameters:

path (str) – The path to be expanded.

Returns:

The expanded path.

Return type:

str

nncore.utils.path.abs_path(path)[source]

Parse absolute path from a relative path.

Parameters:

path (str) – Path to the file or directory.

Returns:

The parsed absolute path.

Return type:

str

nncore.utils.path.rel_path(path)[source]

Parse relative path from an absolute path.

Parameters:

path (str) – Path to the file or directory.

Returns:

The parsed relative path.

Return type:

str

nncore.utils.path.dir_name(path)[source]

Parse directory name from a path.

Parameters:

path (str) – Path to the file or directory.

Returns:

The parsed directory name.

Return type:

str

nncore.utils.path.base_name(path)[source]

Parse base filename from a path.

Parameters:

path (str) – Path to the file or directory.

Returns:

The parsed base filename.

Return type:

str

nncore.utils.path.split_path(path)[source]

Split directory name and filename of a path.

Parameters:

path (str) – Path to the file or directory.

Returns:

The parsed directory name and filename.

Return type:

str

nncore.utils.path.split_ext(path)[source]

Split name and extension of a path.

Parameters:

path (str) – Path to the file or directory.

Returns:

The splitted name and extension.

Return type:

tuple[str]

nncore.utils.path.pure_name(path)[source]

Parse pure filename from a path.

Parameters:

path (str) – Path to the file

Returns:

The parsed pure filename.

Return type:

str

nncore.utils.path.pure_ext(path)[source]

Parse file extension from a path.

Parameters:

path (str) – Path to the file.

Returns:

The parsed file extension.

Return type:

str

nncore.utils.path.join(*args)[source]

Combine strings into a path.

Parameters:

*args (str) – The strings to be combined.

Returns:

The combined path.

Return type:

str

nncore.utils.path.is_file(path, raise_error=False)[source]

Check whether a file exists.

Parameters:

  • path (str) – Path to the file.

  • raise_error (bool, optional) – Whether to raise an error if the file is not found. Default: False.

Returns:

Whether the file exists.

Return type:

bool

nncore.utils.path.is_dir(path, raise_error=False)[source]

Check whether a directory exists.

Parameters:

  • path (str) – Path to the directory.

  • raise_error (bool, optional) – Whether to raise an error if the directory is not found. Default: False.

Returns:

Whether the directory exists.

Return type:

bool

nncore.utils.path.ls(path=None, ext=None, skip_hidden_files=True, join_path=False)[source]

List all files in a directory.

Parameters:

  • path (str | None, optional) – Path to the directory. If not specified, the current working path '.' will be used. Default: None.

  • ext (list[str] | str | None, optional) – The file extension or list of file extensions to keep. If specified, all the other files will be discarded. Default: None.

  • skip_hidden_files (bool, optional) – Whether to discard hidden files whose filenames start with ‘.’. Default: True.

  • join_path (bool, optional) – Whether to return the joined path of files. Default: False.

Returns:

The list of files.

Return type:

list

nncore.utils.path.find(path, pattern, sort=True)[source]

Recursively search for files in a directory.

Parameters:

  • path (str) – Path to the directory.

  • pattern (str) – The pattern of file names.

  • sort (bool, optional) – Whether to sort the results. Default: True.

Returns:

The list of found files.

Return type:

list

nncore.utils.path.rename(old_path, new_path)[source]

Rename a file or directory.

Parameters:

  • old_path (str) – Old path to the file or directory.

  • new_path (str) – New path to the file or directory.

nncore.utils.path.cp(src, dst, symlink=True)[source]

Copy files on the disk.

Parameters:

  • src (str) – Path to the source file or directory.

  • dst (str) – Path to the destination file or directory.

  • symlink (bool, optional) – Whether to create a new symlink instead of copying the file it points to. Default: True.

nncore.utils.path.mv(src, dst)[source]

Move files on the disk.

Parameters:

  • src (str) – Path to the source file or directory.

  • dst (str) – Path to the destination file or directory.

nncore.utils.path.mkdir(dir_name, raise_error=False, keep_empty=False, modify_path=False)[source]

Create a leaf directory and all intermediate ones.

Parameters:

  • dir_name (str) – Path to the directory.

  • raise_error (bool, optional) – Whether to raise an error if the directory exists. Default: False.

  • keep_empty (bool, optional) – Whether to keep the directory empty. Default: False.

  • modify_path (bool, optional) – Whether to add '_i' (where i is an accumulating integer starting from 0) to the end of the path if the directory exists. Default: False.

Returns:

Path to the actually created directory.

Return type:

str

nncore.utils.path.same_dir(old_path, new_path)[source]

Parse another file or directory in the same directory.

Parameters:

  • old_path (str) – Old path to the file or directory.

  • new_path (str) – New relative path to the file or directory.

nncore.utils.path.remove(path, raise_error=False)[source]

Remove a file or directory.

Parameters:

  • path (str) – Path to the file or directory.

  • raise_error (bool, optional) – Whether to raise an error if the file is not found. Default: False.

Create a symlink from source to destination.

Parameters:

  • src (str) – Source of the symlink.

  • dst (str) – Destination of the symlink.

  • overwrite (bool, optional) – Whether to overwrite the old symlink if exists. Default: True.

  • raise_error (bool, optional) – Whether to raise an error if the platform does not support symlink. Default: False.

Progress Bar

class nncore.utils.progress.ProgressBar(iterable=None, num_tasks=None, percentage=False, active=None)[source]

A progress bar which showing the state of a progress.

Parameters:

  • iterable (iterable | None, optional) – The iterable object to decorate with a progress bar. Default: None.

  • num_tasks (int | None, optional) – The number of expected iterations. If not specified, the length of iterable object will be used. Default: None.

  • percentage (bool, optional) – Whether to display the percentage instead of raw task numbers. Default: False.

  • active (bool | None, optional) – Whether the progress bar is active. If not specified, only the main process will show the progree bar. Default: None.

Example

>>> for item in ProgressBar(range(10)):
...     # do processing

>>> prog_bar = ProgressBar(num_tasks=10)
>>> for item in range(10):
...     # do processing
...     prog_bar.update()

Registry

class nncore.utils.registry.Registry(name, parent=None, children=None)[source]

A registry to map strings to objects.

Records in the self.items maintain the registry of objects. For each record, the key is the object name and the value is the object itself. The method self.register can be used as a decorator or a normal function.

Parameters:

  • name (str) – Name of the registry.

  • parent (list[str] | str | None, optional) – The parent registry of list of parent registries. Default: None.

  • children (list[str] | str | None, optional) – The children registry of list of children registries. Default: None.

Example

>>> backbones = Registry('backbone')
>>> @backbones.register()
>>> class ResNet(object):
...     pass

>>> backbones = Registry('backbone')
>>> class ResNet(object):
...     pass
>>> backbones.register(ResNet)
nncore.utils.registry.build_object(cfg, parent, default=None, args=[], **kwargs)[source]

Build an object from a dict.

The dict must contain a key type, which is a indicating the object type. Remaining fields are treated as the arguments for constructing the object.

Parameters:

  • cfg (any) – The object, object config or object name.

  • parent (any) – The module or a list of modules which may contain the expected object.

  • default (any, optional) – The default value when the object is not found. Default: None.

  • args (list, optional) – The argument list used to build the object.

Returns:

The constructed object.

Return type:

any

Timer

class nncore.utils.timer.Timer[source]

A flexible timer class.

reset()[source]

Reset the timer.

is_paused()[source]

Check whether the timer is paused.

pause()[source]

Pause the timer.

resume()[source]

Resume the timer.

seconds()[source]

Return the total number of seconds since the reset of the timer, excluding the time when the timer is paused.

minutes()[source]

Return the total number of minutes since the reset of the timer, excluding the time when the timer is paused.

hours()[source]

Return the total number of hours since the reset of the timer, excluding the time when the timer is paused.

Misc

nncore.utils.misc.recursive(key=None, type='list')[source]

A syntactic sugar to make a function recursive. This method is expected to be used as a decorator.

Parameters:

  • key (str | None, optional) – The name of the argument to iterate. If not specified, the first argument of the function will be used. Default: None.

  • type (str, optional) – The type of returned object. Expected values include 'list', 'tuple', and 'dict'. Default: 'list'.

Example

>>> @recursive()
>>> def func(num):
...     return num + 1
...
>>> # Equals to:
>>> def func(num):
...     if isinstance(num, (list, tuple, range)):
...         return [func(n) for n in num]
...     else:
...         return num + 1
...
>>> @recursive(key='name', type='dict')
>>> def func(value, name):
...     return dict(name=value)
...
>>> # Equals to:
>>> def func(value, name):
...     if isinstance(name, (list, tuple, range)):
...         out_dict = dict()
...         for n in name:
...             out_dict.update(func(value, n))
...         return out_dict
...     else:
...         return dict(name=value)