Misc_utils

Misc system & data process utils

Usage:

>>> import misc_utils as utils
>>> utils.func_name()  # to call functions in this file

misc_utils.misc_utils.cmd(shell)

Run a shell and return results.

Args:

misc_utils.misc_utils.color_print(text='', color=0, end='\n')

Print colored text.

Parameters:

• text (str) – text to print.
• color (int) –
  • 0 black
  • 1 red
  • 2 green
  • 3 yellow
  • 4 blue
  • 5 cyan (like light red)
  • 6 magenta (like light blue)
  • 7 white
• end (str) – end string after colored text.

Example

>>> color_print('yellow', 3)

misc_utils.misc_utils.file_lines(filename, prefix='', offset=0, max_num=0)

Load a text file and parse the content as a list of strings.

Parameters:

• filename (str) – Filename.
• prefix (str) – The prefix to be inserted to the begining of each item.
• offset (int) – The offset of lines.
• max_num (int) – The maximum number of lines to be read, zeros and negatives mean no limitation.

Returns:

A list of strings.

Return type:

list[str]

misc_utils.misc_utils.format_num(num: int) → str

Add comma in every three digits (return a string).

Parameters:num (int) – a number.

Examples

>>> format_num(10000)  # 10,000
>>> format_num(123456789)  # 123,456,789

misc_utils.misc_utils.format_time(seconds)

Convert seconds to formatted time string.

Parameters:seconds (int) – second number.

Examples

>>> format_time(10)  # 10s
>>> format_time(100)  # 1m
>>> format_time(10000)  # 2h 47m
>>> format_time(1000000)  # 11d 13h 47m

misc_utils.misc_utils.gambling(prob, total=1.0)

Return True in a given probability :param prob: chance to return True. :type prob: float :param total: total, default 1.0. :type total: float

Returns:(randomly) True or False. Return type:(bool)

misc_utils.misc_utils.get_dict_value(data, key)

说明:

通过字符串’key1.key2.key3’访问data[key1][key2][key3]。

Parameters:

• data – dict or list
• key – str ‘key1.key2.key3’ format

Returns:

data[key1][key2][key3]

misc_utils.misc_utils.get_dir_name(path)

Get parent directory name.

Args

path(str): file’s abs path.

Returns

dirname.

Example

>>> get_dir_name('root/train/0001.jpg')  # mode/train
>>> get_dir_name(get_dir_name('root/train/0001.jpg'))  # root

misc_utils.misc_utils.get_file_ext(path)

Example

>>> get_file_ext('train/0001.jpg')  # .jpg

misc_utils.misc_utils.get_file_name(path)

Get filename by path (without extension).

Args

path(str): file’s abs path.

Returns

filename (without extension).

Example

>>> get_file_name('train/0001.jpg')  # 0001

misc_utils.misc_utils.get_file_paths_by_pattern(pattern='*', folder=None)

Get a file path list matched given pattern.

Parameters:

• pattern (str) – a pattern to match files.
• folder (str) – searching folder.

Returns

(list of str): a list of matching paths.

Examples

>>> get_file_paths_by_pattern('*.png')  # get all *.png files in folder
>>> get_file_paths_by_pattern('*rotate*')  # get all files with 'rotate' in name

misc_utils.misc_utils.get_logger(f='log.txt', mode='w', level='info', print_stream=True)

Get a logger.

Parameters:

• f (str) – log file path.
• mode (str) – ‘w’ or ‘a’.
• level (str) – ‘debug’ or ‘info’.
• print_stream (bool) – if print to terminal or not.

Returns:

A logger.

Example

>>> logger = get_logger(level='debug')
>>> logger.info("test")

misc_utils.misc_utils.get_time_stamp(add_offset=0)

Get time_zone+0 unix time stamp (seconds)

Parameters:add_offset (int) – bias added to time stamp Returns:time stamp seconds Return type:(str)

misc_utils.misc_utils.get_time_stamp_by_format_str(time_str: str, fmt='%Y/%m/%d %H:%M:%S', timezone=8)

Get timestamp by formatted time string.

Parameters:

• time_str (str) – string in fmt format.
• fmt (str) – format.
• timezone (int) – time zone.

Returns:

time stamp

Return type:

(str)

Example

>>> get_time_stamp_by_format_str('2020/01/01 15:30:00')
>>> # 1577863800

misc_utils.misc_utils.get_time_str(time_stamp=None, fmt='%Y/%m/%d %H:%M:%S', timezone=8, year_length=4)

Get formatted time string.

Parameters:

• time_stamp (str) – linux time string (seconds).
• fmt (str) – string format.
• timezone (int) – time zone.
• year_length (int) – 2 or 4.

Returns:

formatted time string.

Return type:

(str)

Example

>>> get_time_str()
>>> # 2020/01/01 13:30:00

misc_utils.misc_utils.hash(length=8)

Return a random hash-like string such as a6b3c47f. :param length: length of hash :type length: int

Returns:(randomly) a hash-like string. Return type:(bool)

misc_utils.misc_utils.is_file_image(filename)

Return if a file’s extension is an image’s.

Parameters:filename (str) – file path. Returns:if the file is image or not. Return type:(bool)

misc_utils.misc_utils.is_image_file(filename)

Return if a file’s extension is an image’s.

Parameters:filename (str) – file path. Returns:if the file is image or not. Return type:(bool)

misc_utils.misc_utils.mean(data: list, prec=3)

Calc mean value of a list.

Parameters:

• data (list) – a list.
• prec (int) – round precision.

Returns:

(float) mean value.

Example

>>> mean([1, 2, 3, 4])
>>> # 2.5

misc_utils.misc_utils.no_need_to_recur(item, list_max_show=5, indents=0)

说明:

太复杂的类型不需要继续递归, 例如torch.Tensor([24, 3, 256, 256])

misc_utils.misc_utils.p(obj)

Recursively print list, tuple or dict items

Parameters:obj (list, tuple or dict) – a list, tuple or dict to print.

misc_utils.misc_utils.preview(obj, depth=2, dict_max_show=8, text_max_show=20, list_max_show=5, key=None, indents=0)

预览结构复杂的变量 Preview large object :param obj: Any type of object, dict, list, set, np.ndarray, torch.Tensor, or anything :param depth: int, 递归深度 :param dict_max_show: int, 字典类型最多显示的项目数 :param text_max_show: int, 文本类型最多显示的字数 :param list_max_show: int, 列表类型最多显示的项目数 :param key: str, 指定预览的key, 可用用key1.key2.key3的格式 :param indents: 递归使用的参数, 调用时无需指定

misc_utils.misc_utils.print_args(args)

Print args parsed by argparse.

Parameters:args – args parsed by argparse.

Example

>>> parser = argparse.ArgumentParser()
>>> args = parser.parse_args()
>>> print_args(args)

misc_utils.misc_utils.progress_bar(current, total, pre_msg=None, msg=None)

Render a progress_bar in terminal.

Preview

Training… Step: [=======>… 26/100 ………..] ETA: 0s | loss: 0.45

Parameters:

• current (int) – current counter, range in [0, total-1].
• total (int) – total counts.
• pre_msg (str) – message before the progress bar.
• msg (str) – message after the progress bar.

Example

>>> for i in range(100):
>>>     progress_bar(i, 100, 'Training...', 'loss:0.45')

misc_utils.misc_utils.safe_key(dic: dict, key, default=None)

Return dict[key] if dict has the key, in case of KeyError.

Parameters:

• dic (dict) – a dictionary.
• key (usually str or int) – key.
• default – default return value.

Returns:

dic[key] if key in dic else default.

misc_utils.misc_utils.save_file_lines(filename, lines)

Load a text file and parse the content as a list of strings.

Parameters:

• filename (str) – Filename.
• prefix (str) – The prefix to be inserted to the begining of each item.
• offset (int) – The offset of lines.
• max_num (int) – The maximum number of lines to be read, zeros and negatives mean no limitation.

Returns:

A list of strings.

Return type:

list[str]

misc_utils.misc_utils.split_underline(str, end_num, start_num=0, token='_', keep_ex=True)

split a string by token and return a part of it.

Parameters:

• str (str) – string to handle with.
• end_num (int) – end of kept parts.
• start_num (int) – start of kept parts.
• token (str) – split by which token.
• keep_ex (bool) – whether to keep original extension.

Example

>>> split_underline('abc_123_t134567_cam1.jpg', 2)
>>> # abc_123.jpg

misc_utils.misc_utils.to_string(obj, last_comma=False)

Convert to string in one line.

Parameters:

• obj (list, tuple or dict) – a list, tuple or dict to convert.
• last_comma (bool) – add a comma at last.

Returns:

(str) string.

Example

>>> to_string([1, 2, 3, 4], last_comma=True)
>>> # 1, 2, 3, 4,
>>> to_string({'a': 2,'b': 4})
>>> # a=2, b=4

misc_utils.misc_utils.toggle_list_dict(obj)

Convert list of dict to dict of list, and vice versa.

Parameters:obj – a list or a dict. Returns:converted type of obj.

Example

>>> toggle_list_dict([{'a': 3}, {'a': 5}, {'a': 7}])
>>> # {'a': [3, 5, 7]}
>>> toggle_list_dict({'a': [3, 5, 7]})
>>> # [{'a': 3}, {'a': 5}, {'a': 7}]
>>> k, v = toggle_list_dict({1: 2, 3: 4})
>>> # k=[1, 3], v=[2, 4]

misc_utils.misc_utils.try_make_dir(folder)

Make a directory when ignoring FileExistsError.

Parameters:folder (str) – directory path.