Utilities¶
load_yaml_config¶
-
dabstract.utils.load_yaml_config(filename: str, path: str, walk: bool = False, verbose: bool = True, post_process: Callable = None, post_process_kwargs: Dict = {}, unpack: bool = False, **kwargs) → Union[str, Dict, Any]¶ Load a yaml configuration file with some additional functionality For example do:
$ data = load_yaml_config(filename=path_to_dir, path=path_to_yaml, walk=True/False, $ post_process=dataset_from_config, **kwargs)
This loader adds additional constructors which can be used in your yaml file. These include the joining of strings witj !join:
For example; !join [str1, str2] -> 'str1tr2'
Or the joining with underscores with !usjoin:
!usjoin[str1, str2] -> 'str1_str2'
Or the joining of paths with !pathjoin:
!pathjoin[str1, str2] -> 'str1/str2'
Similarly one can inject string or int into the configuration from an external source using !kwargs:
In python: load_yaml_config(..., **{'item1': {'item12': 2) In yaml file !kwargs [item1,item12]Finally, one can initialise classes using the !class constructor:
!class [path.to.module]
- Parameters
- filenamestr
filename of the .yaml file
- pathstr
path to a folder containg the .yaml file. If walk==True, one can place .yaml in a nested folder
- walkbool
Walk through the nested folder
- verbosebool
Allow print statements
- post_processCallable
Place here a factory function that accepts the yaml input configuration. Identical as post_process(load_yaml_config).
- post_process_kwargsDict
Any kwargs you want to provide to the postp_process Callable
- unpackbool
Whether or not to unpack to yaml configuration dict before applying it to post_process()
- kwargsdict
kwargs will be used by the !kwargs constructor to inject parameters from your code flow into the configuration.
- Returns
- Any
str_in_list¶
-
dabstract.utils.str_in_list(lst: List, string: str) → List[int]¶ Get indices of a string in a list
- Parameters
- lstlist
list of strings
- stringstr
string to search for
- Returns
- List[int]
listnp_combine¶
-
dabstract.utils.listnp_combine(lst: List, method: str = 'concatenate', axis: int = 0, allow_error: bool = False) → numpy.ndarray¶ Concatenate or stack a list of numpy along with error handling
- Parameters
- lstlist
list of np arrays
- methodstr
‘concatenate’ or ‘stack’
- axisint
axis to concat or stack over
- allow_errorbool
allow for error handling. If op does not succes, list is provided
- Returns
- np.array OR list of np.array in case of error
listdictnp_combine¶
-
dabstract.utils.listdictnp_combine(lst: List, method: str = 'concatenate', axis: int = 0, keep_nested: bool = False, allow_error: bool = False) → Dict[str, Union[numpy.ndarray, List]]¶ Concatenate or stack a list of dictionaries contains numpys along with error handling
- Parameters
- lstlist
list of dicts containings np arrays
- methodstr
‘concatenate’ or ‘stack’
- axisint
axis to concat or stack over
- keep_nestedbool
keep nested structure of list or not
- allow_errorbool
allow for error handling. If op does not succes, list is provided
- Returns
- np.array OR list of np.array in case of error
unique_list¶
-
dabstract.utils.unique_list(seq: List) → List¶ Get unique entries in a list
- Parameters
- seq: list
sequence you want to unique-fy
- Returns
- list of unique values
flatten_nested_lst¶
-
dabstract.utils.flatten_nested_lst(nested_lst: List[List]) → List¶ Flatten a nested list
- Parameters
- nested_lstlist
A nested list
- Returns
- flattened list
any2str¶
-
dabstract.utils.any2str(obj: Any) → str¶ Convert anything to a string
- Parameters
- objAny
anything to convert to a str rep
- Returns
- string
filter_data¶
-
dabstract.utils.filter_data(data: Iterable, sel_vect: Union[List[int], numpy.ndarray], squeeze: bool = False) → Iterable¶ Filter any sequential data object based on indices
- Parameters
- dataIterable
Iterable object to be filtered
- sel_vectList[int] or np.ndarray
indices to filter
- squeezebool
squeeze output when numpy
- Returns
- Filtered iterable
safe_import_module¶
-
dabstract.utils.safe_import_module(module_string: str) → object¶ Import module with error handling
- Parameters
- module_stringstr
module string to import
- Returns
- imported module
combs_numpy¶
-
dabstract.utils.combs_numpy(values: Union[List, numpy.ndarray]) → numpy.ndarray¶ All possible combinations of numpy entries
- Parameters
- valuesUnion[List, np.ndarray]
All unique examples to get combinations from
- Returns
- np.ndarray
combs_list¶
-
dabstract.utils.combs_list(values: Union[List, numpy.ndarray]) → List¶ All possible combinations of list entries
- Parameters
- valuesUnion[List, np.ndarray]
All unique examples to get combinations from
- Returns
- List
pprint_ext¶
-
dabstract.utils.pprint_ext(str: str, dict: Dict, np_precision: int = 2) → None¶ pprint with np precision specification and title
- Parameters
- strstr
title to print
- dictDict
Dictionary to print
- np_precisionint
precision of numpy print
safe_len¶
-
dabstract.utils.safe_len(var: Any) → int¶ safely get length
- Parameters
- varAny
Any object you want the length from
- Returns
- int
stringlist2ind¶
-
dabstract.utils.stringlist2ind(strlist: List[str]) → numpy.ndarray¶ list to unique indices
- Parameters
- strlistList[str]
List of strings to indices
- Returns
- List[int]