darwin.utils package

Submodules

darwin.utils.flatten_list module

darwin.utils.flatten_list.flatten_list(list_of_lists: List[Any]) → List[Any][source]

Flattens a list of lists into a single list.

Parameters:

list_of_lists (List[List[Any]]) – The list of lists to flatten.

Returns:

The flattened list.

Return type:

List[Any]

darwin.utils.get_item_count module

darwin.utils.get_item_count.get_item_count(dataset_dict: Dict) → int[source]

Returns the number of items in the dataset.

Parameters:

dataset_dict (Dict) – The dataset dictionary.

Returns:

The number of items in the dataset.

Return type:

int

darwin.utils.utils module

Contains several unrelated utility functions used across the SDK.

darwin.utils.utils.is_extension_allowed_by_filename(filename: str) → bool[source]

Returns whether or not the given video or image extension is allowed.

Parameters:

filename (str) – The filename.

Returns:

Whether or not the given extension of the filename is allowed.

Return type:

bool

darwin.utils.utils.is_image_extension_allowed_by_filename(filename: str) → bool[source]

Returns whether or not the given image extension is allowed.

Parameters:

filename (str) – The image extension.

Returns:

Whether or not the given extension is allowed.

Return type:

bool

darwin.utils.utils.is_file_extension_allowed(filename: str) → bool[source]

Returns whether or not the given image extension is allowed.

Parameters:

filename (str) – The name of the file.

Returns:

Whether or not the given extension is allowed.

Return type:

bool

darwin.utils.utils.urljoin(*parts: str) → str[source]

Take as input an unpacked list of strings and joins them to form an URL.

Parameters:

parts (str) – The list of strings to form the url.

Returns:

The url.

Return type:

str

darwin.utils.utils.is_project_dir(project_path: Path) → bool[source]

Verifies if the directory is a project from Darwin by inspecting its structure.

Parameters:

project_path (Path) – Directory to examine

Returns:

Is the directory a project from Darwin?

Return type:

bool

darwin.utils.utils.get_progress_bar(array: List[AnnotationFile], description: str | None = None) → Iterable[ProgressType][source]

Get a rich a progress bar for the given list of annotation files.

Parameters:

  • array (List[dt.AnnotationFile]) – The list of annotation files.

  • description (Optional[str], default: None) – A description to show above the progress bar.

Returns:

An iterable of ProgressType to show a progress bar.

Return type:

Iterable[ProgressType]

darwin.utils.utils.prompt(msg: str, default: str | None = None) → str[source]

Prompt the user on a CLI to input a message.

Parameters:

  • msg (str) – Message to print.

  • default (Optional[str], default: None) – Default values which is put between [] when the user is prompted.

Returns:

The input from the user or the default value provided as parameter if user does not provide one.

Return type:

str

darwin.utils.utils.find_files(files: List[str | Path], *, files_to_exclude: List[str | Path] = [], recursive: bool = True, sort: bool = False) → List[Path][source]

Retrieve a list of all files belonging to supported extensions. The exploration can be made recursive and a list of files can be excluded if desired.

Parameters:

  • files (List[dt.PathLike]) – List of files that will be filtered with the supported file extensions and returned.

  • files_to_exclude (List[dt.PathLike]) – List of files to exclude from the search.

  • recursive (bool) – Flag for recursive search.

  • sort (bool) – Flag for sorting the files naturally, i.e. file2.txt will come before file10.txt.

Returns:

List of all files belonging to supported extensions. Can’t return None.

Return type:

List[Path]

darwin.utils.utils.secure_continue_request() → bool[source]

Asks for explicit approval from the user. Empty string not accepted.

Returns:

True if the user wishes to continue, False otherwise.

Return type:

bool

darwin.utils.utils.persist_client_configuration(client: Client, default_team: str | None = None, config_path: Path | None = None) → Config[source]

Authenticate user against the server and creates a configuration file for him/her.

Parameters:

  • client (Client) – Client to take the configurations from.

  • default_team (Optional[str], default: None) – The default team for the user.

  • config_path (Optional[Path], default: None) – Specifies where to save the configuration file.

Returns:

A configuration object to handle YAML files.

Return type:

Config

darwin.utils.utils.validate_file_against_schema(path: Path) → List[source]
darwin.utils.utils.validate_data_against_schema(data) → List[source]
darwin.utils.utils.attempt_decode(path: Path) → dict[source]
darwin.utils.utils.load_data_from_file(path: Path) → Tuple[dict, AnnotationFileVersion][source]
darwin.utils.utils.parse_darwin_json(path: Path, count: int | None = None) → AnnotationFile | None[source]

Parses the given JSON file in v7’s darwin proprietary format. Works for images, split frame videos (treated as images) and playback videos.

Parameters:

  • path (Path) – Path to the file to parse.

  • count (Optional[int]) – Optional count parameter. Used only if the β€˜s image sequence is None.

Returns:

An AnnotationFile with the information from the parsed JSON file, or None, if there were no annotations in the JSON.

Return type:

Optional[dt.AnnotationFile]

Raises:

OutdatedDarwinJSONFormat – If the given darwin video JSON file is missing the β€˜width’ and β€˜height’ keys in the β€˜image’ dictionary.

darwin.utils.utils.stream_darwin_json(path: Path) → PersistentStreamingJSONObject[source]

Returns a Darwin JSON file as a persistent stream. This allows for parsing large files without loading them entirely into memory.

Parameters:

path (Path) – Path to the file to parse.

Returns:

A stream of the JSON file.

Return type:

PersistentStreamingJSONObject

darwin.utils.utils.get_image_path_from_stream(darwin_json: PersistentStreamingJSONObject, images_dir: Path, annotation_filepath: Path, with_folders: bool = True) → Path[source]

Returns the path to the image file associated with the given darwin json file. Compatible with Darwin JSON V2, as well as releases in folders and flat structures.

Parameters:

  • darwin_json (PersistentStreamingJSONObject) – A stream of the JSON file.

  • images_dir (Path) – Path to the directory containing the images.

  • with_folders (bool) – Flag to determine if the release was pulled with or without folders.

  • annotation_filepath (Path) – Path to the annotation file. Used if loading the JSON as a stream fails.

Returns:

Path to the image file.

Return type:

Path

darwin.utils.utils.is_stream_list_empty(json_list: PersistentStreamingJSONList) → bool[source]
darwin.utils.utils.get_annotations_in_slot(slot_name: str, annotations: Sequence[Annotation | VideoAnnotation]) → List[Annotation | VideoAnnotation][source]
darwin.utils.utils.make_keyframe_annotation(annotation_type: str | None, annotation_data: Dict | None, name: str, slot_names: List[str]) → Annotation[source]
darwin.utils.utils.update_annotation_data(main_annotation_data: Dict[str, Any], annotation_type: str | None, annotation_data: Dict | None) → Tuple[str | None, Dict | None][source]
darwin.utils.utils.get_annotation_type_and_data(frame: Dict, annotation_type: str, annotation_data: Dict) → Tuple[str | None, Dict | None][source]

Returns the type of a given video annotation and its data.

darwin.utils.utils.split_video_annotation(annotation: AnnotationFile) → List[AnnotationFile][source]

Splits the given video AnnotationFile into several video AnnotationFile``s, one for each ``frame_url.

Parameters:

annotation (dt.AnnotationFile) – The video AnnotationFile we want to split.

Returns:

A list with the split video AnnotationFiles.

Return type:

List[dt.AnnotationFile]

Raises:

AttributeError – If the given AnnotationFile is not a video annotation, or if the given annotation has no frame_url attribute.

darwin.utils.utils.parse_slot_names(annotation: dict) → List[str][source]
darwin.utils.utils.ispolygon(annotation: AnnotationClass) → bool[source]

Returns whether or not the given AnnotationClass is a polygon.

Parameters:

annotation (AnnotationClass) – The AnnotationClass to evaluate.

Return type:

True is the given AnnotationClass is a polygon, False otherwise.

darwin.utils.utils.convert_polygons_to_sequences(polygons: List[List[Dict[str, float]] | List[List[Dict[str, float]]]], height: int | None = None, width: int | None = None, rounding: bool = True) → List[List[int | float]][source]

Converts a list of polygons, encoded as a list of dictionaries of into a list of nd.arrays of coordinates.

Parameters:

  • polygons (Iterable[dt.Polygon]) – Non empty list of coordinates in the format [{x: x1, y:y1}, ..., {x: xn, y:yn}] or a list of them as [[{x: x1, y:y1}, ..., {x: xn, y:yn}], ..., [{x: x1, y:y1}, ..., {x: xn, y:yn}]].

  • height (Optional[int], default: None) – Maximum height for a polygon coordinate.

  • width (Optional[int], default: None) – Maximum width for a polygon coordinate.

  • rounding (bool, default: True) – Whether or not to round values when creating sequences.

Returns:

sequences – List of arrays of coordinates in the format [[x1, y1, x2, y2, …, xn, yn], …, [x1, y1, x2, y2, …, xn, yn]]

Return type:

List[ndarray[float]]

Raises:

ValueError – If the given list is a falsy value (such as []) or if it’s structure is incorrect.

darwin.utils.utils.convert_xyxy_to_bounding_box(box: List[int | float]) → Dict[str, float][source]

Converts a list of xy coordinates representing a bounding box into a dictionary. This is used by in-platform model training.

Parameters:

box (List[Union[int, float]]) – List of arrays of coordinates in the format [x1, y1, x2, y2]

Returns:

Bounding box in the format {x: x1, y: y1, h: height, w: width}.

Return type:

BoundingBox

Raises:

ValueError – If box has an incorrect format.

darwin.utils.utils.convert_polygons_to_mask(polygons: List, height: int, width: int, value: int | None = 1) → ndarray[source]

Converts a list of polygons, encoded as a list of dictionaries into an nd.array mask.

Parameters:

  • polygons (list) – List of coordinates in the format [{x: x1, y:y1}, ..., {x: xn, y:yn}] or a list of them as [[{x: x1, y:y1}, ..., {x: xn, y:yn}], ..., [{x: x1, y:y1}, ..., {x: xn, y:yn}]].

  • height (int) – The maximum height for the created mask.

  • width (int) – The maximum width for the created mask.

  • value (Optional[int], default: 1) – The drawing value for upolygon.

Returns:

ndarray mask of the polygon(s).

Return type:

ndarray

darwin.utils.utils.chunk(items: List[Any], size: int) → Iterator[Any][source]

Splits the given list into chunks of the given size and yields them.

Parameters:

  • items (List[Any]) – The list of items to be split.

  • size (int) – The size of each split.

Yields:

Iterator[Any] – A chunk of the of the given size.

darwin.utils.utils.is_unix_like_os() → bool[source]

Returns True if the executing OS is Unix-based (Ubuntu or MacOS, for example) or False otherwise.

Returns:

True for Unix-based systems, False otherwise.

Return type:

bool

darwin.utils.utils.has_json_content_type(response: Response) → bool[source]

Returns True if response has application/json content type or False otherwise.

Returns:

True for application/json content type, False otherwise.

Return type:

bool

darwin.utils.utils.get_response_content(response: Response) → Any[source]

Returns json content if response has application/json content-type, otherwise returns text.

Returns:

Json or text content.

Return type:

Any

darwin.utils.utils.get_annotation_files_from_dir(path: Path) → Iterator[str][source]

Returns an iterator of all the JSON annotation files in the given directory. Ignores the .v7/metadata.json properties manifest file if present.

Parameters:

path (Path) – The directory to search for JSON annotation files.

Returns:

An iterator of all the JSON annotation files in the given directory.

Return type:

Iterator[str]

darwin.utils.utils.convert_sequences_to_polygons(sequences: List[List[int] | List[float]], height: int | None = None, width: int | None = None) → Dict[str, List[List[Dict[str, float]]]][source]

Converts a list of polygons, encoded as a list of dictionaries of into a list of nd.arrays of coordinates. This is used by the backend.

Parameters:

  • sequences (List[Union[List[int], List[float]]]) – List of arrays of coordinates in the format [x1, y1, x2, y2, ..., xn, yn] or as a list of them as [[x1, y1, x2, y2, ..., xn, yn], ..., [x1, y1, x2, y2, ..., xn, yn]].

  • height (Optional[int], default: None) – Maximum height for a polygon coordinate.

  • width (Optional[int], default: None) – Maximum width for a polygon coordinate.

Returns:

Dictionary with the key path containing a list of coordinates in the format of [[{x: x1, y:y1}, ..., {x: xn, y:yn}], ..., [{x: x1, y:y1}, ..., {x: xn, y:yn}]].

Return type:

Dict[str, List[dt.Polygon]]

Raises:

ValueError – If sequences is a falsy value (such as []) or if it is in an incorrect format.

Module contents