darwin package

Subpackages

Submodules

darwin.backend_v2 module

darwin.backend_v2.inject_default_team_slug(method: Callable) → Callable[source]: Injects team_slug if not specified

class darwin.backend_v2.BackendV2(client: Client, default_team)[source]

Bases: object

register_data(*args, **kwargs) → Callable[source]

sign_upload(*args, **kwargs) → Callable[source]

confirm_upload(*args, **kwargs) → Callable[source]

fetch_items(*args, **kwargs) → Callable[source]

archive_items(*args, **kwargs) → Callable[source]

restore_archived_items(*args, **kwargs) → Callable[source]

move_to_stage(*args, **kwargs) → Callable[source]

get_dataset(*args, **kwargs) → Callable[source]

get_workflow(*args, **kwargs) → Callable[source]

delete_items(*args, **kwargs) → Callable[source]

export_dataset(*args, **kwargs) → Callable[source]

get_exports(dataset_slug, *, team_slug: str | None = None)[source]

post_comment(*args, **kwargs) → Callable[source]

import_annotation(*args, **kwargs) → Callable[source]

register_items(*args, **kwargs) → Callable[source]

darwin.cli module

darwin.cli.main() → None[source]

Executes the main function of program.

Raises:

Unauthorized – If the API key with which the use is authenticated does not grant access for the given action.
Unauthenticated – If a given action needs authentication and you are not authenticated.
InvalidTeam – If you are trying to use a team that is not specified in the configuration file. To fix this please authenticate with the given team first.
requests.exceptions.ConnectionError – If there is a connection issue.

darwin.cli_functions module

darwin.cli_functions.validate_api_key(api_key: str) → None[source]

Validates the given API key. Exits the application if it fails validation.

Parameters:: api_key (str) – The API key to be validated.

darwin.cli_functions.authenticate(api_key: str, default_team: str | bool | None = None, datasets_dir: Path | None = None) → Config[source]

Authenticate the API key against the server and creates a configuration file for it.

Parameters:

api_key (str) – API key to use for the client login.
default_team (Optional[bool]) – Flag to make the team the default one. Defaults to None.
datasets_dir (Optional[Path]) – Dataset directory on the file system. Defaults to None.

Returns:

Config
A configuration object to handle YAML files.

darwin.cli_functions.current_team() → None[source]: Print the team currently authenticated against.

darwin.cli_functions.list_teams() → None[source]: Print a table of teams to which the client belong to.

darwin.cli_functions.set_team(team_slug: str) → None[source]

Switches the client to the selected team and persist the change on the configuration file.

Parameters:: team_slug (str) – Slug of the team to switch to.

darwin.cli_functions.set_compression_level(compression_level: int) → None[source]

Change the compression level of text/json contents sent to Darwin APIs and persist the change on the configuration file.

Can be in range from 0 - no compression, to 9 - best compression. By default, 0 is used.

Parameters:: compression_level (int) – Compression level to use.

darwin.cli_functions.create_dataset(dataset_slug: str) → None[source]

Creates a dataset remotely. Exits the application if the dataset’s name is already taken or is not valid.

Parameters:: dataset_slug (str) – Slug of the new dataset.

darwin.cli_functions.local(team: str | None = None) → None[source]

Lists synced datasets, stored in the specified path.

Parameters:: team (Optional[str]) – The name of the team to list, or the default one if no team is given. Defaults to None.

darwin.cli_functions.path(dataset_slug: str) → Path[source]

Returns the absolute path of the specified dataset. Exits the application if the dataset does not exist locally.

Parameters:: dataset_slug (str) – The dataset’s slug.
Returns:: The absolute path of the dataset.
Return type:: Path

darwin.cli_functions.url(dataset_slug: str) → None[source]

Prints the url of the specified dataset. Exits the application if no dataset was found.

Parameters:: dataset_slug (str) – The dataset’s slug.

darwin.cli_functions.export_dataset(dataset_slug: str, include_url_token: bool, name: str, annotation_class_ids: List[str] | None = None, include_authorship: bool = False, version: str | None = None) → None[source]

Create a new release for the dataset.

Parameters:

dataset_slug (str) – Slug of the dataset to which we perform the operation on.
include_url_token (bool, default: False) – If True includes the url token, if False does not.
name (str) – Name of the release.
annotation_class_ids (Optional[List[str]], default: None) – List of the classes to filter.
include_authorship (bool, default: False) – If True include annotator and reviewer metadata for each annotation.
version (Optional[str], default: None) – When used for V2 dataset, allows to force generation of either Darwin JSON 1.0 (Legacy) or newer 2.0. Ommit this option to get your team’s default.

darwin.cli_functions.pull_dataset(dataset_slug: str, only_annotations: bool = False, folders: bool = True, video_frames: bool = False, force_slots: bool = False, ignore_slots: bool = False, no_folders: bool = False, retry: bool = False, retry_timeout: int = 600, retry_interval: int = 10) → None[source]

Downloads a remote dataset (images and annotations) in the datasets directory. Exits the application if dataset is not found, the user is not authenticated, there are no releases or the export format for the latest release is not supported.

Parameters:

dataset_slug (str) – Slug of the dataset to which we perform the operation on.
only_annotations (bool) – Download only the annotations and no corresponding images. Defaults to False.
folders (bool) – Recreates the folders in the dataset. Defaults to True.
video_frames (bool) – Pulls video frames images instead of video files. Defaults to False.
force_slots (bool) – Pulls all slots of items into deeper file structure ({prefix}/{item_name}/{slot_name}/{file_name})
no_folders (bool) – Does not recreate the folders in the dataset. Defaults to False.
retry (bool) – If True, will repeatedly try to download the release if it is still processing until the timeout is reached.
retry_timeout (int) – If retrying, total time to wait for the release to be ready for download
retry_interval (int) – If retrying, time to wait between retries of checking if the release is ready for download.

darwin.cli_functions.split(dataset_slug: str, val_percentage: float, test_percentage: float, seed: int = 0) → None[source]

Splits a local version of a dataset into train, validation, and test partitions.

Parameters:

dataset_slug (str) – Slug of the dataset to which we perform the operation on.
val_percentage (float) – Percentage in the validation set.
test_percentage (float) – Percentage in the test set.
seed (int) – Random seed. Defaults to 0.

darwin.cli_functions.list_remote_datasets(all_teams: bool, team: str | None = None) → None[source]

Lists remote datasets with its annotation progress.

Parameters:

all_teams (bool) – If True, lists remote datasets from all teams, if False, lists only datasets from the given Team.
team (Optional[str]) – Name of the team with the datasets we want to see. Uses the default Team is non is given. Defaults to None.

darwin.cli_functions.remove_remote_dataset(dataset_slug: str) → None[source]

Remove a remote dataset from the workview. The dataset gets archived. Exits the application if no dataset with the given slug were found.

Parameters:: dataset_slug (str) – The dataset’s slug.

darwin.cli_functions.dataset_list_releases(dataset_slug: str) → None[source]

Lists all the releases from the given dataset. Exits the application if no dataset with the given slug were found.

Parameters:: dataset_slug (str) – The dataset’s slug.

darwin.cli_functions.upload_data(dataset_identifier: str, files: List[str | Path | LocalFile] | None, files_to_exclude: List[str | Path] | None, fps: int, path: str | None, frames: bool, extract_views: bool = False, handle_as_slices: bool = False, preserve_folders: bool = False, verbose: bool = False, item_merge_mode: str | None = None) → None[source]

Uploads the provided files to the remote dataset. Exits the application if no dataset with the given name is found, the files in the given path have unsupported formats, or if there are no files found in the given Path.

Parameters:

dataset_identifier (str) – Slug of the dataset to retrieve.
files (List[Union[PathLike, LocalFile]]) – List of files to upload. Can be None.
files_to_exclude (List[PathLike]) – List of files to exclude from the file scan (which is done only if files is None).
fps (int) – Frame rate to split videos in.
path (Optional[str]) – If provided; files will be placed under this path in the v7 platform. If preserve_folders is True then it must be possible to draw a relative path from this folder to the one the files are in, otherwise an error will be raised.
frames (bool) – Specify whether the files will be uploaded as a list of frames or not.
extract_views (bool) – If providing a volume, specify whether to extract the orthogonal views or not.
handle_as_slices (bool) – Whether to upload DICOM files as slices
preserve_folders (bool) – Specify whether or not to preserve folder paths when uploading.
verbose (bool) – Specify whether to have full traces print when uploading files or not.
item_merge_mode (Optional[str]) –
If set, each file path passed to files_to_upload behaves as follows: - Paths pointing directly to individual files are ignored - Paths pointing to folders of files will be uploaded according to the following mode rules.
Note that folders will not be recursively searched, so only files in the first level of the folder will be uploaded:
”slots”: Each file in the folder will be uploaded to a different slot of the same item.

”series”: All .dcm files in the folder will be concatenated into a single slot. All other files are ignored.

”channels”: Each file in the folder will be uploaded to a different channel of the same item.

darwin.cli_functions.dataset_import(dataset_slug: str, format: str, files: List[str | Path], append: bool, class_prompt: bool = True, delete_for_empty: bool = False, import_annotators: bool = False, import_reviewers: bool = False, overwrite: bool = False, use_multi_cpu: bool = False, cpu_limit: int | None = None) → None[source]

Imports annotation files to the given dataset. Exits the application if no dataset with the given slug is found.

Parameters:

dataset_slug (str) – The dataset’s slug.
format (str) – Format of the export files.
files (List[PathLike]) – List of where the files are.
append (bool, default: True) – If True it appends the annotation from the files to the dataset, if False it will override the dataset’s current annotations with the ones from the given files. Incompatible with delete-for-empty.
delete_for_empty (bool, default: False) – If True will use empty annotation files to delete all annotations from the remote file. If False, empty annotation files will simply be skipped. Only works for V2 datasets. Incompatible with append.
import_annotators (bool, default: False) – If True it will import the annotators from the files to the dataset, if available. If False it will not import the annotators.
import_reviewers (bool, default: False) – If True it will import the reviewers from the files to the dataset, if . If False it will not import the reviewers.
overwrite (bool, default: False) – If True it will bypass a warning that the import will overwrite the current annotations if any are present. If False this warning will be skipped and the import will overwrite the current annotations without warning.
use_multi_cpu (bool, default: False) – If True it will use all multiple CPUs to speed up the import process.
cpu_limit (Optional[int], default: Core count - 2) – The maximum number of CPUs to use for the import process.

darwin.cli_functions.list_files(dataset_slug: str, statuses: str | None, path: str | None, only_filenames: bool, sort_by: str | None = 'updated_at:desc') → None[source]

List all file from the given dataset. Exits the application if it finds unknown file statuses, if no dataset with the given slug is found or if another general error occurred.

Parameters:

dataset_slug (str) – The dataset’s slug.
statuses (Optional[str]) – Only list files with the given statuses. Valid statuses are: ‘annotate’, ‘archived’, ‘complete’, ‘new’, ‘review’.
path (Optional[str]) – Only list files whose Path matches.
only_filenames (bool) – If True, only prints the filenames, if False it prints the full file url.
sort_by (Optional[str]) – Sort order for listing files. Defaults to ‘updated_at:desc’.

darwin.cli_functions.set_file_status(dataset_slug: str, status: str, files: List[str]) → None[source]

Sets the status of the given files from the given dataset. Exits the application if the given status is unknown or if no dataset was found.

Parameters:

dataset_slug (str) – The dataset’s slug.
status (str) – The new status for the files.
files (List[str]) – Names of the files we want to update.

darwin.cli_functions.delete_files(dataset_slug: str, files: List[str], skip_user_confirmation: bool = False) → None[source]

Deletes the files from the given dataset. Exits the application if no dataset with the given slug is found or a general error occurs.

Parameters:

dataset_slug (str) – The dataset’s slug.
files (List[str]) – The list of filenames to delete.
skip_user_confirmation (bool, default: False) – If True, skips user confirmation, if False it will prompt the user.

darwin.cli_functions.validate_schemas(location: str, pattern: bool = False, silent: bool = False, output: Path | None = None) → None[source]

Validate function for the CLI. Takes one of 3 required key word arguments describing the location of files and prints and/or saves an output

Parameters:

location (str) – str path to a folder or file location to search
pattern (bool, optional) – glob style pattern matching, by default None
silent (bool, optional) – flag to set silent console printing, only showing errors, by default False
output (Optional[Path], optional) – filename for saving to output, by default None

darwin.cli_functions.dataset_convert(dataset_identifier: str, format: str, output_dir: str | Path | None = None) → None[source]

Converts the annotations from the given dataset to the given format. Exits the application if no dataset with the given slug exists or no releases for the dataset were previously pulled.

Parameters:

dataset_identifier (str) – The dataset identifier, normally in the “<team-slug>/<dataset-slug>:<version>” form.
format (str) – The format we want to convert to.
output_dir (Optional[PathLike], default: None) – The folder where the exported annotation files will be. If None it will be the inside the annotations folder of the dataset under ‘other_formats/{format}’.

darwin.cli_functions.convert(format: str, files: List[str | Path], output_dir: Path) → None[source]

Converts the given files to the specified format.

Parameters:

format (str) – The target format to export to.
files (List[PathLike]) – List of files to be converted.
output_dir (Path) – Folder where the exported annotations will be placed.

darwin.cli_functions.post_comment(dataset_slug: str, filename: str, text: str, x: float = 1, y: float = 1, w: float = 1, h: float = 1) → None[source]

Creates a comment box with a comment for the given file in the given dataset.

Parameters:

dataset_slug (str) – The slug of the dataset the item belongs to.
filename (str) – The filename to receive the comment.
text (str) – The comment.
x (float, default: 1) – X value of the top left coordinate for the comment box.
y (float, default: 1) – Y value of the top left coordinate for the comment box.
w (float, default: 1) – Width of the comment box.
h (float, default: 1) – Height of the comment box.

Raises:

NotFound – If the Dataset was not found.

darwin.cli_functions.report_annotators(dataset_slugs: list[str], start: datetime, stop: datetime, group_by: list[AnnotatorReportGrouping], pretty: bool) → None[source]

Prints an annotators report in CSV format.

Parameters:

dataset_slugs (list[str]) – Slugs of datasets to include in the report.
start (datetime.datetime) – Timezone aware report start DateTime.
stop (datetime.datetime) – Timezone aware report end DateTime.
group_by (list[AnnotatorReportGrouping]) – Non-empty list of grouping options for the report.
pretty (bool) – If True, it will print the output in a Rich formatted table.

darwin.cli_functions.help(parser: ArgumentParser, subparser: str | None = None) → None[source]

Prints the help text for the given command.

Parameters:

parser (argparse.ArgumentParser) – The parser used to read input from the user.
subparser (Optional[str]) – Actions from the parser to be processed. Defaults to None.

darwin.cli_functions.print_new_version_info(client: Client | None = None) → None[source]

Prints a message informing the user of a new darwin-py version. Does nothing if no new version is available or if no client is provided.

Parameters:: client (Optional[Client]) – The client containing information about the new version. Defaults to None.

darwin.cli_functions.extract_video_artifacts(source_file: str, output_dir: str, storage_key_prefix: str, *, fps: float = 0.0, segment_length: int = 2, repair: bool = False, save_metadata: bool = True) → None[source]

Generate video artifacts (segments, sections, thumbnail, frames manifest).

Parameters:

source_file (str) – Path to input video file
output_dir (str) – Output directory for artifacts
storage_key_prefix (str) – Storage key prefix for generated files
fps (float, optional) – Desired output FPS (0.0 for native), by default 0.0
segment_length (int, optional) – Length of each segment in seconds, by default 2
repair (bool, optional) – Whether to attempt to repair video if errors are detected, by default False
save_metadata (bool, optional) – Whether to save metadata to a file, by default False

darwin.client module

exception darwin.client.JobPendingException[source]

Bases: Exception

Raised when a requested job is not finished or failed.

darwin.client.log_rate_limit_exceeded(retry_state: RetryCallState)[source]

darwin.client.retry_if_status_code_429(retry_state: RetryCallState)[source]

darwin.client.retry_if_status_code_429_or_5xx(retry_state: RetryCallState) → bool[source]

Determines if a request should be retried based on the response status code.

Retries on: - Rate limit (429) - Server errors (500, 502, 503, 504)

Parameters:: retry_state (RetryCallState) – The current state of the retry mechanism
Returns:: True if the request should be retried, False otherwise
Return type:: bool

darwin.client.log_retry_error(retry_state: RetryCallState) → None[source]

Logs information about why a request is being retried.

Parameters:: retry_state (RetryCallState) – The current state of the retry mechanism

class darwin.client.Client(config: Config, default_team: str | None = None, log: Logger | None = None)[source]

Bases: object

list_local_datasets(team_slug: str | None = None) → Iterator[Path][source]

Returns a list of all local folders which are detected as dataset.

Parameters:: team_slug (Optional[str]) – The team slug of the dataset. Defaults to None.
Returns:: Iterator of all local datasets
Return type:: Iterator[Path]

list_remote_datasets(team_slug: str | None = None) → Iterator[RemoteDatasetV2][source]

Returns a list of all available datasets with the team currently authenticated against.

Parameters:: team_slug (Optional[str]) – The team slug of the dataset. Defaults to None.
Returns:: List of all remote datasets
Return type:: Iterator[RemoteDataset]

get_remote_dataset(dataset_identifier: str | DatasetIdentifier) → RemoteDatasetV2[source]

Get a remote dataset based on its identifier.

Parameters:: dataset_identifier (Union[str, DatasetIdentifier]) – Identifier of the dataset. Can be the string version or a DatasetIdentifier object.
Returns:: Initialized dataset.
Return type:: RemoteDataset
Raises:: NotFound – If no dataset with the given identifier was found.

create_dataset(name: str, team_slug: str | None = None) → RemoteDataset[source]

Create a remote dataset.

Parameters:

name (str) – Name of the dataset to create.
team_slug (Optional[str]) – Team slug of the team the dataset will belong to. Defaults to None.

Returns:

The created dataset.

Return type:

RemoteDataset

archive_remote_dataset(dataset_id: int, team_slug: str) → None[source]

Archive (soft delete) a remote dataset.

Parameters:

dataset_id (int) – Id of the dataset to archive.
team_slug (str) – Team slug of the dataset.

fetch_remote_classes(team_slug: str | None = None) → List[Dict[str, Any]][source]

Fetches all remote classes on the remote dataset.

Parameters:: team_slug (Optional[str]) – The team slug of the dataset. Defaults to None.
Returns:: None if no information about the team is found, a List of Annotation classes otherwise.
Return type:: Dict[str, UnknownType]
Raises:: ValueError – If no team was found.

update_annotation_class(class_id: int, payload: Dict[str, Any]) → Dict[str, Any][source]

Updates the AnnotationClass with the given id.

Parameters:

class_id (int) – The id of the AnnotationClass to update.
payload (Dict[str, UnknownType]) – A dictionary with the changes to perform.

Returns:

A dictionary with the result of the operation.

Return type:

Dict[str, UnknownType]

create_annotation_class(dataset_id: int, type_ids: List[int], name: str) → Dict[str, Any][source]

Creates an AnnotationClass.

Parameters:

dataset_id (int) – The id of the dataset this AnnotationClass will belong to originaly.
type_ids (List[int]) – A list of type ids for the annotations this class will hold.
name (str) – The name of the annotation class.

Returns:

A dictionary with the result of the operation.

Return type:

Dict[str, UnknownType]

fetch_remote_attributes(dataset_id: int) → List[Dict[str, Any]][source]

Fetches all attributes remotely.

Parameters:: dataset_id (int) – The id of the dataset with the attributes we want to fetch.
Returns:: A List with the attributes, where each attribute is a dictionary.
Return type:: List[Dict[str, UnknownType]]

load_feature_flags(team_slug: str | None = None) → None[source]

Loads in memory the set of current features enabled for a team.

Parameters:: team_slug (Optional[str]) – Team slug of the team the dataset will belong to. Defaults to None.

get_team_features(team_slug: str) → List[Feature][source]

Gets all the features for the given team together with their statuses.

Parameters:: team_slug (Optional[str]) – Team slug of the team the dataset will belong to. Defaults to None.
Returns:: List of Features for the given team.
Return type:: List[Feature]

feature_enabled(feature_name: str, team_slug: str | None = None) → bool[source]

Returns whether or not a given feature is enabled for a team.

Parameters:

feature_name (str) – The name of the feature.
team_slug (Optional[str]) – Team slug of the team the dataset will belong to. Defaults to None.

Returns:

False if the given feature is not enabled OR the team was not found. True otherwise.

Return type:

bool

get_datasets_dir(team_slug: str | None = None) → str[source]

Gets the dataset directory of the specified team or the default one

Parameters:: team_slug (Optional[str]) – Team slug of the team the dataset will belong to. Defaults to None.
Returns:: Path of the datasets for the selected team or the default one, or None if the Team was not found.
Return type:: str
Raises:: ValueError – If no team was found.

set_datasets_dir(datasets_dir: Path, team_slug: str | None = None) → None[source]

Sets the dataset directory of the specified team or the default one.

Parameters:

datasets_dir (Path) – Path to set as dataset directory of the team.
team_slug (Optional[str]) – Team slug of the team the dataset will belong to. Defaults to None.

annotation_types() → List[Dict[str, Any]][source]

Returns a list of annotation types.

Returns:: A list with the annotation types as dictionaries.
Return type:: List[Dict[str, UnknownType]]

get_annotators_report(dataset_ids: list[int], start: datetime, stop: datetime, group_by: list[AnnotatorReportGrouping], team_slug: str | None = None) → Response[source]

Gets annotators the report for the given dataset.

Parameters:

dataset_ids (int) – Ids of the datasets to include in the report.
start (datetime.datetime) – Timezone aware report start DateTime
stop (datetime.datetime) – Timezone aware report end DateTime
group_by (list[AnnotatorReportGrouping]) – Non-empty list of grouping options for the report.
team_slug (Optional[str]) – Team slug of the team the dataset will belong to. Defaults to None.

Returns:

The raw response of the report (CSV format) or None if the Team was not found.

Return type:

Response

Raises:

ValueError – If no team was found. If start or stop parameters are not timezone aware. If no group_by options provided.

poll_pending_report_job(team_slug: str, job_id: str) → ReportJob[source]

fetch_binary(url: str) → Response[source]

Fetches binary data from the given url via a stream.

Parameters:: url (str) – The full url to download the binary data.
Returns:: request’s Response object.
Return type:: Response

classmethod local(team_slug: str | None = None) → Client[source]

Factory method to use the default configuration file to init the client

Parameters:: team_slug (Optional[str]) – Team slug of the team the dataset will belong to. Defaults to None.
Returns:: The initialized client.
Return type:: Client

classmethod from_config(config_path: Path, team_slug: str | None = None) → Client[source]

Factory method to create a client from the configuration file passed as parameter

Parameters:

config_path (str) – Path to a configuration file to use to create the client
team_slug (Optional[str]) – Team slug of the team the dataset will belong to. Defaults to None.

Returns:

The initialized client.

Return type:

Client

classmethod from_guest(datasets_dir: Path | None = None) → Client[source]

Factory method to create a client and access datasets as a guest.

Parameters:: datasets_dir (Optional[Path]) – String where the client should be initialized from (aka the root path). Defaults to None.
Returns:: The initialized client.
Return type:: Client

classmethod from_api_key(api_key: str, datasets_dir: Path | None = None) → Client[source]

Factory method to create a client given an API key.

Parameters:

api_key (str) – API key to use to authenticate the client
datasets_dir (Optional[Path]) – String where the client should be initialized from (aka the root path). Defaults to None.

Returns:

The initialized client.

Return type:

Client

static default_api_url() → str[source]

Returns the default api url.

Returns:: The default api url.
Return type:: str

static default_base_url() → str[source]

Returns the default base url.

Returns:: The default base url.
Return type:: str

property newer_darwin_version: Tuple[int, int, int] | None

property api_v2: BackendV2

get_team_properties(team_slug: str | None = None, include_property_values: bool = True) → List[FullProperty][source]

create_property(team_slug: str | None, params: FullProperty | Dict[str, Any]) → FullProperty[source]

update_property(team_slug: str | None, params: FullProperty | Dict[str, Any]) → FullProperty[source]

get_external_storage(team_slug: str | None = None, name: str | None = None) → ObjectStore[source]

Get an external storage connection by name.

If no name is provided, the default team’s external storage connection will be returned.

Parameters:

team_slug (Optional[str]) – The team slug.
name (Optional[str]) – The name of the external storage connection.

Returns:

The external storage connection with the given name.

Return type:

Optional[ObjectStore]

Raises:

ValueError – If no external storage connection is found in the team.
ValueError – If no name is provided and the default external storage connection is read-only.
ValueError – If provided connection name is read-only.

list_external_storage_connections(team_slug: str) → List[ObjectStore][source]

Returns a list of all available external storage connections.

Parameters:: team_slug (str) – The team slug.
Returns:: A list of all available external storage connections.
Return type:: List[ObjectStore]

darwin.config module

class darwin.config.Config(path: str | Path | None = None)[source]

Bases: object

Handles YAML configuration files

get(key: str | List[str], default: Any | None = None) → Any[source]

Gets the value defined by key.

Parameters:

key (Union[str, List[str]]) – The key where the value to be fetched is stored. It can be formatted as a simple string, or as a path/like/string to fetch nested values.
default (Optional[Any]) – A default value in case the given key returns None. Defaults to None.

Returns:

The value stored by the key.

Return type:

Any

put(key: str | List[str], value: Any, save: bool = True) → None[source]

Sets the value for the specified key.

Parameters:

key (Union[str, List[str]]) – The key where the value is going to be stored. It can be formatted as a simple string, or as a path/like/string to fetch nested values.
value (Any) – The value to be stored.
save (bool) – If True, persists the value in the FileSystem. Defaults to True.

set_team(team: str, api_key: str, datasets_dir: str) → None[source]

Stores the API key and Dataset directory for the given team.

Parameters:

team (str) – The name of the team.
api_key (str) – The API key the user has to do actions in the given team.
datasets_dir (str) – The directory to stores datasets from the given team.

set_default_team(team: str) → None[source]

Sets the given team as the default one.

Parameters:: team (str) – The team’s slug.
Raises:: InvalidTeam – If the given team is not in the configuration file. Authenticate with this team first to avoid this issue.

set_compression_level(level: int) → None[source]

Sets the given compression level globally.

Parameters:: level (int) – The compression level.
Raises:: InvalidCompressionLevel – Compression level is out of supported range. Use number from 0 to 9 to avoid this issue.

set_global(api_endpoint: str, base_url: str, default_team: str | None = None) → None[source]

Stores the url to access teams. If a default team is given, it also stores that team as the globaly default one.

Parameters:

api_endpoint (str) – The ‘/api’ endpoint from V7’s API.
base_url (str) – The base URL for V7 together with protocol.
default_team (Optional[str]) – The default team’s slug. Defaults to None.

get_team(team: str | None = None, raise_on_invalid_team: bool = True) → Team | None[source]

Returns the Team object from the team with the given slug if an authentication with an API key was performed earlier.

Parameters:

team (Optional[str]) – The Team’s slug. If none is given, searches for the default team instead. Defaults to None.
raise_on_invalid_team (bool) – If True, raises if no team is found, if False returns None instead. Defaults to True.

Returns:

The team or None if no API key for the team was found and raise_on_invalid_team is False.

Return type:

Optional[Team]

Raises:

InvalidTeam – If the user has not been authenticated with an API key for they given team.

get_default_team(raise_on_invalid_team: bool = True) → Team | None[source]

Returns the default Team if one is defined.

Parameters:: raise_on_invalid_team (bool) – If True, raises if no default team is found, if False returns None instead. Defaults to True.
Returns:: The team or None if no default team is set and raise_on_invalid_team is False.
Return type:: Optional[Team]
Raises:: InvalidTeam – If the user has not set a default team.

get_all_teams() → List[Team][source]

Returns a list of all teams saved in the configuration file.

Returns:: The list of teams saved.
Return type:: List[Team]

darwin.datatypes module

class darwin.datatypes.Success(value)[source]

Bases: Enum

An enumeration.

SUCCESS = 1

FAILURE = 2

PARTIAL_SUCCESS = 3

UNDETERMINED = 4

class darwin.datatypes.JSONType(**kwargs: Dict[str, Any])[source]

Bases: object

to_json() → Dict[str, Any][source]

classmethod from_json(json: Dict[str, Any]) → JSONType[source]

classmethod from_dict(json: Dict[str, Any]) → JSONType[source]

darwin.datatypes.sorted_nested_lists(obj: Any) → Any[source]

class darwin.datatypes.SubAnnotationType(value)[source]

Bases: str, Enum

An enumeration.

TEXT = 'text'

ATTRIBUTES = 'attributes'

INSTANCE_ID = 'instance_id'

INFERENCE = 'inference'

DIRECTIONAL_VECTOR = 'directional_vector'

MEASURES = 'measures'

class darwin.datatypes.Team(default: bool, slug: str, datasets_dir: str, api_key: str, selected: bool = False)[source]

Bases: object

Definition of a V7 team.

default: bool: If this is the default Team or not.

slug: str: This team’s slug.

datasets_dir: str: The path to the directory of all datasets this teams contains.

api_key: str: The API key used to authenticate for this Team.

selected: bool = False: If this is the currently active Team. Defaults to False.

class darwin.datatypes.Feature(name: str, enabled: bool)[source]

Bases: object

Structured payload of a Feature record on V7 Darwin.

name: str: The name of this Feature.

enabled: bool: Whether or not this Feature is enabled Disabled Features do nothing, as if they didn’t exist.

class darwin.datatypes.AnnotationClass(name: str, annotation_type: Literal['bounding_box', 'polygon', 'ellipse', 'cuboid', 'segmentation', 'raster_layer', 'mask', 'keypoint', 'tag', 'line', 'skeleton', 'table', 'string', 'graph'], annotation_internal_type: str | None = None)[source]

Bases: object

Represents an AnnocationClass from an Annotation.

name: str: The name of this AnnotationClass.

annotation_type: Literal['bounding_box', 'polygon', 'ellipse', 'cuboid', 'segmentation', 'raster_layer', 'mask', 'keypoint', 'tag', 'line', 'skeleton', 'table', 'string', 'graph']: The type of this AnnotationClass.

annotation_internal_type: str | None = None: The V7 internal type of this AnnotationClass. This is mostly used to convert from types that are known in the outside world by a given name, but then are known inside V7’s lingo by another.

class darwin.datatypes.SubAnnotation(annotation_type: SubAnnotationType, data: Any)[source]

Bases: object

Represents a subannotation that belongs to an AnnotationClass.

annotation_type: SubAnnotationType: The type of this SubAnnotation.

data: Any: Any external data, in any format, relevant to this SubAnnotation. Used for compatibility purposes with external formats.

class darwin.datatypes.AnnotationAuthorRole(value)[source]

Bases: Enum

An enumeration.

ANNOTATOR = 'annotator'

REVIEWER = 'reviewer'

class darwin.datatypes.AnnotationAuthor(name: str, email: str)[source]

Bases: object

Represents an annotation’s author

name: str: Name of the author

email: str: Email of the author

class darwin.datatypes.Annotation(annotation_class: ~darwin.datatypes.AnnotationClass, data: ~typing.Any, subs: ~typing.List[~darwin.datatypes.SubAnnotation] = <factory>, slot_names: ~typing.List[str] = <factory>, annotators: ~typing.List[~darwin.datatypes.AnnotationAuthor] | None = None, reviewers: ~typing.List[~darwin.datatypes.AnnotationAuthor] | None = None, id: str | None = None, properties: list[~darwin.future.data_objects.properties.SelectedProperty] | None = None)[source]

Bases: object

Represents an Annotation from an Image/Video.

annotation_class: AnnotationClass: The AnnotationClass from this Annotation.

data: Any: Any external data, in any format, relevant to this Annotation. Used for compatibility purposes with external formats.

subs: List[SubAnnotation]: List of SubAnnotations belonging to this Annotation.

slot_names: List[str]: V2 slots this annotation belogs to

annotators: List[AnnotationAuthor] | None = None: Authorship of the annotation (annotators)

reviewers: List[AnnotationAuthor] | None = None: Authorship of the annotation (reviewers)

id: str | None = None

properties: list[SelectedProperty] | None = None

get_sub(annotation_type: str) → SubAnnotation | None[source]

Returns the first SubAnnotation that matches the given type.

Parameters:: annotation_type (str) – The type of the subannotation.
Returns:: A SubAnnotation found, or None if none was found.
Return type:: Optional[SubAnnotation]

class darwin.datatypes.VideoAnnotation(annotation_class: ~darwin.datatypes.AnnotationClass, frames: ~typing.Dict[int, ~typing.Any], keyframes: ~typing.Dict[int, bool], segments: ~typing.List[~typing.List[int]], interpolated: bool, slot_names: ~typing.List[str] = <factory>, annotators: ~typing.List[~darwin.datatypes.AnnotationAuthor] | None = None, reviewers: ~typing.List[~darwin.datatypes.AnnotationAuthor] | None = None, id: str | None = None, properties: list[~darwin.future.data_objects.properties.SelectedProperty] | None = None, hidden_areas: ~typing.List[~typing.List[int]] = <factory>)[source]

Bases: object

Represents an Annotation that belongs to a Video.

annotation_class: AnnotationClass: The AnnotationClass from this VideoAnnotation.

frames: Dict[int, Any]: A dictionary of frames for this VideoAnnotation.

keyframes: Dict[int, bool]: The keyframes for this VideoAnnotation. Keyframes are a selection of frames from the frames attribute.

segments: List[List[int]]: A list of Segment's.

interpolated: bool: Whether this VideoAnnotation is interpolated or not.

slot_names: List[str]: V2 slots this annotation belogs to

annotators: List[AnnotationAuthor] | None = None: Authorship of the annotation (annotators)

reviewers: List[AnnotationAuthor] | None = None: Authorship of the annotation (reviewers)

id: str | None = None

properties: list[SelectedProperty] | None = None

hidden_areas: List[List[int]]: A list of HiddenArea's.

get_data(only_keyframes: bool = True, post_processing: Callable[[Annotation, Any], Any] | None = None) → Dict[source]

Return the post-processed frames and the additional information from this VideoAnnotation in a dictionary with the format:

{
    "frames": {
        # Post-Processed Frames here
    },
    "segments": [
        # Segments here
    ]
    "interpolated": True
}

Parameters:

only_keyframes (bool, default: True) – Whether or not to return only the keyframes. Defaults to True.
post_processing (Optional[Callable[[Annotation, Any], Any]], default: None) – If given, it processes each frame through the given Callabale before adding it to the returned dictionary. Defaults to None.

Returns:

A dictionary containing the processed frames, the segments of this VideoAnnotation and whether or not it is interpolated.

Return type:

Dict[str, Any]

class darwin.datatypes.Slot(name: 'Optional[str]', type: 'str', source_files: 'List[SourceFile]', thumbnail_url: 'Optional[str]' = None, width: 'Optional[int]' = None, height: 'Optional[int]' = None, frame_count: 'Optional[int]' = None, frame_urls: 'Optional[List[str]]' = None, fps: 'Optional[float]' = None, metadata: 'Optional[Dict[str, UnknownType]]' = None, frame_manifest: 'Optional[List[Dict[str, UnknownType]]]' = None, segments: 'Optional[List[Dict[str, UnknownType]]]' = None, upload_id: 'Optional[str]' = None, reason: 'Optional[str]' = None)[source]

Bases: object

name: str | None: Unique slot name in the item.

type: str: Type of slot, e.g. image or dicom

source_files: List[SourceFile]: Original upload information for the slot

thumbnail_url: str | None = None: Thumbnail url to the file

width: int | None = None: Width in pixel

height: int | None = None: Height in pixels

frame_count: int | None = None: How many sections (eg. frames) does this slot have

frame_urls: List[str] | None = None: A url for each of the existing sections.

fps: float | None = None: Frames per second

metadata: Dict[str, Any] | None = None: Metadata of the slot

frame_manifest: List[Dict[str, Any]] | None = None: Frame Manifest for video slots

segments: List[Dict[str, Any]] | None = None: Segments for video slots

upload_id: str | None = None: Upload ID

reason: str | None = None: The reason for blocking upload of this slot, if it was blocked

class darwin.datatypes.SourceFile(file_name: 'str', url: 'Optional[str]' = None)[source]

Bases: object

file_name: str: File name of source file

url: str | None = None: URL of file

class darwin.datatypes.AnnotationFileVersion(major: int = 1, minor: int = 0, suffix: str = '')[source]

Bases: object

Version of the AnnotationFile

major: int = 1

minor: int = 0

suffix: str = ''

class darwin.datatypes.Property(name: str, type: Literal['multi_select', 'single_select', 'text', 'attributes', 'instance_id', 'directional_vector'], required: bool, property_values: list[dict[str, Any]], granularity: PropertyGranularity = PropertyGranularity.section, description: str | None = None)[source]

Bases: object

Represents a property of an annotation file.

name: str

type: Literal['multi_select', 'single_select', 'text', 'attributes', 'instance_id', 'directional_vector']

required: bool

property_values: list[dict[str, Any]]

description: str | None = None

granularity: PropertyGranularity = 'section'

class darwin.datatypes.PropertyClass(name: 'str', type: 'str', description: 'Optional[str]', color: 'Optional[str]' = None, sub_types: 'Optional[list[str]]' = None, properties: 'Optional[list[Property]]' = None)[source]

Bases: object

name: str

type: str

description: str | None

color: str | None = None

sub_types: list[str] | None = None

properties: list[Property] | None = None

darwin.datatypes.parse_property_classes(metadata: dict[str, Any]) → list[PropertyClass][source]

Parses the metadata file and returns a list of PropertyClass objects.

Parameters:: metadata (dict[str, Any]) – The metadata file.
Returns:: A list of PropertyClass objects.
Return type:: list[PropertyClass]

darwin.datatypes.split_paths_by_metadata(path, dir: str = '.v7', filename: str = 'metadata.json') → tuple[Path, list[PropertyClass] | None][source]

Splits the given path into two: the path to the metadata file and the path to the properties

Parameters:: path (Path) – The path to the export directory.
Returns:: A tuple containing the path to the metadata file and the list of property classes.
Return type:: tuple[Path, Optional[list[PropertyClass]]]

class darwin.datatypes.AnnotationFile(path: ~pathlib.Path, filename: str, annotation_classes: ~typing.Set[~darwin.datatypes.AnnotationClass], annotations: ~typing.Sequence[~darwin.datatypes.Annotation | ~darwin.datatypes.VideoAnnotation], item_properties: list[dict[str, ~typing.Any]] | None = None, is_video: bool = False, image_width: int | None = None, image_height: int | None = None, image_url: str | None = None, workview_url: str | None = None, seq: int | None = None, frame_urls: ~typing.List[str] | None = None, remote_path: str | None = None, slots: ~typing.List[~darwin.datatypes.Slot] = <factory>, image_thumbnail_url: str | None = None, dataset_name: str | None = None, version: ~darwin.datatypes.AnnotationFileVersion = <factory>, item_id: str | None = None, frame_count: int | None = None)[source]

Bases: object

Represents a file containing annotations. Mostly useful when trying to import or export annotations to/from darwin V7.

path: Path: Path to the file.

filename: str: Name of the file containing the annotations.

annotation_classes: Set[AnnotationClass]: Set of all AnnotationClasses this file contains. Used as a way to know in advance which AnnotationClasses this file has without having to go through the list of annotations.

annotations: Sequence[Annotation | VideoAnnotation]: List of VideoAnnotations or Annotations.

item_properties: list[dict[str, Any]] | None = None

is_video: bool = False: Whether the annotations in the annotations attribute are VideoAnnotation or not.

image_width: int | None = None: Width of the image in this annotation.

image_height: int | None = None: Height of the image in this annotation.

image_url: str | None = None: URL of the image in this annotation.

workview_url: str | None = None: URL of the workview for this annotation.

seq: int | None = None: Sequence for this annotation.

frame_urls: List[str] | None = None: URLs for the frames this AnnotationFile has.

remote_path: str | None = None: Remote path for this Annotation's file in V7’s darwin.

slots: List[Slot]

image_thumbnail_url: str | None = None: URL of the image’s thumbnail in this annotation.

dataset_name: str | None = None: Dataset name

version: AnnotationFileVersion

item_id: str | None = None

frame_count: int | None = None

property full_path: str

Returns the absolute path of this file.

Returns:: The absolute path of the file.
Return type:: str

darwin.datatypes.make_bounding_box(class_name: str, x: float, y: float, w: float, h: float, subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a bounding box annotation. x, y, w and h are rounded to 3 decimal places when creating the annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
x (float) – The top left x value where the bounding box will start.
y (float) – The top left y value where the bounding box will start.
w (float) – The width of the bounding box.
h (float) – The height of the bounding box.
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotation``s for this ``Annotation.

Returns:

A bounding box Annotation.

Return type:

Annotation

darwin.datatypes.make_tag(class_name: str, subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a tag annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotation``s for this ``Annotation.

Returns:

A tag Annotation.

Return type:

Annotation

darwin.datatypes.make_polygon(class_name: str, point_paths: List[List[Point]] | List[Point], bounding_box: Dict | None = None, subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a polygon annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
point_paths (List[List[Point]] | List[Point]) –
Either a list of points that comprises a polygon or a list of lists of points that comprises a complex polygon. A complex polygon is a polygon that is defined by >1 path.

A polygon should be defined by a List[Point] and have a format similar to:

… code-block:: python

[
{“x”: 1, “y”: 0}, {“x”: 2, “y”: 1}

]

A complex polygon should be defined by a List[List[Point]] and have a format similar to:
```
[
    [
        {"x": 1, "y": 0},
        {"x": 2, "y": 1}
    ],
    [
        {"x": 3, "y": 4},
        {"x": 5, "y": 6}
    ]
    # ... and so on ...
]
```
bounding_box (Optional[Dict], default: None) – The bounding box that encompasses the polyong.
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotation``s for this ``Annotation.

Returns:

A polygon Annotation.

Return type:

Annotation

darwin.datatypes.make_complex_polygon(class_name: str, point_paths: List[List[Dict[str, float]]], bounding_box: Dict | None = None, subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a complex polygon annotation. Complex polygons are those who have holes and/or disform shapes. This is used by the backend.

Parameters:

class_name (str) – The name of the class for this Annotation.
point_paths (List[List[Point]]) –
A list of lists points that comprises the complex polygon. This is needed as a complex polygon can be effectively seen as a sum of multiple simple polygons. The list should have a format similar to:
```
[
    [
        {"x": 1, "y": 0},
        {"x": 2, "y": 1}
    ],
    [
        {"x": 3, "y": 4},
        {"x": 5, "y": 6}
    ]
    # ... and so on ...
]
```
bounding_box (Optional[Dict], default: None) – The bounding box that encompasses the polyong.
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotation``s for this ``Annotation.

Returns:

A complex polygon Annotation.

Return type:

Annotation

darwin.datatypes.make_keypoint(class_name: str, x: float, y: float, subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a keypoint, aka point, annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
x (float) – The x value of the point.
y (float) – The y value of the point.
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotation``s for this ``Annotation.

Returns:

A point Annotation.

Return type:

Annotation

darwin.datatypes.make_line(class_name: str, path: List[Dict[str, float]], subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a line annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
point_path (List[Point]) –
A list of points that comprises the polygon. The list should have a format similar to:
```
[
    {"x": 1, "y": 0},
    {"x": 2, "y": 1}
]
```
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotation``s for this ``Annotation.

Returns:

A line Annotation.

Return type:

Annotation

darwin.datatypes.make_skeleton(class_name: str, nodes: List[Dict[str, Any]], subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a skeleton annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.

nodes (List[Node]) –

List of Nodes that comprise the skeleton. Each Node will have a format similar to:

{
    "name": "1",
    "occluded": false,
    "x": 172.78,
    "y": 939.81
}

subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotations for this Annotation.

Returns:

A skeleton Annotation.

Return type:

Annotation

darwin.datatypes.make_ellipse(class_name: str, parameters: Dict[str, float | Dict[str, float]], subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns an Ellipse annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
parameters (EllipseData) –
The data needed to build an Ellipse. This data must be a dictionary with a format similar to:
```
{
    "angle": 0.57,
    "center": {
        "x": 2745.69,
        "y": 2307.46
    },
    "radius": {
        "x": 467.02,
        "y": 410.82
    }
}
```
Where:
- angle: float is the orientation angle of the ellipse.
- center: Point is the center point of the ellipse.
- radius: Point is the width and height of the ellipse, where x represents the width and y represents height.
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotation``s for this ``Annotation.

Returns:

An ellipse Annotation.

Return type:

Annotation

darwin.datatypes.make_cuboid(class_name: str, cuboid: Dict[str, Dict[str, float]], subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a Cuboid annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
parameters (CuboidData) –
The data needed to build a Cuboid. This data must be a dictionary with a format similar to:
```
{
    "back": {"h": 381.25, "w": 1101.81, "x": 1826.19, "y": 1841.44},
    "front": {"h": 575.69, "w": 1281.0, "x": 1742.31, "y": 1727.06}
}
```
Where:
- back: Dict[str, float] is a dictionary containing the x and y of the top left corner Point, together with the width w and height h to form the back box.
- front: Dict[str, float] is a dictionary containing the x and y of the top left corner Point, together with the width w and height h to form the front box.
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotations for this Annotation.

Returns:

A cuboid Annotation.

Return type:

Annotation

darwin.datatypes.make_table(class_name: str, bounding_box: Dict[str, float], cells: List[Dict[str, Any]], subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a table annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
bounding_box (BoundingBox) – Bounding box that wraps around the table.

cells (List[Dict[str, Any]]) –

Actual cells of the table. Their format should be similar to:

[
    {
        "bounding_box": {
            "h": 189.56,
            "w": 416.37,
            "x": 807.58,
            "y": 1058.04
        },
        "col": 1,
        "col_span": 1,
        "id": "778691a6-0df6-4140-add9-f39806d950e9",
        "is_header": false,
        "row": 1,
        "row_span": 1
    }
]

subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotations for this Annotation.

Returns:

A table Annotation.

Return type:

Annotation

darwin.datatypes.make_simple_table(class_name: str, bounding_box: Dict[str, float], col_offsets: List[float], row_offsets: List[float], subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a simple table annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
bounding_box (BoundingBox) – Bounding box that wraps around the table.
col_offsets (List[float]) – List of floats representing the column offsets.
row_offsets (List[float]) – List of floats representing the row offsets.
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotations for this Annotation.

Returns:

A simple table Annotation.

Return type:

Annotation

darwin.datatypes.make_string(class_name: str, sources: List[Dict[str, Any]], subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a string annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.

data (Any) –

The data needed to build a String. This data must be a list with a format similar to:

[
    {
        "id": "8cd598b5-0363-4984-9ae9-b15ccb77784a",
        "ranges": [1, 2, 5]
    },
    {
        "id": "6d6378d8-fd02-4518-8a21-6d94f0f32bbc",
        "ranges": null
    }
]

subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotations for this Annotation.

Returns:

A string Annotation.

Return type:

Annotation

darwin.datatypes.make_graph(class_name: str, nodes: List[Dict[str, str]], edges: List[Dict[str, str]], subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a graph annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.

nodes (List[Dict[str, str]]) –

Nodes of the graph. Should be in following format:

[
    {
        "id": "91bb3c24-883a-433b-ae95-a6ee7845bea5",
        "name": "key"
    },
    {
        "id": "5a0ceba1-2e26-425e-8579-e6013ca415c5",
        "name": "value"
    }
]

edges (List[Dict[str, str]]) –

Edges of the graph. Should be in following format:

[
    {
        "end": "value",
        "start": "key"
    }
]

subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotations for this Annotation.

Returns:

A graph Annotation.

Return type:

Annotation

darwin.datatypes.make_mask(class_name: str, subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a mask annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotation``s for this ``Annotation.

Returns:

A mask Annotation.

Return type:

Annotation

darwin.datatypes.make_raster_layer(class_name: str, mask_annotation_ids_mapping: Dict[str, str], total_pixels: int, dense_rle: List[int], subs: List[SubAnnotation] | None = None, slot_names: List[str] | None = None) → Annotation[source]

Creates and returns a raster_layer annotation.

Parameters:

class_name (str) – The name of the class for this Annotation.
mask_annotation_ids_mapping (Dict[str, str]) –
Mapping of mask annotations ids to unique small integers used in the dense_rle. Should be in following format: .. code-block:: javascript

{
“91bb3c24-883a-433b-ae95-a6ee7845bea5”: 1, “5a0ceba1-2e26-425e-8579-e6013ca415c5”: 2

}
total_pixels (int) – Total number of pixels in a corresponding image.
dense_rle (int) –
Run length encoding of all masks in the raster layer. Should be in following format: .. code-block:: javascript

[0, 5, 1, 15, 2, 10]
subs (Optional[List[SubAnnotation]], default: None) – List of SubAnnotations for this Annotation.

Returns:

A raster_layer Annotation.

Return type:

Annotation

darwin.datatypes.make_instance_id(value: int) → SubAnnotation[source]

Creates and returns an instance id sub-annotation.

Parameters:: value (int) – The value of this instance’s id.
Returns:: An instance id SubAnnotation.
Return type:: SubAnnotation

darwin.datatypes.make_attributes(attributes: List[str]) → SubAnnotation[source]

Creates and returns an attributes sub-annotation.

Parameters:: value (List[str]) – A list of attributes. Example: ["orange", "big"].
Returns:: An attributes SubAnnotation.
Return type:: SubAnnotation

darwin.datatypes.make_text(text: str) → SubAnnotation[source]

Creates and returns a text sub-annotation.

Parameters:: text (str) – The text for the sub-annotation.
Returns:: A text SubAnnotation.
Return type:: SubAnnotation

darwin.datatypes.make_opaque_sub(type: str, data: Any) → SubAnnotation[source]

Creates and returns a opaque sub-annotation.

Parameters:

type (str) – Type of this sub-annotation
data (Any) – Data for this sub-annotation.

Returns:

A text SubAnnotation.

Return type:

SubAnnotation

darwin.datatypes.make_keyframe(annotation: Annotation, idx: int) → Dict[str, int | Annotation][source]

Creates and returns a KeyFrame.

Parameters:

annotation (Annotation) – The annotation for the keyframe.
idx (int) – The id of the keyframe.

Returns:

The created Keyframe.

Return type:

KeyFrame

darwin.datatypes.make_video_annotation(frames: Dict[int, Any], keyframes: Dict[int, bool], segments: List[List[int]], interpolated: bool, slot_names: List[str], properties: list[SelectedProperty] | None = None, hidden_areas: List[List[int]] | None = None) → VideoAnnotation[source]

Creates and returns a VideoAnnotation.

Parameters:

frames (Dict[int, Any]) – The frames for the video. All frames must have the same annotation_class.name value.
keyframes (Dict[int, bool]) – Indicates which frames are keyframes.
segments (List[Segment]) – The list of segments for the video.
interpolated (bool) – If this video annotation is interpolated or not.

Returns:

The created VideoAnnotation.

Return type:

VideoAnnotation

Raises:

ValueError – If some of the frames have different annotation class names.

class darwin.datatypes.MaskTypes[source]

Bases: object

Palette: alias of Dict[str, int]

Mode: alias of Literal[‘index’, ‘grey’, ‘rgb’]

TypeOfRender: alias of Literal[‘raster’, ‘polygon’]

CategoryList: alias of List[str]

ExceptionList: alias of List[Exception]

UndecodedRLE: alias of List[int]

ColoursDict: alias of Dict[str, int]

RgbColors: alias of List[int]

HsvColors: alias of List[Tuple[float, float, float]]

RgbColorList: alias of List[List[int]]

RgbPalette: alias of Dict[str, List[int]]

RendererReturn: alias of Tuple[List[Exception], ndarray[Any, dtype[_ScalarType_co]], List[str], Dict[str, int]]

class darwin.datatypes.AnnotationMask(id: 'str', name: 'str', slot_names: 'List[str]' = <factory>)[source]

Bases: object

id: str

name: str

slot_names: List[str]

validate() → None[source]

class darwin.datatypes.RasterLayer(rle: 'MaskTypes.UndecodedRLE', mask_annotation_ids_mapping: 'Dict[str, int]', slot_names: 'List[str]' = <factory>, total_pixels: 'int' = 0)[source]

Bases: object

rle: List[int]

mask_annotation_ids_mapping: Dict[str, int]

slot_names: List[str]

total_pixels: int = 0

validate() → None[source]

class darwin.datatypes.ManifestItem(frame: 'int', absolute_frame: 'Optional[int]', segment: 'int', visibility: 'bool', timestamp: 'float', visible_frame: 'Optional[int]')[source]

Bases: object

frame: int

absolute_frame: int | None

segment: int

visibility: bool

timestamp: float

visible_frame: int | None

class darwin.datatypes.SegmentManifest(slot: 'str', segment: 'int', total_frames: 'int', items: 'List[ManifestItem]')[source]

Bases: object

slot: str

segment: int

total_frames: int

items: List[ManifestItem]

class darwin.datatypes.ObjectStore(name: str, prefix: str, readonly: bool, provider: str, default: bool)[source]

Bases: object

Object representing a configured conection to an external storage locaiton

name

The alias of the storage connection

Type:: str

prefix

The directory that files are written back to in the storage location

Type:: str

readonly

Whether the storage configuration is read-only or not

Type:: bool

provider

The cloud provider (aws, azure, or gcp)

Type:: str

default

Whether the storage connection is the default one

Type:: bool

class darwin.datatypes.StorageKeyDictModel(*, storage_keys: Dict[str, List[str]])[source]

Bases: BaseModel

storage_keys: Dict[str, List[str]]

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class darwin.datatypes.StorageKeyListModel(*, storage_keys: List[str])[source]

Bases: BaseModel

storage_keys: List[str]

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class darwin.datatypes.TeamPropertyLookups(annotation_properties: 'Dict[Tuple[PropertyName, AnnotationClassId], FullProperty]', item_properties: 'Dict[PropertyName, FullProperty]', _client: 'Client', _team_slug: 'str')[source]

Bases: object

annotation_properties: Dict[Tuple[PropertyName, AnnotationClassId], FullProperty]

item_properties: Dict[PropertyName, FullProperty]

classmethod from_team(client: Client, team_slug: str) → TeamPropertyLookups[source]

refresh() → None[source]

class darwin.datatypes.ReportJob(*, id: str, status: str, format: str, url: str | None, team_id: int)[source]

Bases: BaseModel

id: str

status: str

format: str

url: str | None

team_id: int

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class darwin.datatypes.AnnotatorReportGrouping(value)[source]

Bases: str, Enum

An enumeration.

ANNOTATORS = 'annotators'

DATASETS = 'datasets'

STAGES = 'stages'

darwin.doc_enum module

class darwin.doc_enum.DocEnum(value)[source]

Bases: Enum

Documenting Enums in Python is not supported by many tools. Therefore this class was created to support just that. It is basically a hack to allow Enum documentation. See more here <https://stackoverflow.com/a/50473952/1337392>

darwin.exceptions module

exception darwin.exceptions.DarwinException(*args: Any, **kwargs: Dict[str, Any])[source]

Bases: Exception

Generic Darwin exception.

Used to differentiate from errors that originate in our code, and those that originate in third-party libraries.

Extends Exception and adds a parent_exception field to store the original exception.

Also has a combined_exceptions field to store a list of exceptions that were combined into

parent_exception: Exception | None = None

combined_exceptions: List[Exception] | None = None

classmethod from_exception(exc: Exception) → DarwinException[source]

Creates a new exception from an existing exception.

Parameters:: exc (Exception) – The existing exception.
Returns:: The new exception.
Return type:: DarwinException

classmethod from_multiple_exceptions(exceptions: List[Exception], echo: bool = False, console: Console | None = None) → DarwinException[source]

Creates a new exception from a list of exceptions.

Parameters:: exceptions (List[Exception]) – The list of exceptions.
Returns:: The new exception.
Return type:: DarwinException

exception darwin.exceptions.GracefulExit(*args: Any, **kwargs: Dict[str, Any])[source]

Bases: DarwinException

Used to exit the program gracefully.

message: str

exception darwin.exceptions.IncompatibleOptions[source]

Bases: Exception

Used when a combination of options has one or more options that are not compatible between them. An option is not compatible with another if any combination from their set of possibilities returns an unspecified result.

exception darwin.exceptions.MissingDependency[source]

Bases: Exception

Used for when one of the optional install dependencies are missing

exception darwin.exceptions.UnrecognizableFileEncoding[source]

Bases: Exception

Used when a we try to decode a file and all decoding algorithms fail.

exception darwin.exceptions.Unauthenticated[source]

Bases: Exception

Used when a user tries to perform an action that requires authentication without being authenticated.

exception darwin.exceptions.InvalidLogin[source]

Bases: Exception

Used when a user tries to log in with invalid credentials.

exception darwin.exceptions.InvalidTeam[source]

Bases: Exception

Used when a team is not found or has no valid API key.

exception darwin.exceptions.InvalidCompressionLevel(level: int)[source]

Bases: Exception

Used when compression level is invalid.

exception darwin.exceptions.MissingConfig[source]

Bases: Exception

Used when the configuration file was not found.

exception darwin.exceptions.UnsupportedExportFormat(format: str)[source]

Bases: Exception

Used when one tries to export an annotation into a format that is not supported.

exception darwin.exceptions.NotFound(name: str)[source]

Bases: Exception

Used when a given resource is not found.

exception darwin.exceptions.UnsupportedFileType(path: Path)[source]

Bases: Exception

Used when a given does not have a supported video or image extension.

exception darwin.exceptions.InsufficientStorage[source]

Bases: Exception

Used when a request to a server fails due to insufficient storage.

exception darwin.exceptions.NameTaken[source]

Bases: Exception

Used when one tries to create an entity and the name of that entity is already taken.

exception darwin.exceptions.ValidationError[source]

Bases: Exception

Used when a validation fails.

exception darwin.exceptions.Unauthorized[source]

Bases: Exception

Used when a user tries to perform an action without having the necessary permissions.

exception darwin.exceptions.OutdatedDarwinJSONFormat[source]

Bases: Exception

Used when one tries to parse a video with an old darwin format that is no longer compatible.

exception darwin.exceptions.RequestEntitySizeExceeded[source]

Bases: Exception

Used when a request fails due to the URL being too long.

exception darwin.exceptions.MissingSchema(message: str)[source]

Bases: Exception

Used to indicate a problem loading or finding the schema

exception darwin.exceptions.AnnotationFileValidationError(parent_error: ValidationError, file_path: Path)[source]

Bases: Exception

Used to indicate error while validation JSON annotation files.

exception darwin.exceptions.UnknownAnnotationFileSchema(file_path: Path, supported_versions: List[AnnotationFileVersion], detected_version: AnnotationFileVersion)[source]

Bases: Exception

Used to indicate error when inferring schema for JSON annotation file.

exception darwin.exceptions.UnknownExportVersion(version: str)[source]

Bases: Exception

Used when dataset version is not recognized.

exception darwin.exceptions.UnsupportedImportAnnotationType(import_type: str, annotation_type: str)[source]

Bases: Exception

Used when one tries to parse an annotation with an unsupported type.

exception darwin.exceptions.DataloopComplexPolygonsNotYetSupported[source]

Bases: Exception

Used when one tries to parse an annotation with a complex polygon.

exception darwin.exceptions.ExportException(*args: Any, **kwargs: Dict[str, Any])[source]: Bases: DarwinException

exception darwin.exceptions.ExportException_CouldNotAssembleOutputPath(*args: Any, **kwargs: Dict[str, Any])[source]: Bases: ExportException

exception darwin.exceptions.ExportException_CouldNotBuildOutput(*args: Any, **kwargs: Dict[str, Any])[source]: Bases: ExportException

exception darwin.exceptions.ExportException_CouldNotWriteFile(*args: Any, **kwargs: Dict[str, Any])[source]: Bases: ExportException

darwin.item module

class darwin.item.DatasetItem(*, id: int, filename: str, status: str, archived: bool, filesize: int, dataset_id: int, dataset_slug: str, seq: int, current_workflow_id: int | None = None, path: str, slots: List[Any], layout: Dict, current_workflow: Dict[str, Any] | None = None)[source]

Bases: BaseModel

DatasetItem represents files that can be images or videos which belong to a dataset.

id: int: The id of this DatasetItem.

filename: str: The filename of this DatasetItem.

status: str: The status of this DatasetItem. It can be "archived", "error", "uploading", "processing", "new", "annotate", "review" or "complete".

archived: bool: Whether or not this item was soft deleted.

filesize: int: The size of this DatasetItem's file in bytes.

dataset_id: int: The id of the Dataset this DatasetItem belongs to.

dataset_slug: str: The slugified name of the Dataset this DatasetItem belongs to.

seq: int: The sequential value of this DatasetItem in relation to the Dataset it belongs to. This allows us to know which items were added first and is used mostly for sorting purposes.

current_workflow_id: int | None: The id of this DatasetItem’s workflow. A None value means this DatasetItem is new and was never worked on, or was reset to the new state.

path: str: The darwin path to this DatasetItem.

slots: List[Any]: The names of each slot in the item, most items have a single slot corresponding to the file itself. only used for v2 dataset items

layout: Dict: Information about the slot layout of the item including type, version, and slot names

current_workflow: Dict[str, Any] | None: Metadata of this DatasetItem’s workflow. A None value means this DatasetItem is new and was never worked on, or was reset to the new state.

property full_path: str: The full POSIX relative path of this DatasetItem.

classmethod parse(raw: Dict[str, Any], dataset_slug: str = 'n/a') → DatasetItem[source]

Parses the given dictionary into a DatasetItem.

Parameters:: raw (Dict[str, Any]) – The dictionary to parse.
Returns:: A dataset item with the parsed information.
Return type:: DatasetItem
Raises:: ValidationError – If any of the keys from the given dictionary do not have the correct format or are missing.

model_config: ClassVar[ConfigDict] = {}: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

darwin.item_sorter module

class darwin.item_sorter.SortDirection(value)[source]

Bases: DocEnum

The sorting direction of items.

ASCENDING = 'asc'

DESCENDING = 'desc'

classmethod parse(direction: str) → SortDirection[source]

Parses the given direction and returns the corresponding sort Enum.

Parameters:: direction (str) – The direction of the sorting order. Can be ‘asc’ or ‘ascending’, ‘desc’ or ‘descending’.
Returns:: The Enum representing a sorting direction.
Return type:: SortDirection
Raises:: ValueError – If the direction given is invalid.

class darwin.item_sorter.ItemSorter(field: str, direction: SortDirection)[source]

Bases: object

Represents sorting for list of items.

Parameters:

field (str) – The name of the field to be sorted.
direction (SortDirection) – The direction of the sort.

classmethod parse(sort: str | ItemSorter) → ItemSorter[source]

Parses the sorting given into an ItemSorter, capable of being used by Darwin.

Parameters:: sort (Union[str, ItemSorter]) – The sort order. If it is a str, it will be parsed, otherwise it returns the ItemSorter.
Returns:: A parsed ItemSorter representing a sorting direction.
Return type:: ItemSorter
Raises:: ValueError – If the given sort parameter is invalid.

darwin.options module

class darwin.options.Options[source]

Bases: object

Has functions to parse CLI options given by the user.

parse_args() → Tuple[Namespace, ArgumentParser][source]

Parses and validates the CLI options.

Returns:: The tuple with the namespace and parser to use.
Return type:: Tuple[Namespace, ArgumentParser]

darwin.path_utils module

darwin.path_utils.construct_full_path(remote_path: str | None, filename: str) → str[source]

Returns the full Darwin path (in Posix form) of the given file, is such exists.

Parameters:

remote_path (Optional[str]) – The remote path to this file, if it exists.
filename (str) – The name of the file.

Returns:

The full Darwin path of the file in Posix form.

Return type:

str

darwin.path_utils.deconstruct_full_path(filename: str) → Tuple[str, str][source]

Returns a tuple with the parent folder of the file and the file’s name.

Parameters:: filename (str) – The path (with filename) that will be deconstructed.
Returns:: A tuple where the first element is the path of the parent folder, and the second is the file’s name.
Return type:: Tuple[str, str]

darwin.path_utils.parse_metadata(path: Path) → dict[source]

Returns the parsed metadata file.

Parameters:: path (Path) – The path to the metadata file.
Returns:: The parsed metadata file.
Return type:: dict

darwin.path_utils.is_properties_enabled(path: Path, dir: str = '.v7', filename: str = 'metadata.json', annotations_dir: str = 'annotations') → Path | bool[source]

Returns whether the given export directory has properties enabled.

Parameters:

path (Path) – The path to the export directory.
dir (str, optional) – The name of the .v7 directory, by default “.v7”
filename (str, optional) – The name of the metadata file, by default “metadata.json”
annotations_dir (str, optional) – The name of the annotations directory, by default “annotations”

Returns:

Whether the given export directory has properties enabled.

Return type:

bool

darwin.validators module

Holds functions that convert backend errors into a pythonic format the application can understand.

darwin.validators.name_taken(code: int, body: Dict[str, Any]) → None[source]

Validates if a request to the backend errored out with a NameTaken error.

Parameters:

code (int) – The response code.
body (Dict[str, Any]) – The response body.

Raises:

NameTaken – If both code and body indicate that the server request errored due to a name being already taken.

darwin.validators.validation_error(code: int, body: Dict[str, Any]) → None[source]

Validates if a request to the backend errored out with a Validation error.

Parameters:

code (int) – The response code.
body (Dict[str, Any]) – The response body.

Raises:

ValidationError – If both code and body indicate that the server request errored because it failed validation.

darwin package

Subpackages

Submodules

darwin.backend_v2 module

darwin.cli module

darwin.cli_functions module

darwin.client module

darwin.config module

darwin.datatypes module

darwin.doc_enum module

darwin.exceptions module

darwin.item module

darwin.item_sorter module

darwin.options module

darwin.path_utils module

darwin.validators module

Module contents