Datasets

Manage your DataGym.ai Datasets and their images with our Python API

The Dataset object in our Python API Wrapper is a representation of the Datasets you already know from DataGym. This object includes the Images that belongs to your DataGym Datasets. Datasets can be accessed through the Client object introduced in the Getting Started page.
Fetch Datasets from DataGym.ai
Before we look at the Dataset object in our Python API Wrapper, we first have to learn how to get the data from DataGym.ai. There are multiple ways to fetch Datasets from DataGym's backend.
Get all Datasets
datasets = client.get_datasets()
get_datasets returns all Datasets in a list.
Get a Dataset by its name
dummy_dataset = client.get_dataset_by_name(dataset_name="Dummy_Dataset")
get_dataset_by_name returns a specific Dataset by its unique name
Manage Datasets
Create new Datasets
To create a new Dataset on DataGym.ai, you can use the create_dataset method of the Client class. Therefore, you have to specify a Dataset name and an optional short description. In this example we use the owner ID of our Dummy Project. 
new_dataset = client.create_dataset(  
                                      name="My first Dataset"
                                      short_description="This is Optional"
                                   )
create_dataset returns the newly created Dataset
Upload Images to a Dataset via URL
The create_images_from_urls method of the Client class helps you to add Images to your Datasets. The methods requires a List of image URLs and a Dataset ID.
images_to_upload = [ 
                         "<IMAGE_URL_JPG_1>", 
                         "<IMAGE_URL_JPG_2>", 
                         "<IMAGE_URL_JPG_3>"
                   ]
​
images_created = client.create_images_from_urls(dataset_id=new_dataset.id,
                                                image_url_set=images_to_upload)
create_images_from_urls returns a list of errors that may have occurred during the Image upload.
output:
[
 {
  'imageUploadStatus': 'SUCCESS',
  'imageUrl': '<IMAGE_URL_JPG_1>'
 },
 {
  'imageUploadStatus': 'SUCCESS',
  'imageUrl': '<IMAGE_URL_JPG_2>'
 },
 {
  'imageUploadStatus': 'SUCCESS',
  'imageUrl': '<IMAGE_URL_JPG_3>'
 }
]
Add a Dataset to a Project
The add_dataset method of the Client class enables you to connect a Dataset to a Project. This method requires a respective Dataset ID and Project ID. Therefore, the example below fetches these objects before it executes the add_dataset method:
new_dataset = client.get_dataset_by_name("My first Dataset") 
empty_project = client.get_project_by_name("Some empty Project")
​
added_success = client.add_dataset(dataset_id=new_dataset.id,
                                   project_id=empty_project.id)
add_dataset returns True if connecting the Dataset was successful
Remove a Dataset from a Project
The remove_dataset method of the Client class allows you to remove a Dataset from a DataGym.ai Project. This method requires a Dataset ID and Project ID. Therefore, the example below fetches these objects before it executes the remove_dataset method:
new_dataset = client.get_dataset_by_name("My first Dataset") 
empty_project = client.get_project_by_name("Some empty Project")
​
removed_success = client.remove_dataset(dataset_id=new_dataset_1.id,
                                        project_id=empty_project.id)
remove_dataset returns True if removing the Dataset was successful
The Dataset object
Dataset Attributes
Datasets in the Python API are modeled after DataGym's Datasets and, therefore, inherit the same attributes you already know from your DataGym.ai Datasets.
>>>> print(dummy_dataset)
<Dataset {
    'id': '<DATASET_ID>', 
    'name': 'Dummy_Dataset', 
    'short_description': 'This is short', 
    'timestamp': 1583828520627, 
    'owner': '<OWNER_ID>', 
    'images': <List[Image] with 10 elements>
}>
As you can see, a Dataset object also contains a list of its images, which are represented as Image objects. 
Dataset helper methods
A Dataset can hold any number of images. To simplify the access to the respective Image objects, the Dataset object provides a variety of helper methods.
Get Images by name
In some use-cases you might only be interested in a specific set of images. The get_images_by_name method can reduce the effort of searching for these images.
images = dataset.get_images_by_name(image_name="satellite_img_01.jpg")
get_images_by_name returns a list of Images that match the search term
You can also use regular expressions to start a broader search and return only images that fit into a specific naming pattern. For example, let's get all satellite images from your project:
images = dataset.get_images_by_name(image_name="satellite.*", regex=True)
output
[    
    <Image {
        'id': '<ID_2>', 
        'image_name': 'satellite_img_01.jpg', 
        'image_type': 'SHAREABLE_LINK', 
        'timestamp': 1583831745119
    }>, 
    <Image {
        'id': '<ID_2>', 
        'image_name': 'satellite_img_01.jpg', 
        'image_type': 'SHAREABLE_LINK', 
        'timestamp': 1583831714382
    }>
]
​

Python API - Previous

Labeled data

Next - Python API

Images

Last modified 2yr ago