# Import Data DataGym can import your labeled data. How this works and the requirements for this task are described in this document. ### The requirements: * If you use our API, you need a valid API Token. How to create one is described [here](https://docs.datagym.ai/documentation/api-token/manage-api-token). * The **project id** is also required. This id can be found within the URL of the project's detail page or can be found with the API. Try the endpoint to [list all Projects](https://docs.datagym.ai/documentation/api-token/api#get-all-projects). * In addition, the internal ids of all affected images are required. They can be caught via the API. Try the endpoint to [get a dataset by its id](https://docs.datagym.ai/documentation/api-token/api#get-dataset-by-id). {% hint style="info" %} To use the import feature the import data must match the label configuration. {% endhint %} ### The import: You can import the labeled data either via our python package using the public api or upload the JSON formatted file from the project overview page. ![Import & export labeled data is available within the project home page.](https://622290927-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LzaIa2HtqaLgqpL5TFA%2F-M5g-lNiW8B1wWqoC63N%2F-M5g1d2MxH53AvwwMPPF%2Fdg-import-project-data.png?alt=media\&token=9bb95c79-6136-480e-8311-6cbb9752de8f) The import data format is similar to the export format and described in the following sections. #### Example label configuration: As an example to visualize the import, the following configuration is used. It's the same configuration used in the [**Export Data**](https://docs.datagym.ai/documentation/project/export-data) section of this documentation: ![](https://622290927-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LzaIa2HtqaLgqpL5TFA%2F-M6cwIfsIR1w6wTCh4rG%2F-M1zP0lTmFi8Y8M4qVnN%2Fdg-export-example-config.png?alt=media\&token=4d3326fe-dd04-4734-b32a-f1f6b0112d59) Within the exported data the **label\_classes** look like the following snippet from the export data section: {% code title="label\_classes.json" %} ```javascript { "label_classes": [ { "class_name": "bus", "geometry_type": "rectangle" }, { "class_name": "car", "geometry_type": "polygon" }, { "class_name": "car_type", "classification_type": "select" }, { "class_name": "weather", "classification_type": "radio" }, { "class_name": "daytime", "classification_type": "select" } ] } ``` {% endcode %} The used class names are required for the import. Also, the used geometry type or classification type defines the import format. {% hint style="warning" %} Classifications of **type option** are exported as *type radio* if they contain *up to three options* otherwise they are exported as *type select*. The section "**label\_classes**" is not part of the import JSON, but useful to understand how the import works. {% endhint %} Lets look into the expected data: #### The import format: In its simplest form, the import looks like this: ```javascript [ { "internal_media_ID": "", "keepData": false, "global_classifications": {}, "labels": {} } ] ``` | Property | Description | | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | internal\_media\_ID | The internal id to identify the media. | | keepData |

If keepData is equal to false, all already existing labels for the current Image will be deleted after the labels upload.
If keepData is equal to true, all new labels will be added to the already existing labels for the current Image.

Default value for keepData is true

| | global\_classifications | Uses the label\_classes to describe the image. | | labels | Uses the class names to describe a geometry within the image. | To do a successful import a valid media\_id in the internal\_media\_ID node is required. Either of the other three properties is optional. #### Schema validation To see if your import Json is valid, you can validate it against this JSON Schema. ```javascript { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Json Import Schema", "description": "Validate the user input", "definitions": { "rectangle": { "type": "array", "items": { "$ref": "#/definitions/point_def_rectangle" }, "minItems": 1, "maxItems": 1 }, "polygon": { "type": "array", "items": { "$ref": "#/definitions/point_def_not_rectangle" }, "minItems": 3 }, "line": { "type": "array", "items": { "$ref": "#/definitions/point_def_not_rectangle" }, "minItems": 2 }, "point": { "type": "array", "items": { "$ref": "#/definitions/point_def_not_rectangle" }, "minItems": 1, "maxItems": 1 }, "numberOrString": { "oneOf": [ { "type": "number", "minimum": 0 }, { "type": "string", "pattern": "^[0-9]+$" } ] }, "point_def_not_rectangle": { "type": "object", "properties": { "x": { "$ref": "#/definitions/numberOrString" }, "y": { "$ref": "#/definitions/numberOrString" } }, "minProperties": 2, "maxProperties": 2, "required": [ "x", "y" ] }, "point_def_rectangle": { "type": "object", "properties": { "x": { "$ref": "#/definitions/numberOrString" }, "y": { "$ref": "#/definitions/numberOrString" }, "w": { "$ref": "#/definitions/numberOrString" }, "h": { "$ref": "#/definitions/numberOrString" } }, "required": [ "x", "y", "w", "h" ] }, "classification": { "type": "array", "items": [ { "oneOf": [ { "type": "string" }, { "type": "null" } ] }, { "oneOf": [ { "type": "string" }, { "type": "object", "patternProperties": { "": { "$ref": "#/definitions/classification" } } } ] } ], "additionalItems": { "type": "string" } }, "geometry": { "type": "array", "items": { "oneOf": [ { "$ref": "#/definitions/point_def_not_rectangle" }, { "$ref": "#/definitions/point_def_rectangle" } ] } } }, "type": "array", "minItems": 1, "items": { "type": "object", "properties": { "internal_media_ID": { "type": "string" }, "keepData": { "type": "boolean" }, "global_classifications": { "type": "object", "patternProperties": { "": { "$ref": "#/definitions/classification" } } }, "labels": { "type": "object", "patternProperties": { "": { "type": "array", "items": { "type": "object", "properties": { "geometry": { "$ref": "#/definitions/geometry" }, "rectangle": { "$ref": "#/definitions/rectangle" }, "polygon": { "$ref": "#/definitions/polygon" }, "point": { "$ref": "#/definitions/point" }, "line": { "$ref": "#/definitions/line" }, "image_segmentation": { "type": "array" }, "classifications": { "type": "object", "patternProperties": { "": { "$ref": "#/definitions/classification" } } }, "nested_geometries": { "patternProperties": { "": { "type": "array", "items": { "type": "object", "properties": { "geometry": { "$ref": "#/definitions/geometry" }, "rectangle": { "$ref": "#/definitions/rectangle" }, "polygon": { "$ref": "#/definitions/polygon" }, "point": { "$ref": "#/definitions/point" }, "line": { "$ref": "#/definitions/line" }, "classifications": { "type": "object", "patternProperties": { "": { "$ref": "#/definitions/classification" } } }, "additionalProperties": false } } } } } }, "additionalProperties": false } } } } } } } ``` For validation you can either use one of the [validators](https://json-schema.org/implementations.html) from the official JSON Schema page. Those are available for several programming languages and the console. Or you can use one of many JSON Schema online validators. #### Properties explained Lets have a look into the **global\_classifications** property: This property may look like the following snippet: {% code title="global\_classifications.json" %} ```javascript { "global_classifications": { "weather": [ "sunny", { "daytime": [ "day" ] } ] } } ``` {% endcode %} This example can also be found within the Export data page. Nested classifications are possible. If a value is not defined use **null**. {% hint style="info" %} The possible object keys and their values are defined in the label configuration. {% endhint %} Lets have a look into the **labels** object: The snippet may look like the following: {% code title="labels\_snippet.json" %} ```javascript { "labels" : { "bus": [], "car": [] } } ``` {% endcode %} This object holds all classes of geometries defined within the label configuration and found within your image. Each entry is a list of geometries. Each geometry in such a list consists of two entries: | Property | Description | | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | geometry | This section defines the area of interest within the image. The structure depends on the type of geometry. (*Deprecated, use the appropriate entry from one of the properties below*) | | polygon | Use this property if you want to import a polygon | | line | Use this property if you want to import a line | | point | Use this property if you want to import a point | | rectangle | Use this property if you want to import a rectangle | | classifications | This section is similar to the **global\_classifications** section described above. | | *nested\_geometries* | This section holds an array of individual geometries (like those above) | | image\_segmentation | *You cant import image segmentation values yet* | Let's define a car to look at: {% code title="car.json" %} ```javascript { "car": [ { "polygon": [ { "x": 57, "y": 412 }, { "x": 54, "y": 463 }, { "x": 130, "y": 524 } ], "classifications": { "car_type": [ null ] } } ] } ``` {% endcode %} The car's geometry type is defined as a polygon. See the *label\_classes.json* or the **label configuration** as reference. To define a polygon a list of points is required. You need to enter at least three points for a valid polygon, but you can enter as many as needed. {% code title="line.json" %} ```javascript { "street": [ { "line": [ { "x": 57, "y": 412 }, { "x": 54, "y": 463 } ], "classifications": {} } ] } ``` {% endcode %} This is an example for a line. Lines are defined as polylines, therefore they need to have at least two points but can have as many as needed. {% code title="point.json" %} ```javascript { "poi": [ { "point": [ { "x": 57, "y": 412 } ], "classifications": {} } ] } ``` {% endcode %} A point geometry may only have one point containing x and y values. A **bus** may look like the following snippet: {% code title="bus.json" %} ```javascript { "bus": [ { "rectangle": [ { "x": 272, "y": 302, "w": 324, "h": 169 } ], "classifications": {} } ] } ``` {% endcode %} The geometry properties now define a rectangle with the start coordinates **x** & **y** and the properties **w** & **h** for its width and height. In addition, the **classifications** object is empty because *bus* has no classifications.