Edit

Manage datasets

A dataset is a container for documents and all related meta information, such as:

• Labels
• Schema
• OCR settings
• Statistics, and so on

To create and manage different types of datasets, go to AutoML > Datasets. Control Tower updates the list every time you add a dataset.

In the Datasets list, you can find the following information about datasets.

Parameter	Description
Title	Dataset name you specify as you create a dataset. Each dataset name is a link to the associated dataset details page.
Type	Dataset type: tabular or document.
Volume	Number of rows in case of a tabular dataset or the number of documents in case of a document dataset.
Author	Name or alias of a user who created the dataset.
Created	Date and time (in the dd.mm.yyyy, hh:mm format) when the dataset was created.

The flow to manage a particular dataset depends on its type:

• Tabular: a CSV-based dataset used for classification models. Mind that a CSV file must contain the document content in the first column and gold values in the second one. Additional columns are passed as is.
• Document: a dataset containing TXT, TIFF, PDF, or any image files. The type can be used for classification and information extraction models.

Filter and search

You can filter the list of datasets based on their type. Click the Type filter button and specify the dataset type: Document or Tabular. The list is filtered accordingly. To turn off an active filter, click it again.

You can sort datasets by their titles, types, author, or the creation date and time by clicking a column name. Sorting by volume is not available.

You can also search for a particular dataset by its title or author using the Search box in the top-right corner.

Manage tabular datasets

View tabular dataset details

To view the detailed information about a tabular dataset, go to AutoML > Datasets and click the dataset's title in the list or open its Actions menu and select View details.

This brings you to the dataset details page:

For a tabular dataset, the details page can comprise the following meta information:

Parameter	Description
Created	Date and time (dd.mm.yyyy, hh:mm) when the dataset was created.
Author	Name or alias of a user who created the dataset.
Description	Dataset description (if any).
Data type	Tabular.
Columns	Number of columns in the dataset CSV file.
Rows	Number of rows in the dataset CSV file.
Data source	Data source CSV file (if any). You can download the file by clicking the Download link next to the CSV file.

Create dataset

To create a tabular dataset, follow the steps below:

1. In the top-right corner of the Datasets page, click the plus (+) button and select Create new.

   

2. On the displayed page, specify the dataset name and add a description if needed.

   

3. Select Tabular.

4. Click Create. A new tabular dataset is added to the dataset list, and the dataset details page appears on the screen:

   

   Before you upload a dataset, the Columns and Rows data items show 0, and the Data source one is empty.

5. To upload a dataset file, click the plus (+) button in the top-right corner of the details page.

6. In the file selection dialog, specify the path to your CSV file and click Open to start the upload.

   caution

   Before uploading, check that your CSV file contains the document content in the first column and gold values in the second one. Additional columns are passed as is.

As soon as your dataset file is uploaded successfully, you can see the exact number of columns and rows on the dataset details page. The uploaded CSV file is available as Data source.

Copy dataset

Instead of creating a dataset from scratch, you can make a copy of an existing one and modify it to your needs. To copy, proceed in one of the following ways:

• Go to AutoML > Datasets, select the dataset you want to copy, open the Actions menu, and click Copy.

  

• In the top-right corner of the dataset details page, open the Actions menu and select Copy.

  

Once you complete the copy action, a new dataset appears in the Dataset list. It has the same name as the original plus the copy word in the end. Proceed to rename and modify the copy as you think appropriate.

Rename dataset

You can rename a dataset in one of the two ways:

• Go to AutoML > Datasets, select a dataset you want to rename, and click Actions > Rename.

  

• In the top right corner of the dataset details page, open the Actions menu and select the Rename option.

  

In the window that follows, in the Dataset name field, specify a new dataset name and click Save.

Modify dataset

To override the dataset with an updated CSV file, do as follows:

1. Go to AutoML > Datasets, and click the title of the dataset to update.

2. On the tabular dataset page, click the plus (+) button.

   

3. In the window that follows, specify the path to the new dataset file and click Open to start the upload.

Export dataset

To export a tabular dataset as a bundle (a zip archive), find the dataset you want to export in the Datasets list, open its Actions menu, and click Export.

For a tabular dataset, the resulting archive contains the following files:

• dataset.json comprises full dataset representation, including schema, properties, tags, and so on.

• observations.csv contains observations, or feature values, (if any) in the CSV format.

• meta.json includes the technical information associated with the bundle, such as creation date, author, the WorkFusion platform version.

Import dataset

To add a tabular dataset to the list of available ones, you can also import it as a bundle similar to that created with the Export function. Each imported dataset must have a unique ID.

To import a tabular dataset, in the top-right corner of the Datasets list page, click the plus (+) button and select Import.

Above the Datasets list, you will see the import progress bar.

The dataset import comprises two stages:

1. Uploading: the dataset is being uploaded to Control Tower. At the stage, you can cancel it by clicking the x button.

2. Importing: the dataset is being deployed to your environment, and you can no longer cancel the import.

As soon as the import is completed successfully, you get a success message, and the dataset appears in the list. Otherwise, you get a warning or an error message.

Manage document datasets

View document dataset details

To view the detailed data about a document dataset, go to AutoML > Datasets and click its title in the list or open its Actions menu and select View details.

This brings you to the document dataset details page:

On the Overview tab, you can find the following document dataset details:

Data item	Description
Created	Date and time (dd.mm.yyyy, hh:mm) when the dataset was created.
Author	Name or alias of a user who created the dataset.
Description	Dataset description (if any).
Data type	Document.
Documents	Number of documents in the dataset.

Create dataset

To create a document dataset, follow the steps below:

1. In the top-right corner of the Datasets page, click the plus (+) button and select Create new.

   

2. In the Dataset name field, specify a name for the dataset you are creating. Add a description if needed.

   

3. Select Document.

4. Click Create. A new document dataset appears in the dataset list, and the dataset details page is displayed on the screen:

   

5. Navigate to the Documents tab. Before you upload any documents, the page is empty.

6. To upload required files, drag them to the Documents area or click Browse, specify the path to your dataset files, and click Open in the window that follows.

   

   Alternatively, in the top-right corner, click the plus (+) button. It is also convenient to use the option to add more documents to the already uploaded ones. For a document dataset, you can upload TXT, TIFF, PDF, or any image files.

   As soon as your documents are uploaded, you can see them on the Documents tab.

   

   On the Overview tab, in the Documents column, you can find the total number of uploaded documents.

   

Copy dataset

Instead of creating a document dataset from scratch, you can make a copy of an existing one and changes its content to your needs. The procedure for creating a copy of a document dataset is identical to that of the tabular one. Refer to Tabular datasets | Copy dataset.

Rename dataset

The procedure for renaming a document dataset is the same as for renaming a tabular one. For more information, refer to Tabular datasets | Rename dataset.

Export dataset

To export a document dataset as a bundle (a zip archive), find the dataset you want to export in the Datasets list, open its Actions menu, and click Export.

For a document dataset, the resulting archive contains the following files:

• dataset.json comprises a full dataset representation, including schema, properties, tags, and so on.

• observations.csv contains observations, or feature values, (if any) in the CSV format.

• meta.json includes the technical information associated with the bundle, such as creation date, author, the WorkFusion platform version.

• The files folder contains the documents included in the dataset.

Import dataset

The flow to import a document dataset is the same as for importing the tabular type. Refer to Manage tabular datasets | Import dataset.

Manage documents

To view the documents in a selected document dataset, go to AutoML > Datasets and select a dataset. Open the dataset details page and navigate to the Documents tab.

The following information on each document is available:

Parameter	Description
Document	Document name.
Added	Date and time (in the dd.mm.yyyy, hh:mm format) when the document was added to the dataset.
Labels	Labels associated with the document.
Status	Document status: Unlabeled or Labeled.

Control Tower updates the list every time you add documents. To view all dataset documents, use the scroll bar.

Sort and filter documents

You can sort documents by their names, date of adding, labels, and status by clicking the appropriate column.

To filter documents by status, click Status and select either Unlabeled or Labeled. The documents are filtered accordingly. To turn off an active filter, click it again.

You can also search the list for a particular document by its name. In the Search box in the top right corner, specify a document name. The results are displayed automatically.

Upload documents

To add more documents, click the plus (+) button in the top-right corner of the Documents tab.

In the window that follows, specify the path to a new document and click Open.

Remove documents

You can remove documents in the following ways:

• Go to the Documents tab, select a document, open its Actions menu, and click Remove.

  

• To perform a bulk removal, on the Documents tab, select the documents you want to remove, click the blue tick button next to the Status filter, and click Remove.

  

  note

  The blue tick button is not active until you select at least one document.

  If you use the checkbox to select all documents (next to the Document column title), only those documents that are visible on the page are selected. The number of selected documents out of their total quantity is indicated next to the Status filter. Use the scroll bar to see all available documents and select more.

  

  To cancel the selection, deselect the checkbox next to the Document column title, as shown in the figure above.

  The selected documents are removed, and you see a confirmation message in the bottom-right corner of the screen.

  

Set up labels

If labels are not configured for your dataset, a yellow icon is displayed next to the Labels tab title.

To add labels, go to the Labels tab of the dataset details page and follow the steps below:

1. Drag the needed labels from the left panel to the right one.

   

   As soon as a label is moved to the right panel, its detailed setup opens in a separate window.

   

   For Text field components, go to the Data tab, open the Answer Type menu, and specify the field type for your model.

   

   For other controls, leave the default setup.

2. Finish the setup and click Save. The labels you added are visible on the right.

To modify any label settings, move the pointer over a label and select one of the available actions:

• Edit: Edit form component parameters.
• Move: Move a form component within the form.
• Edit JSON: Edit form component parameters in a JSON representation. The Preview option allows viewing edits as they will look in the form.
• Copy: Copy a form component.
• Remove: Delete a form component from the form.

caution

Mind that changing label names (Edit > Display tab > Label) can result in loss of labeled data as the names are used for labeling.

Label documents

To label your dataset documents, go to the Documents tab and perform the following actions:

1. Select the documents you want to label, click the blue tick button next to the Status filter, and choose the Label option.

   

   note

   The blue tick button is not active until you select at least one document.

   If you use the checkbox to select all documents (next to the Document column title), only those documents that are visible on the page are selected. The number of selected documents out of their total quantity is indicated next to the Status filter. Use the scroll bar to see all available documents and select more.

   

   important

   If a previously labeled document is passed to labeling, the previously labeled data is restored.

2. On the page that follows, label your document. For more instructions on labeling, see Start tagging.

   

3. To save, click Finish. If you perform bulk labeling, proceed to the next document in a queue by clicking Next. To complete the labeling procedure and save the progress for your labeled documents, select Finish.

   

   Labeled data is stored in the dataset. On the Documents tab, the Status for the processed documents is changed to Labeled .

   

caution

Up to 10 users in parallel can label documents in a dataset using the default memory configuration for dataset management and labeling services.