Skip to main content
Version: 10.2.9

Administer Data Stores

The Data Store module is a web interface for managing database (DB) tables in the Work.AI platform. A field scheme defines the Data Store table structure and column data format.

You can use Data Stores to keep lookup tables for multi-value Answer options, save Business Process (BP) results, and query Data Stores from Bot Configs.

To manage Data Stores, configure the permissions for managing Data Stores:

  1. Go to System Settings > Role Management and select the user to modify the permissions.

  2. On the Edit Role screen, select the Manage Data Stores permission and click Save.

Manage Data Store​

To start managing Data Stores, go to Advanced > Data Stores.

Create Data Store​

To create a Data Store, follow the instruction below:

  1. Click the Create Data Store button.

  2. Enter a unique name and description to the Name and Description fields to distinguish the Data Store. Names can contain alphanumeric characters, underscores, and spaces.

  3. To define the Data Store structure, select one of the following options:

    • In the Field Scheme drop-down box, select a field scheme. To manage field schemes, go to System Settings > Field Schemes or click the Edit link to the right of the drop-down box.
    • Select Upload File and add your CSV file. The first row should contain column headers. Mind that column names cannot contain spaces.

  4. Click the Save button. After saving the Data Store, you cannot select another field scheme or source CSV file.

  5. Add data to the Data Store:

    • Upload a CSV file by clicking Upload Data. To create a data file, click the Download Data button and populate the downloaded file containing all necessary columns. When you finish populating the file, upload it.

      You can choose the data format (CSV, XLSX) and data set (All Records, Matching Filter) for downloading.

    • Alternatively, you can add records manually one by one by clicking Add Record.

View and edit Data Store​

To view or edit a Data Store, click the Data Store name in the grid.

You can perform the following operations with records:

  • View and filter by column values: set columns to display in the grid.
  • Add.
  • Edit: hover over the appropriate record number and click Edit. Save the record after editing.
  • Upload a file:
    • If new file column names coincide with the Data Store columns, all new records append to the Data Store.
    • Otherwise, map columns from the file to the Data Store.
  • Delete.

Delete Data Store​

To perform a bulk Data Store deletion, follow the steps below:

  1. Go to Advanced > Data Stores.

  2. Select Data Stores in the grid and click Delete.

  3. Confirm the deletion by clicking Delete.

Use filters​

For quick navigation through the Data Stores, use filters. There are two types of them:

  • Predefined to filter out Data Stores by a username

  • Advanced to set custom criteria for filtering

    To add a parameter to your filter, use the Select a Filter... drop-down list. After you add the first parameter to the filter, you can add or delete the parameters using the plus (+) or minus (-) sign. Available parameters are as follows:

    • Name is the text containing the Data Store name.
    • Description is the text containing the Data Store description. With Name and Description, you can use contains and does not contain or equals to and not equals to options.
    • Author is the name of the user who created the Data Store. In the Author field, you can specify several authors and define the contains and does not contain options.
tip

To match the search results with all options defined in your filter, use Match All of The Following.

Use Data Store​

Answers​

You can use a Data Store to create a re-usable lookup list for task Answer, for example, List of Countries.

Instead of adding options manually, you can use records from a Data Store for the following Answer Types:

  • Free Text
  • Select Multi
  • Select One
  • Check One
  • Check Multi
  • Taxonomy

Select the Input from Data Store checkbox and choose an item from the list.

To use a predefined list in Answers, create a Data Store with two columns:

  • id: a unique identifier
  • name: a label to show in the drop-down list

Currency​

All currencies used as an answer for the Currency Answer Type are stored in a system Data Store called System Currency. In the Data Store, the currency code represents the currency id and name that contain the currency code and name.

The names appear in the drop-down list where you can select a currency when processing a task.

Bot Configs​

  1. Create a Bot Task Operation (ETL) with a Data Store Answer. Using the Data Store plugin, you can use SQL queries to read or modify the Data Store content.

    To create a Data Store, use the insert-datastore plugin. See the full list of Bot Config plugins.

  2. Add the Bot Task to your BP and set up its options.

    1. Select the Data Store name.
    2. Map Data Elements to the variables.

Business Processes​

To automatically load the BP Final Results to a Data Store, select the Copy results to Data Store checkbox. This way, you won't have to generate a BP Snapshot and store it on your local computer or in the cloud.

Business Process export​

If a BP uses Data Stores in its steps, you can include the Data Stores in the package during the export procedure and then create them in another Control Tower after you import the package.

Global variables​

All global variables are stored in one Data Store with the Global Variables name. You can use these variable values in the var-global plugin.

Data Store Query​

Manual Task​

You can develop a custom Manual Task that uses Data Store API calls from JavaScript. In such Manual Tasks, you should create a set of Data Store Queries. For more details, refer to Add allowed Data Store query.

Data Store for model training​

During the model training, a special Data Store is created to keep the information about each training step together with its results. A record in the Data Store represents the results per model training.

Manage field schemes​

A field scheme defines Data Store column names and content types. To start managing field schemes, go to System Settings > Field Schemes.

To create a field scheme, follow the steps below:

  1. Click the Create Field Scheme button.
  2. Enter a unique Name and Description to distinguish the field scheme.
  3. Add Answers to the field scheme. The step is similar to adding Answers on the Task Design tab. The field scheme allows setting the data format for each Data Store column by using specific Answer Types, for example, Email and Number.
  4. Click the Save button.
note

For a Data Store created from a file, an appropriate field scheme is created automatically.

Optimize Data Store performance​

The Data Store functionality is implemented using a relational database. To optimize performance for a significant amount of data, think about indexes.

There are two ways of managing Data Store indexes:

  1. Set up indexes within the Bot Task code. See the examples in Data Store plugins.
  2. Contact your Operations team to create an index in MSSQL with the given table and index details.

Also, see the article about data purging.

Audit Data Store​

You can track all actions with Data Stores in the ds.data_store_audit table in DB. The table contains information about the operation type, the Data Store where it appeared, the number of affected records, when and who performed the action, and related comments.

The following operations are logged to the table:

  • CREATE_DATASTORE
  • DELETE_DATASTORE
  • MODIFY_DATASTORE
  • INSERT_ROW
  • MODIFY_ROW
  • DELETE_ROW
  • DELETE_COLUMN
note

Only operations performed via API are logged to the table.

troubleshooting

For troubleshooting tips, refer to the following support guides: