next

Edit

Data Purge API

Authentication

For Data Purge API, form-based authentication is required. For details, read about the WorkFusion REST API security.

Two permissions are essential for working with Data Purge API:

  • Run Data Purge enables using API methods to submit executions and get statuses or events. Thus, with the Run Data Purge permission only, third-party users can already trigger Data Purge, though without access to sensitive system information. The permission is available for the Admin and Developer roles.

  • Manage Data Purge allows utilizing the entire range of Data Purge API methods. The permission is only available for the Admin role.

To enable the permissions, follow the instructions in the Role management guide.

All URLs are to begin with the following pattern:

https://control.tower.host/api/v1

Database Data Purge

GET /data-purges

Returns a list of all available database Data Purge configurations. Each returned configuration is an object comprising the following data:

  • uuid: unique identifier of the Data Purge configuration.
  • active: enabled or disabled state.
  • creationDate: timestamp to indicate when a specific configuration was created.
  • executionId: identifier of the last Data Purge run (if any).
  • customSql: only for ds schema purges, the name of the underlying custom SQL script.
  • storedProcedureName: for purging schemas other than ds, the name of the underlying stored procedure.
  • parameters: parameters for the underlying stored procedure (if any).
  • submitProcedureDate: timestamp to indicate when the underlying stored procedure (if any) was submitted.
  • lastExecutionPoint and nextExecutionPoint: timestamps for the last and next purge runs accordingly, for scheduled configurations.
  • name: Data Purge configuration name.
  • description: additional details (if any) to describe the cleanup operation.
  • cronExpression: execution schedule (if any).
  • status: Data Purge configuration status.

Query parameters

  • page: requested page number.
  • size: page size.
  • sort: sorting criterion. Available values : NAME, DESCRIPTION, ACTIVE, LAST_EXECUTION_POINT, NEXT_EXECUTION_POINT, CREATION_DATE.
  • sortDirection: sorting order. Available values: ASC (ascending order), DESC (descending order).

Successful response example

[
  {
    "active": true,
    "creationDate": 0,
    "cronExpression": "string",
    "customSql": "string",
    "description": "string",
    "executionId": 0,
    "lastExecutionPoint": 0,
    "name": "string",
    "nextExecutionPoint": 0,
    "parameters": [
      {
        "name": "string",
        "value": "string"
      }
    ],
    "status": "QUEUED",
    "storedProcedureName": "string",
    "submitProcedureDate": 0,
    "uuid": "string"
  }
]

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

GET /data-purges/{uuid}

Returns information about a particular database Data Purge configuration, including:

  • active: enabled or disabled state.
  • creationDate: timestamp to indicate when a specific configuration was created.
  • submitProcedureDate: timestamp to indicate when the underlying stored procedure (if any) was submitted.
  • executionId: identifier of the last Data Purge run (if any).
  • name: Data Purge configuration name.
  • description: additional details (if any) to describe the cleanup operation.
  • customSql: only for ds schema purges, the name of the underlying custom SQL script.
  • storedProcedureName: for purging schemas other than ds, the name of the underlying stored procedure.
  • parameters: parameters for the underlying stored procedure (if any).
  • lastExecutionPoint and nextExecutionPoint: timestamps for the last and next purge runs accordingly, for scheduled configurations.
  • cronExpression: execution schedule (if any).
  • status: Data Purge configuration status.

Path parameter

  • uuid: the UUID for the database Data Purge configuration you want to get information about. Required.

Successful response example

{
  "active": true,
  "creationDate": 0,
  "cronExpression": "string",
  "customSql": "string",
  "description": "string",
  "executionId": 0,
  "lastExecutionPoint": 0,
  "name": "string",
  "nextExecutionPoint": 0,
  "parameters": [
    {
      "name": "string",
      "value": "string"
    }
  ],
  "status": "QUEUED",
  "storedProcedureName": "string",
  "submitProcedureDate": 0,
  "uuid": "string"
}

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

GET /data-purges/procedures

Returns a list of available stored procedures to create database Data Purge configurations for schemas other than ds.

No parameters are required for the API method.

Successful response example

[
  "string"
]

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

GET /data-purges/procedures/{sp_name}

Returns the parameters for a specific stored procedure.

Path parameter

sp_name: name of the stored procedure for which you want to get parameters. Required.

Successful response example

{
  "parameters": [
    "string"
  ]
}

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

GET /data-purges/{uuid}/events

Returns all events for a particular database Data Purge configuration. Each event object includes the following information:

  • eventDate: date when the event occurred.
  • level: log level.
  • description: event details.

Path parameter

  • uuid: UUID of the requested database Data Purge configuration. Required.

Query parameter

  • level: log level. Optional.

    Available values: DEBUG, INFO, WARNING, ERROR. The default value is INFO.

Successful response example

[
  {
    "configurationUuid": "string",
    "description": "string",
    "eventDate": "2021-01-26T09:49:34.235Z",
    "executionId": 0,
    "level": "INFO"
  }
]

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

GET /data-purges/{data_purge_uuid}/events/{execution_id}

Returns events for a particular execution of a database Data Purge configuration. Each returned event object comprises the following information:

  • eventDate: date when the event occurred.
  • level: log level.
  • description: event details.

Path parameters

  • data_purge_uuid: UUID of the requested Data Purge configuration. Required.
  • execution_id: ID of the Data Purge run for which event information is requested. Required.

Query parameter

  • level: log level. Optional. Available values: DEBUG, INFO, WARNING, ERROR. The default value is INFO.

Successful response example

[
  {
    "configurationUuid": "string",
    "description": "string",
    "eventDate": "2021-01-26T09:20:25.552Z",
    "executionId": 0,
    "level": "INFO"
  }
]

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

GET /data-purges/{uuid}/statuses/{executionId}

Returns the current statuses for a particular execution of a specific database Data Purge configuration. The following statuses are possible:

  • QUEUED: database Data Purge execution is in the queue at the execution engine.
  • PROCESSING: database Data Purge execution is in progress.
  • FAILED: database Data Purge run failed on the database side.
  • DONE: database Data Purge was completed successfully.

Path parameters

  • uuid: UUID of the requested database Data Purge configuration. Required.
  • executionId: ID of the database Data Purge execution for which status information is requested. Required.

Successful response example

[
  {
    "configUuid": "string",
    "executionId": 0,
    "status": "QUEUED",
    "submitDate": "2021-01-26T09:43:43.271Z"
  }
]

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

POST /data-purges

Creates a database Data Purge configuration based on a stored procedure or custom SQL, as specified in the request body.

Once the configuration is created, it is submitted to a queue under a unique ID (uuid). From the queue, you can launch it using POST /data-purges/{data_purge_uuid}/start.

A successful response returns the following information about the configuration:

  • uuid: unique identifier.
  • creationDate: creation timestamp.
  • status: current status. For a new Data Purge, the status is QUEUED.

For scheduled database Data Purges, a successful response body can also include a timestamp for the next execution (nextExecutionPoint).

Request body

For the endpoint, the request body is a required DataPurge object to be saved to Control Tower (CT) database. The object comprises the following parameters:

  • active: defines whether the created database Data Purge is enabled or disabled. Data Purges must be enabled to be executed. So, set the value to true.
  • cronExpression: schedule for running the configuration. With the parameter specified, the database Data Purge is run according to the schedule once you submit POST /data-purges/{data_purge_uuid}/start. Set to null to create an unscheduled configuration.
  • storedProcedureName: name of the stored procedure for data cleanup. Used for purging any database schema other than the ds one. For the ds schema, set it to null and specify the customSql parameter instead.
  • customSql: custom SQL code for performing a Data Store cleanup. Set it to null and specify the storedProcedureName parameter instead to create a configuration for purging schemas other than the ds one.
  • description: additional details to describe the created Data Purge.
  • name: any random name defined for the configuration.
  • parameters: includes stored procedure parameters if the created configuration is based on one—that means storedProcedureName is specified. Otherwise, set to null.

Request body example

{
  "active": true,
  "cronExpression": "string",
  "customSql": "string",
  "description": "string",
  "name": "string",
  "parameters": [
    {
      "name": "string",
      "value": "string"
    }
  ],
  "storedProcedureName": "string"
}

Successful response example

{
  "active": true,
  "creationDate": 0,
  "cronExpression": "string",
  "customSql": "string",
  "description": "string",
  "executionId": 0,
  "lastExecutionPoint": 0,
  "name": "string",
  "nextExecutionPoint": 0,
  "parameters": [
    {
      "name": "string",
      "value": "string"
    }
  ],
  "status": "QUEUED",
  "storedProcedureName": "string",
  "submitProcedureDate": 0,
  "uuid": "string"
}

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

POST /data-purges/{data_purge_uuid}/start

Launches a database Data Purge from the specified configuration without a schedule. Once the method is called, the stored procedure or custom SQL underlying the configuration is executed, and an execution ID is returned.

Path parameter

data_purge_uuid: UUID of the database Data Purge configuration to be executed. Required.

Request body

The request body is a required parameters array. It specifies the names and values of the parameters for the underlying stored procedure if the Data Purge is to be executed for a schema other than the ds one. For ds schema purges based on custom SQLs, set the array elements to null.

Request body example

[
  {
    "name": "string",
    "value": "string"
  }
]

Successful response example

{
  "executionId": 0
}

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

PUT /data-purges/{uuid}

Updates an existing database Data Purge configuration.

A successful response returns the following information about the configuration:

  • uuid: unique identifier
  • executionId: last execution ID
  • creationDate: creation timestamp
  • status: current status

For scheduled database Data Purges, a successful response body can also include timestamps for the last (lastExecutionPoint) and next (nextExecutionPoint) executions of the configuration.

Path parameter

  • uuid: UUID of the database Data Purge configuration to be updated. Required.

Request body

A required DataPurge object to be updated on the CT side, comprising the same elements as described for POST /data-purges.

Request body example

{
  "active": true,
  "cronExpression": "string",
  "customSql": "string",
  "description": "string",
  "name": "string",
  "parameters": [
    {
      "name": "string",
      "value": "string"
    }
  ],
  "storedProcedureName": "string"
}

Successful response example

{
  "active": true,
  "creationDate": 0,
  "cronExpression": "string",
  "customSql": "string",
  "description": "string",
  "executionId": 0,
  "lastExecutionPoint": 0,
  "name": "string",
  "nextExecutionPoint": 0,
  "parameters": [
    {
      "name": "string",
      "value": "string"
    }
  ],
  "status": "QUEUED",
  "storedProcedureName": "string",
  "submitProcedureDate": 0,
  "uuid": "string"
}

Failed response example

{
  "code": "string",
  "errors": [
    {
      "message": "string"
    }
  ],
  "message": "string"
}

DELETE /data-purges/{uuid}

Deletes an existing database Data Purge configuration.

Path parameter

  • uuid: UUID for the database Data Purge configuration to be deleted. Required.

S3 (MinIO) Data Purge

GET /s3-data-purges/{uuid}

Returns the following information about a specific S3 Data Purge configuration:

  • uuid: unique identifier assigned to the S3 Data Purge configuration.
  • name: name defined for the configuration.
  • description: additional details about the S3 Data Purge.
  • cronExpression: cron expression for scheduled execution.
  • bucket: S3 bucket to clean up.
  • path: path prefix for the S3 bucket to clean up.
  • recursive: if true, the purge cleans up sub-directories.
  • fileNamePattern: Unix-like pattern to filter the files for subsequent cleaning.
  • maxFileAge: files older than the specified value are to be deleted.
  • lastFileModifiedTime: timestamp when the configuration was last modified.
  • sessionDuration: purge session duration.
  • accessKeyAlias: S3 access key alias in Secrets Vault.
  • secretKeyAlias: S3 secret key alias in Secrets Vault.
  • active: defines whether the created S3 Data Purge is enabled or disabled
  • lastExecutionPoint or nextExecutionPoint: date when the configuration was executed last or to be executed next, accordingly, if the configuration has a schedule set (cronExpression).
  • creationDate: date when the Data Purge was created.
  • status: current status of the Data Purge execution.

Path parameters

  • uuid: UUID of the Data Purge configuration. Required.

Successful response example

{
  "uuid": "string",
  "name": "string",
  "description": "string",
  "cronExpression": "string",
  "bucket": "string",
  "path": "string",
  "recursive": true,
  "fileNamePattern": "string",
  "maxFileAge": "string",
  "lastFileModifiedTime": 0,
  "sessionDuration": 0,
  "accessKeyAlias": "string",
  "secretKeyAlias": "string",
  "active": true,
  "lastExecutionPoint": 0,
  "nextExecutionPoint": 0,
  "creationDate": "string",
  "status": "string"
}

GET /s3-data-purges/{data_purge_uuid}/statuses/{execution_id}

Returns all statuses for a specific execution of a Data Purge configuration:

  • QUEUED: execution is waiting in the queue at the execution engine.
  • PROCESSING: execution is in progress.
  • FAILED: execution failed at the database side.
  • DONE: execution was completed successfully.
  • TIMEOUT: execution timeout for the Data Purge is exceeded.
  • PARTIAL_SUCCESS: Data Purge removed only some of the files matching the specified criteria.

Path parameters

  • data_purge_uuid: UUID of the S3 Data Purge configuration. Required.
  • execution_id: identifier of a specific Data Purge execution. Required.

Successful response example

{
  "configUuid": "string",
  "executionId": 0,
  "startTime": "string",
  "endTime": "string",
  "status": "string"
}

POST /s3-data-purges

Creates a Data Purge configuration to clean up an S3 bucket (MinIO).

Request body

A required DataPurge object to be saved to the Control Tower (CT) database, comprising the following parameters:

  • name: any random name defined for the configuration.
  • description: additional details about the created S3 Data Purge.
  • cronExpression: cron expression for scheduled execution.
  • bucket*: S3 bucket to clean up.
  • path*: path prefix for the S3 bucket to clean up.
  • recursive*: if true, the purge cleans up sub-directories.
  • fileNamePattern*: Unix-like pattern to filter the files for cleaning.
  • maxFileAge*: files older than the specified date are to be deleted.
  • lastFileModifiedTime: timestamp when the configuration was last modified.
  • sessionDuration*: purge session duration.
  • accessKeyAlias: S3 access key alias in Secrets Vault.
  • secretKeyAlias: S3 secret key alias in Secrets Vault.
  • active: defines whether the created S3 Data Purge is enabled or disabled. Data Purges must be enabled to be executed. So, set the value to true.

The parameters marked with the asterisk (*) are required.

Request body example

{
  "name": "string",
  "description": "string",
  "cronExpression": "string",
  "bucket": "string",
  "path": "string",
  "recursive": true,
  "fileNamePattern": "string",
  "maxFileAge": "string",
  "lastFileModifiedTime": 0,
  "sessionDuration": 0,
  "accessKeyAlias": "string",
  "secretKeyAlias": "string",
  "active": true
}

Successful response example

{
  "uuid": "string",
  "name": "string",
  "description": "string",
  "cronExpression": "string",
  "bucket": "string",
  "path": "string",
  "recursive": true,
  "fileNamePattern": "string",
  "maxFileAge": "string",
  "lastFileModifiedTime": 0,
  "sessionDuration": 0,
  "accessKeyAlias": "string",
  "secretKeyAlias": "string",
  "active": true,
  "lastExecutionPoint": 0,
  "nextExecutionPoint": 0,
  "creationDate": "string",
  "status": "string"
}

POST /s3-data-purges/{data_purge_uuid}/start

Launches an S3 Data Purge based on the specified configuration without using the scheduler and returns its execution ID.

Path parameters

  • data_purge_uuid: UUID of the S3 Data Purge configuration. Required.

Request body

The request body is a required Data Purge execution parameters array as shown in the example below.

Request body example

{
  "path": "string",
  "fileNamePattern": "string",
  "maxFileAge": "string",
  "lastFileModifiedTime": 0,
  "sessionDuration": "string"
}

Successful response example

{
  "executionId": 0
}

POST /s3-data-purges/{data_purge_uuid}/toggle

Lets you enable or disable an S3 Data Purge.

Path parameters

  • data_purge_uuid: UUID of the S3 Data Purge configuration. Required.

Request body

When the request body parameter is set to true, the Data Purge configuration specified in the path parameter is enabled. Otherwise, it is disabled.

Request body example

{
  "active": true
}

PUT /s3-data-purges/{uuid}

Updates an existing S3 Data Purge configuration.

Path parameter

  • uuid: UUID of the S3 Data Purge configuration to be updated. Required.

Request body

A required DataPurge object to be updated on the CT side, comprising the same elements as described for POST /s3-data-purges.

Request body example

{
  "name": "string",
  "description": "string",
  "cronExpression": "string",
  "bucket": "string",
  "path": "string",
  "recursive": true,
  "fileNamePattern": "string",
  "maxFileAge": "string",
  "lastFileModifiedTime": 0,
  "sessionDuration": 0,
  "accessKeyAlias": "string",
  "secretKeyAlias": "string",
  "active": true
}

Successful response example

{
  "uuid": "string",
  "name": "string",
  "description": "string",
  "cronExpression": "string",
  "bucket": "string",
  "path": "string",
  "recursive": true,
  "fileNamePattern": "string",
  "maxFileAge": "string",
  "lastFileModifiedTime": 0,
  "sessionDuration": 0,
  "accessKeyAlias": "string",
  "secretKeyAlias": "string",
  "active": true,
  "lastExecutionPoint": 0,
  "nextExecutionPoint": 0,
  "creationDate": "string",
  "status": "string"
}

DELETE /s3-data-purges/{uuid}

Deletes an existing S3 Data Purge configuration.

Path parameter

  • uuid: UUID of the S3 Data Purge configuration to be deleted. Required.
← Data Store REST APISecrets Vault API →