Start Business Process from Data Source

A Data Source is a mechanism providing input data and configuration for your Business Processes (BPs). The input format is best suited for those use cases where the data resides on an external source, such as an SFTP Server or Amazon S3 Cloud.

To start a BP from a Data Source, you need the following:

• Definition UUID of an existing BP
• Input data file (CSV) uploaded to an external server
• Configuration file (CCF) uploaded to an external server
• Data Source containing the external server credentials and path to the configuration file
• Schedule, the number of occurrences per Data Source update

Create configuration file

The CCF configuration file contains all BP parameters needed to start a new run as parameter-value pairs. To download a sample CCF file, click the link.

The table below lists the parameters and values from a CCF file.

Parameter	Value	Description
ACTIVE	TRUE	Default parameter.
DATAFILE	input_file.csv	It is the relative path to the input CSV data file in the S3 storage, the full path inside your bucket. The server folder with the CCF file is the root.
DATAFILE_SIZE	-1	Default parameter.
FILE_SEPARATOR	COMMA; PIPE; SEMICOLON; TAB	Separator used in the input file.
ENV	DEVELOPMENT; PRODUCTION	Environment.
BLOCK_SIZE	1	Block size.
CAMPAIGN_UUID	79eea42a-127e-4613-83a2-7fbe80ba0a38	Create a BP (Campaign) in WorkFusion and copy the campaign UUID or copy the UUID of an existing Campaign.
tags	datasource, test	List of tags.
SNAPSHOT_TYPE	XLSX; CSV	Snapshot format.
OUTPUT_DIR	results/	Relative path for storing BP snapshots. The server folder with the CCF file is the root.
STREAM_TYPE	Immediately PermanentOpenTask	Immediately: all hits are created on the endpoint. PermanentOpenTask: a specific quantity is displayed. If the value is not specified, the setting defaults to Immediately.
STREAM_VALUE	10	Works if STREAM_TYPE=PermanentOpenTask. The default value is 10.
STREAM_THRESHOLD	1 (if an absolute value) 25% (if percentage)	The setting defaults to 100% if the value is not specified.

Upload files to server

Upload the input file (CSV) and configuration file (CCF) to an external server (FTPS, S3, SFTP). Remember to check the READ and WRITE permissions for the file and folder on the external server.

Create Data Source

To start managing Data Sources, go to the Control Tower main menu and then navigate to System Settings > Data Sources.

To create a Data Source, follow the instructions below:

1. Click the Create Data Source button.

   

   The platform automatically generates a UUID for the new Data Source. You can use this UUID in API calls or schedules.

2. Enter the appropriate Name and Description.

3. Select a Data Source type from the appropriate drop-down box:

   • S3: Amazon's Cloud Server
   • FTPS: File Transfer Protocol and SSL encryption
   • SFTP: Secure File Transfer Protocol based on SSH

4. Depending on the connection type, enter Configuration Parameters:

   • S3

     • accessKey: Access Key ID.
     • secretKey: Secret Access Key.
     • bucket: destination bucket in S3.
     • keyPrefix: remote folder. Note that keyPrefix should end with the slash symbol (/).

   • SFTP

     • host: hostname to establish a connection over the SFTP protocol
     • port: port number to connect to on the SFTP server
     • login: SFTP Server account user name
     • password: SFTP Server account password
     • remote_folder: directory on the SFTP server to start the session

   • FTPS

     • host: hostname to establish a connection over the FTPS protocol
     • port: port number to connect to on the FTPS server
     • login: FTPS server account user name
     • password: FTPS server account password
     • remote_folder: directory on the FTPS server to start the session

5. Click Save to save the Data Source.

Create schedule

To create a schedule for BP runs, follow the steps below:

1. Go to Advanced > Schedules, click the Create button.

   

2. On the Create Schedule page, find and select the Business Process or Manual Task in the Task or Process Definition drop-down box.

   

3. In the Input Data drop-down box, choose Data Source. In the field that appears next to the drop-down box, select the Data Source you created.

4. Set other schedule parameters as required. Click Save.

After the schedule is created, the specified BP starts automatically. The platform polls the Data Source for any input files at a specified interval (for instance, every 10 minutes). If such a file exists, the platform runs it.

View result

To verify that your BP started, log in to the external server and open the folder with the CCF file. Verify that {name}.ccf.uploaded file was created.

When a BP is completed, the following happens:

• The {name}.ccf.done file appears.
• A result file is created in the appropriate folder (OUTPUT_DIR parameter of the CCF file).

tip

If a Bot Task is not started, make sure that the Bot Config for the step doesn't contain the Bot Source value with Availability options.