Skip to main content
Version: 2.4.2

Evaluate historical data

The NSS skill has an out-of-the-box solution to validate the model's performance and quality on the client's historical data. A pre-built connector reads the client's report files from a screening system and sends the data to the NSS Business Process (BP) for model evaluation. The report file resulting from the BP execution is saved on MinIO S3 for further data and model decision analysis.

Requirements​

Mind the following considerations when working with the historical data evaluation solution:

  • The input file with historical should be in the CSV structured format.

  • The report columns can have any names.

  • The NSS column mapper file should be completed before the BP is started.

  • A historical data input file row represents full alert information: customer (input) data, hit data, and (optionally) human resolution on the hit.

Workflow details​

The NSS skill includes a template format file you should use to map your historical data columns with the NSS standard format. Before running a BP, fill the file with actual column names and update it in MinIO S3. The format file location is doc-upload/name_sanctions_screening/nss_input_format.csv.

Knowing the mapping between the historical data file and NSS standard format, the NSS BP transforms the original data into a format understandable for NSS and sends it for model evaluation.

Your task is to provide a historical data file in the CSV format and upload it to MinIO S3 as shown below.

Usage instructions​

To use the historical data evaluation feature, follow the steps below:

  1. Download the input format file from MinIO S3: doc-upload/name_sanctions_screening/nss_input_format.csv.

  2. Map the historical data file columns and the standard NSS input format. To do that, update the input format file downloaded in Step 1 with the client column names.

  3. Upload the input format file back to MinIO S3.

  4. Make sure that the original historical data file is in the CSV format and has flat structured data.

  5. Upload the historical data file to MinIO S3 to any folder of your choice.

  6. Configure the NSS skill to work with the historical data provider. For details, see Install and configure.

    As you set Input data location on the Input step, you can specify one of the following:

    • The path to the file, in which case the BP processes only the specific file.

      Example: If you set s3://doc-upload/name_sanctions_screening/data/data2.csv, the NSS skill processes the data2.csv file.

    • The path to the bucket or folder, in which case the BP takes all existing CSV files under the bucket or folder. The files are processed in parallel, and only one consolidated QC report is generated.

      Example: If you set s3://doc-upload/name_sanctions_screening/data/, the NSS skill processes all CSV files in the data folder and CSV files from the data_october folder.

  7. Open a Business Process and start it without input data.

Sample data preparation​

The section details a sample flow to prepare data for the NSS BP execution:

  1. Once the example_historical_data.csv file is received from a client for PoC evaluation, upload it to Minio S3.

    This file contains all required information for the NSS BP execution:

    • Alert ID in column A

    • Customer information (input data) in columns from B to E

    • Hit information in columns from F to J

    • Manual review decision in column K

  2. Prepare the NSS format file and upload it to MinIO S3. See the sample file.

    The NSS format file can look like this:

Last updated on