Understand NSS data flow

The guide is intended to help you understand the Evelyn data flow:

• How data is handled within its Business Processes

• How data is stored

• How data is parsed

• How decision score is calculated

Schema definitions

The section lists the input and output data objects and their properties used for input and output contracts in Evelyn Business Processes.

Input and processing data

ScreenedData

The object represents the input data received for screening.

Property	Type	Description
id	string	ScreenedData ID
category	string	ScreenedData category
client_id	string	Technical personal identifier
entityType	EntityType	ScreenedData entity type
gender	Gender	ScreenedData gender
name	NameEntity	ScreenedData name
dates	List<DateEntity>	ScreenedData dates
locations	List<LocationEntity>	ScreenedData locations
aliases	List<NameEntity>	ScreenedData aliases
additional_data	Map<String, String>	ScreenedData additional data

WleData

The object represents the watchlist entity (WLE) information received from a screening system.

Property	Type	Description
id	string	ScreenedData ID
category	string	ScreenedData category
entityType	EntityType	ScreenedData entity type
gender	Gender	ScreenedData gender
name	NameEntity	ScreenedData name
dates	List<DateEntity>	ScreenedData dates
locations	List<LocationEntity>	ScreenedData locations
aliases	List<NameEntity>	ScreenedData aliases
additional_data	Map<String, String>	ScreenedData additional data

EntityType

Type	Values
Enum<String>	INDIVIDUAL ORGANIZATION SECURITIES

Gender

Type	Values
Enum<String>	MALE FEMALE OTHER

NameEntity

Property	Type	Description
fullName	string	Full name (for example, Thomas A. Anderson)
firstName	string	First name (for example, Thomas)
lastName	string	Last name (for example, Anderson)
middleName	string	Middle name (for example, A. )
derived	string	The name derived in TickerLookupTask
source	string	The source where the name was received from (for example, TickerLookupTask)

DateEntity

Property	Type	Description
original	DetailedDate	Original date
derived	DetailedDate	Date obtained or calculated during processing
source	string	The source of information

DetailedDate

Property	Type	Description
type	string	Type of date (for example, DOB)
full	string	Full date (for example, 03/03/2023)
year	string	Year (for example, 2023)
month	string	Month (for example, 03)
day	string	Day (for example, 15)

LocationEntity

Property	Type	Description
original	Location	Original location
enriched	List<Location>	List of enriched locations
source	string	Source of enriched locations

Location

Property	Type	Description
type	string	Type of location
country	string	Location country
state	string	Location state
city	string	Location city
street	string	Location street
address	string	Full location address
latitude	string	Location latitude
longitude	string	Location longitude

Screening results

AlertDecision

Property	Type	Description
id	string	Alert decision ID
resolution	AlertResolution	Alert resolution
explanation	string	Detailed decision explanation
wleDecisions	WleDecision	WLE decision

AlertResolution

Type	Values
Enum<String>	NO_HITS ALL_FP NEED_REVIEW

WleDecision

Property	Type	Description
id	string	WLE decision ID
htmlReportLink	string	Link to the HTML report
resolution	WleResolution	WLE resolution
explanation	string	Detailed decision explanation
nameDecision	NameDecision	Name decision
dateDecision	DateDecision	Date decision
locationDecision	LocationDecision	Location decision
entityTypeDecision	EntityTypeDecision	Entity type decision
genderDecision	GenderDecision	Gender decision
manualReviewStatus	Enum<String>	TO_REVIEW SKIPPED REVIEWED

WleResolution

Type	Values
Enum<String>	FALSE_POSITIVE MORE_INFORMATION_NEEDED

NameDecision

Property	Type	Description
decision	FactorDecision	Model decision
score	double	Model decision score
input	string	Input value
wle	string	WLE value

DateDecision

Property	Type	Description
decision	FactorDecision	Model decision
proximity	integer	Distance between the compared input and WLE dates
input	string	Input date
wle	string	WLE date

LocationDecision

Property	Type	Description
decision	FactorDecision	Model decision
proximity	double	Distance between the compared input and WLE locations
input	string	Input location
wle	string	WLE location

EntityTypeDecision

Property	Type	Description
decision	FactorDecision	Model decision
input	string	Input entity type
wle	string	WLE entity type

GenderDecision

Property	Type	Description
decision	FactorDecision	Model decision
input	string	Input gender
wle	string	WLE gender

FactorDecision

Type	Values
Enum<String>	NOT_COMPARED BAD_INFO STRONG_MISMATCH WEAK_MISMATCH STRONG_MATCH

FircoMetaData

Property	Type	Description
uuid	string, uuid	UUID
systemId	string	Firco system ID
messageId	string	Firco message ID
businessUnit	string	Firco business unit
unit	string	Firco unit
extRecordId	string	Firco external record ID
fpStatus	string	Firco False Positive (FP) status
nmiStatus	string	Firco Need More Information (NMI) status

Data Store structure

The section represents the structure and content of the Data Store tables storing the data used in Evelyn processes and analytics.

Main processing tables

Transaction table

Data Store name: ds_uc_B_AML_PEP_transaction_v3_0

Column	Type	Index	Description
uuid	UUID	+	Transaction UUID
start_time	Date Time		Transaction processing start time
end_time	Date Time		Transaction processing end time
status	String		Transaction status: NEW, PROCESSING, READY_FOR_REPORT, ERROR, COMPLETED
start_bp_uuid	UUID		UUID of the BP where the transaction was processed
bp_start_time	Date Time		BP start time
error_message	String		Error stack trace in case of an error
error_step	String		The step where the transaction failed
dw_variation	String		DW title
dw_version	String		DW version
model_version	String		Version of the model configured in the DW user interface
ext_record_id	String		REST request ID
auto_qc	Bool		Marks if the transaction was selected for the Auto quality check feature

Master record table

Data Store name: ds_uc_B_AML_PEP_master_record_v3_0

Column	Type	Index	Description
uuid	UUID	+	Transaction UUID
bp_run_id	UUID		BP UUID
ext_record_id	String		REST request ID

Ticker enrichment table

Data Store name: ds_uc_B_AML_PEP_tickers_v3_0

Column	Type	Index	Description
uuid	nvarchar(36), UUID
symbol	nvarchar(255), String	+	Ticker symbol
company_name	nvarchar(Max), String		Company name associated with the ticker

Ticker enrichment configuration table

Data Store name: ds_uc_B_AML_PEP_configuration_v3_0

If you enable email notifications from the Ticker Lookup Business Process during the name enrichment configuration, populate the Data Store with the required technical details:

Configuration name	Configuration value
smtp.host	SMTP host
smtp.port	SMTP port
smtp.credentialsAlias	Alias of SecureEntry that stores the username and password for authentication
smtp.from	Sender's email address
smtp.authEnabled	true or false to enable or disable authentication
env	Environment identifier used in Subject of the email

Tracking request status table

Data Store name: ds_uc_B_AML_PEP_request_status_v3_0

Column	Type	Description
UUID	nvarchar(36)
TRANSACTION_ID	nvarchar(36)	Transaction identifier created by Evelyn
REQUEST_ID	nvarchar(255)	External request identifier
STATUS	nvarchar(255)	Request status: IN_PROGRESS READY_FOR_MT MT_COMPLETED FINISHED ERROR
MESSAGE	nvarchar(MAX)

QC report aggregation table

Data Store name: ds_uc_B_AML_PEP_qc_report_data_v3_0

Column	Type	Index	Description
uuid	nvarchar(36)
bp_run_id	nvarchar(36)	+	BP UUID
transaction_id	nvarchar(36)	+	Transaction UUID
screened_data_json	nvarchar(MAX)		Screened data JSON from the Quality Check (QC) report
wle_list	nvarchar(MAX)		WLE data JSON from the QC report
alert_decision	nvarchar(MAX)		Model and review decision for the transaction
auto_qc_indicator	bit		If the transaction was selected for Automatic Quality Check (AutoQC)

Screening system tables

Firco Trust

• Data Store name: ds_uc_B_AML_PEP_firco_configuration_v3_0

  Column	Type	Index	Description
  uuid	nvarchar(36), UUID
  batch_size	int, String		Firco Trust processing batch size
  wait_time	int, String		Wait time between batches in milliseconds

• Data Store name: ds_uc_B_AML_PEP_firco_messages_v3_0

  Column	Type	Index	Description
  uuid	nvarchar(36), UUID
  system_id	nvarchar(255), String	+	Firco Trust message system ID
  message_id	nvarchar(255), String		Firco Trust message ID
  unit	nvarchar(MAX), String		Firco Trust message unit
  business_unit	nvarchar(255), String		Firco Trust message business unit
  transaction_id	nvarchar(36), String	+	Transaction UUID
  start_bp_uuid	nvarchar(36), String	+	BP run UUID
  ext_record_id	nvarchar(255), String		REST request UUID
  status	nvarchar(255), String	+	Firco message processing status

Dow Jones Risk and Compliance

• Data Store name: ds_uc_B_AML_PEP_alerts_cache_v3_0

  Column	Type	Index	Description
  uuid	nvarchar(36), UUID
  alert_id	nvarchar(36)		Alert ID
  alert_update_date	nvarchar(255)		Alert last updated date
  alert_response	nvarchar(MAX)		Alert Decision Justification (DJ) response

• Data Store name: ds_uc_B_AML_PEP_search_cache_v3_0

  Column	Type	Index	Description
  uuid	nvarchar(36), UUID
  search_string	nvarchar(255)	+	DJ Factiva search request
  search	nvarchar(MAX)		DJ Factiva search response

Thomson Reuters

• Data Store name: ds_uc_b_aml_pep_countries_v3_0

  Column	Type	Index	Description
  uuid	nvarchar(36), UUID
  iso2_code	varchar(2)	+	ISO 2 country code
  iso3_code	varchar(3)	+	ISO 3 country code
  name	varchar(56)		Full country name

• Data Store name: ds_uc_b_aml_pep_us_states_v3_0

  Column	Type	Index	Description
  uuid	nvarchar(36), UUID
  iso2_code	varchar(2)		ISO 2 US state code
  name	varchar(56)	+	US state name

Analytics tables

• Data Store name: ds_uc_B_AML_PEP_alert_analytics_v3_0

  Column	Type	Index	Description
  uuid	nvarchar(36), UUID
  transaction_uuid	nvarchar(36), UUID	+	Transaction UUID
  alert_id	nvarchar(MAX), String		Alert ID
  alert_category	nvarchar(MAX), String		Alert category
  alert_entity_type	nvarchar(255), String		Alert entity type
  alert_dw_decision	nvarchar(255), String		NSS model decision
  alert_dw_explanation	nvarchar(MAX), String		NSS model exlanation
  alert_mt_decision	nvarchar(255), String		Manual review decision
  alert_mt_explanation	nvarchar(MAX), String		Manual review explanation
  worker_full_name	nvarchar(255), String		Reviewer's full name
  worker_id	nvarchar(36), UUID		Reviewer ID
  html_report_link	nvarchar(MAX), String		HTML report link if enabled
  csv_report_link	nvarchar(MAX), String		QC report link if enabled
  autoQC	nvarchar(5), bool		The alert was selected for AutoQC
  ticker	nvarchar(5), bool		The alert name was enriched with ticket information

• Data Store name: ds_uc_B_AML_PEP_hit_analytics_v3_0

  Column	Type	Index	Description
  uuid	nvarchar(36), UUID
  transaction_uuid	nvarchar(36), UUID	+	Transaction UUID
  wle_id	nvarchar(255), String	+	WLE ID
  wle_category	nvarchar(255), String		WLE category
  wle_entity_type	nvarchar(255), String		WLE type
  wle_dw_decision	nvarchar(255), String		NSS model decision
  wle_dw_explanation	nvarchar(255), String		NSS model explanation
  wle_mt_decision	nvarchar(255), String		Manual review decision
  wle_mt_explanation	nvarchar(255), String		Manual review exlanation
  worker_full_name	nvarchar(255), String		Reviewer's full name
  worker_id	nvarchar(36), UUID		Reviewer ID
  name_matching_score			Original name matching score
  wle_decision_score			Hit decision score

Data parsing based on format file

During Firco Trust screening, Evelyn can process the following data feeds to retrieve data fields for alert disposition:

• Security
• SSI
• Brokers
• Employees
• Investors
• KYC

Preparing data for screening

Before screening, send the data to be parsed in the RAW format within the MessageData tag of a Firco message. A Firco message contains two XML tags from which data can be extracted:

• <fir:MessageData>
• <fir:CustomerInfo>

Prepare one or both of the following CSV format file types, depending on how the incoming message is structured:

• Pipe-separated: defines which pipe-delimited value maps to each NSS field. The index and nss_column headers are required; any other headers are ignored.
• Fixed-length: defines each field by its start position and length and maps that segment of the raw message to the corresponding NSS field.

Name each file after the entity it applies to:

• Firco Unit: <unit>_format.csv
• Firco Business Unit: <bunit>_format.csv

In S3 Manager, upload the files to the directory that matches its format:

• Pipe-separated: doc-upload/name_sanctions_screening/firco_message_pipeseparator_format_files/
• Fixed-length: doc-upload/name_sanctions_screening/firco_message_fixedlength_format_files/

For example, you have the following configuration:

• Firco Unit: CIF1
• Firco Business Unit: SECURITY

For each format type, Evelyn checks its S3 directory for the following files in order and uses the first available file:

• CIF1_format.csv
• SECURITY_format.csv

info

Filenames are not case-sensitive.

Mapping process

When a Firco message arrives, Evelyn resolves the format file for its unit or business unit from S3. The resolved file is cached per worker to improve performance, and all messages for that unit are mapped with it.

Based on the message format, one mapper in the processing chain handles the message:

• The pipe-separated mapper (PipeSeparatedFormatFileMapper) processes pipe-delimited messages and takes priority.
• The fixed-length mapper (FixedLengthFormatFileMapper) processes positional messages whose fields are defined by a start position and length.

The selected mapper then does the following:

1. Extracts raw data from <fir:MessageData> in the Firco response.
2. Reads the field definitions from the corresponding format file.
3. Maps each delimited value or fixed-length segment to the corresponding field in ScreenedData.
4. Merges information from <fir:CustomerInfo> into ScreenedData.

As a result, ScreenedData contains information from both sources.

Collections such as Addresses, Dates, and Aliases are included. If ScreenedData.name is missing due to issues with MessageData, the name from CustomerInfo is used instead. In other cases, names from CustomerInfo are added to Aliases to avoid overwriting. Evelyn ensures that no duplicate entries are created.

Required and optional format files

In the Firco Trust configuration, you can make pipe-separated and fixed-length format files required independently.

• If one or both format types are required, every required file must be present in S3. If a required file is missing, processing stops with an exception.
• If neither format type is required, Evelyn still searches S3 for both files. If at least one is found, Evelyn attempts to map the message; a mapping failure stops processing with an exception.
• If neither format type is required and no matching format file is found, Evelyn uses CustomerInfo to map ScreenedData.

Handling messages that cannot be parsed

If an incoming Firco message cannot be read or parsed, for example, because a name or other required information is missing, Evelyn handles it once instead of retrying repeatedly. This prevents an unreadable message from consuming processing resources or leaving the alert or its stored data in an unclear state. Evelyn does the following:

1. Returns the alert to Firco for manual review with the note Failed parsing incoming message. Please review the alert manually. A reviewer can then supply the missing information or take the appropriate business action.
2. Marks the transaction as failed and sets it aside. The transaction is skipped for the rest of the screening flow, so Evelyn does not produce screening results, a model score, or a Quality Check report for it.
3. Deletes the stored Firco data created for the transaction, including temporary or partial records, so no stale or orphaned data remains in the system.

Successfully parsed messages are not affected and continue through the standard screening, scoring, and reporting flow.

Alias handling

Alias names are used by Evelyn to screen multiple names for an entity. Each alias is treated the same as the primary input name when calculating the similarity score. Additionally, each name in the screened data is compared with each name in the WLE data.

If an alias contains null or empty values, it does not contribute to the screening process but can still consume resources during comparison. To avoid this, all empty or null values should be filtered out during input data parsing.

Having duplicates among aliases also leads to unnecessary resource consumption. Furthermore, if the primary name (input_full_name or hit_matched_name) is enriched in subsequent steps and a duplicate exists in the aliases, it can cause inconsistencies.

During parsing of both input screened data and WLE data:

• All null or empty alias values are removed.

• All duplicate aliases are filtered out, including those duplicating the primary name.

Decision score calculation

The logic for calculating the decision score is as follows:

• If the decision is False positive, only fields used in discounting are included in the final score.
• If the decision is Need more information, all fields that do not contain BAD_INFO or NOT_COMPARED are included in the final score.