Understand CLDS calculations

Continuous Learning Decision Reapplication (CLDS) uses historical analyst decisions to determine whether Tara can reapply a decision to a matching live hit.

For instructions on preparing and uploading historical decision files, see Apply continuous learning.

Trace CLDS workflow

CLDS works in two phases that change different parts of the same record:

• Historical ingest runs when you upload historical decision files. Tara groups matching hits, creates a record for each hit pattern, and builds the evidence for it: the release count, the releasing analysts, the all-released flag, the latest analyst comment, and the release and escalation streaks.

• Live screening runs when a hit is screened while decision reapplication is enabled. Tara reads the evidence and decides whether to clear the hit, escalate it, or let it continue through standard processing. It updates only the hash status, the enable date, and the thresholds used in the evaluation.

Live decisions do not add to the historical evidence. The release count and analyst list of a hit pattern stay the same no matter how many hits Tara clears against it. They change only when you upload historical decisions again.

Calculate hit hash

FUF suffix-based tag parsing does not change the hit hash used by CLDS and decision reapplication.

A CLDS hash represents a hit pattern, not an individual message. Hits from different messages use the same CLDS record when their hash inputs normalize to the same value.

Tara uses these hit fields in the following fixed order:

1. MESSAGE_SUB_TYPE
2. TAG_NAME
3. HIT_TEXT
4. HIT_TYPE
5. TAG_CONTENT
6. SANCTION_DATA_NAME
7. SANCTION_DATA_PRIMARY_NAME
8. SANCTION_DATA_COUNTRY
9. SANCTION_DATA_ENTRY_ID

The following settings change the hash inputs:

• decisionReapplication.matchTagContent includes TAG_CONTENT. If the value is missing during ingest, Tara derives it from the message content and tag name.

• decisionReapplication.matchMessageSubtype includes MESSAGE_SUB_TYPE.

• decisionReapplication.tagsDataGrid adds selected message tags as sorted KEY:VALUE pairs.

• input.valuesToRemove removes configured placeholder values, such as Unknown, N/A, or *. A single-character entry, such as *, also matches a value made up entirely of that character, ignoring spaces.

Tara reduces SANCTION_DATA_COUNTRY to the first country and converts it to an ISO alpha-2 code. It then converts values to lowercase, removes punctuation and separators, normalizes whitespace, and uses - for empty values. The normalized values and the selected message tags form the hash_id. Sanction type, city, date of birth, and state are not part of the hash.

For live hits, if the sanction country contains several values separated by pipes, such as RUSSIAN FEDERATION|RUSSIA, Tara uses the first value. If the sanction country is empty, Tara uses the country from the hit's locations.

Tara applies the same fields, settings, and normalization rules to live hits, so a live hit and a historical hit of the same pattern produce the same hash_id.

Select message tags for NATIVE content

When the content format is NATIVE, Tara selects the message tag list from decisionReapplication.tagsDataGrid by message type: NATIVE with SWF uses the SWIFT_MT tag list, and NATIVE with ISO_20022 uses the SWIFT_MX tag list.

Historical files cannot carry the message type for NATIVE content. Such rows are hashed without message tags and do not match live hits when decisionReapplication.tagsDataGrid lists tags for SWIFT_MT or SWIFT_MX. Provide this history with an explicit SWIFT_MT or SWIFT_MX content format. See Historical file columns.

tip

Before Tara 4.1.1, individually ingested records with NATIVE content and the SWF or ISO_20022 message type were hashed without message tags and never matched live hits. Re-ingest such records so that they start matching.

caution

Do not change decisionReapplication.matchTagContent, decisionReapplication.matchMessageSubtype, decisionReapplication.tagsDataGrid, or input.valuesToRemove after ingesting historical data. A change produces a different hash for the same logical hit. Existing records are not recalculated and no longer match live hits.

Build historical evidence

Determine which decisions count

Tara excludes the following rows from CLDS evidence:

• Rows from unpaired files. A messages file without its matching hits file, or a hits file without its messages file, is recorded as FAILED in the file analytics Data Store and skipped.

• Rows that do not contain the message ID, hit ID, decision, and decision date. For the complete input requirements, see Historical file columns.

• Decisions older than decisionReapplication.numDaysToLookBack and decisions whose date cannot be parsed. The default lookback period is 365 days. These decisions are still saved to the reapplication Data Store, so it can hold more history than the CLDS analytics.

• Decisions that were already counted. Tara uses the message ID, decision date, and hit hash to prevent the same evidence from being counted twice, so re-uploading the same file does not change the counters.

Different messages with the same hit pattern contribute separately to the same CLDS record. The same message decided on another date is counted again. If one message contains two hits with the same hit pattern and decision date, only the first hit in file order is counted, and the second is discarded without a report. When the two hits carry conflicting decisions, only the first decision is represented.

caution

Placeholder removal is applied to the hash_id but not to the deduplication key. Rows that differ only by removable placeholders, such as Unknown and N/A, can therefore have different deduplication keys but update the same CLDS record. This can increase the record's counters more than expected.

Apply release and escalation decisions

In the merged file, RESOLVE counts as a release. Any other value, normally NO_DECISION, counts as an escalation. Tara derives these values from decisionReapplication.resolveTypes. When hit-level and message-level decisions are both available, the hit-level decision takes precedence.

Field	Release	Escalation
hash_counter	Set to 1 for a new record; otherwise, increase by 1.	Set to 0 for a new record; otherwise, do not change.
analyst_names	Add the analyst if not already present.	Do not add the analyst.
is_all_released	Set to true for a new record; otherwise, keep the current value.	Set to false.
hash_comment	Replace the comment if the new comment is not blank.	Replace the comment if the new comment is not blank.
last_released_times	Set to 1 for a new record; otherwise, increase by 1.	Reset to 0.
last_escalated_times	Reset to 0.	Set to 1 for a new record; otherwise, increase by 1.

As a result:

• hash_counter counts releases only. A hit pattern with 3 releases and 50 escalations has hash_counter = 3.

• analyst_names contains distinct releasing analysts only. An analyst who only escalated the hit pattern never appears, and an analyst who released it ten times counts once.

• After one escalation, is_all_released remains false. Later releases cannot restore it. Only dynamic resolve can qualify the hit pattern again.

• last_released_times and last_escalated_times are current streaks, not totals. Each resets the other, so at most one is greater than 0.

• hash_comment holds one comment, not a history. The latest non-blank comment wins, whether it came from a release or an escalation.

info

Tara processes rows in file order and does not sort them by decision date. Upload historical decisions in chronological order so that the release and escalation streaks represent the latest decisions.

Evaluate live hit

Live screening never creates a CLDS record. If no record matches the hash of the live hit, Tara returns NA and the hit continues through standard processing.

CLDS evaluation is skipped entirely when decisionReapplication.isEnabled is disabled or when upstream processing failed for the message.

The decisionReapplication.level setting controls how results are applied:

• hit evaluates and clears each qualifying hit independently. Cleared hits are removed from the message, and the remaining hits continue through standard processing. This is the default value, which is also used when the setting is missing or empty.

• Any other value resolves the message only when every hit returns FALSE_POSITIVE.

Tara evaluates each hit separately in both modes, and records the evaluation against the hit pattern even when the hit is not cleared.

Apply decision order

For an existing hash, Tara applies these checks in the following order:

Order	Check	Result
1	Automatic escalation is enabled and the hash is manually disabled.	ESCALATE
2	Automatic escalation is enabled, is_all_released is false, and hash_counter is 0.	ESCALATE
3	A screened country is on the configured blacklist.	NA
4	The hash satisfies the false-positive rules.	FALSE_POSITIVE
5	No earlier condition applies.	NA

The first matching check determines the result. Therefore, a manually disabled hash is not re-evaluated while automatic escalation is enabled.

An escalated hit receives NO_DECISION as its suggested action. For the blacklisted-country check, Tara searches the screened country, address, locations, and country codes extracted from identifiers. A hit skipped because of a blacklisted country carries the reason Skipped: found blacklisted country <COUNTRY>, and the hash is left unchanged.

Check false-positive evidence

A hash qualifies as a false positive when all of the following are true:

• hash_counter is at least decisionReapplication.minimumCounter.

• The number of distinct releasing analysts is at least decisionReapplication.minimumAnalysts after exact names from decisionReapplication.excludeAnalyst are removed.

• No dynamic escalation applies.

• Either is_all_released is true or dynamic resolve applies.

A blank analyst list or an empty counter does not satisfy the corresponding requirement.

Understand hash status

The is_hash_enabled field has three meaningful states:

• Empty: not yet evaluated, no longer qualified, or suspended by a dynamic escalation. Historical ingest never enables a hash, so a newly ingested record starts in this state and the next matching hit runs the full evaluation.

• true: CLDS can clear matching hits when no earlier check prevents it.

• false: disabled. Only a manual override sets this value. With automatic escalation enabled, matching hits are escalated without evaluation.

Tara acts on the status as follows:

Status	Behavior
Empty	Evaluate the current evidence. If the hash qualifies, set the status to true, set hash_enabled_date, and return FALSE_POSITIVE. Otherwise, return NA. In both cases, store the thresholds used in the evaluation.
true	Return FALSE_POSITIVE without a write when the configured thresholds have not changed. If they changed, evaluate the evidence again and store the new thresholds. If the hash no longer qualifies, clear the status and the enable date and return NA.
false	Return NA without a write unless dynamic resolve triggers re-evaluation and automatic escalation is disabled.

When changed thresholds disqualify an enabled hash, Tara clears the status instead of setting it to false. The next matching hit therefore runs a new evaluation instead of being escalated automatically.

Live screening updates only the hash status, the enable date, and the stored thresholds. It does not change the historical counters, analyst names, comment, or streaks.

Apply dynamic changes

When decisionReapplication.dynamicEnabled is enabled, the following actions occur:

• An escalation streak that reaches decisionReapplication.dynamicEscalateTimes suspends an enabled hash. Tara refuses the hit and clears is_hash_enabled and hash_enabled_date. A suspended hash resumes clearing hits only after a release streak reaches decisionReapplication.dynamicResolveTimes and the configured minimums are met. While decisionReapplication.dynamicResolveTimes is unset or 0, the hash stays suspended until an operator enables it with a hash status change file.

• A release streak that reaches decisionReapplication.dynamicResolveTimes can allow a disabled hash, or a hash with an earlier escalation, to qualify again.

An unset or 0 threshold disables the corresponding dynamic check. A threshold of 1 acts on the latest decision.

For example, with decisionReapplication.dynamicEscalateTimes set to 3:

• After three consecutive escalations, an enabled hash is suspended and the hit is refused.

• If decisionReapplication.dynamicResolveTimes is also 3, one later release does not resume the hash. After three consecutive releases, the hash is enabled again and the hit is cleared.

caution

• A dynamic escalation does not overwrite a manual disable. A hash set to false by an operator stays false.
• When automatic escalation is enabled, a hash is suspended by a dynamic escalation in the same way, and the hit is not escalated automatically.
• Dynamic resolve can override a manual disable only when automatic escalation is disabled. When automatic escalation is enabled, the earlier escalation check applies first.

Review examples

The following examples assume one shared hit pattern, decisionReapplication.minimumCounter and decisionReapplication.minimumAnalysts set to 2, dynamic changes disabled, and automatic escalation enabled:

Historical decisions	Resulting evidence	Expected live behavior
Three releases by two distinct analysts	hash_counter = 3, is_all_released = true, release streak 3	Qualifies when the configured minimums are met. The first matching hit is cleared and enables the hash; later hits are cleared without further writes while the configuration is unchanged.
Three escalations	hash_counter = 0, is_all_released = false, escalation streak 3, empty analyst list	Escalated automatically when automatic escalation is enabled. Otherwise, the hit returns NA because no analyst released the hit pattern.
Release, release, escalation, release	hash_counter = 3, is_all_released = false, release streak 1	Does not qualify under the all-released rule. Every later hit is evaluated again and refused. Dynamic resolve can qualify it after a sufficient release streak.
Three releases by two distinct analysts; after the hash is enabled, decisionReapplication.minimumAnalysts is raised to 3	Unchanged	The next hit returns NA because two analysts are fewer than three. The status and enable date are cleared, so the hit continues through standard processing and is not escalated automatically.

Configure CLDS

The following settings control live screening:

• decisionReapplication.isEnabled: the master switch. When it is disabled, CLDS is skipped.
• decisionReapplication.level: selects hit-level or message-level reapplication. When the setting is missing or empty, hit-level reapplication is used.
• decisionReapplication.minimumCounter: the minimum release count required to qualify.
• decisionReapplication.minimumAnalysts: the minimum number of distinct releasing analysts required to qualify.
• decisionReapplication.excludeAnalyst: comma-separated analyst names excluded from the count.
• decisionReapplication.escalate: enables automatic escalation.
• decisionReapplication.checkBlacklistedCountries: enables the blacklisted-country check.
• model.blacklistedCountries and model.additionalBlackListedCountrySource: define the blacklisted countries and the optional additional source.
• decisionReapplication.dynamicEnabled: allows release and escalation streaks to affect hash eligibility.
• decisionReapplication.dynamicResolveTimes: the release-streak length that can qualify a hash or resume a suspended hash. An unset or 0 value disables the check.
• decisionReapplication.dynamicEscalateTimes: the escalation-streak length that suspends an enabled hash. An unset or 0 value disables the check.

The following settings control historical ingest:

• decisionReapplication.numDaysToLookBack: the age limit for decisions included in CLDS evidence. The default is 365 days.
• decisionReapplication.resolveTypes: the source decision types treated as releases.
• decisionReapplication.copyMessageCommentsToHits: copies message comments to hit rows in multi-hit messages.
• s3.bucket: the source bucket for historical decision files. When it is not configured, Tara uses the default PSS bucket.

The following settings define the hash inputs and therefore affect both phases. Do not change them after ingesting historical data:

• decisionReapplication.matchTagContent: includes TAG_CONTENT in the hash.
• decisionReapplication.matchMessageSubtype: includes MESSAGE_SUB_TYPE in the hash.
• decisionReapplication.tagsDataGrid: selects the message tags appended to the hash, by message format and type.
• input.valuesToRemove: the placeholder values removed before hashing.

Override hash status

Use a hash_status_change*.csv file to enable or disable an existing hash manually. Only TRUE, ignoring case, enables the hash; any other non-blank value, such as 1 or yes, disables it.

An override records the operator and the time the file was processed. It does not create a missing hash, change the historical evidence, or set hash_enabled_date. For the file requirements, see Upload hash status change.

Each row must contain the hash ID, the status, and the operator. Rows with a blank value are skipped, and a hash ID that does not exist is reported as not found. The results are recorded in the file analytics Data Store:

• A SUCCESS row when at least one hash was changed.
• A FAILED row when at least one hash was not found, or when the file has no rows that can be processed.

A single file can produce both a SUCCESS and a FAILED row.

Interpret CLDS results

Tara copies the complete CLDS record onto the screened message, so reports can show why a hit was or was not cleared. In the CLDS report, the historical_reference column lists the historical alert and hit IDs that contribute to the CLDS decision, and the hash_map column lists all values used to calculate the hash code. Keep the following in mind when reading the values:

• The stored minimum thresholds show the configuration in force at the most recent evaluation. Their presence proves that the hash was evaluated, not that it qualified.
• hash_enabled_date is set only when CLDS itself enables a hash. A hash enabled by an override has a status of true and no enable date.
• The enable date is cleared, not archived, when a threshold change disqualifies a hash or a dynamic escalation suspends it. Capture the date outside CLDS if you need a history of enablement.
• The override timestamp is taken once per file, so all hashes changed by the same file share it. It stays empty for hashes that no operator has ever changed, and it is not a general last-modified timestamp.
• The operator name identifies the most recent override only. It is never cleared, so it can remain next to a status that CLDS later set itself.
• Unpopulated values are not consistent. A record created by an escalation stores an empty analyst_names string, and a record created from a blank comment stores an empty hash_comment, while the status, dates, operator, and thresholds are empty. When you query for hit patterns with no analysts, check for both. To find hit patterns that were never evaluated, query for an empty status.
• The is_auto_disabled column is removed from the hash analytics Data Store. Reports, SQL scripts, or extracts that still reference it fail.

When a CLDS result is not what you expected, check the following symptoms and causes:

Symptom	Likely cause
A live hit matches no CLDS record, although history was uploaded for it.	A hash input setting changed after ingest, so the same logical hit now produces a different hash. Reload the history with the current settings.
hash_counter is lower than the number of uploaded decisions.	Escalations do not increase the counter, decisions outside the lookback period are excluded, and repeated uploads of the same decision are counted once. In rare cases, an update that fails after several retries is skipped, so the counters can fall behind the reapplication data.
hash_counter is higher than expected.	Rows that differ only by removable placeholder values were counted separately. See the deduplication caution above.
A hit pattern with only releases does not clear hits.	The release count or the number of distinct releasing analysts is below the configured minimum. Check whether decisionReapplication.excludeAnalyst removes one of the names.
A hit pattern with many releases does not clear hits.	One escalation set is_all_released to false permanently. Only dynamic resolve can qualify it again.
A hash reports true but its hits are not cleared.	The hits are skipped because of a blacklisted country.
A previously enabled hash has an empty status and no enable date.	A dynamic escalation streak suspended the hash, or changed thresholds disqualified it. A suspended hash resumes after decisionReapplication.dynamicResolveTimes consecutive releases.
An analyst is missing from the analyst list.	The analyst only escalated the hit pattern, or the name is listed in decisionReapplication.excludeAnalyst. Exclusion matching is exact, so spelling and capitalization must match the historical file.
The stored comment describes a true positive on a hit pattern that clears hits.	The comment field keeps only the latest non-blank comment in file order, and an escalation comment can replace a release comment. For multi-hit messages, the comment can also be a copy of the message-level comment when decisionReapplication.copyMessageCommentsToHits is enabled.