Skip to main content
Version: 3.4.1

Explore decision components for entity matching

Thresholds​

Thresholds can be set while configuring the model. For details, refer to Install and configure | Configure model.

Date of birth​

Date of birth (DOB) mismatch is true if any of the following is true:

  • Screened DOB and sanctioned DOB are not None, and the difference between them is greater than date_of_birth_threshold_days.

  • Additional DOB and sanctioned DOB are not None, and the difference between them is greater than date_of_birth_threshold_days.

In case a partial date is provided, the model calculates the minimum distance between dates to compare with the threshold.

tip

1956 versus 1957 is considered a match. The minimum difference between the dates is 1 day. 1956 versus 1958 is considered a mismatch. The minimum difference between the dates is 366 days.

Multiple DOB formats are supported, but the standard format mm/dd/yyyy should be used. The Business Process converts received DOB into the mm/dd/yyyy format and sends it to the model.

Supported formats are as follows:

Full date formats
  • MM/dd/yy
  • M/d/yy
  • M/d/yyyy
  • MM/dd/yyyy
  • MMM. d yy
  • MMM dd, yy
  • MMMMM dd, yy
  • MMM. dd, yy
  • MMM. dd yy
  • MMM. d, yy
  • MMM d yy
  • MMM-dd-yy
  • dd-MMM-yy
  • MMM d, yy
  • yyyy-MM-dd
  • MMM-dd-yyyy
  • dd-MMM-yyyy
  • MMMMM dd yyyy
  • MMMMM dd, yyyy
  • MMM. dd, yyyy
  • MMM. dd yyyy
  • MMM dd yyyy
  • MMM dd, yyyy
  • MMM. d, yyyy
  • MMM. d yyyy
  • MMM d, yyyy
  • MMM d yyyy
  • MMMMM d
  • MMM. d
  • MMM d
  • M/d
  • MM/dd
  • MM-dd
  • dd-MMM
  • MMM-dd
  • MMM dd
  • MMM. dd
  • MMMMM dd
  • MM.dd.yyyy
  • yy/MM/dd
  • yyyy/MM/dd
  • MMMM dd, yyyy
Partial formats
  • MMMM,yyyy
  • MMMM,yy
  • MMM,yyyy
  • MMM,yy
  • MMMM yyyy
  • MMMM yy
  • MMM yy
  • MMM yyyy
  • MM/yy
  • MM/yyyy
  • M/yy
  • M/yyyy
  • MM yy
  • yyyy-MM
  • yyyy/MM
  • yyyy-M
  • yyyy
  • yy

Country mismatch​

The model returns country_mismatch if the screened entity's country is different from the country of the sanctioned entity. As of now, the model only uses the provided country parameter to determine a mismatch.

tip

screenedData.country="GBR" versus sanctionedData.country="Iran" produces a mismatch. The model can accept country abbreviations as well as full country name. If values for comparison are missing, it is neither a match nor a mismatch.

The provided address, city, state, and postal information are utilized to identify the location using the libpostal Python library and geocoder software (if configured).

To consider neighboring countries for a potential match, you can configure the country matcher with the neighbours=True parameter. In this case, USA versus MEX will be a match.

Address mismatch​

Address match or mismatch is determined by comparing all addresses on the screened side against all addresses on the sanctioned side. The following options are available:

  • Literal string comparison. If all the tokens of an address are the same, the addresses must be the same.

  • Geocoder API. If any placeID of a screened address matches any placeID of a sanctioned address, the addresses must be the same.

  • In-house address verification API compares country, state, city, street, and postal code separately and returns match or mismatch per address element. Then, Tara makes overall match or mismatch decision and sets them in rules.

To set up address matching, refer to Install and configure | Configure model.

Primary name and aliases​

For a hit to be considered a name mismatch, the name extracted from the hit text (the NER-predicted name in the tagContent field) must not match any of the sanctioned names: the sanctionData fields name, primaryName, or any of the alsoKnownAs name fields (aliases).

Three values are computed:

  • wle_name_score: the probability that the screened name matches name

  • wle_primary_name_score: the probability that the screened name matches primaryName

  • wle_aliases_score: the probability that the screened name matches any of the alsoKnownAs name fields

If a sanctioned name field is empty, the value for that field is zero. If all values are below the threshold (or the screened name does not match any of the sanctioned entity's known names), it is a name mismatch.

Data validation rules​

Data validation is one of the important components in the model's decision-making process. Invalid requests and missing or corrupted data must be accounted for.

Request with no hits​

If a request does not contain hits, the PSS Business Process returns NO_DECISION for every hit with the comment: Message does not have any actionable alerts.

Invalid request​

If a request cannot be parsed or does not contain required fields, the PSS Business Process returns NO_DECISION with the Invalid Input comment followed by error details. For more information, refer to Providing input.

Name mismatch​

The decision model contains a unified name matcher that uses all available name inputs, such as a screened name and a predicted name, to compare against sanctioned names and aliases. It returns the original name and the processed name with the highest matching score.

The name matcher uses a pretrained model and a set of rules to make the best prediction.

The name matcher score is calculated according to entity types. For example, two individuals are compared differently than two companies.

Input example
{
   "id":"1",
   "type":"PAYMENT_TRANSFER",
   "hits":[
      {
         "id":"1",
         "tagContent":"Alice Smith",
         "tagName":null,
         "tagType":"FREETEXT",
         "hitStartIndex":null,
         "hitEndIndex":null,
         "hitText":null,
         "screenedData":null,
         "additionalData":{
            "type":"INDIVIDUAL",
            "name":"Alice B"
         },
         "sanctionData":{
            "name":"Alice Mason",
            "type":"INDIVIDUAL",
            "alsoKnownAs":[
               {
                  "name":"Alice Baker",
                  "type":"STRONG"
               }
            ]
         }
      }
   ]
}

Hit text portion​

Sometimes, you can have the exact location in the tag content where a hit occurred. The HitStartIndex and HitEndIndex fields are utilized in this case. For example, if the tag content is "Sherlock Holmes" and the hit was triggered due to "Sherlock", the HitstartIndex and HitendIndex are 0 and 7, respectively. The model uses these indexes to extract the hit text to make a decision. The hit text might be explicitly provided using the hitText field.

If the input contains hitText, the model applies the input value as hitText. Otherwise, a function is used to generate HitstartIndex and HitendIndex.

In some cases, the provided hitText might refer to multiple entities in the tag content. For example, if hitText is "Gandhi" and tag content is "Monika Gandhi lives in 221-North Street Gandhi-Nagar Gujrat", HitText can refer both to the individual name or the address. Therefore, in these scenarios, the hit is split into multiple hits, each resolved independently against the provided sanctioned object, and results are merged to provide the final adjudication.

Text preprocessing​

When dealing with free-text data, the content of the screening data is unpredictable. Some text preprocessing must be done before sending the screening context to Name Entity Recognition (NER), Entity Classifier (EC), and the name matcher to achieve better results. Currently, the following preprocessors are used:

  • Convert text to ASCII

  • Replace punctuation with spaces or remove it

  • Lowercase text

  • Filter out salutations and endings (such as Mr., Mrs., Jr., and so on)

  • Filter out legal endings (such as LTD, CO, LLC, and so on)

  • Filter out stop-list tokens

Company stop words​

See possible scenarios below:

  • "Star Entertainment LLC" → "Star Entertainment"

    Company stop words are legal endings and identifiers like "LLC" and "Limited Liability Company". When a name contains two or more tokens without company stop words, company stop words are removed. For example, "Star Entertainment LLC" and "Star Entertainment Limited Liability Company" become "Star Entertainment" for name matching.

  • "Star LLC" → "Star LLC"

    When a name is only one token and a company stop word, the company stop word is not removed. For example, "Star LLC" remains "Star LLC" and "Star Limited Liability Company" remains "Star Limited Liability Company". Company stop words can also appear at the beginning of a name, so "Joint Stock Company Star" remains "Joint Stock Company Star".

  • "Star LLC" versus "Star Entertainment" → Name mismatch

    The company stop word can cause a name mismatch. For example, "Star LLC" versus "Star Entertainment" is a name mismatch because "LLC" does not match "Entertainment".

  • "Star LLC" versus "Star INC" → Name match

    If both screened and sanctioned names have company stop words and the names are the same without the company stop words, they are considered a potential name match and will be escalated. For example, "Star LLC" and "Star INC" are a name match even though "LLC" does not match "INC" because, without company stop words, "Star" matches "Star".

No hit found in content​

Sometimes, a sanctioned name does not match any tokens inside the screened content. If no screened data is provided, the predicted name is not a fuzzy match, and it does not contain any blacklisted terms, the model uses the hitText field. Most likely, Tara returns ether the Name Match or Cannot resolve hit... narrative.

Additionally, company names need to have a legal company stop word as an ending. Examples include Ltd, llc, s.r.o, SARL. Either ss-name or the screened name (ideally both) must have a recognized company stop word as the last token. For example, "John stock" is a valid company name but "Long John Silver" is not.

There is a global list of legal company stop words recognized by the model. The same list is referenced throughout the Work.AI ecosystem.

Partial name mismatch​

If the company or individual names are not an exact match and not different enough to be a full name mismatch, the model outputs partial_name_mismatch. It can be used in combination with other factors like location_mismatch or date_of_birth_mismatch to increase the automation rate.

Blacklisted names and entities​

Entries in blacklists cause a hit to be escalated (return NO_DECISION) when found. Currently, the following blacklisted entities are used:

  • Blacklisted countries found in the country field: AFG, BDI, CAF, TCD, COD, COG, ERI, IRN, IRQ, CUB, PRK, LBY, SOM, SSD, SDN, SYR, VEN, YEM

  • Blacklisted company names found in text: BNP, Paribas, Novatek, Sberbank, Lukoil, isis

  • Blacklisted keywords found in text iranian, iran, syria, venezuela

Type mismatch​

Tara outputs a type mismatch when the screened entity is of a completely different type from the sanctioned entity. This is allowed as a stand-alone RESOLVE decision, for example, when the sanctioned entity is an organization like "Smith Enterprises Incorporated", but the screened entity is an individual like "Payment to John Smith". It can also occur when the sanction alert is triggered by something else entirely, such as a street address like "123 Smith Street". To reduce errors, NER and EC must agree on the entity type.

The sanctioned entity type is provided by the sanctionData.type field that can be INDIVIDUAL, ORGANIZATION, COUNTRY, LOCATION, VESSEL, or AIRCRAFT.

Type mismatch example 1
{
  "id": "123",
  "type": "PAYMENT_TRANSFER",
  "content": null,
  "contentFormat": null,
  "hits": [
    {
      "id": "123",
      "tagContent": "Payment to John Smith",
      "tagName": null,
      "tagType": "FREETEXT",
      "hitStartIndex": null,
      "hitEndIndex": null,
      "hitText": null,
      "screenedData": null,
      "additionalData": null,
      "sanctionData": {
        "name": "Smith Enterprises Incorporated",
        "type": "ORGANIZATION",
        "dob": null,
        "city": null,
        "country": "international",
        "state": null,
        "sourceEntryId": "123",
        "source": "eulist",
        "listType": "sanction list",
        "additionalInfo": null,
        "alsoKnownAs": []
      }
    }
  ]
}

If no screened entity type is provided, it is predicted from the tag content and hit text. For example, "Payment to John Smith" with a hit on "Smith" means the screened entity is identified as "John Smith" and labeled as INDIVIDUAL.

Type mismatch example 2
{
  "id": "123",
  "type": "PAYMENT_TRANSFER",
  "content": null,
  "contentFormat": null,
  "hits": [
    {
      "id": "123",
      "tagContent": "Payment to John Smith",
      "tagName": null,
      "tagType": "FREETEXT",
      "hitStartIndex": null,
      "hitEndIndex": null,
      "hitText": "Smith",
      "screenedData": null,
      "additionalData": null,
      "sanctionData": {
        "name": "Smith Enterprises Incorporated",
        "type": "ORGANIZATION",
        "dob": null,
        "city": null,
        "country": "international",
        "state": null,
        "sourceEntryId": "123",
        "source": "eulist",
        "listType": "sanction list",
        "additionalInfo": null,
        "alsoKnownAs": []
      }
    }
  ]
}

If the screened entity type is provided, Tara checks that the screened name matches the hit text to ensure the hit is actually on the screened entity. For example, if the screened name is "John Smith", but the hit is on "Star Shipping", the screened type of INDIVIDUAL is not used.

To minimize the error rate, Tara does not output a type mismatch if:

  • The screened name and sanctioned name are an exact match.

  • The sanctioned entity is ORGANIZATION, and the screened name is only one token (after removing initials) and does not contain salutations (for example, "J Smith" instead of "Mr. J Smith").

  • The sanctioned entity is INDIVIDUAL, and the screened name is only one token (after removing initials) and does not contain company abbreviations (for example, "J Bank" instead of "J Bank Inc.").

The model can return a type mismatch prediction when the hit occurs on a location, for example, the individual "Garry Adams" hit on the "123 Adams Street Brooklyn USA" location.

Type mismatch hit on location
{
    "id": "123",
    "type": "PAYMENT_TRANSFER",
    "content": null,
    "contentFormat": null,
    "hits": [
        {
            "id": "123",
            "tagContent": "message sent from bank 123 Adams Street Brooklyn USA",
            "tagName": null,
            "tagType": "FREETEXT",
            "hitStartIndex": null,
            "hitEndIndex": null,
            "hitText": "Adams",
            "screenedData": null,
            "additionalData": null,
            "sanctionData": {
                "name": "Garry Adams",
                "primaryName": "Garry Adams",
                "type": "INDIVIDUAL",
                "dob": null,
                "city": null,
                "country": "international",
                "state": null,
                "sourceEntryId": "123",
                "source": "eulist",
                "listType": "sanction list",
                "additionalInfo": null,
                "alsoKnownAs": []
            }
        }
    ]
}

Tara can return a type mismatch prediction when the hit occurs on an arbitrary word, for example, the organization "Message Services" hit on the "message" non-entity.

Type mismatch hit on other words
{
    "id": "123",
    "type": "PAYMENT_TRANSFER",
    "content": null,
    "contentFormat": null,
    "hits": [
        {
            "id": "123",
            "tagContent": "TM INVESTMENT PTY LTD ATF - this message sent from bank",
            "tagName": null,
            "tagType": "FREETEXT",
            "hitStartIndex": null,
            "hitEndIndex": null,
            "hitText": "message",
            "screenedData": null,
            "additionalData": null,
            "sanctionData": {
                "name": "Message Services",
                "primaryName": "Message Services LLC",
                "type": "ORGANIZATION",
                "dob": null,
                "city": null,
                "country": "international",
                "state": null,
                "sourceEntryId": "123",
                "source": "eulist",
                "listType": "sanction list",
                "additionalInfo": null,
                "alsoKnownAs": []
            }
        }
    ]
}

ID mismatch​

ID mismatch is identified if the hit is on something that resembles an ID. The NER model is utilized to identify certain tokens as IDs. If the hit is on an ID, name matching is no longer used to compare text. Instead, the hit is compared with a list of IDs provided in the sanctions data, if present.

"hits":[
    {
      "id":"1",
      "tagContent":" /IDC- 88-AS55513662 John Doe",
      "hitText" : "555",
      "tagType":"FREETEXT",
      "screenedData":{
        "ID": "",
        "type":"INDIVIDUAL",
        "name":"John Doe"

      },
      "additionalData":null,
      "sanctionData":{
        "name":"Alice Doe",
        "type":"INDIVIDUAL",
        "source":"Sanctions",
        "listType":"WORLD_COMPLIANCE",
        "primaryName":"Alice Baker",
        "codeMap":{
          "BIC":"555-3150-3t50",
          "IBAN":"666-555",
          "NID":"777",
          "PASSPORT_ID":"888"
        }
      }
    },

In the example above, hitText is 555, which appears to be inside the token 88-AS55513662 mentioned in the tag content. The model identifies this token as a possible ID and attempts to compare it with the list of IDs provided in sanctionData inside the codeMap object. In this case, an ID mismatch is returned as output since none of the provided code matches the token.

Additionally, the codeMap object can also be passed into screened or additional data separately and used to produce an ID match or ID mismatch decision, for example:

{
  "id": "123",
  "type": "PAYMENT_TRANSFER",
  "content": null,
  "contentFormat": null,
  "hits": [
    {
      "id": "1",
      "tagContent": " /IDC- 88-AS55513662 John Doe",
      "tagType": "FREETEXT",
      "screenedData": {
        "ID": "",
        "type": "INDIVIDUAL",
        "name": "John Doe",
        "codeMap": {
          "BIC": "555-3150-3t50",
          "IBAN": "666-555",
          "NID": "777",
          "PASSPORT_ID": "888"
        }
      },
        "additionalData": null,
        "sanctionData": {
          "name": "Alice Doe",
          "type": "INDIVIDUAL",
          "source": "Sanctions",
          "listType": "WORLD_COMPLIANCE",
          "primaryName": "Alice Baker",
          "codeMap": {
            "BIC": "555-3150-3t50",
            "IBAN": "666-555",
            "NID": "777",
            "PASSPORT_ID": "888"
          }
        }
      }
  ]
}

If the IDs are explicitly provided, the model uses this information instead of attempting to identify a possible ID within the tag content. The model then tries to match the input IDs against those provided in the watchlist data. If one or more IDs match, the model returns an ID match decision; otherwise, it returns an ID mismatch decision.

Last updated on