Skip to main content
Version: 4.6

Analyze results

Along with the out-of-the-box operational analytics of the Work.AI platform, the Adverse Media Monitoring (AMM) AI Agent includes the following pre-built dashboards for real-time continuous monitoring of the solution by operations managers:

The dashboards allow the following:

  • Get insights into the solution's effectiveness

  • Track overall performance

  • Search for improvement opportunities, identify and troubleshoot issues

To access the dashboards, log in to Control Tower and click Analytics > Dashboards.

Adverse Media Monitoring dashboard​

The Adverse Media Monitoring dashboard focuses on processing time and quality. It allows you to track the following metrics:

  • Automation rates
  • Average time spent on assignments
  • Matched and missed False Positive and True Positive statuses in a table and diagram
  • Article-specific statistics
  • Screened entity analysis
  • Investigation statuses across countries

Filtering options allow you to display data for specific Business Process instances, AI Agent versions or variations, and model versions. Additionally, you can filter screening results by the time period:

The dashboard includes the views described in the sections below.

  • Entity investigated shows the overall number of entities processed daily with breakdown by the investigation status. The view allows you to evaluate and understand the volumes and trends of investigations to be performed on a daily basis and its relative volumes in all searches.

  • Entities eligible for straight through processing shows how many entities were screened that could have been closed automatically by Evan.

  • Entities with no results shows how many entities were screened that had zero results.

Article-related metrics​

  • Articles shows how many articles were processed daily with breakdown by the investigation status. The view allows you to evaluate and understand the volumes and trends of investigations to be performed on a daily basis and its relative volumes in all searches.

  • Article automation rate shows how many articles were automatically dispositioned by Evan. This metric helps evaluate the efficiency of the ML model in reducing manual workload. It includes the following metrics:

    • Total automation rate: the overall percentage of articles automatically processed
    • ML automation rate: the percentage of articles automated by the machine learning model
    • ML automation rate (ES): the automation rate for articles in Spanish
    • ML automation rate (EN): the automation rate for articles in English

  • Article count shows the diagram of False Positive, Missed False Positive, Missed True Positive and True Positive articles distributed across time. It includes the following metrics:

    • False Positive Articles: percentage of articles where the ML model correctly identified a False Positive.
    • Missed False Positive Articles: percentage of articles where the ML model suggested Needs Investigation, but the manual review identified them as False Positive.
    • Missed True Positive Articles: percentage of articles where the ML model suggested False Positive, but the manual review identified them as True Positive or Needs Investigation.
    • True Positive Articles: percentage of articles that were correctly identified by the ML model as True Positive or Needs Investigation, or articles were correctly left for manual review.

  • Average articles per entity shows how many articles are returned, on average, for an entity.

  • Article count by reviewer provides visibility into the distribution of article reviews among different team members, allowing for workload assessment and performance monitoring.

  • Article automation: the pie chart provides a detailed breakdown of how articles are being processed, showing the distribution between automated and manual processing. This visualization helps identify trends in automation effectiveness over time.

  • Article ML adjudication reason: shows the reasons the ML model provides when making automated decisions about articles. This helps understand the ML decision-making patterns and identify areas for model improvement.

Manual review metrics​

  • Average time spent on assignment displays the summary information about the average time (in minutes) per assignment for assignments submitted daily over a selected period.

    From the example below, you can see that, over seven days, the average manual time is 1 minute 38 seconds per task. You can see the manual processing step behavior is stable, and there are no dependencies on task volumes—hence, the team works well.

Cost centre details​

The visualization shows the distribution of processing costs across different aspects of the Adverse Media Monitoring workflow. This helps in understanding resource allocation and identifying potential areas for cost optimization.

Investigation status​

From the view, you can get the following information:

  • Top locations for screened entities

  • Which locations give specific statuses: False Positive, True Positive, Needs Investigation, No results found.

Adverse Media Monitoring Investigation Details dashboard​

The dashboard allows you to review the investigation history. You can search by various parameters and choose columns to display.

Adverse Media Monitoring Article Details dashboard​

The dashboard allows you to review all processed articles. You can search by various parameters and choose columns to display.

Adverse Media Monitoring Blocked URLs​

The Adverse Media Monitoring Blocked URLs dashboard is part of the Blocked URL Management mechanism for managing URLs in the AMM ignore list. From the dashboard, users can add or remove URLs in the ignore list as part of the approval process for any blocked URLs based on user feedback from human-in-the-loop tasks.

The dashboard displays the following key information:

  • Blocked URL: specific ignored URLs
  • Status: current state of the URL (enabled/disabled)
  • Added by: users who added the URL to the ignore list
  • Created date: when the URL was blocked
  • Ignore type: whether the block applies to a specific page for an entity, any entity, or the entire website

The table on the left lists all links from the AMM ignore list, with their status displayed in the Status column:

  • Enabled: the link is excluded from the AMM search results.

  • Disabled: the link is included in the AMM search results.

The following actions are available:

  • To remove a URL from the AMM search results, choose the row corresponding to the link and click the ENABLE button in the Action column.

  • To add a URL to the AMM search results, choose the row corresponding to the link and click the DISABLE button in the Action column.

tip

For more information on how the ignore list works, see the guide.

Last updated on