9.4

Edit

Execution Queue monitoring

Business Process Analysis

Collecting Analytics via Control Tower

Prior to diving into SQL internals, it is suggested to follow instructions outlined on the BP Performance Tuning page.

Business Process Bot Record Execution Attempt

Configuration:

Re-loadable file without restart of WF: ${catalina.home}/conf/reloadable-wf.properties

Main properties:

bot.record.execution.track.failed.attempts=true
bot.record.execution.track.released.attempts=false

Behavior

When bot record execution fails and track flag is turned on (bot.record.execution.track.failed.attempts=true) then it is expected to have record in BotRecordExecutionAttempt), e.g.

How it works:

id    hitId   failed  executionStart          executionEnd
1   1       1       2018-08-14 14:20:49.179 2018-08-14 14:20:50.83

When bot record execution does not call export plugin and track flag is turned on( bot.record.execution.track.released.attempts=true), then it is expected to have record in BotRecordExecutionAttempt, e.g.

id    hitId   failed  executionStart          executionEnd
2   2       0       2018-08-14 15:20:49.179 2018-08-14 15:20:50.838

SQL check:

Executable Bot Records

SELECT * FROM BotRecordExecutionAttempt where hitId=?

Bot Record Execution Queue

SQL check

Executable bot records can be found via SQL query:

Executable Bot Records

set @global_retry_threshold = 10;
set @now = convert_tz(now(), @@session.time_zone, '+00:00');

SELECT c.title as bp_name, r.rootRunUUID as instance_uuid, r.title as step_title, ah.id as record_id
FROM Run r
JOIN Campaign c on c.id = r.campaign_id
JOIN RunDescription rd ON r.description_id = rd.id 
join AwsHit ah ON ah.run_id = r.id
WHERE r.executingType = 'MACHINE' AND r.status = 'MACHINE_PROCESSING' AND 
      ah.processingStatus = 'SUBMITTED' AND ah.hitOrigination='INTERNAL'
                    AND ((rd.maximumProcessingCount IS NOT NULL AND ah.processesAttemptCount <= rd.maximumProcessingCount)
                          OR (rd.maximumProcessingCount IS NULL AND ah.processesAttemptCount <= @global_retry_threshold)) 
                     AND ah.minimumExecutionDate <= @now;

Executable Bot Records (+bot source and record processing info)

SELECT NOW(),r.executingType, r.rootRunUUID, r.title, c.title, mcs.name, mcs.threadCount, mcs.batchSize, mcs.prioritizeEarlierRuns, ah.id, ah.creationTime, ah.expirationDate, ah.processingStatus, ah.prevProcessingStatus, ah.lastProcessingTime, ah.processesAttemptCount, ah.sandboxInd, ah.run_id, ah.completionDate, ah.submissionDate, ah.uuid, ah.sortIndex, ah.minimumExecutionDate, ah.splitted
FROM Run r
join RunDescription rd on r.description_id = rd.id
join Campaign c on r.campaign_id = c.id
join AwsHit ah on ah.run_id = r.id
left join CampaignMap cm on cm.id = r.campaignMap_id
left join Campaign stepCamp on stepCamp.id = cm.campaign
left join MACHINE_CONFIG mc on (mc.id = stepCamp.machineConfigId OR mc.id =
c.machineConfigId)
left join MachineCampaignSource mcs on mcs.id = mc.source_id
WHERE ((ah.processingStatus in ('READY_TO_SUBMIT', 'SUBMITTED') AND
 ((rd.maximumProcessingCount IS NOT NULL
AND ah.processesAttemptCount <= rd.maximumProcessingCount)
 OR (rd.maximumProcessingCount IS NULL
AND ah.processesAttemptCount <= 10))) OR ah.advancedNextStep = NULL)
AND EXISTS (select step.id from Run step where step.rootRunUUID = r.rootRunUUID
and step.status IN ('PROCESSING', 'MACHINE_PROCESSING'))
and r.executingType = 'MACHINE'
order by r.rootRunUUID, cm.stepPosition;

Example output:
bp_nameinstance_uuidstep_titlerecord_id
BP Empty Step0ddc361d-cb2d-4d44-8a59-f890c4959ae6Empty step35414

REST check (available since 8.4.0)

Using endpoint api/v1/monitoring/bots/sources:

Overview Report

curl -X GET http://localhost:8080/workfusion/api/v1/monitoring/bots/sources --user <user>:<password>

{
    "Member [127.0.0.1]:5702 - 932122e0-ec6e-45d5-aaa2-82ff959c1549 this": [
        {
            "uuid": "anonymous-source-0ddc361d-cb2d-4d44-8a59-f890c4959ae6",
            "threads": 5,
            "runningTaskCount": 1,
            "waitingWriteTaskCount": 0,
            "queuedTaskCount": 0,
            "runningTasks": [],
            "waitingWriteTasks": [],
            "queuedTasks": []
        }
    ]
}

Detailed Source Report

curl -X GET http://localhost:8080/workfusion/api/v1/monitoring/bots/sources/anonymous-source-0ddc361d-cb2d-4d44-8a59-f890c4959ae6 --user <user>:<password>

{
    "Member [127.0.0.1]:5702 - 932122e0-ec6e-45d5-aaa2-82ff959c1549 this": [
        {
            "uuid": "anonymous-source-0ddc361d-cb2d-4d44-8a59-f890c4959ae6",
            "threads": 5,
            "runningTaskCount": 1,
            "waitingWriteTaskCount": 0,
            "queuedTaskCount": 0,
            "runningTasks": [
                {
                    "taskUuid": "0ddc361d-cb2d-4d44-8a59-f890c4959ae6",
                    "businessProcessUuid": "0ddc361d-cb2d-4d44-8a59-f890c4959ae6",
                    "hitUuid": "75acc244-6eee-4be6-8775-0021f95afd94",
                    "executionTime": 673452,
                    "queueTime": 673471
                }
            ],
            "waitingWriteTasks": [],
            "queuedTasks": []
        }
    ]
}

BP Lifecycle Task Queue

Tasks are processed via thread pool of configurable size (default is 3). Only one task from BP instance is processed simultaneously. So for large number of simultaneous BP instances it can be increased.

workfusion.properties

#how many parallel threads to use for BP state transitions, e.g. human HIT submission, step completion, step transition, etc
bp.step.transition.pool.size=3

Task produces:

  • Step stats update:

    Stats Update

    select c.title as process_name, r.rootRunUUID as instance_uuid, r.title as step_name, r.id as step_id
from Run r
join Campaign c on c.id = r.campaign_id
where r.`status` in ('MACHINE_PROCESSING', 'PROCESSING') and r.submissionSize > r.competedSubmissions
order by r.rootRunUUID;
    process_nameinstance_uuidstep_namestep_id
    Task With Workforce Data027a0804-258f-485b-86ee-47148f67e2d9Task With Workforce Data452
    Automation Process: CZPffQnpbn0579c068-bc1d-41c3-8cb1-507e052b5aa5(Generic)Upload Training Set4
    BP Empty Step0ddc361d-cb2d-4d44-8a59-f890c4959ae6Empty step600

  • Manual step record transition:

    Manual Step Transition

    set @global_retry_threshold = 20;
SELECT DISTINCT e.runUUID FROM AwsHitProcessingEvent e WHERE e.processingCount < @global_retry_threshold;

  • Submission of manual tasks to crowds

    Crowd Submission

    SELECT root.uuid as instance_uuid, r.title as step_name, r.uuid as step_uuid
FROM StreamDescription sd 
JOIN Run r ON r.uuid = sd.objectUUID 
JOIN Run root on r.rootRunUUID = root.uuid 
WHERE sd.status = 'OPEN' AND r.status IN ('READY_TO_PROCESS', 'PROCESSING') 
      AND (root.businessProcessStatus IS NULL OR root.businessProcessStatus = 'PROCESSING')
ORDER BY sd.priority DESC
    instance_uuidstep_namestep_uuid
    8f94b6aa-579d-471f-a0ca-23b3b442c7a7[Regression] Gold accuracy8f94b6aa-579d-471f-a0ca-23b3b442c7a7

  • Bot step transition/completion

    Bot Step Transition

    SELECT c.title as bp_name, r.rootRunUUID as instance_uuid, r.title as step_name, r.id AS step_id
FROM Run r
JOIN Campaign c on c.id = r.campaign_id
WHERE r.submissionSize > 0 AND r. STATUS = 'MACHINE_PROCESSING' AND r.executingType = 'MACHINE'
AND (
    EXISTS (
        SELECT hit.id
        FROM AwsHit hit
        WHERE hit.run_id = r.id
        AND hit.processingStatus = 'DONE'
        AND hit.advancedNextStep IS NULL
    ) OR (r.competedSubmissions = r.submissionSize)
)
    bp_nameinstance_uuidstep_namestep_id
    Split-join bot000c92d8-00cc-49c4-add1-fa3cd5d7855dFinal Results17592
    Split-join botd6054a53-1331-426e-bf40-fe8586729f5fFinal Results19504

Performance metrics

Available in Useful logs and how to check them#performance-metrics.log.

← PreviousNext →