Edit

Result data and snapshots

The Data page is intended for viewing the Manual Task (MT) results. To access it, select Manual Task > View All, and click a completed MT in the list. Under the Results menu on the left, select Data.

The displayed results grid has the following columns:

• columns from the original Data file

• three columns for each Answer:

  • Answer code; for details, read Questions and Answers.

  • Answer confidence—a qualitative parameter showing the number of workers with coincident Answers per Record.

    For instance, when the answer confidence value is "2 out of 3", this means that three workers have completed the Record, and two of them provided coincident Answers.

  • Answer accuracy—a parameter determining the probability that the majority Answer is correct. The calculation is based on the previous worker statistics in similar tasks.

Worker accuracy is the percentage of all worker Answers that coincided with the Answers provided by the majority of other workers.

Low confidence

When the answer confidence is low (1 of N), the accuracy for that Record is not calculated. To address the situation, use one of the following solutions:

• Copy the task and upload a Data file containing all Records with low confidence.
• Repeat the same actions as in the previous solution and edit the set of worker Qualifications required to accept the task.
• Repeat the same actions as in the previous solution and edit the target Crowd for the task. For example, you can use your qualified internal workers.
• Create a Moderation Task to verify the Answers.

View

The grid with Records has horizontal scrolling, and you can manage the number of displayed columns by clicking the cogwheel in the top right corner.

If you do not see some Records, check the applied filters by clicking the Show button in the top right corner or check the Choose Status drop-down box.

To view Gold Results, click the Choose View dropdown and select the appropriate element.

Export

To export MT data, follow the steps:

1. Select or create a filter and apply it.
2. Depending on the format needed, click the Export to Excel or Export to Csv button.

Snapshots

To save the intermediate or final MT results in Excel, click the Generate Snapshot button. No filters are applied to snapshots.

If you have problems with viewing exported UTF-8 data in Excel, read Import and export UTF-8 data.

The following rules apply to snapshots:

• MT snapshots are generated automatically after the MTs are completed.
• If an MT is not completed, the snapshot contains only completed records.
• For an expired MT, snapshots contain answers for completed records and input data for records without answers.
• Snapshots for Qualification Task are empty as correct answers are already known.

Gold Data

You can use provided Answers to create Qualification Tasks by converting Records to Gold Data:

1. Selecting appropriate Records.
2. Clicking the Convert to Gold button.

To undo the operation:

1. In the Choose View drop-down box, select Gold Results.
2. Select appropriate Records and click the Delete Gold Questions button.

Preview Answers

To view the details of a Record, click the appropriate "eye" icon in the third column.

The following popup screen appears:

To preview a Record in a new window, click the External URL link at the top.

Worker Answers

To compare Answers from all workers, click the Worker Answers tab.

To display additional Answer and Record parameters, click the Show Extra Columns or Show Original Data link.

The Worker accuracy is a dynamic parameter and can change after a worker completed an MT. You can recalculate Worker accuracies to use the actual value for calculating the Record Accuracy. To initiate the recalculation, click the Actions and select Recalculate Accuracy.

Audit Trail

On the Audit Trail tab, you can view:

• all worker task events
• event duration and time
• payment details
• workers who submitted answers