Basic Concepts for Task Implementation
This topic contains some basic concepts to keep in mind when creating a Manual Task.
Understand the Task Design
Is the Task size optimal?
If Task is too big: split it up into smaller Tasks.
If Task is too small: use Block Size > 1.
How is the Task being completed currently?
See the Results tab.
How to determine correct (valid) Answers?
- Confidence was found (defined by the Adjudication Rule), for example, 2 out of 3.
- See the answers of Workers with high Qualification and good performance results.
Additional questions
- What is the goal of the Task?
- What type of Workers will be suited for the Task?
- How long should the Task take to complete?
- Are there many variables to finding an answer?
- Is the Task subjective?
- Is there a right or wrong answer?
- Will the Task require Adjudication?
Work through the Task on a High Level
- Use one or two data points.
- Define your input data (what do you already know?)
- Define your output data (what are you trying to achieve?)
- Design the Task using the basic sections.
- Decide if the Task requires a Training Test and/or Qualification Task.
- Estimate the costs, determine the Adjudication rules, and Workforce. See the details here.
Sentiment analysis, surveys, or lengthy subjective Tasks do not work well with Gold Data. If you wish to qualify Workers for these types of Tasks, review their work manually, without the aid of the Gold Data.