Edit

Data Analyst responsibilities

From the article, you can learn the following information about the Data Analyst (DA) role:

• Expectations from a person in the role
• General workflow
• Artifacts to produce
• Cooperation with other roles

DA responsibilities and general workflow

Regardless of the Digital Worker type, all Data Analyst responsibilities can be divided into three main stages:

1. Data set collection. The most important stage makes up more than half of all DA work. Any mistakes you make and fail to correct at the stage can entail serious risks in terms of timelines and meeting success criteria.

   As a prerequisite to the stage, DAs should do the following:

   • Get familiar with different aspects of the project
   • Clearly understand the business logic
   • Know the task type: Information Extraction or Classification
   • Identify what fields or classes are to be used, as well as document types and formats, and so on

2. Model training and analysis of results. The stage requires close cooperation between DAs and ML Engineers.

3. Preparation of the final report. At the end of the project, final results should be presented to the customer.

Stage 1. Data set collection

The stage includes the following steps:

1. Get to know the document specifics and business logic thoroughly enough for further data processing.
2. Guarantee an acceptable quality of the data set after OCR conversion.
3. Split documents into batches by some criteria (if it is not done by the customer).
4. Define the tagging logic for each template.
5. Train SMEs:
   • Hold a training session for SMEs.
   • Provide clear and specific instructions for labeling.
6. Prepare Manual Tasks for tagging and assign batches to SMEs.
7. Validate labeling results.
8. Split data sets into training and test ones.

Stage 2. Model training and analysis of results

Model training

The process includes the following steps:

1. Launch model training.
2. Launch IE or Classification on a test set with the assistance of MLEs.
3. Run the statistics aggregation Business Process.

Analysis of results

The process includes the following steps:

1. Analyze mistake types and possible reasons for them.
2. Define rules for post-processing.

ML improvements and retraining

This step is mostly the responsibility of ML Engineers and Data Scientists. However, a Data Analyst can support them in terms of analyzing interim results.

Stage 3. Final report preparation

Analyze the final model results and prepare a report for the customer. For details, refer to the Report topic.