Skip to main content
Version: 10.3.2

Data Analyst responsibilities

From the article, you can learn the following information about the Data Analyst (DA) role:

  • Expectations from a person in the role
  • General workflow
  • Artifacts to produce
  • Cooperation with other roles

DA responsibilities and general workflow​

Regardless of the automation solution type, all Data Analyst responsibilities can be divided into three main stages:

  1. Dataset collection. The most important stage makes up more than half of all DA work. Any mistakes you make and fail to correct at the stage can entail serious risks in terms of timelines and meeting success criteria.

    As a prerequisite to the stage, DAs should do the following:

    • Get familiar with different aspects of the project
    • Clearly understand the business logic
    • Know the task type: Information Extraction or Classification
    • Identify what fields or classes are to be used, as well as document types and formats, and so on

  2. Model training and analysis of results. The stage requires close cooperation between DAs and ML Engineers.

  3. Preparation of the final report. At the end of the project, final results should be presented to the customer.

Stage 1. Dataset collection​

The stage includes the following steps:

  1. Get to know the document specifics and business logic thoroughly enough for further data processing.
  2. Guarantee an acceptable quality of the dataset after OCR conversion.
  3. Split documents into batches by some criteria (if it is not done by the customer).
  4. Define the labeling logic for each template.
  5. Train SMEs:
  6. Prepare Manual Tasks for labeling and assign batches to SMEs.
  7. Validate labeling results.
  8. Split datasets into training and test ones.

Stage 2. Model training and analysis of results​

Model training​

The process includes the following steps:

  1. Launch model training.
  2. Launch information extraction (IE) or classification on a test set with the assistance of machine learning engineers (MLE).
  3. Run the statistics aggregation Business Process.

Analysis of results​

The process includes the following steps:

  1. Analyze mistake types and possible reasons for them.
  2. Define rules for post-processing.

ML improvements and retraining​

This step is mostly the responsibility of MLEs and Data Scientists. However, a Data Analyst can support them in terms of analyzing interim results.

Stage 3. Final report preparation​

Analyze the final model results and prepare a report for the customer. For details, refer to the Report topic.