Data Analyst job
To understand a Data Analyst's expected contribution to a successful project delivery, let's first outline the context of a cognitive automation implementation.
We will overview the project delivery from an entire-team perspective. It is a joint effort to implement an ML Digital Worker, so effective collaboration while understanding one's own contribution is part of the Data Analyst job.
Problem statement
Operations businesses in many industries (banking, insurance, financial services, manufacturing, healthcare) have a current process of handling documents or information. This consists of many manual steps that, all together, build a workflow. A business goal is to reduce manual work and make processes more cost-effective: This problem defines the goal of Operations automation. A manual workflow is transformed into a Business Process when automation is set up. Manual steps in the workflow are replaced by Bot Tasks and a machine learning (AutoML) Business Process. The transformation of manual workflows into Business Processes is performed for each Digital Worker to mimic all specific details, branches of logic and amounts of data to be processed.
Documents in the workflow have to be processed in a certain way, defined by Digital Worker requirements. Usually, data from the documents has to be taken out and then handled in other steps of the workflow. For example, purchase order details (quantity, product, price) have to be entered into internal applications like SAP, CRM systems, and so on. When there are many document types, they can be processed in different ways. For example, data from purchase orders has to be entered into SAP, and data from invoices has to be sent by email in table format. So to best handle documents in this case, it's necessary to detect document type first.
Digital Worker lifecycle
To understand the ML Data Analyst's contribution to a successful project delivery, first we need to understand and properly start practicing project process. The process, with phases described below, has been proven to work and practiced by WorkFusion PoC while delivering complex ML+RPA implementations for customers.
On a high level, each Digital Worker goes via five main phases. Each phase has its requirements, steps and roles involved. The Data Analyst contributes during the Implementation phase.
7 stages of implementation phase
Let's logically structure a typical Digital Worker Implementation in stages defined by separate people doing specific work and handing-off results to the next person:
An implementation consists of ML and RPA parts. RPA deals with well-structured and repeatable processes and develops automation that replaces Manual Tasks. ML development delivers a model that handles unstructured or semi-structured data in the documents. The model is trained on a data set collected from original documents. A high-quality data set is the Data Analyst's main responsibility. The better the quality of a data set is, the higher results of ML.
Implementation can be delivered as PoC (Proof of Concept), Pilot, or Production, depending on the scope of the Digital Worker.
In data set collection, the Data Analyst studies business logic and applicable documents, and defines the rules and corner cases. The Data Analyst works closely with the Subject Matter Experts (SMEs) from the customer/partner side on the Data Set collection step. They have a deep understanding of documents' logic and tag the data set for model training. The Data Analyst trains SMEs to tag the documents and verifies tagging.
The Machine Learning Engineer or Automation Engineer Specialist leads model training and the Data Analyst analyzes ML results and defines rules that help to improve the model. ML results are evaluated by comparing the results provided by people (manual extraction or classification that is considered to be correct) and calculating quality metrics statistics. The Data Analyst calculates interim and final statistics and submits these to the customer in the form of a report.
At the analysis of results stage, the Data Analyst needs to review the model execution results and, if necessary, propose ways to improve. The model is trained on the training set for each training iteration. If there are several iterations, the Data Analyst calculates statistics of each iteration based on evaluation results. The Data Analyst needs to calculate statistics for each iteration and analyzes the delta for tagging iterations and model mistakes.
ML improvements and retraining is handled by the Machine Learning Engineer or Machine Learning Engineer + Data Scientist. This is the most complex stage, as it includes creation of custom model components.
The (first) final report of automation results has to be shared with the customer, together with any reasons why some stats on some fields may fall below success criteria. Moreover, further steps and solutions should be proposed.
In the integration of ML results into RPA stage, structured data made by machine learning is entered into organization's internal systems. Typically, RPA is used for that because corporate applications lack API support and the only way is to have bots simulate user interactions with UIs.
During the final report, analytics helps DAs and business users to see and understand outcomes of automation in production. This may include aggregated numbers showing savings, amount of work done, SLAs, average ML measures, etc.