Multi-document Manual Task

A multi-document Manual Task enables automation for the cases when you process multiple documents as a part of a single transaction or in a batch, both manually and using machine learning. For more details about multi-document Manual Tasks, refer to Multi-document IE Manual Task. The instruction describes the ODF 2 built-in API to prepare and process data for multi-document Manual Tasks.

Generic data model

There is no common Transaction representation in ODF 2, so preparing data for multi-document Manual Tasks is actually transforming some business entities to a representation you can process with Manual Tasks. For now, the JSON representation of the ODF Transaction is used in multi-document Manual Tasks. Thus, ODF 2 contains the MultidocTaskData class for representing the same structure. This class is a wrapper for MultidocTaskDocument that is the ODF 2 representation of Document and MultidocTaskMeta used to pass global answers.

Utilities to define data transformations are represented as MultidocTaskDataService that helps to prepare input data for multi-document Manual Tasks and process the output. The service provides a fluent builder for all data required for a multi-document Manual Task and hides the complexity of object building and working with TaskInput and TaskOutput inside the object.

Prepare data for multi-document Manual Task

To prepare data for a multi-document Manual Task, define a process of transforming a business entity to MultidocTaskData using MultidocTaskDataService.prepareMultidocTaskData() in a given way. See the comments in the code to go through the task input building process:

import java.util.Collection;
import java.util.UUID;
import java.util.stream.Collectors;
import javax.inject.Inject;

import com.workfusion.odf2.client.model.Attachment;
import com.workfusion.odf2.client.module.ClientOdfModule;
import com.workfusion.odf2.client.repository.AttachmentRepository;
import com.workfusion.odf2.compiler.BotTask;
import com.workfusion.odf2.core.cdi.Requires;
import com.workfusion.odf2.core.task.intake.InputEntityProcessorTask;
import com.workfusion.odf2.core.webharvest.service.multidoc.MultidocTaskDataService;
import com.workfusion.odf2.core.webharvest.service.multidoc.dto.MultidocTaskDocument;
import com.workfusion.odf2.core.webharvest.service.multidoc.dto.MultidocTaskGroupField;
import com.workfusion.odf2.core.webharvest.service.multidoc.dto.MultidocTaskMeta;

@BotTask
@Requires(ClientOdfModule.class)
public class MultiDocumentInputProcessorTask implements InputEntityProcessorTask<Attachment> {

    public static final String DOCTYPE_ATTACHMENT = "Attachment";

    private final AttachmentRepository attachmentRepository;

    private final MultidocTaskDataService multidocTaskDataService;

    @Inject
    public MultiDocumentInputProcessorTask(AttachmentRepository emailRepository, MultidocTaskDataService multidocTaskDataService) {
        this.attachmentRepository = emailRepository;
        this.multidocTaskDataService = multidocTaskDataService;
    }

    @Override
    public Collection<Attachment> findInputEntities(UUID transactionId) {
        return attachmentRepository.findAll();
    }

    @Override
    public void processInputEntities(Collection<Attachment> attachments) {

        multidocTaskDataService.prepareMultidocTaskData()
                // 1. Specify a custom title for the Manual Task.
                .withManualTaskTitle("Manual Task title")
                // 2. Provide the Manual Task config JSON from the task builder that should be used for rendering of the Manual Task.
                .withCustomMultiDocumentTaskConfiguration(readTaskConfiguration())
                // 3. Provide a list of documents.
                .withDocuments(attachments.stream()
                        .map(attachment -> MultidocTaskDocument
                                .builder()
                                .withDocumentId(attachment.getUuid().toString())
                                .withDocumentName(attachment.getFileName())
                                .withDocumentType(DOCTYPE_ATTACHMENT)
                                // 3.1 Provide extracted field as string.
                                .withExtractedField("buyer_company_name", attachment.getBuyerCompanyName())
                                // 3.2 Provide group of extracted fields.
                                .withExtractedField("line_item",
                                        MultidocTaskGroupField.builder()
                                            .withGroupOfFields()
                                                    .singleField("line_item_name", attachment.getLineItemName())
                                                    .singleField("tabnumber", attachment.getTabNumber())
                                                    .singleField("line_item_price", attachment.getPrice())
                                                .add()
                                        .build())
                                .build())
                        .collect(Collectors.toList()))
                // 4. Provide global answers.
                .withMeta(MultidocTaskMeta
                        .builder()
                        .withGlobalAnswer("final_desision", "some_other_field_you_want_to_make_global")
                        .build())
                // 5. Put all built objects to task output.
                .putToTaskOutput();
    }

    @Override
    public void saveInputEntities(Collection<Attachment> inputEntities) {
        // nothing to save
    }

    private String readTaskConfiguration() {
        // Read the Manual Task configuration from the file.
    }

}

To render the Manual Task correctly, you need the multi-document task configuration JSON.

As a result of code execution, all required values are added to the Bot Task output.

In the given task, all attachments found in the Data Store are transformed into documents. You can construct any single document using MultidocTaskDocument.ManualTaskDocBuilder:

MultidocTaskDocument getMultidocTaskDocument(Attachment attachment) {
    return MultidocTaskDocument
            .builder()
            .withDocumentId(attachment.getUuid().toString())
            .withDocumentName(attachment.getFileName())
            .withDocumentType(DOCTYPE_ATTACHMENT)
            .withExtractedField("buyer_company_name", attachment.getBuyerCompanyName())
            .withExtractedField("line_item",
                    MultidocTaskGroupField.builder()
                        .withGroupOfFields()
                                .singleField("line_item_name", attachment.getLineItemName())
                                .singleField("tabnumber", attachment.getTabNumber())
                                .singleField("line_item_price", attachment.getPrice())
                            .add()
                    .build())
            .build();
}

Besides the standard for the document ID, name, type, and other fields, you can provide a list of MultidocTaskFields that represents preliminarily extracted fields by the model. In these fields, Manual Task processing results are also saved. There are two types of MultidocTaskField:

• MultidocTaskSingleField represents a single string value and can be used for providing a simple key-value pair:

  .withExtractedField("extracted_field_key", "extracted field value")

• MultidocTaskGroupField is a list of objects represented by key-value pairs, for example, Map<String, String>. To build such structures, you can manually create List<Map<String, String>> or use MultidocTaskGroupField.Builder:

  MultidocTaskGroupField.builder()
      .withGroupOfFields()
          .singleField("line_item_name", attachment.getLineItemName())
          .singleField("tabnumber", attachment.getTabNumber())
          .singleField("line_item_price", attachment.getPrice())
          .add()
      .withGroupOfFields()
          .singleField("line_item_name", attachment.getLineItemName1())
          .singleField("tabnumber", attachment.getTabNumber1())
          .singleField("line_item_price", attachment.getPrice1())
          .add()
  .build()

Read multi-document Manual Task output data

To read the Manual Task output JSON and transform it into ManualTaskData, use ManualTaskDataService.getProcessedData(). This method extracts the JSON representation of ManualTaksData from TaskInput and transforms it into a Java object.

import java.util.Map;
import javax.inject.Inject;

import com.workfusion.odf2.compiler.BotTask;
import com.workfusion.odf2.core.task.generic.GenericTask;
import com.workfusion.odf2.core.webharvest.TaskOutput;
import com.workfusion.odf2.core.webharvest.service.multidoc.MultidocTaskDataService;
import com.workfusion.odf2.core.webharvest.service.multidoc.dto.MultidocTaskData;
import com.workfusion.odf2.core.webharvest.service.multidoc.dto.MultidocTaskField;

@BotTask
public class MultiDocumentOutputProcessorTask implements GenericTask {

    private final MultidocTaskDataService multidocTaskDataService;
    private final TaskOutput taskOutput;

    @Inject
    public MultiDocumentOutputProcessorTask(MultidocTaskDataService multidocTaskDataService, TaskOutput taskOutput) {
        this.multidocTaskDataService = multidocTaskDataService;
        this.taskOutput = taskOutput;
    }

    @Override
    public void run() {
        final MultidocTaskData processedData = multidocTaskDataService.getProcessedData();

        final Map<String, MultidocTaskField> extractedFields = processedData.getDocs().get(0).getExtractedFields();

        extractedFields.forEach(this::processExtractedField);
    }

    private void processExtractedField(String key, MultidocTaskField value) {
        if (value.hasChildren()) {
            value.getChildren().forEach(map -> map.forEach(this::processExtractedField));
        } else {
            taskOutput.setColumn(key, value.getValueAsString());
        }
    }

}

Any document contains extracted fields to store tagging results. Use the MultidocTaskField API to extract data more effectively:

public interface MultidocTaskField {

    /**
     * Provides string value of {@link MultidocTaskSingleField} or string representation of {@link MultidocTaskGroupField}
     * @return string value of {@link MultidocTaskField}
     */
    String getValueAsString();

    /**
     * Provides a list of children for the {@link MultidocTaskGroupField} or emply {@link List} for {@link MultidocTaskSingleField} or if no children are present.
     * @return list of children for {@link MultidocTaskField}
     */
    List<Map<String, MultidocTaskSingleField>> getChildren();

    /**
     *
     * @return true if {@link MultidocTaskField} has children or false if not or if it is an instance of {@link MultidocTaskSingleField}
     */
    boolean hasChildren();

}