Edit

Configure model

To make the most of AutoML SDK, use its components together in one configuration. The main hyper model class located at [ModelPackage]/model/[ModelName]Model.java contains a @HypermodelConfiguration annotation with a reference to the AutoML SDK configuration class.

import com.workfusion.vds.nlp.hypermodel.ie.generic.GenericIeHypermodel;
import com.workfusion.vds.sdk.api.hypermodel.annotation.HypermodelConfiguration;

@HypermodelConfiguration(TrainingIeModelConfiguration.class)
public class TrainingIeModel extends GenericIeHypermodel {
}

While configuring a model, an ML Engineer should not edit the main hyper model class, but focus on TrainingIeModelConfiguration.java as shown in the code example below.

The code below is generated for a model configuration in case the importConfigurationFromGenericModel parameter is set to N. It contains three methods, and each of the methods provides a list of AutoML SDK components specified by type: Feature Extractors, Annotators, and Post-Processors.

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import com.workfusion.vds.sdk.api.hypermodel.annotation.ModelConfiguration;
import com.workfusion.vds.sdk.api.hypermodel.annotation.Named;
import com.workfusion.vds.sdk.api.nlp.annotator.Annotator;
import com.workfusion.vds.sdk.api.nlp.configuration.IeConfigurationContext;
import com.workfusion.vds.sdk.api.nlp.fe.FeatureExtractor;
import com.workfusion.vds.sdk.api.nlp.model.Document;
import com.workfusion.vds.sdk.api.nlp.model.Element;
import com.workfusion.vds.sdk.api.nlp.model.IeDocument;
import com.workfusion.vds.sdk.api.nlp.processing.Processor;
import com.workfusion.vds.sdk.nlp.component.annotator.tokenizer.SplitterTokenAnnotator;

@ModelConfiguration
public class TrainingIeConfiguration {

    @Named("annotators")
    public List<Annotator<Document>> getAnnotators(IeConfigurationContext context) {
        //TODO configure Annotators here
        List<Annotator<Document>> annotators = new ArrayList<>();
        annotators.add(new SplitterTokenAnnotator("\\s"));
        return annotators;
    }

    @Named("featureExtractors")
    public List<FeatureExtractor<Element>> getFeatureExtractors(IeConfigurationContext context) {
        //TODO configure Feature Extractors here
        return Arrays.asList(new FeatureExtractorExample<Element>());
    }

    @Named("processors")
    public List<Processor<IeDocument>> getProcessors() {
        //TODO configure Post-Processors here
        return Arrays.asList(new PostProcessorExample());
    }

}

In case the importConfigurationFromGenericModel parameter is set to Y, the following annotation is added to the configuration class to provide a reference to a generic Information Extraction model.

import com.workfusion.vds.nlp.hypermodel.ie.generic.config.GenericIeHypermodelConfiguration;
import com.workfusion.vds.sdk.api.hypermodel.annotation.Import;
import com.workfusion.vds.sdk.api.hypermodel.annotation.ModelConfiguration;

@ModelConfiguration
@Import(configurations = {
        @Import.Configuration(GenericIeHypermodelConfiguration.class) // reference to a generic Information Extraction model
})
public class TrainingIeConfiguration {
    // ... the same content
}

For Information Extraction ML Engineer analyzes the data set and specifies the list of fields that should be extracted (for example, address, invoice number, email, etc.). During model development different fields will require different logic to extract values from documents and produce features for the ML algorithm. This means different Feature Extractors and Annotators should be applied to different fields.

To deal with this, each method needs an injected instance of the ConfigurationContext class containing information about the current field. This way, the method can produce different sets of components for different fields because the configuration class is executed multiple times, once for each field.

In the example below, the SplitterTokenAnnotator and EntityBoundaryAnnotator Annotators are added to all field configurations. Also, special Annotators are added for the invoice_number and total_amount fields.

For more information, refer to the Configure AutoML SDK topic.

import java.util.ArrayList;
import java.util.List;

import com.workfusion.vds.sdk.api.hypermodel.annotation.ModelConfiguration;
import com.workfusion.vds.sdk.api.hypermodel.annotation.Named;
import com.workfusion.vds.sdk.api.nlp.annotator.Annotator;
import com.workfusion.vds.sdk.api.nlp.configuration.IeConfigurationContext;
import com.workfusion.vds.sdk.nlp.component.annotator.EntityBoundaryAnnotator;
import com.workfusion.vds.sdk.nlp.component.annotator.ner.BaseRegexNerAnnotator;
import com.workfusion.vds.sdk.nlp.component.annotator.tokenizer.SplitterTokenAnnotator;

@ModelConfiguration
public class TrainingIeModelConfiguration {

    @Named("annotators")
    public List<Annotator> annotators(IeConfigurationContext context) {
        List<Annotator> annotators = new ArrayList<>();
        annotators.add(new EntityBoundaryAnnotator());
        annotators.add(new SplitterTokenAnnotator("([$\\$\\s:#_;'])"));
        switch (context.getField().getCode()) {
            case "invoice_number": {
                annotators.add(BaseRegexNerAnnotator.getJavaPatternRegexNerAnnotator("invoice_number", "([\\d]{10,11})"));
                break;
            }
            case "total_amount": {
                annotators.add(BaseRegexNerAnnotator.getJavaPatternRegexNerAnnotator("amount", "[0-9l]{1,3}[,\\.]?[0-9]{2,3}[\\.][0-9]*"));
                break;
            }
       }
    }

}