Configure model

To make the most of AutoML SDK, components need to be used together in one configuration. The main hyper model class located at [ModelPackage]/model/[ModelName]Model.java contains a @HypermodelConfiguration annotation with a reference to the AutoML SDK configuration class.

import com.workfusion.vds.nlp.hypermodel.ie.generic.GenericIeHypermodel;
import com.workfusion.vds.sdk.api.hypermodel.annotation.HypermodelConfiguration;

@HypermodelConfiguration(TrainingIeModelConfiguration.class)
public class TrainingIeModel extends GenericIeHypermodel {
}

While configuring a model, a machine learning engineer (MLE) should not edit the main hyper model class but focus on TrainingIeModelConfiguration.java as shown in the code sample below.

The following code is generated for a model configuration in case the importConfigurationFromGenericModel parameter is set to N. The generated model configuration contains three methods. Each of these methods provides a list of AutoML SDK components specified by type: Feature Extractors, Annotators, and Post-Processors.

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

import com.workfusion.vds.sdk.api.hypermodel.annotation.ModelConfiguration;
import com.workfusion.vds.sdk.api.hypermodel.annotation.Named;
import com.workfusion.vds.sdk.api.nlp.annotator.Annotator;
import com.workfusion.vds.sdk.api.nlp.configuration.IeConfigurationContext;
import com.workfusion.vds.sdk.api.nlp.fe.FeatureExtractor;
import com.workfusion.vds.sdk.api.nlp.model.Document;
import com.workfusion.vds.sdk.api.nlp.model.Element;
import com.workfusion.vds.sdk.api.nlp.model.IeDocument;
import com.workfusion.vds.sdk.api.nlp.processing.Processor;
import com.workfusion.vds.sdk.nlp.component.annotator.tokenizer.SplitterTokenAnnotator;

@ModelConfiguration
public class TrainingIeConfiguration {

    @Named("annotators")
    public List<Annotator<Document>> getAnnotators(IeConfigurationContext context) {
        //TODO configure annotators here.
        List<Annotator<Document>> annotators = new ArrayList<>();
        annotators.add(new SplitterTokenAnnotator("\\s"));
        return annotators;
    }

    @Named("featureExtractors")
    public List<FeatureExtractor<Element>> getFeatureExtractors(IeConfigurationContext context) {
        //TODO configure feature extractors here.
        return Arrays.asList(new FeatureExtractorExample<Element>());
    }

    @Named("processors")
    public List<Processor<IeDocument>> getProcessors() {
        //TODO configure post processors here.
        return Arrays.asList(new PostProcessorExample());
    }

}

In case the importConfigurationFromGenericModel parameter is set to Y, the following annotation is added to the configuration class to provide a reference to the generic Information Extraction model.

import com.workfusion.vds.nlp.hypermodel.ie.generic.config.GenericIeHypermodelConfiguration;
import com.workfusion.vds.sdk.api.hypermodel.annotation.Import;
import com.workfusion.vds.sdk.api.hypermodel.annotation.ModelConfiguration;

@ModelConfiguration
@Import(configurations = {
        @Import.Configuration(GenericIeHypermodelConfiguration.class) // reference to generic information extraction model
})
public class TrainingIeConfiguration {
    // ... the same content
}

For information extraction, an MLE analyzes the dataset and specifies a list of extracted fields (for example, address, invoice number, email, and so on). During model development, different fields will require different logic for extracting values from documents and producing features for the machine learning algorithm. This means different Feature Extractors and Annotators should be applied to different fields.

To deal with this, each method needs an injected instance of the ConfigurationContext class containing information about the current field. This way, the method can produce different sets of components for different fields because the configuration class is executed multiple times, once for each field.

In the example below, SplitterTokenAnnotator and EntityBoundaryAnnotator Annotators are added to all field configurations, and special Annotators are added for the invoice_number and total_amount fields.

For more information, refer to the Create AutoML SDK configuration page.

import java.util.ArrayList;
import java.util.List;

import com.workfusion.vds.sdk.api.hypermodel.annotation.ModelConfiguration;
import com.workfusion.vds.sdk.api.hypermodel.annotation.Named;
import com.workfusion.vds.sdk.api.nlp.annotator.Annotator;
import com.workfusion.vds.sdk.api.nlp.configuration.IeConfigurationContext;
import com.workfusion.vds.sdk.nlp.component.annotator.EntityBoundaryAnnotator;
import com.workfusion.vds.sdk.nlp.component.annotator.ner.BaseRegexNerAnnotator;
import com.workfusion.vds.sdk.nlp.component.annotator.tokenizer.SplitterTokenAnnotator;

@ModelConfiguration
public class TrainingIeModelConfiguration {

    @Named("annotators")
    public List<Annotator> annotators(IeConfigurationContext context) {
        List<Annotator> annotators = new ArrayList<>();
        annotators.add(new EntityBoundaryAnnotator());
        annotators.add(new SplitterTokenAnnotator("([$\\$\\s:#_;'])"));
        switch (context.getField().getCode()) {
            case "invoice_number": {
                annotators.add(BaseRegexNerAnnotator.getJavaPatternRegexNerAnnotator("invoice_number", "([\\d]{10,11})"));
                break;
            }
            case "total_amount": {
                annotators.add(BaseRegexNerAnnotator.getJavaPatternRegexNerAnnotator("amount", "[0-9l]{1,3}[,\\.]?[0-9]{2,3}[\\.][0-9]*"));
                break;
            }
       }
    }

}