Feature Extractors

A Feature Extractor is an AutoML SDK component that has built-in logic to analyze each Token in a Document and produce a set of independent and discriminating features.

A Feature represents a simple question such as "Is the current cell located in the first column?" with a clear non-ambiguous answer "Yes/No" or "1/0". Though, it is also possible to store double values as an answer, such as "0.25" to a question like "How similar is the text of this token to the particular keyword (like "total")?". Features are used by the AutoML algorithm to predict which Tokens should be tagged as correct answers.

Results of a Feature Extractor may look like this: Cell_In_First_Column = 1 or Similarity_To_Total = 0.25.

This stage of the pipeline is crucial as the quality and quantity of the produced features have a major influence on the overall quality of the model.

tip

AutoML SDK contains a built-in set of Feature Extractors available via AutoML SDK API. Make sure to have a look before proceeding with the next steps, such as Feature Extractor creation or Feature engineering.

The following figure shows the way two Feature Extractors (IsFirstNameFE and IsDateFE) increment through the Sentence element "Peter Nilson 17 October 1937". In this example Feature Extractors take one Token into focus at time, and then extract a Feature if it corresponds to the Feature Extractor logic.

As negative values are typically omitted, the final result will look as follows.

Token	FE Value
Peter	IsFirstNameFE = 1
17	IsDate = 1

Create Feature Extractor

If none of the out-of-the-box Feature Extractors satisfy your need, create a new one.

Implement the FeatureExtractor interface that contains one required method extract().

FeatureExtractor implementation can have optional methods annotated with a corresponding Java annotation as well. These methods represent different stages of the FeatureExtractor lifecycle:

• OnInit
• OnDocumentStart
• OnDocumentComplete
• OnDestroy
• And so on

For complete usage information, refer to the FeatureExtractor documentation.

warning

All fields of a Feature Extractor must be serializable. Although, the Feature Extractor does not have to implement the Serializable interface. Keep in mind that lambdas are not serializable by default.

If you cannot avoid using non-serializable fields for some reason, then you can do the following trick:

1. Define a field and keep it null.
2. Initialize the field in the @OnInit method.
3. Do not initialize the field in a constructor.

Feature Extractor with non-serializable date formatter

Your Feature Extractor converts dates from one format to another with the help of DateTimeFormatter that is not serializable. The correct initialization of the Feature Extractor would be as follows.

public class MyFeatureExtractor implements FeatureExtractor {

    private DateTimeFormatter dateTimeFormatter;

    @OnInit
    public void init() {
        dateTimeFormatter = DateTimeFormatter.ofPattern("dd/MM/yyyy");
    }

    /* The Feature Extractor implementation goes here */
}

Feature Extractor with lambda

Let's say you want to create a Feature Extractor that has a function as a class member. In your Feature Extractor, you want to refer to one of the functions using a lambda. Then, to avoid issues with serialization, the code should look as follows:

public class MyFeatureExtractor implements FeatureExtractor {

    private Function<Element, Integer> function;

    @OnInit
    public void init() {
        function = e -> e.getBegin()
    }

    /* The FeatureExtractor implementation goes here */
}

Extract

The extract() method applies the main feature extraction algorithm that analyzes the provided Document structure and extracts features for a specified Token element.

The extract() method is called for each Token and returns a list of features. The extraction process is parallel for all fields in a Document.

The following Feature Extractor produces a feature when a Token contains only digits.

import java.util.ArrayList;
import java.util.Collection;
import java.util.List;

import com.workfusion.vds.sdk.api.nlp.fe.Feature;
import com.workfusion.vds.sdk.api.nlp.fe.FeatureExtractor;
import com.workfusion.vds.sdk.api.nlp.model.Document;
import com.workfusion.vds.sdk.api.nlp.model.Element;

public class DigitsOnlyFE<T extends Element> implements FeatureExtractor<T> {

    private final static String FEATURE_NAME = "DigitsOnly";

    @Override
    public Collection<Feature> extract(Document document, T element) {
        List<Feature> features = new ArrayList<>();
        if (element.getText().matches("[0-9]+")) {
            features.add(new Feature(FEATURE_NAME, 1.0));
        }
        return features;
    }

}

OnInit

OnInit is a method-level annotation for FeatureExtractor used to define a method that should be invoked after the constructor.

A method annotated with OnInit should follow these conventions:

• Called only once when a FeatureExtractor instance is created.
• Accepts a Map<String, Object> of parameters. These parameters are context-specific and typically not required for a common Feature Extractor.
• Accepts a special builder for index caching and type of focus Element, by default—Token. For details, refer to the Indexes and cache section.
• Method is optional.

A method annotated with OnInit is typically used to initialize regex patterns, add dynamic cached indexes, and so on.

warning

• If a class has more than one @OnInit method, an exception is thrown.
• If both a class and its sub-class have the @OnInit method, only the method in the sub-class is invoked. Use super().parentInitMethodName() to invoke the parent method as well.
• In case @OnInit creates complex memory collections, clean them inside the @OnDestroy method.

OnDestroy

A method annotated with OnDestroy follows these conventions:

• Called only once when the information extraction process is fully complete.
• Used to release resources by removing objects or references that the OnDestroy method is holding
• Optional

warning

• If a class has more than one @OnDestroy method, an exception is thrown.
• If both a class and its sub-class have the @OnDestroy method, only the method in the sub-class is invoked. Use super().parentDestroyMethodName() to invoke the parent method as well.

The following example finds the similarity score between the element text and all keywords from a dictionary file. The OnInit and OnDestroy methods are used to read the dictionary file. The target Token is compared with data from the dictionary file using Levenshtein distance as a string similarity algorithm.

See sample code

import java.io.InputStreamReader;
import java.io.Reader;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;

import org.apache.commons.io.IOUtils;

import com.workfusion.vds.nlp.similarity.StringSimilarityUtils;
import com.workfusion.vds.sdk.api.exception.SdkException;
import com.workfusion.vds.sdk.api.nlp.annotation.OnDestroy;
import com.workfusion.vds.sdk.api.nlp.annotation.OnInit;
import com.workfusion.vds.sdk.api.nlp.fe.Feature;
import com.workfusion.vds.sdk.api.nlp.fe.FeatureExtractor;
import com.workfusion.vds.sdk.api.nlp.model.Document;
import com.workfusion.vds.sdk.api.nlp.model.Element;

public class DictionarySimilarityFE<T extends Element> implements FeatureExtractor<T> {

    public static final String FEATURE_NAME = "SimilarityToKeyword";

    private String pathToResource;
    private List<String> keywords = new ArrayList<>();

    public DictionarySimilarityFE(String pathToResource) {
        this.pathToResource = pathToResource;
    }

    @OnInit
    public void init() {
        // read resource from classpath and collect keywords for future processing.
        try (Reader reader = new InputStreamReader(this.getClass().getResourceAsStream(pathToResource))) {
            keywords = IOUtils.readLines(reader);
        } catch (Exception e) {
            throw new SdkException(String.format("Error reading file %s.", pathToResource), e);
        }
    }

    @Override
    public Collection<Feature> extract(Document document, T element) {
        List<Feature> result = new ArrayList<>();
        for (String keyword : keywords) {
            // calculate similarity between element text and keyword using Similarity Utils
            double similarityScore = StringSimilarityUtils.levenshtein(element.getText(), keyword);
            // aggregate constant FEATURE_NAME and keyword into final feature name
            String featureName = String.join("_", FEATURE_NAME, keyword);
            // add a feature
            result.add(new Feature(featureName, similarityScore));
        }
        return result;
    }

    @OnDestroy
    public void destroy() {
        // clean collection after usage
        keywords.clear();
    }
}

OnDocumentStart

A method annotated with OnDocumentStart follows the conventions defined below:

• Called only once for each document before the extract() method
• Accepts a Document and Element type (Token by default) to perform feature extraction
• Optional

Typical usage: If the feature extraction logic requires to analyze large structures or entire documents for each Token, it is a good practice to do pre-calculations on the Document level and the use it with the extract()method to improve the performance.

warning

• If a class has more than one @OnDocumentStart method, an exception is thrown.
• If both a class and its sub-class have the @OnDocumentStart method, only the method in the sub-class is invoked. Use super().parentMethodName() to invoke the parent method as well.
• In case the @OnDocumentStart method creates complex memory collections, clean them inside the @OnDocumentDestroy method.

All class-level variables initialized inside the OnDocumentStart method should be used in the read-only mode inside the extract() method because this method can be executed in multiple threads.

OnDocumentComplete

A method annotated with OnDocumentComplete follows the conventions defined below:

• Called only once for a whole Document to release the resources, while the extract() method is called for all Tokens in a Document
• Optional

warning

• If a class has more than one @OnDocumentComplete method, an exception is thrown.
• If both a class and its sub-class have the @OnDocumentComplete method, then only the method in the sub-class is invoked. Use super().parentMethodName() to invoke the parent method as well.

note

Both OnDocumentStart and OnDocumentComplete are used to prepare or finalize data for a Document. Usually, OnDocumentStart is used to prepare big data from a full document to create a local cache. This data is consumed inside the extract() method to improve the speed of feature extraction.

The following example defines how many times the text from the focus element is found in a Document. The OnDocumentStart method is used to prepare data for extraction.

See sample code

import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import com.workfusion.vds.sdk.api.nlp.annotation.OnDocumentComplete;
import com.workfusion.vds.sdk.api.nlp.annotation.OnDocumentStart;
import com.workfusion.vds.sdk.api.nlp.fe.Feature;
import com.workfusion.vds.sdk.api.nlp.fe.FeatureExtractor;
import com.workfusion.vds.sdk.api.nlp.model.Document;
import com.workfusion.vds.sdk.api.nlp.model.Element;

public class TextOccurrenceInDocument <T extends Element> implements FeatureExtractor<T> {

    public static final String FEATURE_NAME = "TokenMetInDocument";

    private Map<String, Integer> textOccurrence = new HashMap<>();
    private int totalFocusElementNumber;

    @OnDocumentStart
    public void onStart(Document document, Class<T> focusClass) {
        // Find all focus elements (by default, focusElement is Token)
        Collection<T> elements = document.findAll(focusClass);
        // Store a total number of elements in a Document. Used for feature normalization
        totalFocusElementNumber = elements.size();
        Map<String, Integer> textOccurrence = new HashMap<>();
        // Calculate the number of text occurrences in the Document
        for (T element : elements) {
            Integer occurrence = textOccurrence.get(element.getText());
            if (occurrence == null) {
                occurrence = 0;
            } else {
                occurrence++;
            }
            textOccurrence.put(element.getText(), occurrence);
        }
    }

    @Override
    public Collection<Feature> extract(Document document, T element) {
        List<Feature> result = new ArrayList<>();
        // Get a pre-calculated number of text occurrences in the Document
        Integer number = textOccurrence.get(element.getText());
        // We strongly recommend using normalized features (values between 0 and 1). To normalize a feature value, divide it by the total number of elements in a Document.
        double featureValue = (double)number/totalFocusElementNumber;
        // Add a feature
        result.add(new Feature(FEATURE_NAME, featureValue));
        return result;
    }

    @OnDocumentComplete
    public void complete() {
        // Clear the map after usage.
        textOccurrence.clear();
    }
}

Indexes and cache

The feature extraction process analyzes the location and elements around the analyzed focus element. To analyze, use the findCovered() and findCovering() methods on a Document to find all the elements inside or outside the processed element.

These operations usually require significant time to be processed. To improve the performance, add an @Indexesdeclaration on the class-level. In this case, the Document stores the results of all "covered-covering" relations before the feature extraction process. When FeatureExtractor invokes the findCovering/findCovered method and has a corresponding index, it produces the result from the cache without any calculation.

See sample code

import java.util.Collection;
import java.util.List;

import com.workfusion.vds.sdk.api.nlp.annotation.DependsOn;
import com.workfusion.vds.sdk.api.nlp.fe.Feature;
import com.workfusion.vds.sdk.api.nlp.fe.FeatureExtractor;
import com.workfusion.vds.sdk.api.nlp.fe.annotation.FeatureName;
import com.workfusion.vds.sdk.api.nlp.fe.annotation.Index;
import com.workfusion.vds.sdk.api.nlp.fe.annotation.IndexType;
import com.workfusion.vds.sdk.api.nlp.fe.annotation.Indexes;
import com.workfusion.vds.sdk.api.nlp.model.Document;
import com.workfusion.vds.sdk.api.nlp.model.Element;
import com.workfusion.vds.sdk.api.nlp.model.Line;
import com.workfusion.vds.sdk.api.nlp.model.Token;

@DependsOn({Line.class, Token.class})
@Indexes({
    @Index(covering = Line.class, covered = Token.class, type = IndexType.BIDIRECTIONAL)
})
@FeatureName(SimilarityKeysInNextLineFE.FEATURE_NAME)
public class SimilarityKeysInNextLineFE implements FeatureExtractor<Token> {

    public final static String FEATURE_NAME = "similarityInNextLine";

    @Override
    public Collection<Feature> extract(Document document, Token element) {
        // Lines to be taken from cache
        List<Line> lines= document.findCovering(Line.class, element);
        // Create features here
    }
}

Alternatively, you can declare an initialization method inside the FeatureExtractor class.

See sample code

import java.util.Collection;
import java.util.List;

import com.workfusion.vds.sdk.api.nlp.annotation.OnInit;
import com.workfusion.vds.sdk.api.nlp.cache.CacheBuilder;
import com.workfusion.vds.sdk.api.nlp.fe.Feature;
import com.workfusion.vds.sdk.api.nlp.fe.FeatureExtractor;
import com.workfusion.vds.sdk.api.nlp.model.Document;
import com.workfusion.vds.sdk.api.nlp.model.Element;

public class CustomFeatureExtractor<T extends Element, S extends Element> implements FeatureExtractor<T> {

    private Class<S> searchClass;

    public CustomFeatureExtractor(Class<S> searchClass) {
        super();
        // This Feature Extractor can be configured to search for Tokens (Type T) inside the elements with Type S.
        // For example, Table, Line, Cell, Sentence, etc.
        this.searchClass = searchClass;
    }

    @OnInit
    public void init(CacheBuilder builder, Class<T> focusClass) {
        builder.covering(focusClass, searchClass);
    }    

    @Override
    public Collection<Feature> extract(Document document, T element) {
        // Lines will be taken from cache
        List<S> parentElements = document.findCovering(searchClass, element);
        // Create features here
    }
}

It is strongly recommended adding an @Indexes declaration in the following cases:

• A Feature Extractor uses the findAllCovered or findAllCovering methods.
• A Feature Extractor frequently uses findCovered or findCovering applied to different elements. In this case, @Indexes should contain a declaration of the covered-covering element type. Otherwise, the Feature Extractor uses a basic focus type—Token.

See also

• OOTB feature extractors
• Feature engineering