Version: 10.2.4

Start from Archetype

ML archetypes location

caution

Before generating an AutoML SDK project, configure a local environment.

Archetypes are available from WorkFusion's public Nexus. A separate repository: https://repository.workfusion.com/content/repositories/archetypes is dedicated to various Archetypes. ML Archetypes are located in the com.workfusion.ml package.

If you cannot use WorkFusion's public Nexus due to your company's security policy, use on-prem IA Cloud Enterprise's Nexus. It comes with each deployment and includes out-of-the-box ML Archetypes for a specific IA Cloud Enterprise version.

ML quickstart archetypes

The Machine Learning (ML) quickstart Archetype is a Maven project allowing simple model implementation and comprising a module for deployment to the server IA Cloud environment via Asset Bundle migration API.

The following ML quickstart Archetypes are available:

Name	Search Engine	Use when...
`ml-ie-quickstart`	2.0	You build a Java-based Information Extraction model and want to have a minimalistic working example as a starting point. The example can compile, train, and test out of the box.
`ml-classification-quickstart`	2.0	You build a Java-based Classification model and want a minimalistic working example as a starting point. The example can compile, train, and test out of the box.
`ml-python-classification-quickstart-archetype`		You build a Python-based Classification model and want a minimalistic working example as a starting point. The example can compile, train, and test out of the box.

To generate a Maven project with one of the above Archetypes, execute the following command in the command-line interface:

mvn archetype:generate -DarchetypeGroupId=com.workfusion.ml -DarchetypeArtifactId=ml-ie-quickstart -DarchetypeVersion=<ML SDK version>

The ML SDK version value in the above command should correspond to your IA Cloud Enterprise version. See the AutoML SDK and IA Cloud compatibility matrix.

Provide the following parameters for an Archetype in the interactive mode:

groupId: the group identifier of your artifact. Set a base package, for example, com.mycompany.
artifactId: the unique identifier of your artifact. Use only lowercase Latin symbols without spaces and special characters, for example, ml-ie-quick-test.
version: the artifact version. Avoid using SNAPSHOT versions. Use the RELEASE type, for example, 1.0.0.
package: a default package for code generation. Use only lowercase Latin symbols without spaces and special characters, for example, mlieqt.

The generated project stub has two modules:

[ARTIFACT_ID]-ml-sdk contains an ML SDK model configuration, a training set, a test set, and Java executable classes for training and testing.
[ARTIFACT_ID]-package provides a structure for building of the model bundle artifact.

¦   build.groovy
¦   pom.xml
¦   
+---ml-ie-quick-test-ml-sdk
¦   ¦   pom.xml
¦   ¦   
¦   +---input
¦   ¦       invoice0.html
¦   ¦       invoice106.html
¦   ¦       invoice108.html
¦   ¦       invoice113.html
¦   ¦       invoice116.html
¦   ¦       invoice118.html
¦   ¦       invoice122.html
¦   ¦       invoice14.html
¦   ¦       invoice16.html
¦   ¦       invoice20.html
¦   ¦       
¦   +---process
¦   ¦   +---logs
¦   +---src
¦   ¦   +---main
¦   ¦   ¦   +---java
¦   ¦   ¦   ¦   +---mlieqt
¦   ¦   ¦   ¦       +---annotator
¦   ¦   ¦   ¦       ¦       CeoNerAnnotator.java
¦   ¦   ¦   ¦       ¦       
¦   ¦   ¦   ¦       +---config
¦   ¦   ¦   ¦       ¦       Fields.java
¦   ¦   ¦   ¦       ¦       QuickstartModelConfiguration.java
¦   ¦   ¦   ¦       ¦       
¦   ¦   ¦   ¦       +---fe
¦   ¦   ¦   ¦       ¦   +---classification
¦   ¦   ¦   ¦       ¦   ¦       IsInFirstOrLastNLinesInDocumentFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       LinePositionOfKeywordFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       PositionOfKeywordsFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       
¦   ¦   ¦   ¦       ¦   +---general
¦   ¦   ¦   ¦       ¦   ¦       IsCoveredWithParticularNerFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       IsFirstInSentenceFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       IsFirstWordInDocumentFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       IsFitCustomPatternFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       IsPrecededWithCurrencySignFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       IsPrecededWithKeywordsInLineFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       IsUniqueWordInLineFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       IsUpperCaseFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       IsUpperLowerCaseFeatureExtractor.java
¦   ¦   ¦   ¦       ¦   ¦       
¦   ¦   ¦   ¦       ¦   +---table
¦   ¦   ¦   ¦       ¦           IsCoveredWithCellFeatureExtractor.java
¦   ¦   ¦   ¦       ¦           IsLastCellInTableFeatureExtractor.java
¦   ¦   ¦   ¦       ¦           MatchFullColumnOrRowWithSpecifiedHeaderFeatureExtractor.java
¦   ¦   ¦   ¦       ¦           TableColumnIndexFeatureExtractor.java
¦   ¦   ¦   ¦       ¦           TableRowIndexFeatureExtractor.java
¦   ¦   ¦   ¦       ¦           
¦   ¦   ¦   ¦       +---model
¦   ¦   ¦   ¦       ¦       QuickstartModel.java
¦   ¦   ¦   ¦       ¦       
¦   ¦   ¦   ¦       +---processing
¦   ¦   ¦   ¦       ¦       AmountNormalizationPostProcessor.java
¦   ¦   ¦   ¦       ¦       AppendCurrencyPostProcessor.java
¦   ¦   ¦   ¦       ¦       DatePostProcessing.java
¦   ¦   ¦   ¦       ¦       EmailPostProcessor.java
¦   ¦   ¦   ¦       ¦       ProductPostProcessor.java
¦   ¦   ¦   ¦       ¦       RemoveCommasFromPrice.java
¦   ¦   ¦   ¦       ¦       RemoveSInTheBeginningFromPrice.java
¦   ¦   ¦   ¦       ¦       SupplierNamePostProcessor.java
¦   ¦   ¦   ¦       ¦       ToUpperOrLowerCasePostProcessor.java
¦   ¦   ¦   ¦       ¦       
¦   ¦   ¦   ¦       +---run
¦   ¦   ¦   ¦               ModelExecutionRunner.java
¦   ¦   ¦   ¦               ModelTrainingRunner.java
¦   ¦   ¦   ¦               
¦   ¦   ¦   +---resources
¦   ¦   ¦       +---dictionary
¦   ¦   ¦       ¦       email_keywords.txt
¦   ¦   ¦       ¦       invoice_date_keywords.txt
¦   ¦   ¦       ¦       invoice_number_keywords.txt
¦   ¦   ¦       ¦       price_keywords.txt
¦   ¦   ¦       ¦       product_keywords.txt
¦   ¦   ¦       ¦       quantity_keywords.txt
¦   ¦   ¦       ¦       supplier_name.txt
¦   ¦   ¦       ¦       supplier_name_keywords.txt
¦   ¦   ¦       ¦       total_amount_keywords.txt
¦   ¦   ¦       ¦       
¦   ¦   ¦       +---META-INF
¦   ¦   ¦           +---worker
¦   ¦   ¦                   worker-execution.yml
¦   ¦   ¦                   worker-training.yml
¦   ¦   ¦                   
¦   ¦   +---test
¦   ¦       +---java
¦   ¦           +---mlieqt
¦   +---test
¦           invoice144.html
¦           invoice149.html
¦           invoice151.html
¦           invoice155.html
¦           invoice159.html
¦           
+---ml-ie-quick-test-package
    ¦   pom.xml
    ¦   
    +---assembly
    ¦       package-remote.xml
    ¦       
    +---src
        +---main
            +---resources
                ¦   meta-info.json
                ¦   
                +---automl
                    +---artifact
                    +---model

ML raw archetypes

To generate an empty raw project stub in Maven, choose one of the Archetypes from the table below, depending on the model type and Search Engine version.

Name	Search Engine	Use when...
`ml-sdk-ie-se-20-archetype`	2.0	You build a Java-based Information Extraction model based on a generic model or from scratch with SE 2.0.
`ml-sdk-ie-se-30-archetype`	3.0	You build a Java-based Information Extraction model based on a generic model or from scratch with SE 3.0.
`ml-sdk-classification-se-20-archetype`	2.0	You build a Java-based Classification model based on a generic model or from scratch
`ml-sdk-binary-classification-se-20-archetype`	2.0	You build a Java-based Binary Classification model based on a generic model or from scratch.
`ml-sdk-python-ie-archetype`		You want to use a custom Python Information Extraction model in IA Cloud Enterprise.
`ml-python-ie-archetype-v2`		You want to use a pure Python (without Java code) Information Extraction model in IA Cloud Enterprise.
`ml-python-classification-archetype`		You want to use a custom Python Classification model in IA Cloud Enterprise.
`ml-python-classification-archetype-v2`		You want to use a pure Python (without Java code) Classification model in IA Cloud Enterprise.
`ml-python-image-archetype`		You want to use a custom Python image-based model to solve an ML problem of transforming input images in IA Cloud Enterprise.
`ml-python-image-archetype-v2`		You want to use a pure Python (without Java code) image-based model to solve an ML problem of transforming input images in IA Cloud Enterprise.
`ml-sdk-hybrid-aggregation-archetype`		You want to solve a complex ML problem, Information Extraction and Classification, Cascade of classification, and so on, using a single model.

To generate a Maven project with one of the above Archetypes, execute the following command in the command-line interface:

mvn archetype:generate -DarchetypeGroupId=com.workfusion.ml -DarchetypeArtifactId=ml-sdk-ie-se-20-archetype -DarchetypeVersion=<ML SDK version>

The ML SDK version value in the above command should correspond to your IA Cloud Enterprise version. See the AutoML SDK and IA Cloud compatibility matrix.

Provide the following parameters for an Archetype in the interactive mode:

groupId: the group identifier of your artifact.
artifactId: the unique identifier of your artifact.
version: the artifact version.
package: the default package for code generation.
importConfigurationFromGenericModel: y for a generic model configuration or n for an empty model without configuration.
modelClassName: the class name of your model.
modelCode: the pipeline (hyper model) code; only [a-z, 0-9, _, -] are allowed.
modelDescription: the pipeline (hyper model) description used as a hint during automation setup.
modelTitle: the pipeline (hyper model) title used as a value in the automation setup drop-down box.
modelVersion: the pipeline (hyper model) version.

Your pipeline (hyper model) stub is generated. It's a Maven project you can import to your preferred Java IDE.

The generated project stub has two modules:

[ARTIFACT_ID]-ml-sdk contains an ML SDK model configuration, Java executable classes for training and testing.
[ARTIFACT_ID]-package provides a structure for building of the model bundle artifact.

¦   build.groovy
¦   pom.xml
¦   
+---ie20-ml-sdk
¦   ¦   pom.xml
¦   ¦   
¦   +---src
¦       +---main
¦       ¦   +---java
¦       ¦   ¦   +---ie20
¦       ¦   ¦       +---config
¦       ¦   ¦       ¦       Ie20ModelConfiguration.java
¦       ¦   ¦       ¦       
¦       ¦   ¦       +---fe
¦       ¦   ¦       ¦       FeatureExtractorExample.java
¦       ¦   ¦       ¦       
¦       ¦   ¦       +---model
¦       ¦   ¦       ¦       Ie20Model.java
¦       ¦   ¦       ¦       
¦       ¦   ¦       +---processing
¦       ¦   ¦       ¦       PostProcessorExample.java
¦       ¦   ¦       ¦       
¦       ¦   ¦       +---run
¦       ¦   ¦               ModelExecutionRunner.java
¦       ¦   ¦               ModelTrainingRunner.java
¦       ¦   ¦               
¦       ¦   +---resources
¦       ¦       +---META-INF
¦       ¦           +---worker
¦       ¦                   worker-execution-python.yml
¦       ¦                   worker-execution.yml
¦       ¦                   worker-training-python.yml
¦       ¦                   worker-training.yml
¦       ¦                   
¦       +---test
¦           +---java
¦           ¦   +---ie20
¦           ¦       +---fe
¦           ¦               FeatureExtractorExampleTest.java
¦           ¦               
¦           +---resources
+---ie20-package
    ¦   pom.xml
    ¦   
    +---assembly
    ¦       package-remote.xml
    ¦       
    +---src
        +---main
            +---resources
                +---automl
                    +---artifact
                    +---model

Import to IDE

To import a generated Maven project to WorkFusion Studio (Eclipse) or IntelliJ IDEA, follow the steps below:

Choose Import Maven project.
Select the root pom.xml from the directory where you generated the AutoML project with the Archetype.

Check project artifacts

The generated stub should contain the following classes:

<%artifact-name%>Model: class with a custom pipeline (hyper model).
In the class, you can find a basic configuration of your pipeline (hyper model): code, version, title, and so on. It also contains a generated ModelConfiguration annotation with a pre-generated model configuration.
The generated pipeline (hyper model) extends one of the generic models:
- GenericIeHyperModel for IE models
- GenericMultiClassificationHyperModel for Classification models
<%artifact-name%>Configuration: class comprising the processor, featureExtractors, annotators methods where you can define your model configuration.
PostProcessorExample: a post-processor example.
FeatureExtractorExample: a feature extractor example.
ModelTrainingRunner: class for a local model training run.
ModelProcessingRunner: class for a local model execution run to develop or debug post-processing.

ML archetypes location​

ML quickstart archetypes​

ML raw archetypes​

Import to IDE​

Check project artifacts​

ML archetypes location

ML quickstart archetypes

ML raw archetypes

Import to IDE

Check project artifacts