Start from Archetype
ML Archetypes location
caution
Before generating a new project, configure a local environment.
Archetypes are available from WorkFusion's public Nexus. A separate repository—https://repository.workfusion.com/content/repositories/archetypes—is dedicated to various Archetypes. ML Archetypes are located in the
com.workfusion.ml package.
If you cannot use WorkFusion's public Nexus due to your company's security policy, use on-prem IA Cloud Enterprise's Nexus. It comes with each deployment and includes out-of-the-box ML Archetypes for a specific IA Cloud Enterprise version.
ML quickstart Archetypes
The Machine Learning (ML) quickstart Archetype is a Maven project allowing simple model implementation and comprising a module for deployment to the server IA Cloud environment via Asset Bundle API.
The following ML quickstart Archetypes are available:
| Name | Search Engine | Use when... |
|---|---|---|
| ml-ie-quickstart | 2.0 | You build a Java-based Information Extraction model and want to have a minimalistic working example as a starting point. The example can compile, train, and test out of the box. |
| ml-classification-quickstart | 2.0 | You build a Java-based Classification model and want a minimalistic working example as a starting point. The example can compile, train, and test out of the box. |
| ml-python-classification-quickstart-archetype | You build a Python-based Classification model and want a minimalistic working example as a starting point. The example can compile, train, and test out of the box. |
To generate a Maven project with one of the above Archetypes, execute the following command in the command-line interface:
mvn archetype:generate -DarchetypeGroupId=com.workfusion.ml -DarchetypeArtifactId=ml-ie-quickstart -DarchetypeVersion=<ML SDK version>
The ML SDK version value in the above command should correspond to your IA Cloud Enterprise version. See the AutoML SDK and IA Cloud compatibility matrix.
Provide the following parameters for an Archetype in the interactive mode:
groupId: the group identifier of your artifact. Set a base package, for example,com.mycompany.artifactId: the unique identifier of your artifact. Use only lowercase Latin symbols without spaces and special characters, for example,ml-ie-quick-test.version: the artifact version. Avoid using SNAPSHOT versions. Use the RELEASE type, for example,1.0.0.package: a default package for code generation. Use only lowercase Latin symbols without spaces and special characters, for example,mlieqt.
The generated project stub has two modules:
- [ARTIFACT_ID]-ml-sdk contains an ML SDK model configuration, a training set, a test set, and Java executable classes for training and testing.
- [ARTIFACT_ID]-package provides a structure for building of the model bundle artifact.
¦ build.groovy
¦ pom.xml
¦
+---ml-ie-quick-test-ml-sdk
¦ ¦ pom.xml
¦ ¦
¦ +---input
¦ ¦ invoice0.html
¦ ¦ invoice106.html
¦ ¦ invoice108.html
¦ ¦ invoice113.html
¦ ¦ invoice116.html
¦ ¦ invoice118.html
¦ ¦ invoice122.html
¦ ¦ invoice14.html
¦ ¦ invoice16.html
¦ ¦ invoice20.html
¦ ¦
¦ +---process
¦ ¦ +---logs
¦ +---src
¦ ¦ +---main
¦ ¦ ¦ +---java
¦ ¦ ¦ ¦ +---mlieqt
¦ ¦ ¦ ¦ +---annotator
¦ ¦ ¦ ¦ ¦ CeoNerAnnotator.java
¦ ¦ ¦ ¦ ¦
¦ ¦ ¦ ¦ +---config
¦ ¦ ¦ ¦ ¦ Fields.java
¦ ¦ ¦ ¦ ¦ QuickstartModelConfiguration.java
¦ ¦ ¦ ¦ ¦
¦ ¦ ¦ ¦ +---fe
¦ ¦ ¦ ¦ ¦ +---classification
¦ ¦ ¦ ¦ ¦ ¦ IsInFirstOrLastNLinesInDocumentFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ LinePositionOfKeywordFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ PositionOfKeywordsFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦
¦ ¦ ¦ ¦ ¦ +---general
¦ ¦ ¦ ¦ ¦ ¦ IsCoveredWithParticularNerFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ IsFirstInSentenceFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ IsFirstWordInDocumentFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ IsFitCustomPatternFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ IsPrecededWithCurrencySignFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ IsPrecededWithKeywordsInLineFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ IsUniqueWordInLineFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ IsUpperCaseFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦ IsUpperLowerCaseFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ ¦
¦ ¦ ¦ ¦ ¦ +---table
¦ ¦ ¦ ¦ ¦ IsCoveredWithCellFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ IsLastCellInTableFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ MatchFullColumnOrRowWithSpecifiedHeaderFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ TableColumnIndexFeatureExtractor.java
¦ ¦ ¦ ¦ ¦ TableRowIndexFeatureExtractor.java
¦ ¦ ¦ ¦ ¦
¦ ¦ ¦ ¦ +---model
¦ ¦ ¦ ¦ ¦ QuickstartModel.java
¦ ¦ ¦ ¦ ¦
¦ ¦ ¦ ¦ +---processing
¦ ¦ ¦ ¦ ¦ AmountNormalizationPostProcessor.java
¦ ¦ ¦ ¦ ¦ AppendCurrencyPostProcessor.java
¦ ¦ ¦ ¦ ¦ DatePostProcessing.java
¦ ¦ ¦ ¦ ¦ EmailPostProcessor.java
¦ ¦ ¦ ¦ ¦ ProductPostProcessor.java
¦ ¦ ¦ ¦ ¦ RemoveCommasFromPrice.java
¦ ¦ ¦ ¦ ¦ RemoveSInTheBeginningFromPrice.java
¦ ¦ ¦ ¦ ¦ SupplierNamePostProcessor.java
¦ ¦ ¦ ¦ ¦ ToUpperOrLowerCasePostProcessor.java
¦ ¦ ¦ ¦ ¦
¦ ¦ ¦ ¦ +---run
¦ ¦ ¦ ¦ ModelExecutionRunner.java
¦ ¦ ¦ ¦ ModelTrainingRunner.java
¦ ¦ ¦ ¦
¦ ¦ ¦ +---resources
¦ ¦ ¦ +---dictionary
¦ ¦ ¦ ¦ email_keywords.txt
¦ ¦ ¦ ¦ invoice_date_keywords.txt
¦ ¦ ¦ ¦ invoice_number_keywords.txt
¦ ¦ ¦ ¦ price_keywords.txt
¦ ¦ ¦ ¦ product_keywords.txt
¦ ¦ ¦ ¦ quantity_keywords.txt
¦ ¦ ¦ ¦ supplier_name.txt
¦ ¦ ¦ ¦ supplier_name_keywords.txt
¦ ¦ ¦ ¦ total_amount_keywords.txt
¦ ¦ ¦ ¦
¦ ¦ ¦ +---META-INF
¦ ¦ ¦ +---worker
¦ ¦ ¦ worker-execution.yml
¦ ¦ ¦ worker-training.yml
¦ ¦ ¦
¦ ¦ +---test
¦ ¦ +---java
¦ ¦ +---mlieqt
¦ +---test
¦ invoice144.html
¦ invoice149.html
¦ invoice151.html
¦ invoice155.html
¦ invoice159.html
¦
+---ml-ie-quick-test-package
¦ pom.xml
¦
+---assembly
¦ package-remote.xml
¦
+---src
+---main
+---resources
¦ meta-info.json
¦
+---automl
+---artifact
+---model
ML raw Archetypes
To generate an empty raw project stub in Maven, choose one of the Archetypes from the table below, depending on the model type and Search Engine version.
| Name | Search Engine | Use when... |
|---|---|---|
| ml-sdk-ie-se-20-archetype | 2.0 | You build a Java-based Information Extraction model based on a generic model or from scratch with SE 2.0. |
| ml-sdk-ie-se-30-archetype | 3.0 | You build a Java-based Information Extraction model based on a generic model or from scratch with SE 3.0. |
| ml-sdk-classification-se-20-archetype | 2.0 | You build a Java-based Classification model based on a generic model or from scratch |
| ml-sdk-binary-classification-se-20-archetype | 2.0 | You build a Java-based Binary Classification model based on a generic model or from scratch. |
| ml-sdk-python-ie-archetype | You want to use a custom Python Information Extraction model in IA Cloud Enterprise. | |
| ml-python-ie-archetype-v2 | You want to use a pure Python (without Java code) Information Extraction model in IA Cloud Enterprise. | |
| ml-python-classification-archetype | You want to use a custom Python Classification model in IA Cloud Enterprise. | |
| ml-python-classification-archetype-v2 | You want to use a pure Python (without Java code) Classification model in IA Cloud Enterprise. | |
| ml-python-image-archetype | You want to use a custom Python image-based model to solve an ML problem of transforming input images in IA Cloud Enterprise. | |
| ml-python-image-archetype-v2 | You want to use a pure Python (without Java code) image-based model to solve an ML problem of transforming input images in IA Cloud Enterprise. | |
| ml-sdk-hybrid-aggregation-archetype | You want to solve a complex ML problem—Information Extraction and Classification, Cascade of classification, and so on—using a single model. |
To generate a Maven project with one of the above Archetypes, execute the following command in the command-line interface:
mvn archetype:generate -DarchetypeGroupId=com.workfusion.ml -DarchetypeArtifactId=ml-sdk-ie-se-20-archetype -DarchetypeVersion=<ML SDK version>
The ML SDK version value in the above command should correspond to your IA Cloud Enterprise version. See the AutoML SDK and IA Cloud compatibility matrix.
Provide the following parameters for an Archetype in the interactive mode:
groupId: the group identifier of your artifact.artifactId: the unique identifier of your artifact.version: the artifact version.package: the default package for code generation.importConfigurationFromGenericModel:yfor a generic model configuration ornfor an empty model without configuration.modelClassName: the class name of your model.modelCode: the pipeline (hyper model) code; only[a-z, 0-9, _, -]are allowed.modelDescription: the pipeline (hyper model) description used as a hint during automation setup.modelTitle: the pipeline (hyper model) title used as a value in the automation setup drop-down box.modelVersion: the pipeline (hyper model) version.
Your pipeline (hyper model) stub is generated. It's a Maven project you can import to your preferred Java IDE.
The generated project stub has two modules:
- [ARTIFACT_ID]-ml-sdk contains an ML SDK model configuration, Java executable classes for training and testing.
- [ARTIFACT_ID]-package provides a structure for building of the model bundle artifact.
¦ build.groovy
¦ pom.xml
¦
+---ie20-ml-sdk
¦ ¦ pom.xml
¦ ¦
¦ +---src
¦ +---main
¦ ¦ +---java
¦ ¦ ¦ +---ie20
¦ ¦ ¦ +---config
¦ ¦ ¦ ¦ Ie20ModelConfiguration.java
¦ ¦ ¦ ¦
¦ ¦ ¦ +---fe
¦ ¦ ¦ ¦ FeatureExtractorExample.java
¦ ¦ ¦ ¦
¦ ¦ ¦ +---model
¦ ¦ ¦ ¦ Ie20Model.java
¦ ¦ ¦ ¦
¦ ¦ ¦ +---processing
¦ ¦ ¦ ¦ PostProcessorExample.java
¦ ¦ ¦ ¦
¦ ¦ ¦ +---run
¦ ¦ ¦ ModelExecutionRunner.java
¦ ¦ ¦ ModelTrainingRunner.java
¦ ¦ ¦
¦ ¦ +---resources
¦ ¦ +---META-INF
¦ ¦ +---worker
¦ ¦ worker-execution-python.yml
¦ ¦ worker-execution.yml
¦ ¦ worker-training-python.yml
¦ ¦ worker-training.yml
¦ ¦
¦ +---test
¦ +---java
¦ ¦ +---ie20
¦ ¦ +---fe
¦ ¦ FeatureExtractorExampleTest.java
¦ ¦
¦ +---resources
+---ie20-package
¦ pom.xml
¦
+---assembly
¦ package-remote.xml
¦
+---src
+---main
+---resources
+---automl
+---artifact
+---model
Import to IDE
To import a generated Maven project to WorkFusion Studio (Eclipse) or IntelliJ IDEA, follow the steps below:
- Choose Import Maven project.
- Select the root
pom.xmlfrom the directory where you generated the AutoML project with the Archetype.
Check project artifacts
The generated stub should contain the following classes:
<%artifact-name%>Model: class with a custom pipeline (hyper model).In the class, you can find a basic configuration of your pipeline (hyper model): code, version, title, and so on. It also contains a generated
ModelConfigurationannotation with a pre-generated model configuration.The generated pipeline (hyper model) extends one of the generic models:
GenericIeHyperModelfor IE modelsGenericMultiClassificationHyperModelfor Classification models
<%artifact-name%>Configuration: class comprising theprocessor,featureExtractors,annotatorsmethods where you can define your model configuration.PostProcessorExample: a post-processor example.FeatureExtractorExample: a feature extractor example.ModelTrainingRunner: class for a local model training run.ModelProcessingRunner: class for a local model execution run to develop or debug post-processing.