Edit

Configure automatic algorithm selection

The automatic algorithm selection is a mechanism that finds the best classifier for a dataset to apply during model training.

important

• Only for ML SDK v.10.2.4.13 and newer.

• Only Liblinear and Python classifiers can be used.

CompositeClassifier is a new classifier type that is necessary and sufficient for enumerating Machine Learning algorithms. See its code below:

public class CompositeClassifier implements Classifier {

    private final String modelName;
    private final Map<String, Classifier> classifiersMap;

    private CompositeClassifier(Builder<?> builder) {
        this.modelName = builder.modelName;
        classifiersMap = new HashMap<>();
    }

    public String getModelName() {
        return modelName;
    }

    public Map<String, Classifier> getClassifiersMap() {
        return classifiersMap;
    }

    public static class Builder<T extends Builder<T>> {

        protected String modelName;

        public CompositeClassifier build() {
            return new CompositeClassifier(this);
        }

        public T modelName(String name) {
            this.modelName = name;
            return (T) this;
        }

    }

}

Launch automatic algorithm selection

Add CompositeClassifier to your configuration with desired algorithms to start the iteration. See the examples for the Information Extraction and Classification cases.

Information Extraction

@Import(configurations = {
        @Import.Configuration(value = GenericIeHypermodelConfiguration.class)
  })
@ModelConfiguration
public class IeAutoAlgorithmHypermodelConfiguration {

    @Named("mlConfig")
    public Classifier mlConfig() {

        CompositeClassifier compositeClassifier = new CompositeClassifier.Builder<>()
                .modelName("generic.composite")
                .build();

        LiblinearClassifier liblinearClassifier = new LiblinearClassifier.Builder<>()
                .solver(LiblinearClassifier.Solver.L2R_L2LOSS_SVC)
                .quietMode()
                .build();

        compositeClassifier.getClassifiersMap().put("liblinear", liblinearClassifier);

        GradientBoostingClassifier gradientBoostingClassifier = GradientBoostingClassifier.builder()
                .seed(123)
                .build();

        compositeClassifier.getClassifiersMap().put(gradientBoostingClassifier.getModelName(), gradientBoostingClassifier);

        return compositeClassifier;
    }

}

Here, the goal is to find out which classifier is better for your dataset: LiblinearClassifier or GradientBoostingClassifier.

Your class with the model looks as follows:

@ModelDescription(
        code = "information-extraction-auto-algorithm-se-20",
        title = "Information Extraction with auto selected algorithm - Search Engine 2.0",
        description = "Information Extraction - Best Algorithm - Search Engine 2.0 using auto generated FE in categories",
        version = "${project.version}",
        type = ModelType.IE
)
@HypermodelConfiguration(IeAutoAlgorithmHypermodelConfiguration.class)
@Capabilities({SEARCH_ENGINE_2_0, RETRAINING})
public class IeAutoAlgorithmHypermodel extends IeGenericSe20Hypermodel {
}

To execute Automatic Algorithms selection, you can use the standard WorkFusion runners.

Classification

You can apply the Automatic Algorithms selection to the Classification case in the same way:

@Import(configurations = {
        @Import.Configuration(value = GenericMultiClassificationHypermodelConfiguration.class)
})
@ModelConfiguration
public class ClassificationAutoAlgorithmHypermodelConfiguration {

        @Named("mlConfig")
        public Classifier mlConfig() {

            CompositeClassifier compositeClassifier = new CompositeClassifier.Builder<>()
                    .modelName("generic.composite")
                    .build();

            RandomForestClassifier randomForestClassifier = new RandomForestClassifier.Builder<>()
                    .maxFeatures(123)
                    .build();

            compositeClassifier.getClassifiersMap().put("liblinear", randomForestClassifier);

            LogisticRegression logisticRegression = LogisticRegression.builder()
                    .maxIterations(10)
                    .seed(123)
                    .build();

            compositeClassifier.getClassifiersMap().put(logisticRegression.getModelName(), logisticRegression);

            return compositeClassifier;
        }

}

Here, you choose between RandomForestClassifier and LogisticRegression. Your class with the model looks as follows:

@ModelDescription(
        code = "classification-auto-algorithm-se-20",
        title = "Classification with auto selected algorithm - Search Engine 2.0",
        description = "Classification - Best Algorithm - Search Engine 2.0 using auto generated FE in categories",
        version = "${project.version}",
        type = ModelType.CLASSIFICATION
)
@HypermodelConfiguration(ClassificationAutoAlgorithmHypermodelConfiguration.class)
@Capabilities({SEARCH_ENGINE_2_0, RETRAINING})
public class ClassificationAutoAlgorithmHypermodel extends MultiClassClassificationGenericSe20Hypermodel {
}

The best algorithm is applied automatically during training.

note

The score calculated during the parameter selection is the WorkFusion's internal score. It is the average of F1 from all scores based on how well the model predicts BIESO classes. It does not account for pre- and post-processing. That is why it does not correlate with the final model F1 score.