Configure automatic algorithm selection

The automatic algorithm selection is a mechanism that finds the best classifier for a dataset to apply during model training.

info

• Only for AutoML SDK v.10.2.4.13 and newer.

• Only Liblinear and Python classifiers can be used.

CompositeClassifier is a new classifier type that is necessary and sufficient for enumerating Machine Learning algorithms.

See sample code

public class CompositeClassifier implements Classifier {

    private final String modelName;
    private final Map<String, Classifier> classifiersMap;

    private CompositeClassifier(Builder<?> builder) {
        this.modelName = builder.modelName;
        classifiersMap = new HashMap<>();
    }

    public String getModelName() {
        return modelName;
    }

    public Map<String, Classifier> getClassifiersMap() {
        return classifiersMap;
    }

    public static class Builder<T extends Builder<T>> {

        protected String modelName;

        public CompositeClassifier build() {
            return new CompositeClassifier(this);
        }

        public T modelName(String name) {
            this.modelName = name;
            return (T) this;
        }

    }

}

Launch automatic algorithm selection

To start the iteration, add CompositeClassifier to your configuration with desired algorithms.

Information Extraction case

See sample code

@Import(configurations = {
        @Import.Configuration(value = GenericIeHypermodelConfiguration.class)
  })
@ModelConfiguration
public class IeAutoAlgorithmHypermodelConfiguration {

    @Named("mlConfig")
    public Classifier mlConfig() {

        CompositeClassifier compositeClassifier = new CompositeClassifier.Builder<>()
                .modelName("generic.composite")
                .build();

        LiblinearClassifier liblinearClassifier = new LiblinearClassifier.Builder<>()
                .solver(LiblinearClassifier.Solver.L2R_L2LOSS_SVC)
                .quietMode()
                .build();

        compositeClassifier.getClassifiersMap().put("liblinear", liblinearClassifier);

        GradientBoostingClassifier gradientBoostingClassifier = GradientBoostingClassifier.builder()
                .seed(123)
                .build();

        compositeClassifier.getClassifiersMap().put(gradientBoostingClassifier.getModelName(), gradientBoostingClassifier);

        return compositeClassifier;
    }

}

Here, you choose between RandomForestClassifier and LogisticRegression.

See sample code for your class with model

@ModelDescription(
        code = "information-extraction-auto-algorithm-se-20",
        title = "Information Extraction with auto selected algorithm - Search Engine 2.0",
        description = "Information Extraction - Best Algorithm - Search Engine 2.0 using auto generated FE in categories",
        version = "${project.version}",
        type = ModelType.IE
)
@HypermodelConfiguration(IeAutoAlgorithmHypermodelConfiguration.class)
@Capabilities({SEARCH_ENGINE_2_0, RETRAINING})
public class IeAutoAlgorithmHypermodel extends IeGenericSe20Hypermodel {
}

To execute automatic algorithms selection, you can use the standard WorkFusion runners.

Classification case

You can apply the automatic algorithms selection to the Classification case in the same way:

See sample code

@Import(configurations = {
        @Import.Configuration(value = GenericMultiClassificationHypermodelConfiguration.class)
})
@ModelConfiguration
public class ClassificationAutoAlgorithmHypermodelConfiguration {

        @Named("mlConfig")
        public Classifier mlConfig() {

            CompositeClassifier compositeClassifier = new CompositeClassifier.Builder<>()
                    .modelName("generic.composite")
                    .build();

            RandomForestClassifier randomForestClassifier = new RandomForestClassifier.Builder<>()
                    .maxFeatures(123)
                    .build();

            compositeClassifier.getClassifiersMap().put("liblinear", randomForestClassifier);

            LogisticRegression logisticRegression = LogisticRegression.builder()
                    .maxIterations(10)
                    .seed(123)
                    .build();

            compositeClassifier.getClassifiersMap().put(logisticRegression.getModelName(), logisticRegression);

            return compositeClassifier;
        }

}

Here, you choose between RandomForestClassifier and LogisticRegression.

See sample code for your class with model

@ModelDescription(
        code = "classification-auto-algorithm-se-20",
        title = "Classification with auto selected algorithm - Search Engine 2.0",
        description = "Classification - Best Algorithm - Search Engine 2.0 using auto generated FE in categories",
        version = "${project.version}",
        type = ModelType.CLASSIFICATION
)
@HypermodelConfiguration(ClassificationAutoAlgorithmHypermodelConfiguration.class)
@Capabilities({SEARCH_ENGINE_2_0, RETRAINING})
public class ClassificationAutoAlgorithmHypermodel extends MultiClassClassificationGenericSe20Hypermodel {
}

The best algorithm is applied automatically during training.

note

The score calculated during the parameter selection is the WorkFusion's internal score. It is the average of F1 from all scores based on how well the model predicts BIESO classes. It does not account for pre- and post-processing. That is why it does not correlate with the final model F1 score.