Merge and split models

WorkFusion features a merge-and-split utility for information extraction (IE) models trained on different machines or with different fieldsets.

Merge IE models

The merge functionality example is as follows:

File model1Dir = new File(getResource(this.getClass(), "util/model1").toURI());
File tempModel1Folder = Files.newTemporaryFolder();

File model2Dir = new File(getResource(this.getClass(), "util/model2").toURI());
File tempModel2Folder = Files.newTemporaryFolder();

// Merging modifies the input folders, so it is recommended to make a copy.

FileUtils.copyDirectory(model1Dir, tempModel1Folder);
FileUtils.copyDirectory(model2Dir, tempModel2Folder);

File tempModel1InputFolder = new File(tempModel1Folder.getPath() + "/training");
File tempModel2InputFolder = new File(tempModel2Folder.getPath() + "/training");

The merge result is available in tempModel1Folder:

ModelMergeUtil.merge(tempModel1InputFolder.getPath(), tempModel2InputFolder.getPath());

Split IE models

The split functionality example is as follows:

File model1Dir = new File(getResource(this.getClass(), "util/model1").toURI());
File tempModelFolder = Files.newTemporaryFolder();

// Splitting modifies the input folder, so it is recommended to make a copy.

FileUtils.copyDirectory(model1Dir, tempModelFolder);

File tempModelInputFolder = new File(tempModelFolder.getPath() + "/training");

The split results are available in the tempModelFolder and tempModelFolder_split folders created in the same location. Mind to separate features to be kept by a comma.

ModelSplitUtil.split(tempModelInputFolder.getPath(), "order_total");