Skip to main content
Version: 10.2.9

Learn about WorkFusion models

The article highlights models you can access in Work.AI.

AutoML SDK models​

AutoML SDK models (also, AutoML models) are out-of-the-box Work.AI models comprising a combination of AutoML SDK components, such as algorithms and their parameters, annotators, feature extractors, pre- and post-processors.

Compatibility: All Work.AI versions

Hardware requirements: 8 CPUs, 32 GB RAM, disk storage 150 GB for training and inference

Associated Work.AI components:

Use cases​

AutoML models are used for, but not limited to, the following major use cases:

  • Classification means arranging objects in groups or categories based on established criteria. Both binary and multi-class grouping is supported. Examples include email classification, document identification, classification of pages within documents, and approval process.

  • Information extraction (IE) involves extracting structured information, or key facts, from unstructured or semi-structured documents. Examples are extraction of details from invoices, claims, and financial documents.

The following models are intended to support the classification and IE use cases in the following way:

  • Checker models improve information extraction results. The model is trained to identify incorrect values produced by the original model and to post-process them, that is, validate and remove, if needed. For details, read the Checker model topic.

  • Grouping models enable developers to combine existing or new models into a single artifact to reduce the number of Workers and minimize the start time on the cluster. For details, read the Grouping models topic.

LayoutLM models​

LayoutLM is a deep learning model designed for document understanding, particularly in contexts where both textual and layout information is important.

  • Effective for structured document types where both content and layout matter.
  • Handles OCR (optical character recognition) outputs and is robust to layout-related noise in such data.
  • Works well in industries like banking, insurance, legal, and healthcare where document automation is critical.

Compatibility: Work.AI v10.2.9+

Hardware requirements:

  • Training: 1 GPU (NVIDIA V100), 16 CPUs, 32 GB RAM, 100 GB SSD drive
  • Inference: 16 CPUs, 32 GB RAM, 100 GB SSD drive

Associated Work.AI components:

Key features​

  • Multimodal input

    LayoutLM processes both the textual content of documents and their layout information (for instance, the position of text blocks on a page). This makes it well-suited for tasks where the spatial arrangement of text on a page provides additional context (for instance, in forms or tables).

  • 2D position information

    The model incorporates not only the tokenized text but also the 2D coordinates of each token, representing its position on the document page. This spatial information is key in understanding the layout and hierarchy of text elements in documents.

  • Transformer-based architecture

    LayoutLM uses the transformer architecture, similar to models like BERT, but it extends the idea by adding layout-specific embeddings. These embeddings represent the coordinates and spatial relationships between different text elements, allowing the model to capture both the semantic and spatial context of a document.

  • Pretraining

    LayoutLM is pretrained on large datasets of documents using two major objectives: masked language modeling (like BERT) and a new task that predicts the relative positions of words. This pre-training helps the model understand the document structure before fine-tuning it on specific downstream tasks, such as form recognition or document classification.

  • Fine-tuning

    Once pretrained, LayoutLM can be fine-tuned on extracting specific entities, such as dates, names, or other key information, from documents in real-life cases.

Use cases​

  • Invoice processing: extracting fields like invoice numbers, dates, and total amounts from invoices.
  • Form understanding: extracting information from structured forms with complex layouts.
  • Receipt scanning: identifying key fields, like vendor name, date, and total price, from scanned receipts.

LayoutLM vs AutoML​

As compared to AutoML models, LayoutLM ones have the following advantages:

  • Better at processing complex document structures (complex tables, lack of context around fields).
  • Works well with high variability of document templates.
  • Fewer documents required per template: five to ten documents per template for LayoutLM. For AutoML, the requirement is 50 documents.
  • Better results on unknown templates.

Limitations​

  • Depends on OCR quality: errors in text recognition or bounding boxes reduce accuracy.

  • Not ideal for pure natural language processing (NLP) tasks: adds overhead without benefits when layouts are irrelevant.

  • Computational overhead: larger memory and compute requirements due to multimodal embeddings.

LLM models​

LLM (Large Language Model) is an artificial intelligence program that can recognize and generate text, among other tasks. LLMs are pre-trained on large datasets, hence the "large" attribute.

Compatibility: Work.AI v10.2.9+

Use cases​

LLM models are used to enhance the Work.AI functionality and assist in tasks, such as dataset labeling, rule building, and Manual Task design.

Dataset labeling​

LLMs provide zero-shot learning on documents to create a training set. In the resulting set, the LLM attempts to pre-populate fields to extract or classify so that data experts can review them before finalizing the training set for supervised machine learning. For details, read the Manage datasets topic.

For the labeling application, LLM models are available from both WorkFusion and external providers, such as OpenAI and MistralAI. Labeling with a WorkFusion LLM model is enabled out of the box. To enable labeling with OpenAI and MistralAI, follow the guide.

Decision or transformation rule setup​

LLMs assist in developing no-code rules and transformations in a Business Process. The models help to generate complex business logic from simple texts provided by users. For details, read the following guides:

Designing Manual Tasks​

WorkFusion proprietary LLMs assist you in automating form layout generation for labeling tasks. A selected model creates a task form automatically based on its training set or configuration. For details, read Design Manual Tasks via Task Designer Operation.

Model pipelines​

There are two model pipeline (previously known as hyper model) types:

  • A fixed set containing an optimal model configuration created by a data scientist for a specific use case.

  • A Search Engine-based one assuming that an optimal configuration is found by running a Search Engine. Typically, the type requires more time and resources than a fixed set.

Single-model vs multi-model approach​

Under the hood, WorkFusion allows using one of the two approaches for handling complex documents:

  • Single-model
  • Multi-model

Single-model approach​

This approach implies training a single binary model for all fields. It works when you need to extract fields with the same semantic meaning. All fields are equally important and used in a specific use case. These fields are also subject to post-processing.

This approach is optimized for training and execution.

Multi-model approach​

This approach implies training a set of individual binary models per field. It works when the semantic meaning of fields can vary, which means different subsets of fields are used in a specific use case.

This approach is optimized for a higher automation rate and accuracy.

troubleshooting

For troubleshooting tips, refer to the following support guides: