Edit

Infrastructure capacity planning

Summary

A common concern of enterprise is whether the required resources are in place to handle an increase in users or number of transactions. Capacity is defined as the maximum throughput a service, system, or device can handle.

Infrastructure capacity planning is the process of determining the hardware and software resources required to meet processing demand for current moment as well as over future period of time.

Capacity Planning is a part of a wider field of Capacity Management. Capacity Management is concerned with ensuring that cost-effective capacity, which meets or exceeds the needs of the business as established in Service Level Agreements, exists at any given time.

Standard build

Refer to System requirements. The page summarizes the recommended standard specification for the environment to run WorkFusion SPA.

The recommended production configuration is set up to handle the following load¹:

• 40 active Business Processes
  • Total of 4,000,000 transactions a year
• 7 simultaneous OCR jobs
  • Total of 10,000,000 OCR pages a year ²
• 4 simultaneous AutoML prediction jobs
• 1 simultaneous HPO AutoML model training ³
• 5 bots ⁴
• 50 simultaneous (unlimited of named or registered) WorkSpace workers
• 8 simultaneous (unlimited of named or registered) Control Tower users
• 50 simultaneous Analytics users ⁵

¹ There are a number of factors that can impact performance and scalability of the software. Some of the important system variables include business processes design, server configuration, infrastructure tuning, data environment, compute capacity, and networking. The above numbers were established in a closed network lab on automated load tests mimicking realistic production workload.

² 100K pages is typical bundled volume. Additional pages require an extra license to be purchased.

³ AutoML HPO training is performed on Development environment.

⁴ The bot quantity can be easily scaled up, see Horizontal scaling.

⁵ 5 named Analytics users is typical bundled volume. Additional users require an extra license to be purchased.

RPA

Bot number estimation

The first step in planning the RPA capacity is estimating the number of bots required to handle the work.

Horizontal scaling

The RPA scaling is done by horizontally scaling the hardware resources (VDIs or RPA servers) executing RPA tasks.

Scaling to 100 bots is possible purely by adding the RPA resources.

To scale beyond 100 bots, an Application server upgrade may be required. For example, upgrading CPU on Application server to 32 cores will allow horizontal scaling of the RPA component up to 500 bots.

Minimal system requirements for single bot

Single bot is operational with the following system specifications:

• 1 core
• 2 GB RAM
• 10 GB HDD

Production deployment on VDI

In case of bots deployments on VDI technology, each bot has a dedicated VDI.

Typical VDI capacity is as follows:

• 2 cores
• 8 GB RAM
• 50 GB HDD

Production deployment on server

In case of bots deployments on Windows Server, multiple bots can be deployed to a single server.

Typical requirements for 5 bots are as follows:

• 8 cores
• 16 GB RAM
• 250 GB HDD

Typical requirements for 10 bots are as follows:

• 16 cores
• 32 GB RAM
• 500 GB HDD

For this type of deployment you need Terminal Services license.

OCR

OCR engine allows processing of incoming requests in parallel using multiple CPU cores. Requirements for 1 processor are as follows:

• 1 core
• 4 GB RAM
• 10 GB HDD

In the OCR cluster, one CPU is reserved for orchestration (master) activities. On all the remaining CPU Cores, OCR requests can be paralleled:

workers = CPU Cores - 1

Transaction throughput calculation

It's possible to calculate minimal number of required OCR workers knowing the following details:

• TT: transaction time (average time for processing single item)

• N: number of transactions per time interval

• T: time interval (seconds)

  min(workers) = TT * N / T

Example: OCR processing takes 20 seconds (TT = 20) for one-page document. If we want to process 30 documents (N = 30) within 1 minute (T = 60), then we need 20 * 30 / 60 = at least 10 workers.

Scenario A

• 10 seconds per page
• 5 pages per document

Scenario B

• 10 seconds per page
• 10 pages per document

Factors which might affect processing speed are as follows:

• OCR parameters (for example, number of output document types)
• CPU clock speed
• Document complexity
• How efficiently Business Process loads OCR queue: its best when there is always additional 30% documents in queue comparing to the number of workers, example: 32 workers and 40 documents in queue. This can be achieved via setting high number of threads in a Bot Source.

To estimate exact hardware requirements, it's better to run several documents through OCR on test server and check OCR logs for timing details.

Machine Learning

Additional system resources might be required for the following cases:

• Parallel extraction with different models
• Scheduled parallel training

It is possible to setup Mesos cluster on several ML servers and utilize all system resources of the cluster.

Exact CPU and RAM requirements can be estimated during runtime. You should always start with recommended production setup.

Additional things to consider:

• Compute intensive training or extract process is done on demand or with intervals.
• Training can be run in cloud using WorkFusion infrastructure if the input data can be supplied.

ML capacity planning building blocks

Search Engine training

Running Search Engine to produce metamodel requires the following resources for each field:

• 8 cores
• 32 GB RAM
• 2 hours of processing time per field
• 1 TB HDD

Adding more resources enables Search Engine training for multiple fields in parallel.

Metamodel retraining

Retraining a metamodel requires the following resources for each field:

• 4 cores
• 32 GB RAM
• 1 hour of processing time per field

Adding more resources enables metamodel retraining for multiple fields in parallel.

Model prediction

Use of the trained model (multi-field) on an unseen document to generate ML predictions requires the following resources:

• 2 core
• 8 GB RAM
• 5 GB HDD
• A few seconds of the processing time (ranges from milliseconds to a minute depending on the document size) per field (prediction for a document is done sequentially, field by field).

If server resources are available, the clustering engine automatically spins up additional model instances to process multiple incoming requests from the queue concurrently.

Requirements

The requirements may differ based on the training strategy selected. For more information, see ML training strategies.

For detailed calculations, refer to Hardware requirements.

For optimal resources utilization and processing speed, it's beneficial to build an independent capacity planning framework accounting for the infrastructure features, previous studies, load testing results, performance optimizations applied.

For high-volume large-scale deployments, it’s recommended to do additional analysis, have dedicated stages for high scale design, implementation, testing and load testing, configuration, and implementation optimizations.