Information about Data Stores
The table below illustrates on which step which Data Store is created. Keep in mind that in the table only parameters of Data Stores are mentioned.
| BP 1 | BP 2 | BP 3 |
|---|---|---|
| documents_info | da_automation_training_set | documents_distribution |
| documents_statistics | ||
| documents_fields_statistics |
Below you can find detailed information about columns and status in the Data Stores.
documents_info
| system_id | layout | ocr_status | ocr_test | run_uuid | original_document_link | ocr_document_url | train_test |
|---|---|---|---|---|---|---|---|
| 1 | MSNL (layout of the document that was defined during BP 1 or was predefined) | not_processed (means the document was not taken for OCR check) | 25b3019c-29a7-4f4e-ae10-25fecbe05fa3 (UUID of the BP1) | https://s3.amazonaws.com... | https://velcom-s3-ct1.workfusion.com.... | train (those documents which were tagged and were added to automation training set will have this status. Status appears when training in BP3 was launched) | |
| 2 | MSNL | Unacceptable (means the document doesn't have acceptable OCR quality) | true (means the document was taken for OCR check) | 25b3019c-29a7-4f4e-ae10-25fecbe05fa3 | https://s3.amazonaws.com... | https://velcom-s3-ct1.workfusion.com.... | unseen (those documents which were tagged and were selected by bot for extraction step. Status appears when training in BP3 was launched) |
| 3 | MSNL | Acceptable (means the document has acceptable OCR quality) | true | 25b3019c-29a7-4f4e-ae10-25fecbe05fa3 | https://s3.amazonaws.com... | https://velcom-s3-ct1.workfusion.com.... | (those documents which were not selected in any of the steps) |
da_automation_training_set
| tagged_text | train_status | tagged_text_hash | original_document_link | tagged_text_link |
|---|---|---|---|---|
<document xmlns="http://ocr.crowdcomputingsystems.c... | NEW (all documents will have this status from the start of BP3) | ca662f0780bb0b64bad9974176a5976c | link to original document | link to the tagged document |
| EVALUATE (status appear when model training was launched) | ||||
| PRODUCTION (status appear when model training was finished) |
documents_distribution
| system_id | field | documents_percentage | objects_count | group_field | total_documents_count | run_uuid | type | documents_count |
|---|---|---|---|---|---|---|---|---|
| 1 | issue_price | 75.00% (percentage of the documents out of all documents that have defined field) | 15 (number of objects of defined field) | FALSE | 20 (total number of documents in data set) | 68c0de17-9d17-40e0-bb46-26cfb3741bef (UUID of the BP3) | NUMBER (type of the field) | 15 (number of documents that have defined field) |
| 2 | issue_date | 100.00% | 20 | FALSE | 20 | 68c0de17-9d17-40e0-bb46-26cfb3741bef | DATE | 20 |
documents_statistics
| run_uuid | field_name | layout | document_name | automatically_extracted | manually_extracted | decision |
|---|---|---|---|---|---|---|
| 68c0de17-9d17-40e0-bb46-26cfb3741bef (UUID of the BP3) | issue_amount | BARC | https://s3.amazonaws.com... | 800,000 (model extraction) | 800,000 (answer from human task where DA corrects model extraction) | TP |
documents_fields_statistics
| run_uuid | field_name | layout | automation_accuracy | automation_rate |
|---|---|---|---|---|
| 68c0de17-9d17-40e0-bb46-26cfb3741bef (UUID of the BP3) | issue_amount | BARC | 1 (automatically calculated accuracy based on data from parameter document_statistics) | 1 (automatically calculated rate based on data from parameter document_statistics) |