“Through co-innovation with Microsoft AI Co-Innovation Lab KOBE, we built a high-accuracy OCR solution capable of handling the diverse specialized documents used in our operations. This enabled us to preprocess complex document information into a form that can be more readily used as a knowledge asset.
We believe that using trusted knowledge appropriately is an important initiative that not only enables us to deliver value to healthcare institutions, but also contributes to patient health and to improving the quality and efficiency of healthcare.
Working with Kobe Lab, we pursued development with the goal of making this vision a reality, and the team supported us with persistence through to the end.
Under our mission of “Shaping the advancement of healthcare,” we will continue contributing to the further development of healthcare.”
Guided by its long-term vision, "Together for a better healthcare journey," Sysmex Corporation is a global healthcare company committed to solving healthcare challenges worldwide. Its business focuses primarily on in vitro diagnostics, which analyzes samples such as blood and urine. The company also researches and develops testing and diagnostic technologies using technology platforms that measure cells, proteins, and genes. Today, Sysmex provides clinical laboratory instruments, reagents, and related software in more than 190 countries and regions.
To enable more effective use of specialized healthcare information and expertise, this project developed a high-accuracy OCR solution for the wide variety of specialized documents handled in business operations. The solution extracts information while preserving specialized notation and complex document structures, then converts it into structured data suitable for downstream search and use. In doing so, the project enhanced the preprocessing required to turn reliable information into knowledge assets.
Going forward, the team will continue validating and improving the solution through real-world operation, with the goal of achieving even greater accuracy.
Co-Innovation Challenge
Due to the nature of its business, Sysmex handles a wide variety of specialized healthcare documents from Japan and around the world in its daily operations.
Reviewing and organizing the information contained in these documents had traditionally relied mainly on manual work. In recent years, however, Sysmex has also pursued operational efficiencies using Power Platform.
Healthcare documents, however, contain mathematical formulas, superscripts such as ¹², Greek letters such as α, and other special characters. They also use complex layouts that combine charts and tables with vertical and horizontal writing, as well as highly specialized medical and technical terminology. To use these documents as reliable information, Sysmex needed a solution that could preserve both document structure and specialized notation with high accuracy while balancing document quality and operational efficiency.
Examples include technical documents related to in vitro diagnostics, such as hematology, urinalysis, hemostasis, and immunochemistry, as well as documents for the medical robotics business, including the hinotori™ Surgical Robot System.
These documents contain diverse information and complex structures that vary by field. The central theme of this document-processing initiative was therefore to build a high-accuracy OCR solution that could handle such specialized documents and support their transformation into knowledge assets.
Onboarding: Co-Innovation Design Session
Before beginning prototype development, the team conducted an onboarding phase to analyze the identified challenges, perform technical validation, and explore the best approach.
Sysmex first reviewed its existing document-processing flow and identified document patterns that presented challenges in structuring and extraction accuracy. The review showed that while adequate results were being achieved for general documents, there was room to improve accuracy for the specialized notation and complex structures unique to specialist documents.
The two companies then discussed how to advance conventional OCR technology and began examining a new approach for complex medical documents. Kobe Lab evaluated multiple technical options and confirmed that Mistral, one of the frontier models available through Microsoft Foundry and a model with specialized OCR capabilities, was well suited to the Sysmex use case.
In addition to strong multilingual comprehension for documents that mix Japanese and English, Mistral OCR can semantically interpret document structures such as heading hierarchies, tables, and bulleted lists. It is also highly capable of preserving the specialized notation found in technical documents. Whereas conventional OCR is designed to read characters, a key distinction of Mistral OCR is its ability to understand an entire document and output it as structured data. Its ability to integrate securely with existing infrastructure in Azure while delivering both high-quality output and efficient inference performance was another important factor in its selection.
Architecture diagram
Based on these technical validation results, the two companies held multiple Tech Calls and architecture reviews. While preserving the existing Power Platform-based workflow, they jointly designed the system, including how to integrate Mistral OCR on Microsoft Foundry, pass data to downstream processes, and connect with translation and document-management workflows.
After approximately two months of validation and co-innovation design clarified how Mistral OCR could address the challenge, the two companies decided to conduct a five-day prototype development sprint at Kobe Lab to validate its effectiveness in a real-world environment.
In the Lab: Sprint Development
During the five-day prototype development sprint, the team integrated with the Power Platform-based document-processing flow already operated by Sysmex and developed and technically validated a next-generation document-processing solution for specialized healthcare documents.
The existing environment already operated a document-processing flow using Power Platform. In this project, the team evaluated whether incorporating intelligent document-understanding capabilities through Mistral OCR, available in Microsoft Foundry, could enable more advanced document-structure recognition and information extraction.
For the prototype, the team first built a mechanism to call Mistral OCR in Microsoft Foundry from Power Platform through Azure Functions, automating OCR processing for the target files. The team then implemented processing that converted the extracted text and layout information into structured data so it could be used in downstream business processes, creating a solution that could be incorporated into the existing workflow.
Extracting text while preserving document heading hierarchies and paragraph structures
The team also validated accuracy by comparing OCR results with the original documents, improving the character-recognition errors and loss of document structure that had affected the conventional approach. For complex charts and tables, captions, and multi-column layouts in specialized documents, the team repeatedly tested ways to obtain information while preserving document structure rather than merely extracting text.
The team also tested the application of semantic understanding to image regions and implemented an extension that adds descriptive information, or captions, to charts, tables, and images.
Extracting not only text, but also the content and meaning of charts, tables, and images as structured data
In parallel, the team evaluated the translation process, testing translation quality using large language models (LLMs) on specialized healthcare documents. This work identified technical challenges and areas for improvement toward future operations that connect OCR, translation, and document management.
The project emphasized operability as well as processing accuracy. The team also evaluated integration with Power Platform, implementation on Azure Functions, and future directions for functional expansion.
As a result, the team confirmed the potential to improve OCR accuracy and document-structure understanding for complex specialized documents while retaining the existing operational workflow. It also established a clear technical direction for advancing the solution that supports knowledge asset creation. Some functions still have room for improvement, and the team will continue validation and refinement to achieve greater accuracy.
Beyond Technology
Through this project, Sysmex took a major step toward using its diverse specialized documents as reliable knowledge assets.
Specialized healthcare documents are vital information assets that support medical progress and healthcare operations. To use these documents appropriately, the information they contain must be preserved accurately and prepared in an accessible form, because its quality affects subsequent use. The project therefore aimed not merely to improve OCR accuracy, but to build a mechanism for accurately passing on and using the information and knowledge that support healthcare. By processing the diverse information and complex structures in specialized documents with high accuracy, the project advanced an environment that supports reliable use of knowledge.
Significant accuracy improvement over conventional OCR, even for low-quality documents
This project also represented more than technical validation; it embodied co-innovation between Sysmex and Kobe Lab. By combining Sysmex's domain knowledge in medicine and testing with Microsoft's AI and cloud technologies, the team rapidly developed a prototype of an advanced OCR solution capable of handling diverse specialized documents.
From a technical perspective, the team incorporated the analytical methods designed by Sysmex around the characteristics of specialized documents into the system design and confirmed improved AI-based document-structure understanding. This established a path beyond simply digitizing documents toward preparing the information and insight they contain in a form that can be more readily used as organizational knowledge assets.
From a development perspective, the project combined no-code development centered on Power Platform with pro-code development using Microsoft Foundry, Visual Studio Code, and Azure Functions. The team also combined multiple AI technologies to assess ways of addressing challenges that are difficult for a single model to solve. This work clarified an implementation approach for a document-processing solution with the high accuracy and reliability required in healthcare.
Building on these outcomes, the project marked an important first step toward connecting trusted information and knowledge to a better future for healthcare.
Sysmex and Microsoft will continue using AI and digital technologies to maximize the value of information and knowledge in healthcare and pursue initiatives that deliver new value to healthcare professionals and patients.
* hinotori™ is a registered trademark of Medicaroid Corporation. Sysmex serves as the global master distributor of Medicaroid's hinotori™ Surgical Robot System, selling it in Japan and overseas.