Recognizing 7-segment LCD display characters using OCR-IT API

Recognizing 7-segment LCD display characters using OCR-IT API

This question has been asked by a developer on Stackoverflow (source link).  OCR-IT API came to the rescue.

“I’m trying to develop a Windows Phone 8.1 App but I need to recognize some numbers from different Displays.”

ocr-it_api_cloud_sdk_document_conversion_original_image1_7-segement1

ocr-it_api_cloud_sdk_document_conversion_original_image2_7-segement
ocr-it_api_cloud_sdk_document_conversion_original_image3_7-segement

I wish the answer to your question would be “Sure, here it is” with a link to a black-box process-anything OCR tool, but there are several stages and steps to the success are involved, which are best considered separately.

First, there is some work on image pre-processing BEFORE you even consider any OCR. These image samples are very drastically different, and include full range of issues.

SAMPLE 1 has low contrast, so when it is binarized to black and white layer, which most OCR will perform internally at some stage, there are no characters to process. It looks like this after binarization:

ocr-it_api_cloud_sdk_document_conversion_binarized_image_7-segement

See this OCR Blog post for additional details on image pre-processing: http://www.ocr-it.com/guide-to-better-mobile-images-from-cell-phone-camera-for-higher-quality-ocr.

Secondly, the image has no dpi information in the header, which some OCR technologies use to determine appropriate scaling of the image. Without header information, some OCR programs may set some default dpi, which may or may not match your image, thus affecting the OCR result. This is NOT critical, but preferred if this can be implemented at the time of picture creation.

SAMPLE 2 has sufficient contrast and adaptive notarization returns a clear image. It is also missing dpi resolution value in the header.

ocr-it_api_cloud_sdk_document_conversion_binarized_image2_7-segement

SAMPLE 3 has very clear contrast, but it also has no resolution dpi in the header.

Once you have images that are optimized for OCR processing, the next step is to look at OCR technologies.

In general, there in no 7-segment OCR known to me. However, we were able to adapt OCR-IT recognition engine for this specialized task. The OCR ‘out-of-box’ or with default settings is unable to handle this recognition. And it is logical and expected. Why? Because most OCR are written to recognize inseparable pixel pattern for each character. This is related to “character separability” principle used to separate words into separate characters. In other words, inner OCR algorithms look for connected strokes which make up each characters. More powerful commercial OCR allow some breaks in pixel patterns, but they are expected to be minimal to none, like defects in print or scan, which may result in missing character pieces.  OCR-IT API has all necessary tools, so read on…

7-segment display by nature will have multiple breaks in each character, conflicting with the character separability principle.

More powerful OCR technologies have a) more tolerance to breaks in pixel patterns and/or b) have special settings to handle these cases.

SAMPLE 3. This is the easiest sample to process, so I tested it first. Using OCR-IT API, and making a request with default settings, requesting the output to TXT format, I get the following:

ocr-it_api_cloud_sdk_document_conversion_sample3_7-segement_result_default

It appears that OCR is a) segmenting characters into two separate lines, and b) tries to read resulting patters as close as possible to valid characters.

ocr-it_api_cloud_sdk_document_conversion_original_image3_7-segement_segmentation

Based on this quick analysis, making one adjustments to OCR settings results in the following recognition:

ocr-it_api_cloud_sdk_document_conversion_original_image3_7-segement_good-result

The setting that made substantial difference in OCR result is switching from default print type to using “DotMatrix”, which is in the middle of this entire OCR-IT API settings XML:

 <Job>
<InputURL>http://i.stack.imgur.com/wOtFx.jpg</InputURL>
<CleanupSettings>
<Deskew>false</Deskew>
<RemoveGarbage>false</RemoveGarbage>
<RemoveTexture>false</RemoveTexture>
<RotationType>NoRotation</RotationType>
</CleanupSettings>
<OCRSettings>
<PrintType>DotMatrix</PrintType>
<OCRLanguage>English</OCRLanguage>
<SpeedOCR>false</SpeedOCR>
<AnalysisMode>MixedDocument</AnalysisMode>
<LookForBarcodes>false</LookForBarcodes>
</OCRSettings>
<OutputSettings>
<ExportFormat>Text</ExportFormat>
</OutputSettings>
</Job>

The use of DotMatrix print type turned on necessary algorithms to increase tolerance for breaks in character structure, which commonly occurs by nature of dot-matrix printers in dot-matrix prints. Alternatively, a “Typewriter” print type could be used, since character breaks are also expected in typewritten fonts, thus being automatically handled by OCR.

There could be one more change to the API setting to run OCR using “Digits” character set (language), effectively eliminating any possibility of mis-reading 1 as I, etc.

SAMPLE 2. In this sample, the gaps in each character’s structure are much wider. Even standard algorithms for handling DotMatrix or Typerwriter print types cannot accommodate for these wide gaps. The use of all possible setting variations returned something like this:

ocr-it_api_cloud_sdk_document_conversion_original_image2_7-segement_bad-result

Character segmentation seems to be the issue. One technical solutions goes back to image pre-processing. A simple algorithm can be implemented to fill in gaps between each segment of the 7-segment character. It does not have to be very precise, something like this:

ocr-it_api_cloud_sdk_document_conversion_binarized_image2_7-segement_edited

But that is enough to produce a perfect OCR result.

ocr-it_api_cloud_sdk_document_conversion_original_image2_7-segement_good-result

Since it may be unknown in advance which 7-segment LCD display will require filled in gaps, and which does not, I recommend to apply this algorithm to all LCD 7-segment images, with small or large gaps. I would limit the size of the gap to no wider than the width of a segment. Given these screens come in various background and segment colors, this pre-procession algorithm can be substantially simplified if it is performed on binarized (black & white) image.

Overall, this task is possible with OCR and near out-of-box functionality, assuming that some image pre-processing is performed. In general, I believe that image pre-processing is required for any OCR-related project anyway, specific to that project.

If you have any further questions about OCR or image pre-processing, please contact us at support@ocr-it.com.