Handwritten text recognition

Marián Skrip

supervisor: RNDr. Peter Borovanský, PhD.

advisor: Mgr. Peter Gergeľ

Goal

Mobile app for handwritten mathematical text recognition based on neural network technology

Mathpix - https://mathpix.com/

Inspiration

Neural network

Convolutional neural network

My model

• blue - input parameters (data, class) 
• red - consolutional layers
• green - max pooling layer
• gray - output layers (dropout, success and error rate calculation)

Training

Final accuracy: 99%

Light line - instant accuracy, Strong line - average accuracy

Input

Threshold

Character segmentation

Simple mobile app

• Android
• Simple user interface
• Taking photo starts communication with server

Simple mobile app

WolframAlpha

Making some sense of output

Problems

Segmentation of characters

Training data

Main gains

• Bc. Marián Skrip
• Showing capabilities of neural nets

Thanks for your attention