Augmented Reality Equation Plotter

Salman Naqvi
salman.naqvi@stanford.edu

Final Project EE 368 Spring 2013

Uzair Sikora
usikora@stanford.edu

Motivation:
A student in a math, science, or engineering class will undoubtedly come across an equation in the
course of his or her studies that the student will want to quickly visualize. Plotting the equation allows the
student to better understand some property of the real variables the equation relates. However, this may still
be difficult for younger students that are working on building the intuition to be able to visualize simple
equations quickly in their thoughts. In addition, some three dimensional expressions are difficult to picture
in ones thoughts even with the intuition. While there are several tools available to the student to accomplish
this plotting, all involve leaving the context the equation originally appears in; i.e. the notebook. The goal
of this app is to provide the capability for the student to visualize an equation in his or her notes using the
Android platform.
Methods:
We will implement this using a Motorola DROID smartphone. The general approach we will take
is to first detect and identify the equation to be plotted in the image from the camera. Then detect and
identify the axis the user has drawn and any features the user has specified. For example, the presence of
any labeled tic marks and their spacing on the axis, axis labels, or the number of axis drawn. Finally, we
will then update the screen with the plot in the context of the axis defined by the user. This can be broken
down into a number of steps:
1. Equation Detection
a. Binarize the image using locally adaptive thresholding.
b. Segment the individual characters.
c. Use Optical Character Recognition to classify the characters. (Implemented by the
Tesseract OCR engine.) [1]
d. Analyze equation. (Check for consistency, number of dimensions, etc.)
2. Axis Detection
a. Binarize as before.
b. Segment out the axis regions.
c. Determine if there are any other markings around like tic marks by using a
morphological detector.
d. Track the axis location using feature tracking implemented in OpenCV. [2]
3. Plotting
a. Update the users screen with the plot in the correct region with rotation and resizing
applied as necessary.
As a start to the proposed project we will focus on detecting and plotting 2D functions that are
printed on paper as opposed to hand written. As we polish our approach, we will add 3D function
capabilities and hand-written equation detection as time permits.
References
[1] R. Smith, "An Overview of the Tesseract OCR Engine," in Proc. Ninth Int. Conference on Document Analysis
and Recognition (ICDAR), 2007.
[2] Itseez, "Android | OpenCV," 2013. [Online]. Available: http://opencv.org/platforms/android.html.
[3] R. Zanibbi and D. Blostein, "Recognition and retrieval of mathematical expressions," International Journal on
Document Analysis and Recognition (IJDAR), vol. 15, pp. 331-357, 2012.
[4] S. S. Tsai, H. Chen, D. Chen, R. Vedantham, R. Grzeszczuk and B. Girod, "Mobile visual search using image
and text features," in Signals, Systems and Computers (ASILOMAR), 2011 Conference Record of the Forty
Fifth Asilomar Conference on, 2011.