Google PlusFacebookTwitter

Making a Simple OCR Android App using Tesseract

By on Nov 9, 2011 in Open Source | 473 comments

Share On GoogleShare On FacebookShare On Twitter

This post tells you how you can easily make an Android application to extract the text from the image being captured by the camera of your Android phone! We’ll be using a fork of Tesseract Android Tools by Robert Theis called Tess Two. They are based on the Tesseract OCR Engine (mainly maintained by Google) and Leptonica image processing libraries.

Recognizing text using your Android phone. Not exactly the end result of this blog post, but what you could achieve.

Note: These instructions are for Android SDK r19 and Android NDK r7c, at least for the time being (written at this tree). On 64-bit Ubuntu, you may need to install the ia32-libs 32-bit compatibility library. You would also need proper PATH variables added (see Troubleshooting section below).

  1. Download the source or clone this git repository. This project contains tools for compiling the Tesseract, Leptonica, and JPEG libraries for use on Android. It contains an Eclipse Android library project that provides a Java API for accessing natively-compiled Tesseract and Leptonica APIs. You don’t need eyes-two code, you can do without it.
  2. Build this project using these commands (here, tess-two is the directory inside tess-two – the one at the same level as of tess-two-test):
    cd <project-directory>/tess-two
    ndk-build
    android update project --path .
    ant release
  3. Now import the project as a library in Eclipse. File -> Import -> Existing Projects into workspace -> tess-two directory. Right click the project, Android Tools -> Fix Project Properties. Right click -> Properties -> Android -> Check Is Library.
  4. Configure your project to use the tess-two project as a library project: Right click your project name -> Properties -> Android -> Library -> Add, and choose tess-two. You’re now ready to OCR any image using the library.
  5. First, we need to get the picture itself. For that, I found a simple code to capture the image here. After we have the bitmap, we just need to perform the OCR which is relatively easy. Be sure to correct the rotation and image type by doing something like:
    // _path = path to the image to be OCRed
    ExifInterface exif = new ExifInterface(_path);
    int exifOrientation = exif.getAttributeInt(
            ExifInterface.TAG_ORIENTATION,
            ExifInterface.ORIENTATION_NORMAL);
    
    int rotate = 0;
    
    switch (exifOrientation) {
    case ExifInterface.ORIENTATION_ROTATE_90:
        rotate = 90;
        break;
    case ExifInterface.ORIENTATION_ROTATE_180:
        rotate = 180;
        break;
    case ExifInterface.ORIENTATION_ROTATE_270:
        rotate = 270;
        break;
    }
    
    if (rotate != 0) {
        int w = bitmap.getWidth();
        int h = bitmap.getHeight();
    
        // Setting pre rotate
        Matrix mtx = new Matrix();
        mtx.preRotate(rotate);
    
        // Rotating Bitmap & convert to ARGB_8888, required by tess
        bitmap = Bitmap.createBitmap(bitmap, 0, 0, w, h, mtx, false);
    }
    bitmap = bitmap.copy(Bitmap.Config.ARGB_8888, true);
  6. Now we have the image in the bitmap, and we can simply use the TessBaseAPI to run the OCR like:
    TessBaseAPI baseApi = new TessBaseAPI();
    // DATA_PATH = Path to the storage
    // lang = for which the language data exists, usually "eng"
    baseApi.init(DATA_PATH, lang);
    // Eg. baseApi.init("/mnt/sdcard/tesseract/tessdata/eng.traineddata", "eng");
    baseApi.setImage(bitmap);
    String recognizedText = baseApi.getUTF8Text();
    baseApi.end();

    (You can download the language files from here and put them in a directory on your device – manually or by code)

  7. Now that you’ve got the OCRed text in the variable recognizedText, you can do pretty much anything with it – translate, search, anything! ps. You can add various language support by having a preference and then downloading the required language data file from here. You might even put them in the assets folder and copy them to the SD card on start.

To make things easy, and for you to have a better understanding, I have uploaded a simple application on OCR that makes use of Tess Two on Github called Simple Android OCR (for beginners). If you want a full-fledged application, that has a selectable region while capturing the image, translating the text, preferences etc., then you can checkout Robert Theis’ Android OCR application (for intermediate+)!

Updated: 7 October 2012

References

  1. Using Tesseract Tools for Android to Create a Basic OCR App by Robert Theis
  2. Simple Android Photo Capture by MakeMachine
  3. tess-two README

Troubleshooting

Translations

  1. Japanese by datsuns
  2. French by Mathieu

Projects Made By Users

People have made a lot of projects using this tutorial, some of them are:

  1. DatumDroid by Aviral, Devashish and me
  2. MachineRetina by Salman Gadit

If you have made one too, do tell us in the comments below!

473 Comments

  1. konain

    May 12, 2013

    Post a Reply

    i m having a problem,whenever i run my ocr application the emulator stops on a white screen and doesnot display any my layout..

    • Kannan

      February 4, 2014

      I’m also having the same problem. The app gets closed after the picture is taken. Can anybody help to rectify this error?

  2. ramya

    May 15, 2013

    Post a Reply

    Anyone can help me, how to do an NDK Build in Windows?. This is the main task, anyone can tell me the procedure clearly for NDK build on window?..

    • Dharshana

      May 20, 2013

      Use the cgwin building tool. setup the path for NDK jar file. Then u can build.

    • Nauman

      May 20, 2013

      You can use ndk builder in eclipse. If still u r having problem contact me at nauman3128@gmail.com

    • Ramya

      May 23, 2013

      @ DHarshana and Nauman Thanks.successfully i completed NDK buid using windows itself. Its working. But character recognition using this OCR is giving 50 % output only. wat i have to do for improve this?

  3. natesh

    May 17, 2013

    Post a Reply

    Error Shows.

    import com.googlecode.tesseract.android.TessBaseAPI

    TessBaseAPI

    Please help me

    • Nauman

      May 20, 2013

      did u build it first?

  4. francis solo

    May 19, 2013

    Post a Reply

    Thanks for this post.settin up everythin was very straightforward for me.i have a runnin project but the text is sort of garbage.I just want to read 12digits serial numbers.is there a way to train mine to just 0-9.nunerals only and specified font?.thanks

  5. Nauman

    May 20, 2013

    Post a Reply

    do anyone have idea of image processing for optimized results? if so please help me in that.

  6. Ramya

    May 23, 2013

    Post a Reply

    Successfully i completed NDK buid using windows itself. Its working. But character recognition using this OCR is giving 50 % output only. wat i have to do for improve this?

    • Kage

      November 14, 2013

      Here is the same problem! Although everything is ok, result is not perfect.
      How can I repair that?

  7. sachinthana

    June 4, 2013

    Post a Reply

    I have successfully builed the NDK in windows, using cmd. no issues at all/ set NDK and SDK paths correctly. no issues with that too. after i build NDK and trying to execute

    android update project –path .

    gives an error – ‘the project either have no target set or the target is invalid ‘

    somebody please help me with this. thank you !

    • solo king jec

      June 4, 2013

      try this on windows android.bat update project -t 8 -p .

    • sachinthana

      June 4, 2013

      it did not work for the value ’8′. but for the value ’1′.

      the result is

      “updated project.properties
      added file \proguard-project.txt”

      is this the result i should be expecting ? thnx

    • CVE

      November 9, 2013

      Same problem here!

    • Chris Chang

      November 22, 2013

      You can type “android list target” to see those available android targets on your machine. Then, specify the desired target_id to “android.bat update project -t target_id -p .”.

  8. sachinthana

    June 4, 2013

    Post a Reply

    When i Go to the eclipse and try to set the tess-two as the library. The library is now shown in there. i have checked the IsLibrary too. still nothing display as a library to select. please help me

  9. ahmed

    June 5, 2013

    Post a Reply

    hi
    the library es perfecto , but the TextView in top left not show info in live camera , before capture image
    pleas help

  10. sachinthana

    June 7, 2013

    Post a Reply

    I am trying to develop the OCR to identify equations.
    I found a traineddata file that is ‘equ.traineddata’

    Can I use both ‘eng’ and ‘equ’ both tessdata to my app ?

    if so, how ??

    thnx

  11. Hope_amal

    June 19, 2013

    Post a Reply

    Hi all
    I have imported Simple Android OCR application (complete project) , all work fine when tested on my phone but the output text doesn’t fit the text in the photo at all… why is this happening?

  12. C.Eyquem

    July 30, 2013

    Post a Reply

    Hi Gautam,

    I am trying to follow your tutorial (thanks by the way for this, there is not much documentation out there about tesseract). I encounter some issues at the end of the compilation:

    $ ndk-build
    Install : liblept.so => libs/armeabi/liblept.so
    Compile++ thumb : tess <= permdawg.cpp
    jni/com_googlecode_tesseract_android/src/dict/permdawg.cpp: In member function 'void tesseract::Dict::go_deeper_dawg_fxn(char const*, const BLOB_CHOICE_LIST_VECTOR&, int, const CHAR_FRAGMENT_INFO*, bool, WERD_CHOICE*, float*, float*, WERD_CHOICE*, int*, void*)':
    jni/com_googlecode_tesseract_android/src/dict/permdawg.cpp:208:62: error: format not a string literal and no format arguments [-Werror=format-security]
    cc1plus: some warnings being treated as errors

    make: *** [obj/local/armeabi/objs/tess/src/dict/permdawg.o] Error 1

    And I have lots of other warnings during compilation, such as:

    Compile++ thumb : lept <= readfile.cpp
    jni/com_googlecode_leptonica_android/readfile.cpp: In function 'jint Java_com_googlecode_leptonica_android_ReadFile_nativeReadFiles(JNIEnv*, jclass, jstring, jstring)':
    jni/com_googlecode_leptonica_android/readfile.cpp:103:12: warning: converting to non-pointer type 'int' from NULL [-Wconversion-null]
    jni/com_googlecode_leptonica_android/readfile.cpp:109:12: warning: converting to non-pointer type 'int' from NULL [-Wconversion-null]
    jni/com_googlecode_leptonica_android/readfile.cpp: In function 'jint Java_com_googlecode_leptonica_android_ReadFile_nativeReadFile(JNIEnv*, jclass, jstring)':
    jni/com_googlecode_leptonica_android/readfile.cpp:127:12: warning: converting to non-pointer type 'int' from NULL [-Wconversion-null]

    Any idea of how to solve this? I have a fresh download and install of ADT with NDK from less than a week and working on a MacBook.

    Config:

    $ uname -a
    Darwin MacBook-XXX.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64

    $ ndk-build -v
    GNU Make 3.81
    Copyright (C) 2006 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.
    There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
    PARTICULAR PURPOSE.

    This program built for x86_64-apple-darwin

    $ echo $PATH
    /opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Users/christophe/Documents/ADT/ndk:/Users/christophe/Documents/ADT/sdk/tools/:/Users/christophe/Documents/ADT/sdk/platform-tools/

    • Kien

      July 30, 2013

      There is an error on tess-two project, at line 208, file permdawg.cpp :
      jni/com_googlecode_tesseract_android/src/dict/permdawg.cpp:208:62: error: format not a string literal and no format

    • C.Eyquem

      August 1, 2013

      Are we supposed to correct such errors? I have not seen anything about this case on the forums. Isn’t it more related to the compiler or system settings?

    • rmtheis

      August 1, 2013

      Thanks for pointing this out. This problem was introduced with NDK v9. I committed a fix, so please pull the latest tess-two code and try the build again.

    • otidh

      August 21, 2013

      @rmtheis thanks… :)

  13. sujith

    August 2, 2013

    Post a Reply

    I had completed all the step mentioned here to get the out put ..
    but unfortunately an exception occurred after taking the photo…
    so i made little bit change in my application and tested with a picture that manually pushed by me to the sdcard, when i running in emulator shows the following exception


    E/AndroidRuntime(3275): java.lang.ExceptionInInitializerError
    08- at com.datumdroid.android.ocr.simple.SimpleAndroidOCRActivity.onPhotoTaken(SimpleAndroidOCRActivity.java:212)
    08- at com.datumdroid.android.ocr.simple.SimpleAndroidOCRActivity.startCameraActivity(SimpleAndroidOCRActivity.java:122)
    08- at com.datumdroid.android.ocr.simple.SimpleAndroidOCRActivity$ButtonClickHandler.onClick(SimpleAndroidOCRActivity.java:112)
    08- at android.view.View.performClick(View.java:4204)
    ...
    08- at java.lang.System.loadLibrary(System.java:535)
    08 at com.googlecode.tesseract.android.TessBaseAPI.(TessBaseAPI.java:45)

  14. Gorjan

    August 29, 2013

    Post a Reply

    Gautam, I followed all the steps in your tutorial, and everything went according to plan. I successfully imported the tess-two library in eclipse, made it a library project and imported it into my project. But, at runtime, when I try to execute:

    baseApi.init(path, lang);

    I get an exception: ljava/lang/unsatisfiedlinkerror thrown while initializing Lcom/googlecode/tesseract/android/TessBaseAPI

    can you please help me overcome this issue…
    I’m developing a simple OCR receipt scanner, and I’m working on windows8.

  15. sachinthana

    September 5, 2013

    Post a Reply

    Dear Gautam/rmtheis,

    I successfully integrated your advanced OCR. I need this app to identify few certain symbols that are used in mathematics such as pi and sigma. I have successfully followed the instruction of how to train a language in google groups.
    The problem I have is even I trained those symbols, they won’t appear in the OCR as it is. for an example if I scan the pi symbol, the OCR do not display the symbols as it is but it shows the corresponding letter of the keyboard which I used in the font.

    what would cause this problem ?

    thank you !

  16. gour

    September 5, 2013

    Post a Reply

    I have imported Simple Android OCR application (complete project) , all work fine when tested on my phone but the output text doesn’t fit the text in the photo at all… why is this happening?

  17. Jash Bhankharia

    September 10, 2013

    Post a Reply

    I tried your basic as well as intermediate apps you have shared. The basic one doesn’t give accurate results but the intermediate one was giving a better output. Camera focus is having some problems some times in your intermediate app.

    But my question is that i am trying to build an Image to Text Converter using a Cloud(Server). Would you help me with it as i have no idea about cloud computing and the 2nd problem i am facing is that there are going to be diagrams too in the document the app will be converting. How to get the diagram out of it.

  18. Leeon

    September 13, 2013

    Post a Reply

    Hi gautam,
    This is really a nice tutorial for beginners. Can you tell me what can i do for multiple words or text-lines. When i use huge amount words of image, the program-me crashes immediately.

    Thanks.

  19. Abanoub

    October 16, 2013

    Post a Reply

    Hi, how I can detect orientation of text to auto adjust image before OCR I wrote the next line baseApi.setPageSegMode(TessBaseAPI.PageSegMode.PSM_AUTO_OSD); and put the osd.trainddata file in the tessdata folder but it didn’t work. Please respond as soon as you can?

  20. Kroan

    October 22, 2013

    Post a Reply

    Hi there,

    if I try to do TessBaseApi api = new TessBaseApi(); my application crashes.

    Does anyone know why?

    Thank you in advance.

    Greetings

    • Chris Chang

      November 22, 2013

      You can clean project to rebuild it, that will prevent exception like “couldn’t load lept from loader dalvik”.

  21. Brandon

    November 25, 2013

    Post a Reply

    I seem to have a problem running my application. In particular, loading the libraries. I get the following error:
    E/AndroidRuntime(11472): Caused by: java.lang.UnsatisfiedLinkError: Couldn’t load lept: findLibrary returned null

    I have tried cleaning the project and had no results.

    Any ideas?

  22. Nazu

    January 2, 2014

    Post a Reply

    Hey Gautam. Thank you so much for this tutorial. I’m a newbie in android and this tutorial is really what I’m looking for. I have a challenge with building the tess-two project. I can’t seem to get the commands to work. I keep getting the error ‘no target set’ on cmd. I’m not sure whether I should write them one at a time or all of them together and whether to use cygwin or cmd on Windows vista. Please anyone help point me in the right direction.

    • vipul

      March 7, 2014

      I think, You have to give Android target Version no. ( i.e. Android Max version).

  23. Imran

    January 12, 2014

    Post a Reply

    Hey Gautam. I tried to install but ran into difficulties. I’m using Windows and would appreciate some help:

    -Downloaded and extracted NDK
    -Downloaded tess-two as a ZIP file from Github, extracted it.
    -Using CMD I navigated to the tess-two folder, and ran ndk-build from inside the folder. It executed the script, which took about 15 minutes.

    This is now where I’m at. Tried to follow the next step, which is “android update project –path” CMD tells me “android” is not a recognized command. I thought it might be Windows, so I got cygwin and tried the same thing – it tells me “-bash: android: command not found”

    Next I tried to use Eclipse to build tess-two. I set NDK path as the NDK builder. Then I tried to import the tess-two project, but immediately I got the following errors: “Project has no project.properties file! Edit the project properties to set one.”

    I decided to plough on. I tried to set a new Builder for tess-two using NDK (I set the location as the path for ndk-build and set the working directory as $(workspace)/tess-two.

    However upon trying to build tess-two, it gave me the following build error:
    “Unable to resolve target ‘android-8′”. It then refuses to build.

    Where am I going wrong?

    • rmtheis

      January 13, 2014

      Your “android” command was not found because you need to add the Android SDK to your PATH.

  24. Anny

    January 13, 2014

    Post a Reply

    Hi Gautam. I followed all the steps successfully. Everything works fine but I am not getting the exact text as the image contains. Please help me in this issue. Every time it returns something new or unique character. Thanks in advance.

  25. Winnie Lam

    January 17, 2014

    Post a Reply

    Hey Gautam, I cannot translate TRADITIONAL_CHINESE, zh_TW, to SIMPLIFIED_CHINESE, zh_CN. The TRADITIONAL_CHINESE can be scanned and show in screen but the output is remain in TRADITIONAL_CHINESE. The translation problem only occurs in zh_TW to zh_CN.

  26. Radiant

    January 31, 2014

    Post a Reply

    Guys I need help, I don’t know how to build it, there was no ndk-build.cmd inside the tess-two folder

    • MG

      February 8, 2014

      After installing the NDK, you can go in your tess folder until ../tess-two as specified by Gautam.

      With command line put: “ndk-build Path-of-NDK-location” while working for windows command line. After building, go with command line in your SDK folder. (You will see some androids tools.). Type: “android update project -t 1 -p path-of-Tess-project”

      Finally there is the step with ant release but for now it failed for me.

    • MG

      February 8, 2014

      Hi back…correction being in your folder of tess 2 : put : /ndk-build, same thing for ant release : /ant release. Hope it will help. It is all right on Wndows command line in this way.

  27. Edwin

    February 15, 2014

    Post a Reply

    I’m stuck at the step “ant release”
    -BASH: ant: command not found

    • AJ

      February 25, 2014

      Downlaod ant, extract the zip and set system variables to ant/bin folder. Close your command prompt and open it again, type ant -version

  28. Michael D. Nguyen

    February 18, 2014

    Post a Reply

    How to determine if the character belongs to the rectangle box bound?

  29. Karan

    February 19, 2014

    Post a Reply

    Nice blog. Any ideas for getting better results using Tesseract? Please suggest any ideas on pre-processing the image before providing it to Tesseract for recognition..

    • AJ

      March 7, 2014

      Hi, I was away for a bit. I manged to solve my problem. Simply used eclipse and set it as library then it produced a jar file with accompanying classes

  30. AJ

    February 25, 2014

    Post a Reply

    Hello, I am working on a Sencha Touch 2 + Phonegap 3 app and intend to use tess-two as an ocr plugin in phonegap. I have successfully build tess-two. However, the folder is about 400MB++ (way too big), which folder specifically should I copy for my project. Should it be the bin or libs or jni? Any feedback will be greatly appreciated

  31. Trian

    March 3, 2014

    Post a Reply

    Guys, I need help. I can finish step 1 to 3. But I’ve problem that I can’t understand it in step 4 and 5.
    the script in step 4 and 5, where it should be placed? Can anyone explain it, or give the whole script because I’ve stuck in that part.
    Thank you in advance.
    Best Regard.

  32. Leo

    March 3, 2014

    Post a Reply

    I don’t know why but I included the tess-two library and somehow tried to build it but failed (it seems to be built though). And then first I ran it on an emulator, it says SD card was corrupted/unavailable. After I solved the SD card problem, it just popped out a black screen and said OCR Test has stopped. Please help me

  33. sudhir

    March 7, 2014

    Post a Reply

    I have done all your steps but couldn’t get the right result can you please tell me what is the problem.

  34. Nitish

    March 13, 2014

    Post a Reply

    The app works fine with English traineddata(eng.traineddata). But i have to use it for French. i put the fra.traineddata file in assets/tessdata folder but it wont work. the error it shows:

    Installation error: INSTALL_FAILED_INSUFFICIENT_STORAGE
    though there is sufficient storage on my phone.

    Please help.

  35. RA

    March 14, 2014

    Post a Reply

    Hi Gautam, i moved on all steps and it build success fully but i get these error pls help me

    ActivityManager: Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.example.rexdroid/.RexDroidOCRActivity }

  36. RA

    March 19, 2014

    Post a Reply

    This is the error i getting pls help me to solve it……

    [2014-03-19 11:44:17 - Simple-Android-OCR] Installing Simple-Android-OCR.apk…
    [2014-03-19 11:44:37 - Simple-Android-OCR] Success!
    [2014-03-19 11:44:37 - tess-two] Could not find tess-two.apk!
    [2014-03-19 11:44:37 - Simple-Android-OCR] Starting activity com.datumdroid.android.ocr.simple.SimpleAndroidOCRActivity on device emulator-5554
    [2014-03-19 11:44:38 - Simple-Android-OCR] ActivityManager: Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] cmp=com.datumdroid.android.ocr.simple/.SimpleAndroidOCRActivity }

  37. Amol

    March 21, 2014

    Post a Reply

    After installing all s/w I am facing problem in step 2.
    ‘ndk-build’ is not recognize as an internal or external command, operable program or batch file.

  38. Akshay

    March 24, 2014

    Post a Reply

    I have successfully completed all the steps as mentioned above.but,now i am getting error given below:
    tess-two] Failed to create BuildConfig class
    Simple-Android-OCR] Failed to create BuildConfig class
    tess-two] Failed to create BuildConfig class
    Simple-Android-OCR] Failed to create BuildConfig class
    Please Help me.

  39. Pramod

    April 15, 2014

    Post a Reply

    Hello Gautam,

    Great job.

    I have a question for you. Do you think this OCR reader could be used in Google Glass? I want to implement an app using OCR reader for Google glass. Any thoughts?

    Thanks,
    Pramod

Trackbacks/Pingbacks

  1. Tesseract OCR on Android is easier if you Maven-ise it, works on Windows too… | James Elsey - [...] to the lack of general resources available on the subject, I was able to find some good material by …
  2. Using Tesseract Tools for Android to Create a Basic OCR App « Software Projects - [...] an up-to-date tutorial available here, and I’ve collected the code in an Android project that should be easier to …
  3. Tesseracting Android | Giving Eyes To A Machine - [...] was to reconstruct a very simple OCR app. With the help of Gautam’s article from ‘http://gaut.am/making-an-ocr-android-app-using-tesseract/‘ I was able …

Submit a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>