opencv - Easy ways to detect and crop blocks (paragraphs) of text out of image?

Question

I have done some research on the subject, but I think my question is significantly different from what has been asked before.

My PhD thesis deals with OCR-ing an old dictionary and converting the result into an XML-like database automatically. This part I have figured out. However, I'd like to enrich the final result by displaying a fragment of scan used for each entry/headword. As the dictionary is almost 9000 pages long, doing it manually is out of the question.

This is how a random page looks: http://i.imgur.com/X2mPZr0.png

As each entry always equals one paragraph, I would like to find a way to split every image into rectangles with text (no OCR needed) as separate files, like this (without drawing the rectangles): http://i.imgur.com/CWtQD6Q.png

The good thing is that the scans I have are identical in shape and size, and similar in terms of margins/text alignment. Every paragraph always has an identation, too.

The bad thing is that I am mostly a linguist and not much of a programmer. Most of my experience is with Ruby, XML and CSS. And that some paragraphs are only one-line long.

I am aware of some ways do to a similar thing:

• Algorithm to detect presence of text on image
• http://www.danvk.org/2015/01/07/finding-blocks-of-text-in-an-image-using-python-opencv-and-numpy.html
• http://answers.opencv.org/question/27411/use-opencv-to-detect-text-blocks-send-to-tesseract-ios/
• https://github.com/kanaadp/iReader

but they are require significant amount of time for me to learn (especially that I have 0 knowledge in Python) and I don't know if they allow not only for text detection, but also paragraph detection.

Any input/suggestion on the matter would be greatly appreciated, especially newbie-friendly.

Answer 1

I have a few ideas to share... I think I would proceed along these lines:

LOW-RESOLUTION COPY OF ORIGINAL IMAGE JUST FOR REFERENCE

Step 1 - Threshold to Black and White

I think I would use OpenCV's Otsu thresholding for this.

Step 2 - Find vertical black line

I would average the pixels in every column of the image and find the one with the lowest average and that should be the vertical line up the middle. Code below outputs:

Centreline at column: 1635

Step 3 - Split image in two and trim excess white space

Step 4 - Box filter

I would box filter with a 55x45 box that matches the indent at the start of each paragraph then threshold so all paragraph starts are marked with black boxes.

I am pretty new to OpenCV but have coded the above ideas as follows - I m sure lots of it could be made more robust and more efficient so treat it as conceptual ;-)

#include <iostream>
#include <opencv2/opencv.hpp>

using namespace cv;
using namespace std;

int
main(int argc,char*argv[])
{
   // Load image
   Mat orig=imread("page.png",IMREAD_COLOR);

   vector<int> PNGwriteOptions;
   PNGwriteOptions.push_back(CV_IMWRITE_PNG_COMPRESSION);
   PNGwriteOptions.push_back(9);

   // Get greyscale and Otsu-thresholded version
   Mat bw,grey;
   cvtColor(orig,grey,CV_RGB2GRAY);
   threshold(grey,bw,0,255,CV_THRESH_BINARY|CV_THRESH_OTSU);

   // Find vertical centreline by looking for lowest column average - i.e. darkest vertical bar
   Mat colsums;
   reduce(bw,colsums,0,CV_REDUCE_AVG);
   double min,max;
   Point min_loc, max_loc;
   minMaxLoc(colsums,&min,&max,&min_loc,&max_loc);
   cout << "Centreline at column: " << min_loc.x << endl;

   namedWindow("test",CV_WINDOW_AUTOSIZE);

   // Split image into left and right
   Rect leftROI(0,0,min_loc.x,bw.rows);
   Mat  leftbw=bw(leftROI);
   Rect rightROI(min_loc.x+8,0,bw.cols-min_loc.x-8,bw.rows);
   Mat  rightbw=bw(rightROI);
   imshow("test",leftbw);
   waitKey(0); 
   imshow("test",rightbw);
   waitKey(0); 

   // Trim surrounding whitespace off
   Mat Points;
   Mat inverted =  cv::Scalar::all(255) - leftbw;
   findNonZero(inverted,Points);
   Rect bRect=boundingRect(Points);
   Mat lefttrimmed=leftbw(bRect);

   inverted =  cv::Scalar::all(255) - rightbw;
   findNonZero(inverted,Points);
   bRect=boundingRect(Points);
   Mat righttrimmed=rightbw(bRect);

   imwrite("lefttrimmed.png",lefttrimmed,PNGwriteOptions);
   imwrite("righttrimmed.png",righttrimmed,PNGwriteOptions);

   // Box filter with 55x45 rectangle to match size of paragraph indent on left
   Mat lBoxFilt,rBoxFilt;
   boxFilter(lefttrimmed,lBoxFilt,-1,Size(55,45));
   normalize(lBoxFilt,lBoxFilt,0,255,NORM_MINMAX,CV_8UC1);
   threshold(lBoxFilt,lBoxFilt,254,255,THRESH_BINARY_INV);
   imwrite("leftBoxed.png",lBoxFilt,PNGwriteOptions);

}

Just in case you need a hand to build this code - as it seems non-trivial to compile and link anything against it - I made my CMakeLists.txt file like this and stored it in the same directory as the source file. Then I create a sub-directory called build to do an "out-of-source" build in and the build process is:

cd build
cmake ..
make -j 8
./demo

CMakeLists.txt

cmake_minimum_required(VERSION 2.8)
project(demo)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
find_package(OpenCV)
add_executable(demo main.cpp)
target_link_libraries(demo ${OpenCV_LIBS})

Keywords: Image processing, book, margin, spine, centreline, page, crease, fold, gutter, binding, stitching, text, paragraph, detect, detection.