surf - OpenCV 2.4.3 - warpPerspective with reversed homography on a cropped image

Question

When finding a reference image in a scene using SURF, I would like to crop the found object in the scene, and "straighten" it back using warpPerspective and the reversed homography matrix.

Meaning, let's say I have this SURF result:


Now, I would like to crop the found object in the scene:


and "straighten" only the cropped image with warpPerspective using the reversed homography matrix. The result I'm aiming at is that I'll get an image containing, roughly, only the object, and some distorted leftovers from the original scene (as the cropping is not a 100% the object alone).

Cropping the found object, and finding the homography matrix and reversing it are simple enough. Problem is, I can't seem to understand the results from warpPerspective. Seems like the resulting image contains only a small portion of the cropped image, and in a very large size.
While researching warpPerspective I found that the resulting image is very large due to the nature of the process, but I can't seem to wrap my head around how to do this properly. Seems like I just don't understand the process well enough. Would I need to warpPerspective the original (not cropped) image and than crop the "straightened" object?

Any advice?

Answer 1

try this.

given that you have the unconnected contour of your object (e.g. the outer corner points of the box contour) you can transform them with your inverse homography and adjust that homography to place the result of that transformation to the top left region of the image.

1. compute where those object points will be warped to (use the inverse homography and the contour points as input):

   cv::Rect computeWarpedContourRegion(const std::vector<cv::Point> & points, const cv::Mat & homography)
   {
       std::vector<cv::Point2f> transformed_points(points.size());
   
       for(unsigned int i=0; i<points.size(); ++i)
       {
           // warp the points
           transformed_points[i].x = points[i].x * homography.at<double>(0,0) + points[i].y * homography.at<double>(0,1) + homography.at<double>(0,2) ;
           transformed_points[i].y = points[i].x * homography.at<double>(1,0) + points[i].y * homography.at<double>(1,1) + homography.at<double>(1,2) ;
       }
   
       // dehomogenization necessary?
       if(homography.rows == 3)
       {
           float homog_comp;
           for(unsigned int i=0; i<transformed_points.size(); ++i)
           {
               homog_comp = points[i].x * homography.at<double>(2,0) + points[i].y * homography.at<double>(2,1) + homography.at<double>(2,2) ;
               transformed_points[i].x /= homog_comp;
               transformed_points[i].y /= homog_comp;
           }
       }
   
       // now find the bounding box for these points:
       cv::Rect boundingBox = cv::boundingRect(transformed_points);
       return boundingBox;
   }
   

2. modify your inverse homography (result of computeWarpedContourRegion and inverseHomography as input)

   cv::Mat adjustHomography(const cv::Rect & transformedRegion, const cv::Mat & homography)
   {
       if(homography.rows == 2) throw("homography adjustement for affine matrix not implemented yet");
   
       // unit matrix
       cv::Mat correctionHomography = cv::Mat::eye(3,3,CV_64F);
       // correction translation
       correctionHomography.at<double>(0,2) = -transformedRegion.x;
       correctionHomography.at<double>(1,2) = -transformedRegion.y;
   
   
       return correctionHomography * homography;
   }
   

3. you will call something like

cv::warpPerspective(objectWithBackground, output, adjustedInverseHomography, sizeOfComputeWarpedContourRegionResult);

hope this helps =)