java - find archery target in image of different perspectives

Question

I'm trying to find a way to identify an archery target and all of its rings on a photo which might be made of different perspectives:

My goal is to identify the target and later on also where the arrows hit the target to automatically count their score. Presumptions are as follows:

• The camera's position is not fixed and might change
• The archery target might also move or rotate slightly
• The target might be of different size and have different amount of circles
• There might be many holes (sometimes big scratches) in the target

I have already tried OpenCV to find contours, but even with preprocessing (grayscale -> blur (-> threshold) -> edge detection) I still find a few houndred contours which are all distracted by the arrows or other obstacles (holes) on the target, so it is impossible to find a nice circular line. Using Hough to find circles doesn't work either as it will give me weired results as Hough will only find perfect circles and not ellipses.

With preprocessing the image this is my best result so far:

I was thinking about ellipse and circle fitting, but as I don't know radius, position and pose of the target this might be a very cpu consuming task. Another thought was about using recognition from a template, but the position and rotation of the target changes often.

Now I have the idea to follow every line on the image to check if it is a curve and then guess which curves belong together to form a circle/ellipse (ellipse because of the perspective). The problem is that the lines might be intersected by arrows or holes in a short distance so the line would be too short to check if it is a curve. With the smaller circles on the target the chance is high that it isn't recognised at all. Also, as you can see, circle 8, 7 and 6 have no clear line on the left side.

I think it is not neccessary to do perspective correction to achieve this task as long as I can clearly identify all the rings in the target.

I googled a long time and found some thesis which are all not exactly focussed on this specific task and also too mathematical for me to understand.

Is it by any chance possible to achieve this task? Could you share with me an idea how to solve this problem? Anything is very appreciated.

I'm doing this in Java, but the programming language is secondary. Please let me know if you need more details.

Answer 1

for starters see

• Detecting circles and shots from paper target.

If you are using standardized target as on the image ( btw. I use these same too for my bow :) ) then do not cut off the color. You can select the regions of blue red and yellow pixels to ease up the detection. see:

• footprint fitting

From that you need to fit the circles. But as you got perspective then the objects are not circles nor ellipses. You got 2 options:

1. Perspective correction

   Use right bottom table rectangle area as marker (or the whole target). It is rectangle with known aspect ratio. so measure it on image and construct transformation that will change the image so it became rectangle again. There are tons of stuff about this: 3D scene reconstruction so google/read/implement. The basic are based just on De-skew + scaling.

2. Approximate circles by ellipses (not axis aligned!)

   so fit ellipses to found edges instead circles. This will not be as precise but still close enough. see:

   • ellipse fitting

[Edit1] sorry did not have time/mood for this for a while

As you were unable to adapt my approach yourself here it is:

1. remove noise

   you need to recolor your image to remove noise to ease up the rest... I convert it to HSV and detect your 4 colors (circles+paper) by simple tresholding and recolor the image to 4 colors (circles,paper,background) back into RGB space.

   

2. fill the gaps

   in some temp image I fill the gaps in circles created by arrows and stuff. It is simple just scan pixels from opposite sides of image (in each line/row) and stop if hit selected circle color (you need to go from outer circles to inner not to overwrite the previous ones...). Now just fill the space between these two points with your selected circle color. (I start with paper, then blue,red and yellow last):

   

3. now you can use the linked approach

   So find avg point of each color, that is approx circle center. Then do a histogram of radius-es and chose the biggest one. From here just cast lines out of the circle and find where the circle really stops and compute the ellipse semi-axises from it and also update the center (that handles the perspective distortions). To visually check I render cross and circle for each circle into the image from #1:

   

   As you can see it is pretty close. If you need even better match then cast more lines (not just 90 degree H,V lines) to obtain more points and compute ellipse algebraically or fit it by approximation (second link)

C++ code (for explanations look into first link):

picture pic0,pic1,pic2;
    // pic0 - source
    // pic1 - output
    // pic2 - temp
DWORD c0;
int x,y,i,j,n,m,r,*hist;
int x0,y0,rx,ry;    // ellipse
const int colors[4]=// color sequence from center
    {
    0x00FFFF00, // RGB yelow
    0x00FF0000, // RGB red
    0x000080FF, // RGB blue
    0x00FFFFFF, // RGB White
    };

// init output as source image and resize temp to same size
pic1=pic0;
pic2=pic0; pic2.clear(0);

// recolor image (in HSV space -> RGB) to avoid noise and select target pixels
pic1.rgb2hsv();
for (y=0;y<pic1.ys;y++)
 for (x=0;x<pic1.xs;x++)
    {
    color c;
    int h,s,v;
    c=pic1.p[y][x];
    h=c.db[picture::_h];
    s=c.db[picture::_s];
    v=c.db[picture::_v];
    if (v>100)  // bright enough pixels?
        {
        i=25; // treshold
             if (abs(h- 40)+abs(s-225)<i) c.dd=colors[0]; // RGB yelow
        else if (abs(h-250)+abs(s-165)<i) c.dd=colors[1]; // RGB red
        else if (abs(h-145)+abs(s-215)<i) c.dd=colors[2]; // RGB blue
        else if (abs(h-145)+abs(s- 10)<i) c.dd=colors[3]; // RGB white
        else                              c.dd=0x00000000; // RGB black means unselected pixels
        } else                            c.dd=0x00000000; // RGB black
    pic1.p[y][x]=c;
    }
pic1.save("out0.png");
// fit ellipses:
pic1.bmp->Canvas->Pen->Width=3;
pic1.bmp->Canvas->Pen->Color=0x0000FF00;
pic1.bmp->Canvas->Brush->Style=bsClear;
m=(pic1.xs+pic1.ys)*2;
hist=new int[m]; if (hist==NULL) return;
for (j=3;j>=0;j--)
    {
    // select color per pass
    c0=colors[j];
    // fill the gaps with H,V lines into temp pic2
    for (y=0;y<pic1.ys;y++)
        {
        for (x=        0;(x<pic1.xs)&&(pic1.p[y][x].dd!=c0);x++); x0=x;
        for (x=pic1.xs-1;(x>     x0)&&(pic1.p[y][x].dd!=c0);x--);
        for (;x0<x;x0++) pic2.p[y][x0].dd=c0;
        }
    for (x=0;x<pic1.xs;x++)
        {
        for (y=        0;(y<pic1.ys)&&(pic1.p[y][x].dd!=c0);y++); y0=y;
        for (y=pic1.ys-1;(y>     y0)&&(pic1.p[y][x].dd!=c0);y--);
        for (;y0<y;y0++) pic2.p[y0][x].dd=c0;
        }
    if (j==3) continue; // do not continue for border
    // avg point (possible center)
    x0=0; y0=0; n=0;
    for (y=0;y<pic2.ys;y++)
     for (x=0;x<pic2.xs;x++)
      if (pic2.p[y][x].dd==c0)
       { x0+=x; y0+=y; n++; }
    if (!n) continue;               // no points found
    x0/=n; y0/=n;                   // center
    // histogram of radius
    for (i=0;i<m;i++) hist[i]=0;
    n=0;
    for (y=0;y<pic2.ys;y++)
     for (x=0;x<pic2.xs;x++)
      if (pic2.p[y][x].dd==c0)
        {
        r=sqrt(((x-x0)*(x-x0))+((y-y0)*(y-y0))); n++;
        hist[r]++;
        }
    // select most occurent radius (biggest)
    for (r=0,i=0;i<m;i++)
     if (hist[r]<hist[i])
      r=i;
    // cast lines from possible center to find edges (and recompute rx,ry)
    for (x=x0-r,y=y0;(x>=     0)&&(pic2.p[y][x].dd==c0);x--); rx=x; // scan left
    for (x=x0+r,y=y0;(x<pic2.xs)&&(pic2.p[y][x].dd==c0);x++);       // scan right
    x0=(rx+x)>>1; rx=(x-rx)>>1;
    for (x=x0,y=y0-r;(y>=     0)&&(pic2.p[y][x].dd==c0);y--); ry=y; // scan up
    for (x=x0,y=y0+r;(y<pic2.ys)&&(pic2.p[y][x].dd==c0);y++);       // scan down
    y0=(ry+y)>>1; ry=(y-ry)>>1;

    i=10;
    pic1.bmp->Canvas->MoveTo(x0-i,y0);
    pic1.bmp->Canvas->LineTo(x0+i,y0);
    pic1.bmp->Canvas->MoveTo(x0,y0-i);
    pic1.bmp->Canvas->LineTo(x0,y0+i);
    //rx=r; ry=r;
    pic1.bmp->Canvas->Ellipse(x0-rx,y0-ry,x0+rx,y0+ry);
    }
pic2.save("out1.png");
pic1.save("out2.png");
pic1.bmp->Canvas->Pen->Width=1;
pic1.bmp->Canvas->Brush->Style=bsSolid;
delete[] hist;