python - how do i get coordinates of image shown in opencv

Question

Sorry but title doesnt really make sense

i am trying to make an ai that clicks on the ball to make it bounce. for context heres a picture of the application

in the game when you click the ball it goes up and then comes back down and the aim of the game is to keep it up.

i have writen some code that turns the image into a mask with opencv, heres a picture of the result

what i now need to do is find the location of the ball in pixels/coordinates so i can make the mouse move to it and click it. By the way the ball has a margin on the left and right of it so it doesn't just go strait up and down but left and right too. Also the ball isnt animated,just a moving image.

How would i get the ball location in pixels/coordinates so i can move the mouse to it.

heres a copy of my code:

import numpy as np
from PIL import ImageGrab
import cv2
import time
import pyautogui


def draw_lines(img,lines):
    for line in lines:
        coords = line[0]
        cv2.line(img, (coords[0], coords[1]), (coords[2], coords[3]), [255,255,255], 3)

def process_img(original_image):
    processed_img = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
    processed_img = cv2.Canny(processed_img, threshold1=200, threshold2=300)
    vertices = np.array([[0,0],[0,800],[850,800],[850,0]
                         ], np.int32)
    processed_img = roi(processed_img, [vertices])

    # more info: http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html
    #                          edges       rho   theta   thresh         # min length, max gap:        
    lines = cv2.HoughLinesP(processed_img, 1, np.pi/180, 180,      20,         15)
    draw_lines(processed_img,lines)
    return processed_img

def roi(img, vertices):
    #blank mask:
    mask = np.zeros_like(img)
    # fill the mask
    cv2.fillPoly(mask, vertices, 255)
    # now only show the area that is the mask
    masked = cv2.bitwise_and(img, mask)
    return masked
def main():
    last_time = time.time()
    while(True):
        screen =  np.array(ImageGrab.grab(bbox=(0,40, 800, 850)))
        new_screen = process_img(screen)
        print('Loop took {} seconds'.format(time.time()-last_time))
        last_time = time.time()
        cv2.imshow('window', new_screen)
        #cv2.imshow('window2', cv2.cvtColor(screen, cv2.COLOR_BGR2RGB))
        if cv2.waitKey(25) & 0xFF == ord('q'):
            cv2.destroyAllWindows()
            break

def mouse_movement():
    ##Set to move relative to where ball is
    pyautogui.moveTo(300,400)
    pyautogui.click();
main()

Sorry if this is confusing but brain.exe has stopped working :( Thanks

Answer 1

You can do it like this:

1. crop an image of the ball from a screenshot or so, sth. like

img = cv2.imread("screenshot.jpg")
crop_img = img[y:y+h, x:x+w] # you will have to look for the parameters by trial and error

2. use template matching to look where the ball is in your image

3. get the point in the middle of the resulting rectangle and move your mouse there

I hope this helps, if you need more help on how to achieve any of this feel free to ask