Short answer: Solving exactly for whether the two objects intersect is complicated enough to be infeasible for the purpose of collision detection. Discretize your ellipse as an n-sided polygon for some n (depending on how accurate you need to be) and do collision detection with that polygon.
Long answer: If you insist on determining if the smooth ellipse and circle intersect, there are two main approaches. Both involve solving first for the closest point to the circle's center on the ellipse, and then comparing that distance to the circle's radius.
Approach 1: Use a parametrization of the ellipse. Transform your coordinates so that the ellipse is at the origin, with its axes aligned to the x-y axes. That is:
- Center of ellipse: (0,0)
- Center of circle: c = (cx, cy)
- Radius of circle: r
- Radius of x-aligned axis of ellipse: a
- Radius of y-aligned axis of ellipse: b.
The equation of the ellipse is then given by a cos(t), b sin(t). To find the closest point, we want to minimize the square distance
|| (a cos t, b sin t) - c ||^2. As Jean points out, this is "just calculus": take a derivative, and set it equal to 0. Unless I'm missing something, though, solving the resulting (quite nasty) equation for t is not possible analytically, and must be approximated using e.g. Newton's Method.
Plug in the t you find into the parametric equation to get the closest point.
- Pro: Numerical solve is only in one variable,
t.
- Con: You must be able to write down a parametrization of the ellipse, or transform your coordinates so that you can. This shouldn't be too hard for any reasonable representation you have of the ellipse. However, I'm going to show you a second method, which is much more general and might be useful if you have to generalize your problem to, say, 3D.
Approach 2: Use multidimensional calculus. No change of coordinates is necessary.
- Center of circle: c = (cx, cy)
- Radius of cirlce: r
- Ellipse is given by g(x, y) = 0 for a function g. For instance, per Curd's answer you might use g(x,y) = distance of (x,y) from focus 1 + distance of (x,y) from focus 2 - e.
Finding the point on the ellipse closest to the center of the circle can then be phrased as a constrained minimization problem:
Minimize ||(x,y) - c||^2 subject to g(x,y) = 0
(Minimizing the square distance is equivalent to minimizing the distance, and much more pleasant to deal with since it's a quadratic polynomial in x,y.)
To solve the constrained minimization problem, we introduce Lagrange multiplier lambda, and solve the system of equations
2 * [ (x,y) -c ] + lambda * Jg(x,y) = 0
g(x,y) = 0
Here Jg is the gradient of g. This is a system of three (nonlinear) equations in three unknowns: x, y, and lambda. We can solve this system using Newton's Method, and the (x,y) we get is the closest point to the circle's center.
- Pro: No parametrization needs to be found
- Pro: Method is very general, and works well whenever writing g is easier than finding a parametric equation (such as in 3D)
- Con: Requires a multivariable Newton solve, which is very hairy if you don't have access to a numerical method package.
Caveat: both of these approaches technically solve for the point which extremizes the distance to the circle's center. Thus the point found might be the furthest point from the circle, and not the closest. For both methods, seeding your solve with a good initial guess (the center of the circle works well for Method 2; you're on your own for Method 1) will reduce this danger.
Potential Third Approach?: It may be possible to directly solve for the roots of the system of two quadratic equations in two variables representing the circle and ellipse. If a real root exists, the objects intersect. The most direct way of solving this system, again using a numerical algorithm like Newton's Method, won't help because lack of convergence does not necessary imply nonexistence of a real root. For two quadratic equations in two variables, however, there may exist a specialized method that's guaranteed to find real roots, if they exist. I myself can't think of a way of doing this, but you may want to research it yourself (or see if someone on stackoverflow can elaborate.)
