Use two inputs (X and Y) for scikit-learn gaussian kernel in python

Question

I am trying to build a Gaussian kernel for kernel ridge regression using sklearn in python.
For instance, this is the example if I am using a RBF kernel.

from sklearn.datasets import load_iris
from sklearn.gaussian_process import GaussianProcessClassifier
from sklearn.gaussian_process.kernels import RBF
X, y = load_iris(return_X_y=True)
kernel = 1.0 * RBF(1.0)
gpc = GaussianProcessClassifier(kernel=kernel, random_state=0).fit(X, y)
gpc.predict(X)

However, in the __call__ function of the kernel, it seems can take not just X as input, so as an additional input Y (default is None): __call__(X, Y=None, eval_gradient=False). All the examples I found are only taking X as argument for conducting kernel. Is there an example of taking all two arguments X and Y for building kernel, and how the fit and predict functions would look like in that case?

This is the kernel class I am trying to build

class KrrInteractKernel2(GenericKernelMixin, Kernel):
    def __init__(self, z=None):
        self.z = z # this does nothing, _init_ function is required

    def __call__(self, X, Y=None, eval_gradient=False):
        """Return the kernel k(X, Y) and optionally its gradient.
        Parameters
        ----------
        X : array-like of shape (n_samples_X, n_features) or list of object
            Left argument of the returned kernel k(X, Y)
        Y : array-like of shape (n_samples_X, n_features) or list of object, 
            default=None
            Right argument of the returned kernel k(X, Y). If None, k(X, X)
            is evaluated instead.
        eval_gradient : bool, default=False
            Determines whether the gradient with respect to the log of
            the kernel hyperparameter is computed.
            Only supported when Y is None.
        Returns
        -------
        K : ndarray of shape (n_samples_X, n_samples_Y)
            Kernel k(X, Y)
        K_gradient : ndarray of shape (n_samples_X, n_samples_X, n_dims), 
            optional
            The gradient of the kernel k(X, X) with respect to the log of the
            hyperparameter of the kernel. Only returned when eval_gradient
            is True.
        """
        if Y is None:
            Y = X
        elif eval_gradient:
            raise ValueError("Gradient can only be evaluated when Y is None.")

        for i in np.unique(Y):
            x_ = X[np.where(Y == i)]
            n_ = x_.shape[0]
            m_ = x_ @ x_.T / n_
            if i==1:
                K = m_
            else:
                K = block_diag(K, m_)

        if eval_gradient:
            if not self.hyperparameter_constant_value.fixed:
                return (K, np.full((_num_samples(X), _num_samples(X), 1),
                                   self.constant_value,
                                   dtype=np.array(self.constant_value).dtype))
            else:
                return K, np.empty((_num_samples(X), _num_samples(X), 0))
        else:
            return K

    def diag(self, X):
        return np.diag(X)

    def is_stationary(self):
        return False

    def clone_with_theta(self, theta):
        cloned = clone(self)
        cloned.theta = theta
        return cloned

I would like to use X and Y in the fit and predict functions. I was hoping something like this:

kr = KernelRidge(alpha=0.1, kernel=krrInteractKernel)
kr.fit([X,Y], yobs)
kr.predict([X,Y], yobs)

Thank you so much!

question from:https://stackoverflow.com/questions/65829609/use-two-inputs-x-and-y-for-scikit-learn-gaussian-kernel-in-python

Answer 1

Waitting for answers