The kernel trick lets a linear method draw curved boundaries without ever doing the expensive part. The idea: many algorithms, including the support vector machine, touch the data only through dot products between pairs of examples. If you replace every dot product with a kernel function k(x, x′) that equals a dot product in some richer feature space, the algorithm now works in that space, but you never compute the feature vectors themselves.
A concrete case: a polynomial kernel of degree p computes a dot product over all degree-p products of the input coordinates. Writing those products out explicitly would mean a vector that grows combinatorially with the input size. The kernel gets the same number from one dot product and a power. Some kernels go further and correspond to infinite-dimensional feature spaces, which could never be built explicitly at all; the widely used Gaussian (RBF) kernel is one.
Why it mattered
Linear methods are well understood: convex to train, with clean theory. The trick keeps all of that while letting the model fit nonlinear patterns, because the model is linear in the feature space and nonlinear in the original inputs. It is what lets support vector machines (the paper) separate classes that no straight line can, and it carries over to other methods that rely on dot products, such as kernel PCA and Gaussian processes.
The catch
Not every function is a valid kernel; it has to be positive definite, which guarantees a matching feature space exists. And kernel methods compare examples in pairs, so training builds an n × n matrix. That matrix grows with the square of the dataset, which makes kernel methods expensive on very large datasets. Neural networks take the opposite approach, learning their features rather than fixing them in advance through the choice of kernel.