Given two column vectors X = (X1, ..., Xn)′ and Y = (Y1, ..., Ym)′ of random variables with finite second moments, one may define the cross-covariance cov(X, Y) to be the n×m matrix whose ij entry is the covariance cov(Xi, Yj). (Sometimes this is called simply the covariance between X and Y. But sometimes one speaks of the "covariance" of X, intending the n×n matrix of covariances between the pairs of scalar components of X. Sometimes the latter matrix is called the variance of X.)
Canonical correlation analysis seeks vectors a and b such that the real random variables a′ X and b′ Y (where the row-vector a′ is the transpose of the column-vector a) maximize the correlation ρ(a′ X, b′ Y ). The random vectors U = a′ X and V = b′ Y are the first pair of canonical variables. Then one seeks vectors maximizing the same correlation subject to the constraint that they are to be uncorrelated with the first pair of canonical variables; this gives the second pair of canonical variables, etc.