Multivariate normal distribution

A random vector X=(X₁,...,X_n) follows a multivariate normal distribution, also sometimes called a multivariate Gaussian distribution (in honor of Carl Friedrich Gauss, who was not the first to write about the normal distribution), if it satisfies the following equivalent conditions:

• every linear combination Y=a₁X₁ + ... + a_nX_n is normally distributed;

• there is a random vector Z=(Z₁, ..., Z_m), whose components are independent standard normal random variables, a vector μ = (μ₁, ..., μ_n) and an n-by-m matrix A such that X = A Z + μ.

• there is a vector μ and a symmetric, positive semi-definite matrix Γ such that the characteristic function of X is

φ_X(u)=exp(iμ^Tu − (½) u^T Γ u).

The following is not quite equivalent to the conditions above, since it fails to allow for a singular matrix as the variance:

• there is a vector μ=(μ₁, ..., μ_n) and a symmetric, positive semidefinite matrix Γ such that X has density

f_X(x₁, ..., x_n) dx₁ ... dx_n = (det(2πΓ))^−1/2 exp ½((x − μ)^TΓ⁻¹(x − μ)) dx₁...dx_n.

The vector μ in these conditions is the expected value of X and the matrix Γ=A^TA is the covariance matrix of the components X_i. It is important to realize that the covariance matrix must be allowed to be singular. That case arises frequently in statistics; for example, in the distribution of the vector of residuals in ordinary linear regression problems. Note also that the X_i are in general not independent; they can be seen as the result of applying the linear transformation A to a collection of independent Gaussian variables Z.

Proof?

Multivariate Gaussian density

Recall characteristic function of a random vector.

Recall characterizations of gaussian random variables.

Calculate characteristic function of Z in terms of characteristic function of X.

Deduce characteristic functional of X in terms of mean vector and covariance matrix.