# Karhunen-Loève expansions: A primer

Somehow I’ve managed to have nothing to do with Karhunen-Loève expansions of random processes at work in lab. A couple of weeks ago, though, I started working with stochastic collocation for partial differential equations and there they were, Karhunen-Loève expansions. That name, *Karhunen-Loève*, is long enough to promise good entertainment, and as probability theory remains in the list of topics that give me an insta-headache everytime I have to deal with them, it seemed to me, well, not a recipe for a good time. To my surprise, though, after a couple of days of hands-on I can say this is actually a lot of fun (Well, at least *as fun* as probability theory can be in your book). The topic allows going from a state of almost-no-idea-whatsoever to oh-now-I-kinda-get-what’s-going-on in a not painfully long time. Let’s summarize then some of the stuff I’ve learned about K-L expansions and in the meantime practice some .

Let’s say we have a second order centered random process , with (that is, with zero mean, or ). We’ll assume that is continuous in quadratic mean (or *q.m.*), so that we can employ the calculus in q.m. developed by Loève (1977) (for more information on that, and expositions on the topic with actual technical rigour, check that reference and Potthoff, 2010). This process can be represented as an infinite linear combination of orthonormal functions whose coefficients are zero-mean random variables , that is,

As they are orthonormal, the functions satisfy . The coefficients are then given by

as is customary with orthogonal expansions. K-L expansions require the random variables to be mutually uncorrelated, i.e., , with the variance of . What does this say about the random variables and the orthonormal functions? To investigate this, we’ll follow the procedure outlined in this article, starting with using the expression above for the to rewrite the covariance of the random variables:

where is the covariance of (which exists, as is second order). This last expression can be rewritten as

which can be easily satisfied by setting the stuff in the curly brackets equal to . If we let we obtain the following relation:

This result, a Friedholm equation, states the relationship satisfied by each of the and the variance of its corresponding random variable. It is not difficult to see that and are the set of eigenfunctions and eigenvalues of the linear operator

that is, .

The usual form of the K-L expansion is obtained by writing , where the are zero-mean, unit variance random variables. The final result is then

where , satisfy (1) and the random variables are given by

with . It can be proved that the series (2) converges uniformly in . For this, we’ll follow the procedure of Loève (1977) and start introducing Mercer’s theorem: A nonnegative-definite type function continuous in can be expanded as

where , are again the solutions of (1). This series converges absolutely and uniformly of . Covariance functions are positive-definite, so good times. Now, equation (3) implies ; also, we need to define

With these elements, it’s easy to see that

In virtue of Mercer’s theorem, the last term on the RHS converges to as and the LHS converges to zero uniformly on .

Well, that’s gonna be pretty much it for the moment. I just want to add that this entry was brought to you with the help of LaTeX2WP, this super-neat Python script that converts LaTeX file to discernible HTML ready to copy-paste to the WordPress editor. Awesome! Now go and do something fun.

**References**

Adler, R.J. (1990). *An Introduction to Continuity, Extrema and Related Topics for General Gaussian Processes*. Institute of Mathematical Statistics.

Johnson, D. (n.d.). *Karhunen-Loève Expansions*. In *Connexions*. Retrieved March 17, 2011 from http://cnx.org/content/m11259/latest/

Loève, M. (1978). *Probability theory.* Vol. II, 4th ed. New York: Springer-Verlag.

Potthoff, J. (2010). *Sample properties of random fields — III: Differentiability*. *Commun. Stoch. Anal.*, 4, 335-353.

And obviously, the Wikipedia article:

*Karhunen-Loève theorem*. (n.d.). In *Wikipedia*. Retrieved March 17, 2011 from http://en.wikipedia.org/wiki/Karhunen-Loève_theorem

Comments are closed.

Me quedo con mis small Pastor López. Sin embargo, karhunen love parade kicks asses! o rulea!

Hello,

The Karhunen-Loève expansion does NOT require the random variables to be uncorrelated – this is just a special case of the KL-expansion (Gaussian random fields). In general it is wrong to assume uncorrelated random variables.

..see:

Stochastic finite elements: a spectral approach

By Roger G. Ghanem, Pol D. Spanos

page: 23 – after equation 2.30

http://books.google.com/books?id=WzgKyTQQcAwC&printsec=frontcover&dq=spanos&hl=en&ei=4RXJTdPnDMr1-gaS96XSBg&sa=X&oi=book_result&ct=result&resnum=3&ved=0CDUQ6AEwAg#v=onepage&q=karhunen&f=false

Best Regards,

Wolfgang Betz

I was wrong, the random variables are uncorrelated (since they have zero mean) – but not necessarily independent.