Information Theory, Inference, and Learning Algorithms by David MacKay

[Books_] I just discovered this wonderful book by David MacKay. It has a very good interpretation of Occam's razor in Bayesian terms. Very nice examples of Bayesian inference and pitfalls of orthodox statistics. The book is at the intersection of communication, inference, and learning (although very little learning). I am still undecided about the pedagogic value of information theory for teaching inference - for the most part the information theory results can be stated in probability terms and vice versa. It inspired me to replan my machine learning course more on foundations and less on various algorithmic incarnations. The book is freely available on the net at the above address (which is another reason I appreciated it and bought a hardcopy at Amazon).
Related link