NOTE: This is also on YouTube. I am charging a very low price here for practice with lbry.io. A presentation in which I discuss the cause of overfitting in statistics, which is the use of optimisation methods such as maximum likelihood for estimating parameters, when ideally we should use the posterior distribution from Bayesian inference. I give an example of fitting a dataset with a complex model, and show how the maximum likelihood estimate is very atypical of the posterior distribution. There’s one thing I wasn’t completely clear about towards the end of the talk, in the bit with the red and green bars where I discuss trans-dimensional models. The green parts are meant to represent the regions of parameter space that fit the data. The regions that overfit the data will be a tiny subset of the green bars, even in the complex model on the right hand side of the slides. Even if you conditioned on the model all the way on the right, you wouldn’t get overfitting unless you optimised within that model. Links to the things I referred to about foundations of probability: http://aapt.scitation.org/doi/pdf/10.1119/1.1990764 https://www.amazon.com/Probability-Theory-Science-T-Jaynes/dp/0521592712/ref=sr_1_1?ie=UTF8&qid=1509311627&sr=8-1&keywords=probability+theory+logic+of+science http://www.mdpi.com/2075-1680/1/1/38

NOTE: This is available free online elsewhere, but I am charging a very small fee here to get practice with lbry. :-) The coursebook for STATS 331 (Introduction to Bayesian Statistics) at the University of Auckland. (c) Brendon J. Brewer LICENSE: CC-BY-SA.

Starting to look at more features of Bayesian inference, including multiple hypotheses, dependence of the posterior on the prior probabilities, etc.

The first lecture of STATS 331 at the University of Auckland. In this lecture I outline the structure of the course and try to provide some motivation for why this subject is worth studying. The coursebook PDF is also available on lbry.