NOTE: This is also on YouTube. I am charging a very low price here for practice with lbry.io. A presentation in which I discuss the cause of overfitting in statistics, which is the use of optimisation methods such as maximum likelihood for estimating parameters, when ideally we should use the posterior distribution from Bayesian inference. I give an example of fitting a dataset with a complex model, and show how the maximum likelihood estimate is very atypical of the posterior distribution. There’s one thing I wasn’t completely clear about towards the end of the talk, in the bit with the red and green bars where I discuss trans-dimensional models. The green parts are meant to represent the regions of parameter space that fit the data. The regions that overfit the data will be a tiny subset of the green bars, even in the complex model on the right hand side of the slides. Even if you conditioned on the model all the way on the right, you wouldn’t get overfitting unless you optimised within that model. Links to the things I referred to about foundations of probability: http://aapt.scitation.org/doi/pdf/10.1119/1.1990764 https://www.amazon.com/Probability-Theory-Science-T-Jaynes/dp/0521592712/ref=sr_1_1?ie=UTF8&qid=1509311627&sr=8-1&keywords=probability+theory+logic+of+science http://www.mdpi.com/2075-1680/1/1/38

Starting to look at more features of Bayesian inference, including multiple hypotheses, dependence of the posterior on the prior probabilities, etc.

More on parameter estimation... I again forgot to switch the recording back to the laptop screen at one point. The R code I entered is here: https://www.stat.auckland.ac.nz/~brewer/volcanoes.R

In this lecture I briefly review basic concepts in probability, maths, and R programming, that are used throughout the course.