|
|
Stat 180/236
Introduction to Bayesian Statistics
Bayesian statistics enjoys today considerable popularity, as novel
powerful computing tools have eased the application of this elegant
theory. Indeed, almost any statistician finds himself doing some
"Bayesian statistics" occasionally, making it a necessity for anybody
seriously interested in the field to acquire first hand knowledge of this
approach. The goal of this course is precisely to introduce statistical
inference as it can be carried out using the Bayes theorem. We will
consider foundational aspects of statistics as well as computational
issues encountered in applications in order to provide a sound and
complete picture of Bayesian inference.
As the number of courses offered at UCLA and covering the Bayesian approach to statistics is increasing, to avoid overlap with others we will pay particular attention to the understanding of the basis of Bayesian inference.
The emphasis will be on clear
understanding of the concept presented and of their implication for the
practice of statistics. Hence, the examples will be chosen so to keep the
mathematics simple and to illustrate problems of actual relevance (image
reconstruction, protein alignment, etc. ) We will discuss Stein's paradox,
non parametric Bayes and statistical learning.
This is an upper-level undergraduate course: on the one hand, a
previous knowledge of probability and calculus will be assumed; on the
other hand, there will not be space for things like a high-powered
treatment of decision theory or MCMC techniques and too challenging
computations. The course can be taken for graduate credit by
complementing the requirements with a data analysis project.
Following is a tentative syllabus. As the course progress, we will
precise the topics covered to date, grouping them by week of instruction.
- The goals of Bayesian statistics. Subjective interpretation of
probability. Bayes theorem.
Chapter 1 of Gelman et al. or Chapter 1 of Lee.
- Review of random variables and their distributions (ex. binomial
model and introduction of the Beta distribution). Introduction
of the Gibbs sampler algorithm.
Section 2 of the reader: sampling from joint and conditional
distributions
If you need a more extensive review of random variables, the first four chapters of Rice 'Mathematical Statistics and Data Analysis' are great.
Chapter 9 of Lee has a lot of material on Gibbs Sampler. Chapter 11 in Gelman et al. also deals with simulating random variables with a given distribution.
- The concepts and methods of Bayesian statistics as
they appear in Bayes's writings and their generalization with a
decision theory framework. Choice of prior distributions and role of predictive distributions.
Section 1 of the reader:
'The first examples of Bayesian statistics'
Section 3.1 of Lee deals with the Binomial model. Section 2.1 of Gelman et al. aslo deals with this model.
- Prior information on parameters (states of nature). Likelihood of the parameter based on data.
Posterior distribution of the parameter. Loss and risk
functions. Bayesian risk.
Estimation, testing, classification and different loss functions.
Section 3 of the reader:
'Decision theory' Section 7.5 of Lee has some material on decision theory. See also section 7.7. Section 4.1 of Lee has material on hypothesis testing.
- Bayesian calculus. Working out the examples of
Normal-normal, gamma-normal and beta-binomial, etc.
Chapter 3 and 4 of Lee. Note: we do not cover in class all the material and examples here; in particular we do not emphatize reference priors as much and we do not talk about Jeffrey's rule. Nevertheless, you should be able to read and understand most of what is here. Reading these chapters is a good way of preparing for the midterm.
Chapter 2 and 3 of Gelman et al. Same comments apply.
- Choice of the prior distribution and of the statistical model. Exchangeability. Finite exchangeability and representation theorems.
section 4 of the reader
- Hierarchical models. Empirical Bayes. Stein's paradox. Examples from gene expression array models
section 5 of the reader. Chapter 5 of of Gelman et al. Chapter 8 of of Lee.
- Model selection. Notions of smoothness, roughness penalty.
section 6 of the reader; Chapter 4 of of Lee. Section 6.7 of Gelman et al.
- Bayesian image reconstruction. An example of maximum a-posteriori estimate and the use of the Gibbs sampler.
section 8 of the reader
NOTE there will be no class during the following days:
October 6, and November 17 . Instead, we will meet on the
thursday of the respective weeks at 1pm in Boelter 5514.
......
|
......
|