Instructor: Todd Kuffner (kuffner@math.wustl.edu)

Grader: Wei Wang (wwang@math.wustl.edu)

Lecture: 11:30-1:00pm, Tuesday and Thursday, Psychology 249

Office Hours: Monday 3:00-4:00pm, Tuesday/Thursday 1:05-2:00pm in Room 18, Cupples I

Course Overview: This course introduces Bayesian statistical theory and practice. The material will be presented at a level suitable for advanced undergraduate and master's degree students. Topics include: foundations and principles of Bayesian inference, comparisons with frequentist procedures, prior specification, selected computational methods (Markov Chain Monte Carlo), empirical Bayes, Bayesian linear regression and Bayesian model selection. Time permitting, additional topics may be selected by the instructor, such as approximate Bayesian computation or Bayesian nonparametric inference. Emphasis will be given to applications using R.

Prerequisite: It is assumed that students are already familiar with probability at the level of Math 493, and have learned the core concepts of statistical inference. The latter requirement is ideally satisfied by Math 494, but other courses are acceptable. Familiarity with R is essential. A course in computer programming would be helpful. Knowledge of multivariate calculus and linear algebra at the level of Math 233 and Math 309, respectively, is assumed.

Piazza: Make sure to enroll in this course on Piazza.

Textbook: You are encouraged, but not required, to obtain a copy of:

• BDA3 Bayesian Data Analysis, Third Edition, by Gelman, Carlin, Stern, Dunson, Vehtari and Rubin, CRC Press, 2013.

Computing: Familiarity with R is required. You can find many tutorials by clicking here. On the left side under Documentation, select Contributed to see a list of tutorials. A list of Bayesian packages here.

Grades: 30% Homework, 20% for each Midterm, 30% Final

Exams: 2 midterms and 1 final.

Homework: The lowest homework grade will be dropped. Assignments will include applied (using R), theoretical and philosophical (essay-based) problems. I expect to assign roughly one homework for every 3 lectures.  Homework is due at the beginning of class on the specified due date. All homework must be submitted to the instructor to receive credit; homework submitted to the grader will not be accepted without prior approval. See the policy on late homework below.

Final Course Grade: The letter grades for the course will be determined according to the following numerical grades on a 0-100 scale.

A+	[98, 100]	B+	[87, 90)	C+	[77, 80)	D+	[67, 70)	F	[0,60)
A	[93, 98)	B	[83, 87)	C	[73, 77)	D	[63, 67)
A-	[90, 93)	B-	[80, 83)	C-	[70, 73)	D-	[60, 63)

Course Schedule: Future topics tentative and subject to change; will be updated to reflect actual topics covered.

Week 1 01/18-01/22	Theme: What is Bayesian Inference? Principles and Examples of Bayesian Methods; Review of MLE; Bayesian estimation for scalar parameter models R examples: binomial and exponential with conjugate priors
Week 2 01/25-01/29	Theme: What is Bayesian inference? MLE and Bayesian estimation for vector-parameter models; likelihood inference; marginal posterior; interval estimates: credible sets
Week 3 02/01-02/05	Theme: Why Bayesian inference? Philosophy of science; falsifiability; inductive reasoning; interpretations of probability; current controversies (Ioannidis on why most published research findings are false, the backlash against null-hypothesis significance testing) Decision theory; components of statistical decision problems; risk; loss functions; criteria for optimal decision rules; admissibility, minimaxity, unbiasedness, Bayes risk
Week 4 02/08-02/12	Theme: Why Bayesian inference? Deriving Bayes rules from principles of decision theory; Bayes and admissibility/minimaxity; least favorable priors; HPD intervals; Stein's paradox; James-Stein estimation; empirical Bayes interpretation
Week 5 02/15-02/19	Theme: Specifying the prior. Conjugacy; objective Bayes; empirical Bayes; invariance and Jeffreys prior; Kullback-Leibler divergence and the reference prior; probability matching priors; background on Fisher information, orthogonality, prior and posterior independence, exchangeability
Week 6 02/22-02/26	Theme: Asymptotic Analysis Review of deterministic concepts; stochastic convergence; stochastic orders of magnitude; Khintchine's WLLN and Kolmogorov's SLLN; classical CLT; continuous mapping, Levy continuity theorems; uniform convergence; Polya's theorem; Berry-Esseen theorem; Scheffe's lemma Midterm Exam 1
Week 7 02/29-03/04	Theme: Large Sample Properties for Parametric Bayes Review of likelihood asymptotics; multivariate normal distribution; posterior consistency; Bernstein-von Mises theorem; sufficiency, conditionality and likelihood principles
Week 8 03/07-03/11	Theme: Motivations & Tools for Approximate Bayesian Inference Liouville's theorem with bits of complex analysis (holomorphic, meromorphic functions, special functions); Risch algorithm; Monte Carlo methods; random number generation; importance/rejection sampling
Week 9 03/14-03/18	Spring Break
Week 10 03/21-03/25	Theme: Computation Gibbs sampling, Metropolis-Hastings; reversible jump; convergence diagnostics
Week 11 03/28-04/01	Theme: Linear regression. Prior specification, estimation, inference, model diagnostics and model comparison
Week 12 04/04-04/08	Theme: Model Comparison and Hypothesis Testing Bayes factors; Laplace approximation; MCMC estimation of marginal likelihood; relationship to classical approaches Midterm Exam 2
Week 13 04/11-04/15	Theme: Generalized linear models. Review of posterior predictive distributions; generalized linear models; Bayesian binomial and Poisson regression in rstan
Week 14 04/18-04/22	Theme: Hierarchical linear models. Hierarchical linear models; empirical Bayes connections; random and mixed effects models
Week 15 04/25-04/29	Theme: Bayesian nonparametric models. Concepts; prior specification; Dirichlet process; stick-breaking representation
Reading Period 05/02-05/04	Office Hours by appointment.
Final Exam	See Piazza for details.

Other Course Policies: Students are encouraged to look at the Faculty of Arts & Sciences policies.

• Academic integrity: students are expected to adhere to the University's policy on academic integrity.
  
• Auditing: There is an option to audit, but this still involves enrolling in the course. See the Faculty of Arts & Sciences policy on auditing. Auditing students will still be expected to attend all lectures and compete all required coursework and exams.
  
• Collaboration: students are encouraged to discuss homework with one another, but each student must submit separate solutions, and these must be the original work of the student. This also applies to any R code.
• Exam conflicts: Read the University policy.
  
• Late homework: only by prior arrangement.
  
• Missed exams: there are no make-up exams. For valid excused absences with midterm exams - such as medical, family, transportation and weather-related emergencies - the contribution of that midterm to the final course grade will be redistributed equally to the other midterm exam and final exam. Students missing both midterm exams and/or the final exam cannot earn a passing grade for the course.