Instructor:
Todd Kuffner (
kuffner@math.wustl.edu)
Lecture:
1:00-2:30pm, Monday/Wednesday, Cupples I, Room 199
Office
Hours: Monday 8:00-9:00am, Tuesday 4:00-6:00pm; Cupples I Room 18 (basement).
Course
Overview:
This course is intended for Ph.D. students in
Statistics and Mathematics. Math 5061-5062 together form a year-long
sequence
in mathematical statistics leading to the Ph.D. qualifying exam in
statistical theory. The first
semester will cover introductory measure-theoretic probability,
decision theory, notions of optimality, principles of data
reduction, and finite sample estimation and inference. We will discuss
foundational issues, and consider several paradigms for testing, such
as the Neyman-Pearson, neo-Fisherian, and Bayesian approaches. Roughly
half of the first semester is devoted to the measure-theoretic
foundations of probability theory and statistics. The
second
semester will cover asymptotic theory, including convergence in
measure, limit theorems, integral and density approximations, and
higher-order asymptotics. Maximum likelihood, Bayesian, and bootstrap
methods will be considered. Empirical processes, large deviations,
and modern topics (e.g. Bayesian nonparametric asymptotics) will be
introduced as time permits. The style of the
course
is theorem-proof based; applications will not be emphasized, and
examples will be theoretical. Statistical software is not part of the
course.
Prerequisite:
It is assumed that students have taken a first course in real analysis,
probability, and mathematical statistics, and are familiar with basic
topology, multivariate calculus, and matrix algebra. Ph.D. students are
strongly encouraged to enroll in Math 5051 concurrently (Ph.D.-level
measure theory and functional analysis).
If you are undecided about whether or not to take this course, it may be helpful to look at the
Ph.D.
qualifying exam
from the last time I taught the course (2014-2015). This time there
will be more measure theory and probability theory on exams.
Textbook:
There are many excellent books and online resources for the material in
this course. However, no single book is suitable. Due to the cost of
purchasing several books, I will not require that students use any
particular books. The recommended readings for each lecture are
accompanied by sections of three books listed below, but students are
welcome to look at other references for the same material. I will use the same books for Math
5061 and Math 5062. The links give electronic access to two of the
books for Washington University students (logged in to library account)
through SpringerLink, but I also recommend purchasing these books as
they are excellent references for researchers.
- K.B. Athreya and S.N. Lahiri's Measure
Theory and Probability Theory, First Edition, Springer. electronic access errata ; thanks to Soumendra Lahiri for sending this to me!
- E.L. Lehmann and G. Casella's Theory of Point Estimation, Second Edition, Springer. errata more errata
- E.L. Lehmann & J.P. Romano's Testing
Statistical Hypotheses, Third Edition, Springer. electronic access errata
Another suggestion:
Essentials of Statistical
Inference by
G.A. Young and R.L. Smith, Cambridge University Press. This book is
much shorter and not intended as an encyclopedic reference, but it is perhaps the
most clearly-written, insightful treatment of modern statistical
inference.
Homework:
There will be weekly homework assignments. You are strongly encouraged
to write your solutions
in LaTeX. If not, then handwritten submissions must be clear and
organized. Homework will be graded, but solutions will not be provided
to students.
Homework grader: Qiyiwen Zhang (
qiyiwenzhang@wustl.edu)
Blackboard:
During the semester, homework assignments, homework and midterm exam
grades and any other course-related announcements will be posted to
Blackboard or sent by email using Blackboard.
Attendance:
Attendance is required for all lectures. The student who misses a
lecture is responsible for any assignments and/or announcements
made.
Grades:
15% Homework, 20% Midterm 1, 20% Midterm 2, 45% Final
Exams: 2 in-class midterms and 1 final.
The
dates of the exams should not be considered fixed until the first day
of class. What appears on Course Listings may be incorrect.
Homework: There will be weekly homework assignments.
The lowest homework grade will be dropped. If you added the class late and missed the first homework, then that will count as your dropped homework.
Final Course Grade: The letter grades for the course will be determined
according to the following numerical grades on a 0-100 scale.
A+
|
impress me
|
B+
|
[87, 90)
|
C+
|
[77, 80)
|
D+
|
[67, 70)
|
F
|
[0,60)
|
A
|
93+
|
B
|
[83, 87)
|
C
|
[73, 77)
|
D
|
[63, 67)
|
|
|
A-
|
[90, 93)
|
B-
|
[80, 83)
|
C-
|
[70, 73)
|
D-
|
[60, 63)
|
|
|
Other
Course Policies: Students are encouraged to look at the Faculty
of Arts & Sciences
policies.
- Academic integrity: Students
are expected to adhere to the University's policy
on academic
integrity.
- Auditing: There is an
option to audit, but this
still involves enrolling in the course. See the Faculty of Arts &
Sciences policy
on auditing.
Auditing students will still be expected to attend all
lectures and compete all required coursework and exams. A course grade of 75 is required for a successful audit.
- Collaboration: Students
are encouraged to discuss homework with one another, but each student
must submit separate solutions, and these must be the original work of
the student.
- Exam conflicts: Read the
University policy.
The exam dates for this course are posted before the semester begins,
and thus you are expected to be present at all exams.
- Late homework: Only by
prior arrangement. If a valid reason for an exception is not presented at least 36
hours before a homework due date, then it will not be accepted late (a
zero will be given for that assignment).
- Missed exams: There are
no make-up exams. For valid excused absences with midterm exams - such
as medical, family, transportation and weather-related
emergencies - the contribution of that midterm to the final course
grade will be redistributed equally to the other midterm exam and final
exam. Students missing both midterm exams and/or the final exam cannot
earn a passing grade for the course.
Course Schedule: tentative;
will be updated after lecture to reflect what was actually covered;
AL=Athreya & Lahiri, TPE=Theory of Point Estimation (Lehmann &
Casella); TSP=Testing Statistical Hypotheses (Lehmann & Romano)
08/29
|
Lecture 1
Review of set theory; algebras, sigma algebras; Borel sets
Reading: Appendix of AL
HW1 assigned
|
08/31
|
Lecture 2
Measures; extensions
Reading: AL 1.1-1.2
HW1 due
HW2 assigned
|
09/05
|
Labor Day; no class
|
09/07
|
Lecture 3
Completeness; measurable transformations
Reading: AL 1.3-1.4; AL 2.1-2.2
HW2 due Friday 09/09
|
09/12
|
Lecture 4
Induced measures; distribution functions;
Lebesgue and Riemann integration
Reading: AL 2.1-2.4
HW3 assigned
|
09/14
|
Lecture 5
Convergence for measurable functions
Reading: AL 2.5
|
09/19
|
Lecture 6 (guest lecturer)
Important Inequalities (Markov, Chebychev, Cramer, Jensen, Holder, Cauchy-Schwarz, Minkowski)
Reading: AL 3.1
|
09/21
|
Lecture 7 (guest lecturer)
L^p spaces, Banach spaces, and Hilbert spaces
Reading: AL 3.2-3.3
|
09/26
|
Lecture 8
Radon-Nikodym theorem; signed measures
Reading: AL 4.1-4.2
|
09/28
|
Lecture 9
Functions of bounded variation; absolutely continuous function on R; singular distributions; product spaces; product measures
Reading: AL 4.3-4.4, 5.1
|
10/03
|
Lecture 10
Fubini-Tonelli theorems; sample spaces; random variables; Kolmogorov's consistency theorem
Reading: AL 5.2 (note: 5.3-5.8 would be part of an analysis course), 6.1-6.3
|
10/05
|
Lecture 11
Expectation; moment generating functions; pi-lambda Theorem; independence
Reading: AL 6.1-6.3, 7.1
|
10/10
|
Lecture 12
Exchangeability; Representation Theorems; Borel-Cantelli lemmas; Kolmogorov's 0-1 Law
Reading: 7.1-7.2
|
10/12
|
Midterm 1 during class
Material: Lectures 1-10
|
10/17
|
Fall Break; no class
|
10/24
|
Lecture 13
Conditional expectations/probability; regular conditional distributions; Bayesian statistical experiments
Reading: 12.1-12.3
Recommended review material before next lecture: TPE 1.1-1.4
|
10/26
|
Lecture 14
Decision theory; data reduction via sufficiency
Reading: TSH 1.1-1.2, 1.4; TPE 1.6
|
10/31
|
Lecture 15
Exponential families; optimal data reduction via minimal sufficiency and completeness
Reading: TPE 1.5-1.6
|
11/02
|
Lecture 16
More data reduction; risk reduction
Reading: TPE 1.6-1.7, 2.1-2.3
|
11/07
|
Lecture 17
Optimal unbiased and location equivariant estimation; risk unbiasedness
Reading: TPE 2.1-2.3, 3.1-3.3
|
11/09
|
Midterm 2 during class
Material: Lectures 11-17
|
11/14
|
Lecture 18
Bayes estimators and average risk optimality
Reading: TPE 4.1-4.3
|
11/16
|
Lecture 19
Bayes estimators and average risk optimality
Reading: TPE 4.1-4.3
|
11/21
|
Lecture 20
Minimax estimators and worst-case optimality
Reading: TPE 5.1-5.2
|
11/28
|
Thanksgiving Break; no class
|
11/30
|
Lecture 21
Minimax estimators and worst-case optimality
Reading: TPE 5.1-5.2
|
12/05
|
Lecture 22
Minimax estimation; admissibility; simultaneous estimation
Reading: TPE 5.1-5.2, 4.7 (p. 272-277), 5.5 (p. 355-360)
|
12/07
|
Lecture 23
Robust estimation; high-dimensional estimation
Reading: handout
|
12/09
|
Last day of fall semester classes
|
12/19
|
Final Exam scheduled 6:00-8:00pm in Cupples I Room 199
Material: Lectures 1-23
|