ECE 901: Statistical Learning Theory
Prerequisites:Background in applied mathematics, probability, and statistics
Instructor:
Robert Nowak
E-mail: nowak@engr.wisc.edu
Web: http://www.ece.wisc.edu/~nowak/
Phone: 608 265 3914
3627 Engineering Hall
Office Hours: email for appointment
Lectures:
Spring 2007
Time/Place: 11:00-12:15pm Tuesday and Thursday / 2341 Engineering Hall
Course Format:
The course will consist of 15-20 introductory lectures,
followed by readings and discussion of recent developments
Lectures:
Lecture 0 Statistical Decision and Learning Theory
Lecture 1 Elements of Statistical Learning Theory
Lecture 2 Introduction to Classification and Regression
Lecture 3 Introduction to Complexity Regularization
Lecture 4 Denoising in Smooth Function Spaces
Lecture 5 Plug-in Rules and Histogram Classifiers
Lecture 6 Probably Approximately Correct (PAC) Learning
Lecture 7 Chernoff's Bound and Hoeffding's Inequality
Lecture 8 Classification Error Bounds
Lecture 9 Error Bounds in Countably Infinite Models Spaces
Lecture 10 Complexity Regularization
Lecture 11 Decision Trees
Lecture 12 Complexity Regularization for Squared Error Loss
Lecture 13 Maximum Likelihood Estimation
Lecture 14 Maximum Likelihood and Complexity Regularization
Lecture 15 Denoising II: Adapting to Unknown Smoothness
Lecture 17 Minimax Lower Bounds
Homework Problems:
Homework 0 (due Tuesday Feb. 6 at the beginning of class)
Homework 1, rob.mat (right-click browser and "Save as", due Tues. Feb 13 at beginning of class)
Homework 2,(due Tues. Feb 20 at beginning of class)
Homework 3,(due Tues. Feb 27 at beginning of class)
Homework 4,(due Tues. March 6 at beginning of class)
Homework 5,(due Tues. March 13 at beginning of class)
Homework 6,(due Tues. March 20 at beginning of class)
Homework 7,(due Tues. March 27 at beginning of class)
Homework 8,(due Tues. April 10 at beginning of class)
Homework 9,(due Tues. May 1 at beginning of class)
Homework 10,(due Tues. May 10 at beginning of class)
Readings: TBA
Textbooks and References:
A textbook will not be followed in this course. A collection of
notes, relevant papers and materials will be prepared and distributed.
Textbooks recommended for further reading are listed below.
A probabilistic theory of pattern recognition, Devroye, Gyorfi, Lugosi, Springer
Nonparameteric Estimation Theory, Iain Johnstone, unpublished monograph
The Elements of Statistical Learning, Hastie, et al, Springer
An introduction to support vector machines, Cristianini and Shawe-Taylor, Cambridge Press
Combinatorial methods in density estimation, Devroye and Lugosi, Springer
Statistical Learning Theory, Vapnik, Wiley
An Introduction to Computational Learning Theory, Kearns and Vazirani, MIT Press
Empirical Processes in M-Estimation, van de Geer, Cambridge PressGrading and Evaluation:
Grades will be based on course participation, projects, and paper presentations.