| Lecture 1. |
Introduction and Motivation
A pattern exists; we don't know it; we have data to learn it. Netflix; Credit Approval,... | LFD Chapter 1 (The Learning Problem): § 1.1 |
| Lecture 2. |
General Setup and the Perceptron
The liniear separator; types of learning: supervised, reinforcement, unsupervised; a puzzle. | LFD Chapter 1 (The Learning Problem): § 1.1, 1.2 |
| Lecture 3. |
Is Learning Feasible
Can we reach outside the data? Probability to the rescue - Hoeffding's lemma. | LFD Chapter 1 (The Learning Problem): § 1.3 |
| Lecture 4. |
Real Learning is Feasible
Real learning, the 2 step solution. Back to reality: error measures and noisy targets. | LFD Chapter 1 (The Learning Problem): § 1.3, 1.4 |
| Lecture 5. |
Training Versus Testing
Toward an "effective size" for infinite hypothesis sets: the growth function. | LFD Chapter 2 (Training Versus Testing): § 2.1.1 |
| Lecture 6. |
Bounding the Growth Function
A polynomial bound on the growth function and the VC generalization bound. | LFD Chapter 2 (Training Versus Testing): § 2.1.2 |
| Lecture 7. |
The VC Dimension
Approximation versus generalization; bias-variance analysis and learning curves. | LFD Chapter 2 (Training Versus Testing): § 2.1.3, 2.2, 2.3 |
| Lecture 8. |
Linear Classification,
Regression
Non-separable data and the linear model for estimating a real value and the pseudo-inverse. | LFD Chapter 3 (The Linear Model): § 3.1, 3.2 |
| Lecture 9. |
Logistic Regression and
Gradient Descent
Estimating a probability: the cross entropy error and gradient descent minimization. | LFD Chapter 3 (The Linear Model): § 3.3 |
| Lecture 10. |
Non Linear Transforms
Nonlinear hypotheses using the non-linear transform. | LFD Chapter 3 (The Linear Model): § 3.4 |
| Lecture 11. |
Overfitting
When are simpler models better than complex ones? Deterministic and stochastic noise | LFD Chapter 4 (Overfitting): § 4.1 |
| Lecture 12. |
Regularization
Constraining a model toward simpler hypotheses to combat noise. | LFD Chapter 4 (Overfitting): § 4.2 |
| Lecture 13. |
Validation and Model Selection
Estimating out-of-sample error and its use to make high level choices in learning. | LFD Chapter 4 (Overfitting): § 4.3 |
| Lecture 14. |
Three Learning Principles
Occam's razor (choosing hypotheses); sampling bias (getting data); data snooping (handling data). | LFD Chapter 5 (Three Learning Priniciples): § 5 |