Course Archives Theoretical Statistics and Mathematics Unit
Course:
Special Topics 3: Statistical Learning Theory
Level:
Postgraduate
Time:
Currently not offered
Syllabus
Past Exams
Syllabus :
What is learning? Multilayer perceptron as a concrete example of learning. Visualization of learning as multidimensional function approximation, either through training or prior experiences (supervised learning) or through a set of predetermined rules (unsupervised learning).
Supervised learning: Variable types and terminologies, two simple approaches to prediction (1) least squares and (2) nearest neighbors. Structured regression models. Model selection and biasvariance trade off. Linear methods for regression, least square linear regression, Gauss-Markov theorem, multiple regression, subset selection. Linear methods for classification, linear discriminant analysis, logistic regression. Bayesian decision theory.
Overview of neural networks, feed forward and feed back algorithms, an application in handwritten character recognition, specificity vs. generalizability.
Basic ideas of support vector machine, Vapnik-Chervonenkis dimension.
Unsupervised learning: Cluster analysis, principal component analysis, independent component analysis.
Suggested Texts :
1. T. Hastie, R. Tibshirani and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction, 2nd ed., Springer, New York, 2009.
2. R. O. Duda, P. E. Hart and D. G. Stork, Pattern Classification, 2nd ed., John Wiley & Sons Inc., New York, 2001.
3. C. J. C. Burges, A tutorial on support vector machine for pattern recognition, Data Mining and Knowledge Discovery, vol. 2, pp. 121 167, 1998.
Pre-requisite: Linear algebra, basic probability and statistics. The course will be covered in 16 weeks with 3 hour lecture per week. It will have 70% weight on theory and 30% on programming assignments, preferably in MATLAB or Python.
Top of the page
Past Exams
Midterm
19.pdf
Semestral
19.pdf
Supplementary and Back Paper
Top of the page
[
Semester Schedule
][
SMU
] [
Indian Statistical Institute
]