MACHINE LEARNING FOR PATTERN RECOGNITION (2st MODULE)
Learning outcomes of the course unit
Objective of the course is to provide the student with the ability to understand and apply the basic rules of machine learning and, in particular:
- to apply the most common statistical tests in classification among different categories
- to synthesize the structure of the optimal classifier and analyze its error performance
- to apply the most common feature extraction methods from input data
- to apply the most common statistical estimators in machine learning
- to apply the most common clustering algorithms in unsupervised learning
The abilities in applying the above-mentioned knowledge are in particular in the:
- design and performance analysis of classifiers in machine learning
- selection of the most appropriate features to discriminate input categories
- selection of the most appropriate clustering algorithms in the design of unsupervised classifiers
Entry-level courses in linear algebra and probability theory, such as those normally offered in the corresponding 3-year Laurea course, are necessary pre-requisites for this course.
Course contents summary
PART 1: Fundamentals (Bononi)
(follows ref  Ch 1-9)
- problem statement and definitions
- Examples of machine learning problems
2. Basic probability refresher:
- Bayes formula
- conditional density functions
3. Classical Decision Rules
- binary Bayes rule
- M-ary Bayes rule
- receiver operating curve (ROC) and its properties
- Glossary of equivalent terms in Radar detecton theory, hypothesis testing and
4. Linear Algebra refresher
- Unitary and Hermitian matrices
- spectral decomposition (SD)
- covariance matrices and diagonalization
5. Feature extraction
- sufficient statistics
- feature extraction based on eigenvector analysis
6. Quadratic and linear classifiers
- discriminant functions
- classification with Gaussian vectors
- Bounds on classifiers error probability
7. parameter estimation
- maximum likelihood and properties
- bayes estimation: MMSE and MAP
- Bounds on MS error
PART 2: Advanced topics (Cagnoni)
8. Nonparametric estimation
- Parzen density estimation
- k-Nearest-Neighbor algorithm
9. Linear Discriminant Analysis
- Fisher linear classifier
- Support Vector Machines
10. Classifier evaluation:
- Generalization and overfitting (Training/validation/test sets)
- Performance indices, representations curve, confusion matrices
- Classification risk: are all errors equally relevant ?
11. Unsupervised classification and clustering
- K-means and Isodata algorithms
- Self-Organizing Maps
- Learning Vector Quantization
- Kohonen networks
 C. W. Therrien, "Decision, estimation and classification" Wiley, 1989
 C. M. Bishop "Pattern Recognition and Machine Learning", Springer, 2006.
 R O Duda, P, E. Hart, D. G. Stork, "Pattern classification", 2nd Ed., Wiley, 2001
Classroom teaching, 42 hours.
In-class problem solving, 6 hours.
Homework regularly assigned.
Assessment methods and criteria
Part 1, Bononi: Oral only, to be scheduled on an individual basis. When ready, please contact the instructor by email at alberto.bononi[AT]unipr.it and by specifying the requested date. The exam consists of solving some exercises and explaining theoretical details connected with them, for a total time of about 1 hour. You can bring your summary of important formulas in an A4 sheet to consult if you so wish.
Part 2, Cagnoni: A practical project will be assigned, whose results will presented and discussed by the student both as a written report and as an oral presentation.
Bononi: Monday 11:30-13:30 (Scientific Complex, Building 2, floor 2, Room 2/19T).
Cagnoni: by appointment (Scientific Complex, Building 1, floor 2, email cagnoni[AT]ce.unipr.it).