Written exam (in case of a small number of participants: oral tests)

Part 1: basic results in statistical learning theory:
• Statistical setup, estimation and approximation error, consistency
• Negative results: No free lunch theorem, slow rates of convergence
• Consistency of k nearest neighbor algorithms and partitioning algorithms
• Concentration inequalities
• Simple generalization bounds, for example with shattering coefficients and VC dimension
• Advanced generalization bounds, for example using Rademacher complexities, algorithmic stability, sample compression.
• Regularization and its consistency
Part 2: advanced results in statistical learning theory. This part of the lecture
changes, depending on the interests of the audience and the current state of
the art in the field and covers some of the recent results on learning theory. It
could cover topics like online learning, theory of unsupervised learning, theory
of deep learning, etc.

Students get to know the standard tools and approaches in statistical learning
theory. They understand positive and negative results in learning theory, in
particular what are the fundamental limitations of machine learning, and which
properties are important to make a machine learning algorithm work.

The literature for this lecture will be provided at the beginning of the semester. / Students need to know the contents of the basic math classes, in particular
linear algebra and probability theory.