Written exam (in case of a small number of participants: oral tests)

The computational cost of machine learning is almost entirely caused by numerical
computations: Optimization for training and fitting of point estimates;
integration for marginalization and conditioning in probabilistic models; simulation,
i.e. the solution of differential equations for predictions of the future,
and linear algebra as the base case of all of the above. These tasks are often
solved with “black-box” tools, but those who want to build highly performant,
scalable, professional solutions need to know how these tools worn and adapt
them to the specific task. This course introduces basic and advanced tools for
the aforementioned tasks. It develops a holistic view of computation in the
context of, and within the conceptual framework of machine learning, moving
from classic concepts to recent developments.
Apart from mathmatical derivations, the exercises put a focus on practical
programming. In particular, they contain implementations of some content of
the lectures.

Students develop both an intuitive and mathematical understanding of numerical
methods for optimization, integration, linear algebra, and the solution of
differential equation. They know how to adapt the tools to the challenges of
the task at hand, such as high dimensionality, stochasticity in computation,
numerical stability, non-convexity, efficient tuning of algorithmic parameters,
and uncertainty calibration for imprecise computation. Experience in the design
and use of numerical tools is a highly sought-after skill in industry, and
distinguishes the expert engineer from the amateur user.

Literature will be listed at the beginning of the semester. / Linear algebra is a core theme. Knowledge of probabilistic machine learning
is valuable for this course. Prior experience with numerical analysis is helpful
but not required. The practical parts use python and various recent python
libraries.