FAU MoD Lecture Series. Special December 2024

Name: FAU MoD Lecture Series. Special December 2024
Start: 2024-12-03
End: 2024-12-03
Location: Friedrich-Alexander-Universität Erlangen-Nürnberg

Date: Tue. December 03, 2024
Event: FAU MoD Lecture Series (Special double session. December 2024)
Organized by: FAU MoD, the Research Center for Mathematics of Data at Friedrich-Alexander-Universität Erlangen-Nürnberg (Germany)

Session 01: 14:30H
FAU MoD Lecture: The implicit bias phenomenon in deep learning
Speaker: Prof. Dr. Holger Rauhut
Affiliation: Mathematisches Institut der Universität München (Germany)

Abstract. Deep neural networks are usually trained by minimizing a non-convex loss functional via (stochastic) gradient descent methods. Unfortunately, the convergence properties are not very well-understood. Moreover, a puzzling empirical observation is that learning neural networks with a number of parameters exceeding the number of training examples often leads to zero loss, i.e., the network exactly interpolates the data. Nevertheless, it generalizes very well to unseen data, which is in stark contrast to intuition from classical statistics which would predict a scenario of overfitting.

A current working hypothesis is that the chosen optimization algorithm has a significant influence on the selection of the learned network. In fact, in this overparameterized context there are many global minimizers so that the optimization method induces an implicit bias on the computed solution. It seems that gradient descent methods and their stochastic variants favor networks of low complexity (in a suitable sense to be understood), and, hence, appear to be very well suited for large classes of real data.

Initial attempts in understanding the implicit bias phenomen considers the simplified setting of linear networks, i.e., (deep) factorizations of matrices. This has revealed a surprising relation to the field of low rank matrix recovery (a variant of compressive sensing) in the sense that gradient descent favors low rank matrices in certain situations. Moreover, restricting further to diagonal matrices, or equivalently factorizing the entries of a vector to be recovered, shows connection to compressive sensing and l1-minimization.

Session 02: 16:00H
FAU MoD Lecture: Counterintuitive approximations
Speaker: Prof. Dr. Christian Bär
Affiliation: Institut für Mathematik. Universität Potsdam (Germany)

Abstract. The Nash-Kuiper embedding theorem is a prototypical example of a counterintuitive approximation result: any short (but highly non-isometric) embedding of a Riemannian manifold into Euclidean space can be approximated by isometric C¹-embeddings. As a consequence, any surface can be isometrically C¹-embedded into an arbitrarily small ball in ℝ³. For C²-embeddings this is impossible due to curvature restrictions.

I will present a general result which allows for approximations by functions satisfying strongly overdetermined equations on open dense subsets. This will be illustrated by three examples: Lipschitz functions with surprising derivative, surfaces in 3-space with unexpected curvature properties, and a similar statement for abstract Riemannian metrics on manifolds. Our method is based on “cut-oﬀ homotopy”, a concept introduced by Gromov in 1986.

This is based on joint work with Bernhard Hanke.

AUDIENCE

This is a hybrid event (On-site/online) open to: Public, Students, Postdocs, Professors, Faculty, Alumni and the scientific community all around the world.

WHEN

Tue. December 03, 2024 at 14:30H (Berlin time)

WHERE

On-site / Online

[On-site] Friedrich-Alexander-Universität Erlangen-Nürnberg
Felix Klein building. Department Mathematik
Room H13 Johann-Radon-Hörsaal
Cauerstraße 11, 91058 Erlangen
GPS-Koord. Raum: 49.579737N, 11.029862E

[Online] FAU Zoom link
Meeting ID: TBA | PIN code: TBA