by Prof. Vera Kurkova

Institute of Computer Science
Academy of Sciences of the Czech Republic
http://cs.cas.cz/ vera

Generalization capability in learning from data can be investigated in terms
of regularization, which has been used in many branches of applied mathematics
to obtain stable solutions of inverse problems, i.e., problems of nding unknown
causes (such as shapes of functions) of known consequences (such as measured
data). It will be shown that supervised learning modelled in terms of minimizati-
ons of error functionals can be reformulated as inverse problems with solutions
in spaces of functions dened by kernels. Mathematical results from theory of
inverse problems can be applied to propose new types of computational units,
to design stabilizers increasing generalization, and to construct optimal soluti-
ons of learning tasks, which can be used to design learning algorithms based on
solutions of systems of linear equations.

Content:

Learning from data: minimization of the empirical error functional de-
ned by a sample of data and minimization of the expected error functional
dened by a probability distribution, optimizations of error functionals as best
approximations.
Generalization: generalization in learning as a stability of solutions with
respect to small changes of data, penalization of solutions with high-frequency
oscillation, output-weight regularization.
Inverse problems: well and ill-posed problems, well and ill-conditioned
problems, Moore-Penrose pseudosolution, measures of stability, regularization
as improvement of stability, properties of optimal and regularized solutions.
Representation of learning as an inverse problem: typical operators
dening inverse problems, tomography and Radon transform, operators dening
inverse problems modelling learning, characterization of optimal and regularized
solutions, comparison of regularized and non regularized case.
Three reasons for using kernels in machine learning: kernels dene a
class of hypothesis spaces satisfying assumptions needed for application of main
results from theory of inverse problems, kernels dene stabilizers penalizing
various types of high-frequency oscillations, kernels dene transformations of
input space geometry allowing more types of data to be separated linearly.
Learning algorithms based on solutions of inverse problems: neural
network learning as a solution of a system of linear equations, computational
units using tensor products and combinations of kernels, approximate optimi-
zation as complexity reduction, comparison with algorithms operating on ne-
tworks with smaller number of units than the size of the sample of data.

RNDr. Věra Kůrková, DrSc received PhD in mathematics from Charles
University Prague and DrSc (prof.) in theoretical computer science from Aca-
demy of Sciences of the Czech Republic. Since 1990 she works as a scientist in
the Institute of Computer Science, Prague, in 2002-2009 as the Head of the De-
partment of Theoretical Computer Science. She published many journal papers
and book chapters on mathematical theory of neurocomputing and learning and
on nonlinear approximation theory. She is a member of the editorial boards of
Neural Networks and Neural Processing Letters, in past she also served as an
associate editor of IEEE Transactions on Neural Networks. She is a member of
the Board of European Neural Network Society (ENNS), she was general chair
of the conferences ICANN 2008 and ICANNGA 2001.