BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//https://techplay.jp//JP
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALDESC:[20th AIP Open Seminar] Talks by Deep Learning Theory Team 
X-WR-CALNAME:[20th AIP Open Seminar] Talks by Deep Learning Theory Team 
X-WR-TIMEZONE:Asia/Tokyo
BEGIN:VTIMEZONE
TZID:Asia/Tokyo
BEGIN:STANDARD
DTSTART:19700101T000000
TZOFFSETFROM:+0900
TZOFFSETTO:+0900
TZNAME:JST
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
UID:811573@techplay.jp
SUMMARY:[20th AIP Open Seminar] Talks by Deep Learning Theory Team 
DTSTART;TZID=Asia/Tokyo:20210407T150000
DTEND;TZID=Asia/Tokyo:20210407T170000
DTSTAMP:20260423T212815Z
CREATED:20210315T060015Z
DESCRIPTION:イベント詳細はこちら\nhttps://techplay.jp/event/81157
 3?utm_medium=referral&utm_source=ics&utm_campaign=ics\n\nDeep Learning Th
 eory Team (https://aip.riken.jp/labs/generic_tech/deep_learn_theory/?lang
 =en) at RIKEN AIP\n\nSpeaker 1: Taiji Suzuki (30 mins)\nTitle: Overview o
 f Recent Advances of Deep Learning Theory Researches\nAbstract: \nIn this
  talk\, I will overview recent development of our deep learning theory re
 searches. The main problems of deep learning theory are roughly divided i
 nto (1) representation ability\, (2) generalization ability\, and (3) opt
 imization. We have conducted several researches on these issues. As for (
 1) representation ability\, we consider a setting where the true target f
 unction is included in some special function classes such as Besov space\
 , and analyze the approximation ability of deep neural networks. We can s
 how that deep learning can achieve the so-called adaptive approximation. 
 Eventually\, deep learning can achieve better rate of convergence to esti
 mate such a "complicated" functions. As for (2) the generalization abilit
 y\, we briefly introduce the compression ability based generalization abi
 lity\, and we also discuss generalization ability by revealing connection
  to optimization ability. As for (3) the optimization ability\, we discus
 s achievability of the global optimal solution by a gradient descent tech
 nique. In particular\, we consider the global optimality in a mean field 
 regime and discuss generalization error of the solution obtained by gradi
 ent descent type methods.\n\nSpeaker 2: Atsushi Nitanda (30 mins)\nTitle:
  Optimization for two-layer neural networks with quantitative global conv
 ergence analysis\nAbstract: \nThe gradient-based method is known to achie
 ve vanishing training error on overparameterized neural networks\, despit
 e the nonconvexity of the objective function. Recently\, many studies are
  devoted to explaining the global convergence property. A common idea is 
 to utilize overparameterization of neural networks to translate the train
 ing dynamics into function space and exploit the convexity of the objecti
 ve with respect to the function. These approaches are mainly divided into
  two categories: the neural tangent kernel (NTK) and mean-field (MF) regi
 mes which deal with different dynamics switched by the scaling factor of 
 neural networks. In this presentation\, I would like to talk about our re
 cent advances on both regimes for overparameterized two-layer neural netw
 orks. First\, for the NTK regime\, we show that the averaged stochastic g
 radient descent can achieve the fast convergence rate under assumptions o
 n the complexities of the target function and the RKHS associated with th
 e NTK. Second\, for the MF regime\, we give the first quantitative global
  convergence rate analysis by proposing a new method for the entropic reg
 ularized empirical risk minimization in the probability space.\n\nSpeaker
  3: Kenta Oono (30 mins)\nTitle: On over-smoothing of graph neural networ
 ks\nAbstract: \nGraph Neural Networks (GNNs) are a collective term of dee
 p learning models for graph-structured data. Recent studies have empirica
 lly shown that GNNs performed well in many application fields such as bio
 chemistry\, computer vision\, and knowledge graph analysis. However\, the
 oretical characteristics of GNNs are less investigated compared with thos
 e of classical deep learning models such as fully-connected neural networ
 ks or convolutional neural networks. Over-smoothing is one of the challen
 ges of current GNN models\, where representations of nodes a GNN makes be
 come indistinguishable as we increase the number of layers of the GNN. Th
 is problem prevents us from making a GNN model deep. In this talk\, I int
 roduce the over-smoothing problem and explain recent research on it.\n\nS
 peaker 4: Sho Sonoda (30 mins)\nTitle: Functional analysis methods for ne
 ural network theory\nAbstract: \nCharacterization of the typical solution
 s obtained by deep learning is an important open problem in machine learn
 ing theory. The speaker has been addressing this problem from the viewpoi
 nt of functional analysis by using the integral representation of neural 
 networks. The integral representation is known to have a closed-form righ
 t inverse operator\, called the ridgelet transform\, which is related to 
 both the Radon and the wavelet transforms. The speaker has recently shown
  with his collaborators that for the case of ridge regression by finite t
 wo-layer neural networks\, the empirical risk minimizers are given by rid
 gelet transform in the limit of over-parametrization (S-Ishikawa-Ikeda\, 
 AISTATS2021). In this talk\, the speaker will introduce the ridgelet tran
 sform and present recent results to characterize deep learning solutions.
 \n\n\n\nAll participants are required to agree with the AIP Open Seminar 
 Series Code of Conduct.\nPlease see the URL below.\nhttps://aip.riken.jp/
 event-list/termsofparticipation/?lang=en\n\nRIKEN AIP will expect adheren
 ce to this code throughout the event. We expect cooperation from all part
 icipants to help ensure a safe environment for everybody.\n\n
LOCATION:オンライン
URL:https://techplay.jp/event/811573?utm_medium=referral&utm_source=ics&utm
 _campaign=ics
END:VEVENT
END:VCALENDAR