Heaviside function as an activation function

• Autorzy: Hatano Naoya , Ikeda Masahiro , Ishikawa Isao , Sawano Yoshihiro

Identyfikatory

DOI

10.1515/jaa-2022-2012

• Języki publikacji: EN

Abstrakty

EN

The uniform closure of the neural networks with Heaviside activation is specified.

Słowa kluczowe

EN

càdlàg; Heaviside function; uniform norm; neural networks

• Wydawca: De Gruyter
• Czasopismo: Journal of Applied Analysis
• Rocznik: 2023
• Tom: Vol. 29, nr 1
• Strony: 1--2
• Opis fizyczny: Bibliogr. 6 poz.

Twórcy

autor

Hatano Naoya

• Department of Mathematics, Faculty of Science and Technology, Keio University, 3-14-1 Hiyoshi, Kohoku-ku, Yokohama 223-8522
• Center for Advanced Intelligence Project, RIKEN
• Department of Mathematics, Chuo University, 1-13-27, Kasuga, Bunkyo-ku, Tokyo 112-8551, Japan

autor

Ikeda Masahiro

• Center for Advanced Intelligence Project, RIKEN, Nihonbashi 1-chome Mitsui Building, 15th floor, 1-4-1 Nihonbashi, Chuo-ku Tokyo 103-0027
• Department of Mathematics, Faculty of Science and Technology, Keio University, 3-14-1 Hiyoshi, Kohoku-ku, Yokohama 223-8522, Japan

autor

Ishikawa Isao

• Center for Advanced Intelligence Project, RIKEN
• Center for Data Science, Ehime University, 3 Bunkyo-cho, Matsuyama, Ehime 790-8577, Tokyo Japan

autor

Sawano Yoshihiro

• Department of Mathematics, Faculty of Science and Technology, Keio University, 3-14-1 Hiyoshi, Kohoku-ku, Yokohama 223-8522
• Center for Advanced Intelligence Project, RIKEN
• Department of Mathematics, Chuo University, 1-13-27, Kasuga, Bunkyo-ku, Tokyo 112-8551, Japan

Bibliografia

• [1] G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signals Systems 2 (1989), no. 4, 303-314.
• [2] B. Hanin, Universal function approximation by deep neural nets with bounded width and ReLU activations, Mathematics 7 (2019), no. 10, Paper No. 992.
• [3] N. Hatano, M. Ikeda, M. Ishikawa and Y. Sawano, A global universality of two-layer neural networks with ReLU activations, J. Funct. Spaces 2021 (2021), Article ID 6637220.
• [4] P. Kainen, V. Kůrková and A. Vogt, Best approximation by Heaviside perceptron networks, Neural Networks 13 (2000), no. 7, 695-697.
• [5] A. Pinkus, Approximation theory of the MLP model in neural networks, Acta Numer. 8 (1999), 143-195.
• [6] S. Sonoda and N. Murata, Neural network with unbounded activation functions is universal approximator, Appl. Comput. Harmon. Anal. 43 (2017), no. 2, 233-268.

Uwagi

Opracowanie rekordu ze środków MNiSW, umowa nr SONP/SP/546092/2022 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2024).