Hardware implementation of hyperbolic tangent and sigmoid activation functions

• Autorzy: Hajduk Z.

Identyfikatory

DOI

10.24425/124272

• Języki publikacji: EN

Abstrakty

EN

This paper presents the high accuracy hardware implementation of the hyperbolic tangent and sigmoid activation functions for artificial neural networks. A kind of a direct implementation of the functions in a few different versions is proposed and investigated both by software and hardware modeling. A single precision floating point arithmetic is applied. Apart from conventional design style with hardware description language coding, high level synthesis design techniques with the Matlab HDL coder and Xilinx Vivado HLS have also been investigated.

Słowa kluczowe

EN

FPGA; hyperbolic tangent; sigmoid; floating point arithmetic

PL

FPGA; tangens hiperboliczny; arytmetyka zmiennoprzecinkowa; funkcja sigmoidalna

• Wydawca: Polska Akademia Nauk, Wydział IV Nauk Technicznych
• Czasopismo: Bulletin of the Polish Academy of Sciences. Technical Sciences
• Rocznik: 2018
• Tom: Vol. 66, nr 5
• Strony: 563--577
• Opis fizyczny: Bibliogr. 20 poz., rys., wykr., tab.

Twórcy

autor

Hajduk Z.

• Rzeszów University of Technology, ul. Powstańców Warszawy 12, 35-959 Rzeszów, Poland

Bibliografia

• [1] A. Gomperts, A. Ukil, and F. Zurfluh, “Development and Implementation of Parameterized FPGA-Based General Purpose Neural Networks for Online Applications”, IEEE Trans. on Industrial Informatics 7(1), 78‒89 (2011).
• [2] A. Armato, L. Fanucci, G. Pioggia, and D. De Rossi, “Low-error approximation of artificial neuron sigmoid function and its derivative”, Electronics Letters 45(21), 1‒2 (2009).
• [3] V. Tiwari and N. Khare, “Hardware implementation of neural network with Sigmoidal activation functions using CORDIC”, Microprocessors and Microsystems 39, 373‒381 (2015).
• [4] A. Armato, L. Fanucci, E.P. Scilingo, and D. De Rossi, “Lowerror digital hardware implementation of artificial neuron activation functions and their derivative”, Microprocessors and Microsystems 35, 557‒567 (2011).
• [5] M.T. Tommiska, “Efficient digital implementation of the sigmoid function for reprogrammable logic”, IEEE Proc. Comput. Digit. Tech. 150(6), 403‒411 (2003).
• [6] P. Ferreiraa, P. Ribeiroa, A. Antunesa, and F. Morgado Dias, “A high bit resolution FPGA implementation of a FNN with a new algorithm for the activation function”, Neurocomputing 71(1‒3), 71–77 (2007).
• [7] M. Bajger and A. Omondi, “Low-error, High-speed Approximation of the Sigmoid Function for Large FPGA Implementations”, Journal of Signal Processing Systems 52, 137–151 (2008 ).
• [8] T. Orlowska-Kowalska and M. Kaminski, “FPGA Implementation of the Multilayer Neural Network for the Speed Estimation of the Two-Mass Drive System”, IEEE Trns. on Industrial Informatics 7(3), 436‒445 (2011).
• [9] D. Baptista and F. Morgado-Dias, “Low-resource hardware implementation of the hyperbolic tangent for artificial neural networks”, Neural Computing and Applications 23(3), 601‒607 (2013).
• [10] I. del Campo, R. Finker, J. Echanobe, and K. Basterretxea, “Controlled accuracy approximation of sigmoid function for efficient FPGA-based implementation of artificial neurons”, Electronics Letters 49(25), 1598‒1600 (2013).
• [11] A.M. Abdelsalam, J.M. Pierre Langlois, and F. Cheriet, “A Configurable FPGA Implementation of the Tanh Function using DCT Interpolation”, IEEE 25th Annual Int. Symp. on Field Programmable Custom Computing Machines, 168‒171, (2017).
• [12] F. Zhoua, J. Liua, Y. Yua, X. Tiana, H. Liub, Y. Haoa, S. Zhanga, W. Chena, J. Daia, and X. Zhenga, “Field-programmable gate array implementation of a probabilistic neural network for motor cortical decoding in rats”, Journal of Neuroscience Methods 185, 299–306 (2010).
• [13] D. Sanchez-Roman, G. Sutter, S. Lopez-Buedo, I. Gonzalez, F.J. Gomez-Arribas, J. Aracil, and F. Palacios, “High-Level Languages and Floating-Point Arithmetic for FPGA Based CFD Simulations”, IEEE Design & Test of Computers 28(4), 28‒37 (2011).
• [14] Z. Hajduk, “High accuracy FPGA activation function implementation for neural networks”, Neurocomputing (Brief papers) 247, 59‒61 (2017). DOI: 10.1016/j.neucom.2017.03.044
• [15] M. Vajta, “Some remarks on Padé-approximations”, Tempus-Intcom Symposium, Sept. 2000, 1‒6 (2000).
• [16] J. Kluska and Z. Hajduk, “Hardware implementation of P1-TS fuzzy rule-based systems on FPGA”, Proc. Artif. Intell. Soft Comput. 7894, 282‒293 (2013).
• [17] Z. Hajduk, B. Trybus, and J. Sadolewski, “Architecture of FPGA Embedded Multiprocessor Programmable Controller”, IEEE Trans. Ind. Electron. 62(5), 2952–2961 (2015).
• [18] Z. Hajduk, “An FPGA embedded microcontroller”, Microprocessors and Microsystems 38(1), 1–8 (2014).
• [19] C. Maxfield, “MathWorks’ MATLAB now supports HDL code generation”, EETimes, 2012, available at http://www.eetimes.com/document.asp?doc_id=1317035
• [20] M. Santarini, “Xilinx unveils vivado design suite for the next decade of ‘all programmable’ devices”, Xcell Journal 79, 8‒13 (2012).

Uwagi

PL

Opracowanie rekordu w ramach umowy 509/P-DUN/2018 ze środków MNiSW przeznaczonych na działalność upowszechniającą naukę (2019).