i-manager Publications

An Efficient Deep Neural Network with Amplifying Sine Unit for Nonlinear Oscillatory Systems

Faiza Makhdoom*, Jamshaid Ul Rahman**, Naveed Anjum***

* Abdus Salam School of Mathematical Sciences, Government College University, Lahore, Pakistan.

** School of Mathematical Sciences, Jiangsu University, Zhenjiang, China.

*** Department of Mathematics, Government College University, Faisalabad, Pakistan.

Periodicity:April - June'2024
DOI : https://doi.org/10.26634/jme.14.2.21191

Abstract

Nonlinear differential equations play a pivotal role in modeling various physical phenomena, capturing their oscillatory behaviors. Numerous analytic and numerical methods have been developed to obtain precise or approximate solutions for nonlinear dynamics. This study introduces a novel approach utilizing neural network strategies, known for their computational flexibility. This study proposed a novel architecture for deep neural networks that is intended to simulate the nonlinear oscillations of two systems. It has been modified to include an oscillatory activation function called the amplifying sine unit. This study compares the results from our customized neural networks with those from the conventional stable and reliable numerical technique of the Runge-Kutta method of order four. Remarkably, there is striking agreement between the outcomes of the two methods, highlighting the ability of deep neural networks to identify nonlinear dynamical behaviors without requiring explicit conversion of mathematical models into an equation system.

Keywords

Artificial Neural Networks, Deep Learning, Mechanical Vibrations, Nonlinear Dynamical Systems, Amplifying Sine Unit, Oscillatory Systems.

How to Cite this Article?

Makhdoom, F., Rahman, J. U., and Anjum, N. (2024). An Efficient Deep Neural Network with Amplifying Sine Unit for Nonlinear Oscillatory Systems. i-manager’s Journal on Mechanical Engineering, 14(2), 30-40. https://doi.org/10.26634/jme.14.2.21191

References

[1]. Agarap, A. F. (2018). Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375.

[2]. Ain, Q. T., Anjum, N., & He, C. H. (2021). An analysis of time-fractional heat transfer problem using two-scale approach. GEM-International Journal on Geomathematics, 12, 1-10.

[3]. Ain, Q. T., He, J. H., Anjum, N., & Ali, M. (2020). The fractional complex transform: A novel approach to the time-fractional Schrödinger equation. Fractals, 28(07), 2050141.

[4]. Ali, A., Hussain, M., Hussan, M., & Rahman, J. U. (2022). Numerical solution and mathematical modelling of mass transport from medicated stent. Physica Scripta, 97(6), 065709.

[5]. Anjum, N., & Ain, Q. T. (2020). Application of He's fractional derivative and fractional complex transform for time fractional Camassa-Holm equation. Thermal Science, 24(5 Part A), 3023-3030.

[6]. Anjum, N., & He, J. H. (2020a). Analysis of nonlinear vibration of nano/microelectromechanical system switch induced by electromagnetic force under zero initial conditions. Alexandria Engineering Journal, 59(6), 4343-4352.

[7]. Anjum, N., & He, J. H. (2020b). Higher-order homotopy perturbation method for conservative nonlinear oscillators generally and microelectromechanical systems' oscillators particularly. International Journal of Modern Physics B, 34(32), 2050313.

[8]. Anjum, N., Ul Rahman, J., He, J. H., Alam, M. N., & Suleman, M. (2022). An efficient analytical approach for the periodicity of nano/microelectromechanical systems' oscillators. Mathematical Problems in Engineering, 2022(1), 9712199.

[9]. Apicella, A., Donnarumma, F., Isgrò, F., & Prevete, R. (2021). A survey on modern trainable activation functions. Neural Networks, 138, 14-32.

[10]. Arif, F., Majeed, Z., Rahman, J. U., Iqbal, N., & Kafle, J. (2022). Mathematical modeling and numerical simulation for the outbreak of COVID-19 involving loss of immunity and quarantined class. Computational and Mathematical Methods in Medicine, 2022(1), 3816492.

[11]. Bock, S., & Weiß, M. (2019, July). A proof of local convergence for the Adam optimizer. In 2019 international Joint Conference on Neural Networks (IJCNN) (pp. 1-8). IEEE.

[12]. Cao, Y., & Gopaluni, R. B. (2020). Deep neural network approximation of nonlinear model predictive control. IFAC-PapersOnLine, 53(2), 11319-11324.

[13]. Chauhan, V., & Srivastava, P. K. (2019). Computational techniques based on Runge-Kutta method of various order and type for solving differential equations. International Journal of Mathematical, Engineering and Management Sciences, 4(2), 375-386.

[14]. Chen, M., Xue, H., & Cai, D. (2019). Domain adaptation for semantic segmentation with maximum squares loss. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 2090-2099).

[15]. He, C. H., Liu, C., He, J. H., & Gepreel, K. A. (2021b). Low frequency property of a fractal vibration model for a concrete beam. Fractals, 29(05), 2150117.

[16]. He, J. H. (1999). Variational iteration method–a kind of non-linear analytical technique: Some examples. International Journal of Non-Linear Mechanics, 34(4), 699-708.

[17]. He, J. H. (2004). Variational principles for some nonlinear partial differential equations with variable coefficients. Chaos, Solitons & Fractals, 19(4), 847-851.

[18]. He, J. H. (2005). Variational approach to (2+ 1)-dimensional dispersive long water equations. Physics Letters A, 335(2-3), 182-184.

[19]. He, J. H. (2011). A short remark on fractional variational iteration method. Physics Letters A, 375(38), 3362-3364.

[20]. He, J. H., & El-Dib, Y. O. (2020). Homotopy perturbation method for Fangzhu oscillator. Journal of Mathematical Chemistry, 58, 2245-2253.

[21]. He, J. H., El-Dib, Y. O., & Mady, A. A. (2021a). Homotopy perturbation method for the fractal toda oscillator. Fractal and Fractional, 5(3), 93.

[22]. Jais, I. K. M., Ismail, A. R., & Nisa, S. Q. (2019). Adam optimization algorithm for wide and deep neural network. Knowledge Engineering and Data Science (KEDS), 2(1), 41-46.

[23]. Liu, C. Y., Ku, C. Y., & Chen, W. D. (2024). A spacetime RBF-Based DNNs for solving unsaturated flow problems. Mathematics, 12(18), 2940.

[24]. Liu, Y., Wang, X., Wang, L., & Liu, D. (2019). A modified leaky ReLU scheme (MLRS) for topology optimization with multiple materials. Applied Mathematics and Computation, 352, 188-204.

[25]. Lu, D., Suleman, M., Ramzan, M., & Ul Rahman, J. (2021). Numerical solutions of coupled nonlinear fractional KdV equations using He's fractional calculus. International Journal of Modern Physics B, 35(02), 2150023.

[26]. Lu, L., Shin, Y., Su, Y., & Karniadakis, G. E. (2019). Dying relu and initialization: Theory and numerical examples. arXiv preprint arXiv:1903.06733.

[27]. Masana, M., Liu, X., Twardowski, B., Menta, M., Bagdanov, A. D., & Van De Weijer, J. (2022). Class-incremental learning: Survey and performance evaluation on image classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5), 5513-5533.

[28]. Misra, D. (2019). Mish: A self regularized non-monotonic activation function. ar Xiv preprint arXiv:1908.08681.

[29]. Noel, M. M., Trivedi, A., & Dutta, P. (2021). Growing cosine unit: A novel oscillatory activation function that can speedup training and reduce parameters in convolutional neural networks. ar Xiv preprint arXiv:2108.12943.

[30]. Nwankpa, C., Ijomah, W., Gachagan, A., & Marshall, S. (2018). Activation functions: Comparison of trends in practice and research for deep learning. arXiv preprint arXiv:1811.03378.

[31]. Rahman, J. U. (2024). DiffGrad for physics-informed neural networks. arXiv preprint arXiv:2409.03239.

[32]. Rahman, J. U., Danish, S., & Lu, D. (2024a). Oscillator simulation with deep neural networks. Mathematics, 12(7), 959.

[33]. Rahman, J. U., Makhdoom, F., & Lu, D. (2023). Amplifying sine unit: An oscillatory activation function for deep neural networks to recover nonlinear oscillations efficiently. arXiv preprint arXiv:2304.09759.

[34]. Rahman, J. U., Makhdoom, F., Rashid, U., Lu, D., & Akgül, A. (2024b). An Oscillatory Deep Neural Network for Coupled Electrical Circuits. Research Square.

[35]. Raissi, M. (2018). Deep hidden physics models: Deep learning of nonlinear partial differential equations. Journal of Machine Learning Research, 19(25), 1-24.

[36]. Rehman, S., Hussain, A., Rahman, J. U., Anjum, N., & Munir, T. (2022). Modified Laplace based variational iteration method for the mechanical vibrations and its applications. Acta Mechanica et Automatica, 16(2), 98-102.

[37]. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533-536.

[38]. Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Müller, K. R. (2021). Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE, 109(3), 247-278.

[39]. Saxe, A. M., McClelland, J. L., & Ganguli, S. (2013). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv preprint arXiv:1312.6120.

[40]. Shapovalov, V., Kharlamov, P., Oziabkin, A., Gorin, S., Zinovev, V., Kornienko, R., & Mihajluk, A. (2019, December). Application of methods physical and mathematical modeling for a research of nonlinear mechanical systems on the example of the rolling stock. In AIP Conference Proceedings, 2188 (1), 020017. AIP Publishing.

[41]. Sharma, S., Sharma, S., & Athaiya, A. (2017). Activation functions in neural networks. Towards Data Science, 6(12), 310-316.

[42]. Suleman, M., Lu, D., He, J. H., Farooq, U., Hui, Y. S., & Rahman, J. U. (2018). Numerical investigation of fractional HIV model using Elzaki projected differential transform method. Fractals, 26(05), 1850062.

[43]. Suleman, M., Lu, D., Yue, C., Ul Rahman, J., & Anjum, N. (2019). He–Laplace method for general nonlinear periodic solitary solution of vibration equations. Journal of Low Frequency Noise, Vibration and Active Control, 38(3-4), 1297-1304.

[44]. Tan, H. H., & Lim, K. H. (2019, June). Vanishing gradient mitigation with deep learning neural network optimization. In 2019 7th International Conference on Smart Computing & Communications (ICSCC) (pp. 1-4). IEEE.

[45]. Taylanov, N. A., Urinov, S. X. O. G. L., & Abdiev, J. M. O. G. L. (2022). A fourth-order runge-kutta method for numerical solution of the Kuramoto-Sivashinsky equation. Eurasian Scientific Herald, 7, 58–61.

[46]. Tian, D., Ain, Q. T., Anjum, N., He, C. H., & Cheng, B. (2021). Fractal N/MEMS: From pull-in instability to pull-in stability. Fractals, 29(02), 2150030.

[47]. Tripathi, G. C., Rawat, M., & Rawat, K. (2019, October). Swish activation based deep neural network predistorter for RF-PA. In TENCON 2019-2019 IEEE Region 10 Conference (TENCON) (pp. 1239-1242). IEEE.

[48]. Ul Rahman, J., Ali, A., Ur Rehman, M., & Kazmi, R. (2020b). A unit softmax with Laplacian smoothing stochastic gradient descent for deep convolutional neural networks. In Intelligent Technologies and Applications: Second International Conference, INTAP 2019, Bahawalpur, Pakistan, November 6–8, 2019, Revised Selected Papers 2 (pp. 162-174). Springer Singapore.

[49]. Ul Rahman, J., Chen, Q., & Yang, Z. (2020a). Additive parameter for deep face recognition. Communications in Mathematics and Statistics, 8, 203-217.

[50]. Ul Rahman, J., Danish, S., & Lu, D. (2023). Deep neural network-based simulation of sel'kov model in glycolysis: A comprehensive analysis. Mathematics, 11(14), 3216.

[51]. Ul Rahman, J., Lu, D., Suleman, M., He, J. H., & Ramzan, M. (2019). He–Elzaki method for spatial diffusion of biological population. Fractals, 27(05), 1950069.

[52]. Ul Rahman, J., Makhdoom, F., Ali, A., & Danish, S. (2024). Mathematical modeling and simulation of biophysics systems using neural network. International Journal of Modern Physics B, 38(05), 2450066.

[53]. ul Rahman, J., Mohyuddin, M. R., Anjum, N., & Zahoor, S. (2015). Mathematical modelling & simulation of mixing of salt in 3-interconnected tanks. Journal of Advances in Civil Engineering, 1(1), 1-6.

[54]. Wang, K., Chen, J., Song, Z., Wang, Y., & Yang, C. (2021). Deep neural network-embedded stochastic nonlinear state-space models and their applications to process monitoring. IEEE Transactions on Neural Networks and Learning Systems, 33(12), 7682-7694.

[55]. Wang, Q., Ma, Y., Zhao, K., & Tian, Y. (2020). A comprehensive survey of loss functions in machine learning. Annals of Data Science (pp. 1-26).

[56]. Xu, J., Li, Z., Du, B., Zhang, M., & Liu, J. (2020, July). Reluplex made more practical: Leaky ReLU. In 2020 IEEE Symposium on Computers and communications (ISCC) (pp. 1-7). IEEE.

[57]. Yeung, E., Kundu, S., & Hodas, N. (2019, July). Learning deep neural network representations for Koopman operators of nonlinear dynamical systems. In 2019 American Control Conference (ACC) (pp. 4832-4839). IEEE.

If you have access to this article please login to view the article or kindly login to purchase the article

Single Article

	North Americas,UK, Middle East,Europe		India	Rest of world
	USD	EUR	INR	USD-ROW
Online	15	15

ADD TO CART

Options for accessing this content:

If you would like institutional access to this content, please recommend the title to your librarian.
If you already have i-manager's user account: Login above and proceed to purchase the article.
New Users: Please register, then proceed to purchase the article.