ACCELERATION OF NEURAL NETWORK TRAINING IN IMAGE RECOGNITION AND CLASSIFICATION PROBLEMS

The possibility of increasing the efficiency of learning of the neural network that recognizes images is being investigated. Network configuration is made so that all learning examples are recognized. Uses a uniform criterionfor the quality of education. Levenberg-Marquardt algorithm has been chosen as an algorithm to teach the neural network, and Bayesian regularization was applied to improve Levenberg-Marquardt algorithm and make it better usable for practical tasks. In the experimental part, we improve quality of the modified LM algorithm using Bayesian regularization and determine appropriate number of hidden layers to prevent overfitting. The considered algorithms allow not only to speed up the learning process, but also to reduce the number of adjustments of the neural network parameters. The latter property is important when parallelizing the learning process on cluster computing systems.

1. An Ru, LI Wen Jing, Han Hong Gui. QIAO Jun Fei. An Improved Levenberg-Marquardt Algo­rithm with Adaptive Learning Rate for RB F Neural Network. Proceedings of the 35th Chinese Control Conference July 27-29, 2016

2. Henri P. Gavin. The Levenberg-Marquardt method for nonlinear least squares curve-fitting prob­lems. March 22, 2017.

3. Liyan Qi, Xiantao Xiao and Liwei Zhang. A PARAMETER-SELF-ADJUSTING LEVEN­ BERG-MARQUARDT METHOD FOR SOLVING NONSMOOTH EQUATIONS. Journal of Computational Mathematics Vol.34, No.3, 2016, 317-338.

4. Soufiane Haddout and Mbarek Rhazi. Levenberg-Marquardt’s and Gauss-Newton algorithms for parameter optimisation of the motion of a point mass rolling on a turntable. European Journal Of Computational Mechanics Vol. 24 , Iss. 6, 2015

5. Murat Kayri. Predictive Abilities of Bayesian Regularization and Levenberg-Marquardt Algo­rithms in Artificial Neural Networks: A comparative Empirical Study on Social Data. Mathe­matical and Computational Applications, May 2016.

6. Altayeva, A. B., Omarov, B. S., Aitmagambetov, A. Z., Kendzhaeva, B. B., Burkitbayeva, M. A. Modeling and exploring base station characteristics of LTE mobile networks. Life Science Journal pp. 227-233. 2014.

7. Poland J. On the Robustness of Update Strategies for the Bayesian Hyperparameter alpha. Jan Poland, November, 2001.

8. B. S. Omarov, A. Suliman, K. Kushibar K., Journal of Theoretical and Applied Information Technology 91 (2), pp. 238-248, (2016)

9. Omarov, B., Suliman, A., Tsoy, A. Parallel backpropagation neural network training for face recognition. Far East Journal of Electronics and Communications. Volume 16, Issue 4, December 2016, Pages 801-808. (2016)

10. A. Altayeva, B. Omarov, H. C. Jeong, Y. I. Cho. Multi-step face recognition for improving face detection and recognition rate. Far East Journal of Electronics and Communications 16 (3), pp. 471-491, (2016)

11. Satish Saini, Ritu Vijay. 2014. Optimization of Artificial Neural Network Breast Cancer Detec­tion System based on Image Registration Techniques. International Journal of Computer Applications (0975 - 8887) Volume 105 - No. 14, November 2014.

12. Aimi Abdul Nasir, Mohd Yusoff Mashor, and Rosline Hassan. 2013. Classification of Acute Leukaemia Cellsusing Multilayer Perceptron and Simplified Fuzzy ARTMAP Neural Networks. The International Arab Journal of Information Technology, Vol. 10, No. 4, July 2013

13. Devesh Batra, 2014. Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradi­ent Training Algorithms For Image Compression Using MLP . International Journal of Image Processing (IJIP), Volume (8) : Issue (6) : 2014

14. Muhammad Ibn Ibrahimy, Md. Rezwanul Ahsan, Othman Omran Khalifa. 2013. Design and Optimization of Levenberg-Marquardt based Neural Network Classifier for EMG Signals to Identify Hand Motions. MEASUREMENT SCIENCE REVIEW, Volume 13, No. 3, 2013.

15. Abdel-Zaher Ahmed, Eldeib Ayman. 2016. Breast cancer classification using deep belief networks. Expert Systems With Applications, 46 (2016) 139-144

16. Lv, C., Xing, Y., Zhang, J., Na, X., Li, Y., Liu, T., ... & Wang, F. Y. (2018). Levenberg-Marquardt Backpropagation Training of Multilayer Neural Networks for State Estimation of a Safety-Critical Cyber-Physical System. IEEE Transactions on Industrial Informatics, 14 (8), 3436-3446.

17. Flores-Martrnez, N. L., Perez-Perez, M. C. I., Oliveros-Munoz, J. M., Lopez-Gonzalez, M. L., & Jimenez-Islas, H. (2018). Estimation of diffusion coefficients of essential oil of pimenta dioica in edible films formulated with aloe vera and gelatin, using levenberg-marquardt method. Revista Mexicana de Ingenieria Qrnmica, 17 (2), 485-506.

18. Choudhury, A., & Greene, D. (2018). Prognosticating autism spectrum disorder using artificial neural network: Levenberg-marquardt algorithm. Avishek Choudhury, Christopher M Greene. Prognosticating Autism Spectrum Disorder Using Artificial Neural Network: Levenberg-Marquardt Algorithm. Archives of Clinical and Biomedical Research, 2(2018), 188-197.

19. Choudhary, S., Doon, R., & Jha, S. K. (2019). Prediction of the Material Removal Rate and Sur­face Roughness in Electrical Discharge Diamond Grinding Using Best-Suited Artificial Neural Network Training Technique. In Applications of Artificial Intelligence Techniques in Engineering (pp. 487-495). Springer, Singapore.

20. Heravi, A. R., & Hodtani, G. A. (2018). Comparison of the convergence rates of the new Correntropy-based Levenberg-Marquardt (CLM) method and the Fixed-Point Maximum Correntropy (FP-MCC) algorithm. Circuits, Systems, and Signal Processing, 37 (7), 2884-2910.