<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.3 20210610//EN" "JATS-journalpublishing1-3.dtd">
<article article-type="research-article" dtd-version="1.3" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xml:lang="ru"><front><journal-meta><journal-id journal-id-type="publisher-id">kaz29</journal-id><journal-title-group><journal-title xml:lang="ru">Вестник Казахстанско-Британского технического университета</journal-title><trans-title-group xml:lang="en"><trans-title>Herald of the Kazakh-British Technical University</trans-title></trans-title-group></journal-title-group><issn pub-type="ppub">1998-6688</issn><issn pub-type="epub">2959-8109</issn><publisher><publisher-name>Казахстанско-Британский Технический Университет</publisher-name></publisher></journal-meta><article-meta><article-id custom-type="elpub" pub-id-type="custom">kaz29-330</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research Article</subject></subj-group><subj-group subj-group-type="section-heading" xml:lang="ru"><subject>МАШИННОЕ ОБУЧЕНИЕ И АНАЛИЗ ДАННЫХ</subject></subj-group><subj-group subj-group-type="section-heading" xml:lang="en"><subject>DATA SCIENCE AND MACHINE LEARNING</subject></subj-group></article-categories><title-group><article-title>УСКОРЕНИЕ НЕЙРОННОЙ СЕТИ В ПРОБЛЕМАХ РАСПОЗНАВАНИЯ И КЛАССИФИКАЦИИ ИЗОБРАЖЕНИЯ</article-title><trans-title-group xml:lang="en"><trans-title>ACCELERATION OF NEURAL NETWORK TRAINING IN IMAGE RECOGNITION AND CLASSIFICATION PROBLEMS</trans-title></trans-title-group></title-group><contrib-group><contrib contrib-type="author" corresp="yes"><name-alternatives><name name-style="eastern" xml:lang="ru"><surname>Омаров</surname><given-names>Б.</given-names></name><name name-style="western" xml:lang="en"><surname>Omarov</surname><given-names>B.</given-names></name></name-alternatives><xref ref-type="aff" rid="aff-1"/></contrib><contrib contrib-type="author" corresp="yes"><name-alternatives><name name-style="eastern" xml:lang="ru"><surname>Омаров</surname><given-names>Н.</given-names></name><name name-style="western" xml:lang="en"><surname>Omarov</surname><given-names>N.</given-names></name></name-alternatives><bio xml:lang="ru"><p>зам. директора</p></bio><xref ref-type="aff" rid="aff-2"/></contrib><contrib contrib-type="author" corresp="yes"><name-alternatives><name name-style="eastern" xml:lang="ru"><surname>Аккасов</surname><given-names>А.</given-names></name><name name-style="western" xml:lang="en"><surname>Akkasov</surname><given-names>A.</given-names></name></name-alternatives><bio xml:lang="ru"><p>студент</p></bio><xref ref-type="aff" rid="aff-3"/></contrib><contrib contrib-type="author" corresp="yes"><name-alternatives><name name-style="eastern" xml:lang="ru"><surname>Жумамуратов</surname><given-names>М.</given-names></name><name name-style="western" xml:lang="en"><surname>Zhumamuratov</surname><given-names>M.</given-names></name></name-alternatives><bio xml:lang="ru"><p>студент</p></bio><xref ref-type="aff" rid="aff-4"/></contrib></contrib-group><aff-alternatives id="aff-1"><aff xml:lang="ru">Казахстанская инновационная лаборатория при поддержке фонда ЮНИСЕФ; Международный университет информационных технологий<country>Казахстан</country></aff><aff xml:lang="en">College of Computer Science &amp; Information Technology, Universiti Tenaga Nasional; Kazakhstan Innovations Lab supported by UNICEF<country>Kazakhstan</country></aff></aff-alternatives><aff-alternatives id="aff-2"><aff xml:lang="ru">Казахский университет путей сообщения<country>Казахстан</country></aff><aff xml:lang="en">Kazakh University of Railways and Communications<country>Kazakhstan</country></aff></aff-alternatives><aff-alternatives id="aff-3"><aff xml:lang="ru">Международный университет информационных технологий<country>Казахстан</country></aff><aff xml:lang="en">International Information Technology University<country>Kazakhstan</country></aff></aff-alternatives><aff-alternatives id="aff-4"><aff xml:lang="ru">Казахский университет путей сообщения<country>Казахстан</country></aff><aff xml:lang="en">International Information Technology University<country>Kazakhstan</country></aff></aff-alternatives><pub-date pub-type="collection"><year>2019</year></pub-date><pub-date pub-type="epub"><day>19</day><month>11</month><year>2021</year></pub-date><volume>16</volume><issue>3</issue><fpage>469</fpage><lpage>477</lpage><permissions><copyright-statement>Copyright &amp;#x00A9; Омаров Б., Омаров Н., Аккасов А., Жумамуратов М., 2021</copyright-statement><copyright-year>2021</copyright-year><copyright-holder xml:lang="ru">Омаров Б., Омаров Н., Аккасов А., Жумамуратов М.</copyright-holder><copyright-holder xml:lang="en">Omarov B., Omarov N., Akkasov A., Zhumamuratov M.</copyright-holder><license xml:lang="ru" license-type="creative-commons-attribution" xlink:href="https://creativecommons.org/licenses/by/4.0/" xlink:type="simple"><license-p>Данная работа распространяется под лицензией Creative Commons Attribution 4.0.</license-p></license><license xml:lang="en" license-type="creative-commons-attribution" xlink:href="https://creativecommons.org/licenses/by/4.0/" xlink:type="simple"><license-p>This work is licensed under a Creative Commons Attribution 4.0 License.</license-p></license></permissions><self-uri xlink:href="https://vestnik.kbtu.edu.kz/jour/article/view/330">https://vestnik.kbtu.edu.kz/jour/article/view/330</self-uri><abstract><p>Изучается возможность повышения эффективности обучения нейронной сети, распознающей изображения. Конфигурация сети сделана так, чтобы все обучающие примеры были распознаны. Используется единый критерий качества образования. АлгоритмЛевенберга-Марквардта был выбран в качестве алгоритма для обучения нейронной сети, а байесовская регуляризация была применена для улучшения алгоритма Левенберга-Марквардта и его лучшего использованиядля практических задач. В экспериментальной части мы улучшаем качество модифицированного алгоритма LM, используя байесовскую регуляризацию, и определяем соответствующее количество скрытых слоев, чтобы предотвратить переоснащение. Рассмотренные алгоритмы позволяют не только ускорить процесс обучения, но и сократить количество корректировок параметров нейронной сети. Последнее свойство важно при распараллеливании процесса обучения на кластерных вычислительных системах.</p></abstract><trans-abstract xml:lang="en"><p>The possibility of increasing the efficiency of learning of the neural network that recognizes images is being investigated. Network configuration is made so that all learning examples are recognized. Uses a uniform criterionfor the quality of education. Levenberg-Marquardt algorithm has been chosen as an algorithm to teach the neural network, and Bayesian regularization was applied to improve Levenberg-Marquardt algorithm and make it better usable for practical tasks. In the experimental part, we improve quality of the modified LM algorithm using Bayesian regularization and determine appropriate number of hidden layers to prevent overfitting. The considered algorithms allow not only to speed up the learning process, but also to reduce the number of adjustments of the neural network parameters. The latter property is important when parallelizing the learning process on cluster computing systems.</p></trans-abstract><kwd-group xml:lang="ru"><kwd>классификация</kwd><kwd>алгоритм Левенберга-Марквардта</kwd><kwd>нейронные сети</kwd><kwd>регуляризация</kwd></kwd-group><kwd-group xml:lang="en"><kwd>Classification</kwd><kwd>Levenberg-Marquardt Algorithm</kwd><kwd>Neural Networks</kwd><kwd>Regularization</kwd></kwd-group></article-meta></front><back><ref-list><title>References</title><ref id="cit1"><label>1</label><citation-alternatives><mixed-citation xml:lang="ru">An Ru, LI Wen Jing, Han Hong Gui. QIAO Jun Fei. An Improved Levenberg-Marquardt Algo­rithm with Adaptive Learning Rate for RB F Neural Network. Proceedings of the 35th Chinese Control Conference July 27-29, 2016</mixed-citation><mixed-citation xml:lang="en">An Ru, LI Wen Jing, Han Hong Gui. QIAO Jun Fei. An Improved Levenberg-Marquardt Algo­rithm with Adaptive Learning Rate for RB F Neural Network. Proceedings of the 35th Chinese Control Conference July 27-29, 2016</mixed-citation></citation-alternatives></ref><ref id="cit2"><label>2</label><citation-alternatives><mixed-citation xml:lang="ru">Henri P. Gavin. The Levenberg-Marquardt method for nonlinear least squares curve-fitting prob­lems. March 22, 2017.</mixed-citation><mixed-citation xml:lang="en">Henri P. Gavin. The Levenberg-Marquardt method for nonlinear least squares curve-fitting prob­lems. March 22, 2017.</mixed-citation></citation-alternatives></ref><ref id="cit3"><label>3</label><citation-alternatives><mixed-citation xml:lang="ru">Liyan Qi, Xiantao Xiao and Liwei Zhang. A PARAMETER-SELF-ADJUSTING LEVEN­ BERG-MARQUARDT METHOD FOR SOLVING NONSMOOTH EQUATIONS. Journal of Computational Mathematics Vol.34, No.3, 2016, 317-338.</mixed-citation><mixed-citation xml:lang="en">Liyan Qi, Xiantao Xiao and Liwei Zhang. A PARAMETER-SELF-ADJUSTING LEVEN­ BERG-MARQUARDT METHOD FOR SOLVING NONSMOOTH EQUATIONS. Journal of Computational Mathematics Vol.34, No.3, 2016, 317-338.</mixed-citation></citation-alternatives></ref><ref id="cit4"><label>4</label><citation-alternatives><mixed-citation xml:lang="ru">Soufiane Haddout and Mbarek Rhazi. Levenberg-Marquardt’s and Gauss-Newton algorithms for parameter optimisation of the motion of a point mass rolling on a turntable. European Journal Of Computational Mechanics Vol. 24 , Iss. 6, 2015</mixed-citation><mixed-citation xml:lang="en">Soufiane Haddout and Mbarek Rhazi. Levenberg-Marquardt’s and Gauss-Newton algorithms for parameter optimisation of the motion of a point mass rolling on a turntable. European Journal Of Computational Mechanics Vol. 24 , Iss. 6, 2015</mixed-citation></citation-alternatives></ref><ref id="cit5"><label>5</label><citation-alternatives><mixed-citation xml:lang="ru">Murat Kayri. Predictive Abilities of Bayesian Regularization and Levenberg-Marquardt Algo­rithms in Artificial Neural Networks: A comparative Empirical Study on Social Data. Mathe­matical and Computational Applications, May 2016.</mixed-citation><mixed-citation xml:lang="en">Murat Kayri. Predictive Abilities of Bayesian Regularization and Levenberg-Marquardt Algo­rithms in Artificial Neural Networks: A comparative Empirical Study on Social Data. Mathe­matical and Computational Applications, May 2016.</mixed-citation></citation-alternatives></ref><ref id="cit6"><label>6</label><citation-alternatives><mixed-citation xml:lang="ru">Altayeva, A. B., Omarov, B. S., Aitmagambetov, A. Z., Kendzhaeva, B. B., Burkitbayeva, M. A. Modeling and exploring base station characteristics of LTE mobile networks. Life Science Journal pp. 227-233. 2014.</mixed-citation><mixed-citation xml:lang="en">Altayeva, A. B., Omarov, B. S., Aitmagambetov, A. Z., Kendzhaeva, B. B., Burkitbayeva, M. A. Modeling and exploring base station characteristics of LTE mobile networks. Life Science Journal pp. 227-233. 2014.</mixed-citation></citation-alternatives></ref><ref id="cit7"><label>7</label><citation-alternatives><mixed-citation xml:lang="ru">Poland J. On the Robustness of Update Strategies for the Bayesian Hyperparameter alpha. Jan Poland, November, 2001.</mixed-citation><mixed-citation xml:lang="en">Poland J. On the Robustness of Update Strategies for the Bayesian Hyperparameter alpha. Jan Poland, November, 2001.</mixed-citation></citation-alternatives></ref><ref id="cit8"><label>8</label><citation-alternatives><mixed-citation xml:lang="ru">B. S. Omarov, A. Suliman, K. Kushibar K., Journal of Theoretical and Applied Information Technology 91 (2), pp. 238-248, (2016)</mixed-citation><mixed-citation xml:lang="en">B. S. Omarov, A. Suliman, K. Kushibar K., Journal of Theoretical and Applied Information Technology 91 (2), pp. 238-248, (2016)</mixed-citation></citation-alternatives></ref><ref id="cit9"><label>9</label><citation-alternatives><mixed-citation xml:lang="ru">Omarov, B., Suliman, A., Tsoy, A. Parallel backpropagation neural network training for face recognition. Far East Journal of Electronics and Communications. Volume 16, Issue 4, December 2016, Pages 801-808. (2016)</mixed-citation><mixed-citation xml:lang="en">Omarov, B., Suliman, A., Tsoy, A. Parallel backpropagation neural network training for face recognition. Far East Journal of Electronics and Communications. Volume 16, Issue 4, December 2016, Pages 801-808. (2016)</mixed-citation></citation-alternatives></ref><ref id="cit10"><label>10</label><citation-alternatives><mixed-citation xml:lang="ru">A. Altayeva, B. Omarov, H. C. Jeong, Y. I. Cho. Multi-step face recognition for improving face detection and recognition rate. Far East Journal of Electronics and Communications 16 (3), pp. 471-491, (2016)</mixed-citation><mixed-citation xml:lang="en">A. Altayeva, B. Omarov, H. C. Jeong, Y. I. Cho. Multi-step face recognition for improving face detection and recognition rate. Far East Journal of Electronics and Communications 16 (3), pp. 471-491, (2016)</mixed-citation></citation-alternatives></ref><ref id="cit11"><label>11</label><citation-alternatives><mixed-citation xml:lang="ru">Satish Saini, Ritu Vijay. 2014. Optimization of Artificial Neural Network Breast Cancer Detec­tion System based on Image Registration Techniques. International Journal of Computer Applications (0975 - 8887) Volume 105 - No. 14, November 2014.</mixed-citation><mixed-citation xml:lang="en">Satish Saini, Ritu Vijay. 2014. Optimization of Artificial Neural Network Breast Cancer Detec­tion System based on Image Registration Techniques. International Journal of Computer Applications (0975 - 8887) Volume 105 - No. 14, November 2014.</mixed-citation></citation-alternatives></ref><ref id="cit12"><label>12</label><citation-alternatives><mixed-citation xml:lang="ru">Aimi Abdul Nasir, Mohd Yusoff Mashor, and Rosline Hassan. 2013. Classification of Acute Leukaemia Cellsusing Multilayer Perceptron and Simplified Fuzzy ARTMAP Neural Networks. The International Arab Journal of Information Technology, Vol. 10, No. 4, July 2013</mixed-citation><mixed-citation xml:lang="en">Aimi Abdul Nasir, Mohd Yusoff Mashor, and Rosline Hassan. 2013. Classification of Acute Leukaemia Cellsusing Multilayer Perceptron and Simplified Fuzzy ARTMAP Neural Networks. The International Arab Journal of Information Technology, Vol. 10, No. 4, July 2013</mixed-citation></citation-alternatives></ref><ref id="cit13"><label>13</label><citation-alternatives><mixed-citation xml:lang="ru">Devesh Batra, 2014. Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradi­ent Training Algorithms For Image Compression Using MLP . International Journal of Image Processing (IJIP), Volume (8) : Issue (6) : 2014</mixed-citation><mixed-citation xml:lang="en">Devesh Batra, 2014. Comparison Between Levenberg-Marquardt And Scaled Conjugate Gradi­ent Training Algorithms For Image Compression Using MLP . International Journal of Image Processing (IJIP), Volume (8) : Issue (6) : 2014</mixed-citation></citation-alternatives></ref><ref id="cit14"><label>14</label><citation-alternatives><mixed-citation xml:lang="ru">Muhammad Ibn Ibrahimy, Md. Rezwanul Ahsan, Othman Omran Khalifa. 2013. Design and Optimization of Levenberg-Marquardt based Neural Network Classifier for EMG Signals to Identify Hand Motions. MEASUREMENT SCIENCE REVIEW, Volume 13, No. 3, 2013.</mixed-citation><mixed-citation xml:lang="en">Muhammad Ibn Ibrahimy, Md. Rezwanul Ahsan, Othman Omran Khalifa. 2013. Design and Optimization of Levenberg-Marquardt based Neural Network Classifier for EMG Signals to Identify Hand Motions. MEASUREMENT SCIENCE REVIEW, Volume 13, No. 3, 2013.</mixed-citation></citation-alternatives></ref><ref id="cit15"><label>15</label><citation-alternatives><mixed-citation xml:lang="ru">Abdel-Zaher Ahmed, Eldeib Ayman. 2016. Breast cancer classification using deep belief networks. Expert Systems With Applications, 46 (2016) 139-144</mixed-citation><mixed-citation xml:lang="en">Abdel-Zaher Ahmed, Eldeib Ayman. 2016. Breast cancer classification using deep belief networks. Expert Systems With Applications, 46 (2016) 139-144</mixed-citation></citation-alternatives></ref><ref id="cit16"><label>16</label><citation-alternatives><mixed-citation xml:lang="ru">Lv, C., Xing, Y., Zhang, J., Na, X., Li, Y., Liu, T., ... &amp; Wang, F. Y. (2018). Levenberg-Marquardt Backpropagation Training of Multilayer Neural Networks for State Estimation of a Safety-Critical Cyber-Physical System. IEEE Transactions on Industrial Informatics, 14 (8), 3436-3446.</mixed-citation><mixed-citation xml:lang="en">Lv, C., Xing, Y., Zhang, J., Na, X., Li, Y., Liu, T., ... &amp; Wang, F. Y. (2018). Levenberg-Marquardt Backpropagation Training of Multilayer Neural Networks for State Estimation of a Safety-Critical Cyber-Physical System. IEEE Transactions on Industrial Informatics, 14 (8), 3436-3446.</mixed-citation></citation-alternatives></ref><ref id="cit17"><label>17</label><citation-alternatives><mixed-citation xml:lang="ru">Flores-Martrnez, N. L., Perez-Perez, M. C. I., Oliveros-Munoz, J. M., Lopez-Gonzalez, M. L., &amp; Jimenez-Islas, H. (2018). Estimation of diffusion coefficients of essential oil of pimenta dioica in edible films formulated with aloe vera and gelatin, using levenberg-marquardt method. Revista Mexicana de Ingenieria Qrnmica, 17 (2), 485-506.</mixed-citation><mixed-citation xml:lang="en">Flores-Martrnez, N. L., Perez-Perez, M. C. I., Oliveros-Munoz, J. M., Lopez-Gonzalez, M. L., &amp; Jimenez-Islas, H. (2018). Estimation of diffusion coefficients of essential oil of pimenta dioica in edible films formulated with aloe vera and gelatin, using levenberg-marquardt method. Revista Mexicana de Ingenieria Qrnmica, 17 (2), 485-506.</mixed-citation></citation-alternatives></ref><ref id="cit18"><label>18</label><citation-alternatives><mixed-citation xml:lang="ru">Choudhury, A., &amp; Greene, D. (2018). Prognosticating autism spectrum disorder using artificial neural network: Levenberg-marquardt algorithm. Avishek Choudhury, Christopher M Greene. Prognosticating Autism Spectrum Disorder Using Artificial Neural Network: Levenberg-Marquardt Algorithm. Archives of Clinical and Biomedical Research, 2(2018), 188-197.</mixed-citation><mixed-citation xml:lang="en">Choudhury, A., &amp; Greene, D. (2018). Prognosticating autism spectrum disorder using artificial neural network: Levenberg-marquardt algorithm. Avishek Choudhury, Christopher M Greene. Prognosticating Autism Spectrum Disorder Using Artificial Neural Network: Levenberg-Marquardt Algorithm. Archives of Clinical and Biomedical Research, 2(2018), 188-197.</mixed-citation></citation-alternatives></ref><ref id="cit19"><label>19</label><citation-alternatives><mixed-citation xml:lang="ru">Choudhary, S., Doon, R., &amp; Jha, S. K. (2019). Prediction of the Material Removal Rate and Sur­face Roughness in Electrical Discharge Diamond Grinding Using Best-Suited Artificial Neural Network Training Technique. In Applications of Artificial Intelligence Techniques in Engineering (pp. 487-495). Springer, Singapore.</mixed-citation><mixed-citation xml:lang="en">Choudhary, S., Doon, R., &amp; Jha, S. K. (2019). Prediction of the Material Removal Rate and Sur­face Roughness in Electrical Discharge Diamond Grinding Using Best-Suited Artificial Neural Network Training Technique. In Applications of Artificial Intelligence Techniques in Engineering (pp. 487-495). Springer, Singapore.</mixed-citation></citation-alternatives></ref><ref id="cit20"><label>20</label><citation-alternatives><mixed-citation xml:lang="ru">Heravi, A. R., &amp; Hodtani, G. A. (2018). Comparison of the convergence rates of the new Correntropy-based Levenberg-Marquardt (CLM) method and the Fixed-Point Maximum Correntropy (FP-MCC) algorithm. Circuits, Systems, and Signal Processing, 37 (7), 2884-2910.</mixed-citation><mixed-citation xml:lang="en">Heravi, A. R., &amp; Hodtani, G. A. (2018). Comparison of the convergence rates of the new Correntropy-based Levenberg-Marquardt (CLM) method and the Fixed-Point Maximum Correntropy (FP-MCC) algorithm. Circuits, Systems, and Signal Processing, 37 (7), 2884-2910.</mixed-citation></citation-alternatives></ref></ref-list><fn-group><fn fn-type="conflict"><p>The authors declare that there are no conflicts of interest present.</p></fn></fn-group></back></article>
