kaz29Вестник Казахстанско-Британского технического университетаHerald of the Kazakh-British Technical University1998-66882959-8109Казахстанско-Британский Технический Университетkaz29-241Research ArticleРАЗРАБОТКА ПРОГРАММНОГО ОБЕСПЕЧЕНИЯ И ИНЖЕНЕРИЯ ЗНАНИЙSOFTWARE ENGINEERING AND KNOWLEDGE ENGINEERINGМЕТОДЫ РЕГУЛЯРИЗАЦИИ ГЛУБОКОГО ОБУЧЕНИЯ MAX-POOL И DROPOUT ДЛЯ ОБНАРУЖЕНИЯ ДОРОЖНЫХ ЗНАКОВMAX-POOL AND DROPOUT REGULARIZATION DEEP LEARNING TECHNIQUES TO DETECT TRAFFIC SIGNSЕрежепбековА.YerezhepbekovA.

магистрант

Международный университет информационных технологийКазахстанInternational Information Technology UniversityKazakhstan2019131120211634654Copyright © Ережепбеков А., 20212021Ережепбеков А.Yerezhepbekov A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://vestnik.kbtu.edu.kz/jour/article/view/241

Многие водители автомобилей невнимательны к дорожным знакам, которые приводят к несчастным или даже драматическим случаям. Поэтому, чтобы предотвратить такие вещи, в этой статье предлагается использовать технику машинного обучения сверточными нейронными сетями с алгоритмами максимального пула и повторного отсева. В последнее время методика регуляризации отсева находит все большее применение в глубоком обучении. Известно, что для глубоко сверточных нейронных сетей отсеивание хорошо работает в полностью связанных слоях. Однако его влияние на сверточный и объединяющий слои все еще неясно. В этой статье наглядно показано, что отсев максимального пула эквивалентен случайному выбору активации на основе полиномиального распределения во время обучения. Учебный комплект реализован на основе известного немецкого набора данных дорожных знаков и позволяет увидеть разницу между двумя методами регуляризации, поскольку регуляризатор отсева очень эффективен для минимизации переобучения обучающего набора путем случайного отбрасывания входящих и исходящих нейронов. Кроме того, в сочетании с максимальным пулированием для регуляризации отсева может потребоваться больше эпох, чтобы более точно сходиться. Заполнение алгоритма набором данных дорожных знаков делает его полезным для адаптивных систем круиз-контроля в автомобилях, чтобы избежать неприятных и неуклюжих автомобильных аварий. Два метода могут использоваться в тандеме или по отдельности, но в любом случае производительность может быть настроена путем изменения гиперпараметров.

Many car drivers are inattentive to traffic signs which result in unfortunate or even dramatic accidents, so in order to prevent such things this article proposes using machine learning technique convolutional neural networks with max-pool and dropout reqularization algorithms. Recently, a dropout regularization technique has seen increasing use in deep learning. For deep convolutional neural networks, dropout is known to work well in fully-connected layers. However, its effect in convolutional and pooling layers is still not clear. This article illustrates in pythonic manner that max-pooling dropout is equivalent to randomly picking activation based on a multinomial distribution at training time. Training set is implemented upon a famous German traffic sign dataset and to see the difference between two regularization methods. Since, dropout regularizer is very efficient in minimizing the overfitting o f the training set by randomly discarding inbound and outbound neurons. Plus, in mix with max-pooling a dropout regularization might require more epochs to converge more accurately. Feeding the algorithm with traffic sign dataset makes it useful for adaptive cruise control systems in cars to avoid nasty and awkward car accidents. Two methods can be used in tandem or separately but in either case performance can be tuned by changing hyperparameters.

глубокое обучениесверточные нейронные сетимакс-пул отбрасываниерегуляризациятеория БайесатренировкаDeep learningConvolutional neural networksMax-pooling dropoutReferencesBaldi, P., & Sadowski, P. (2014). The dropout learning algorithm. Artificial Intelligence, 210, 78-122.Baldi, P., & Sadowski, P. (2014). The dropout learning algorithm. Artificial Intelligence, 210, 78-122.Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: a review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 1798-1828.Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: a review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 1798-1828.Boureau, Y. L., Ponce J., & LeCun, Y (2010). A theoretical analysis of feature pooling in visual recognition. In Proceedings 27th of International Conference on Machine Learning (ICML 2010).Boureau, Y. L., Ponce J., & LeCun, Y (2010). A theoretical analysis of feature pooling in visual recognition. In Proceedings 27th of International Conference on Machine Learning (ICML 2010).Breiman, L. (1996). Bagging predictors. Machine Learning, 24, 123-140.Breiman, L. (1996). Bagging predictors. Machine Learning, 24, 123-140.Ciresan. D., Meier, U., & Schmidhuber, J. (2012). Multi-column deep neural networks for image classification. In Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recog­nition (CVPR 2012).Ciresan. D., Meier, U., & Schmidhuber, J. (2012). Multi-column deep neural networks for image classification. In Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recog­nition (CVPR 2012).Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., & Bengio, Y. (2013). Maxout networks. In Proceedings of 30th International Conference on Machine Learning (ICML 2013).Goodfellow, I. J., Warde-Farley, D., Mirza, M., Courville, A., & Bengio, Y. (2013). Maxout networks. In Proceedings of 30th International Conference on Machine Learning (ICML 2013).Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313, 504-507.Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. Science, 313, 504-507.Hinton, G. E., Srivastave, N., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaption of feature detectors. arXiv 1207.0580.Hinton, G. E., Srivastave, N., Krizhevsky, A., Sutskever, I. & Salakhutdinov, R. R. (2012). Improving neural networks by preventing co-adaption of feature detectors. arXiv 1207.0580.Springenberg J. T., & Riedmiller M. (2014). Improving deep neural networks with probabilistic maxout units. In Proceedings of 3rd International Conference on Learning Representations (ICLR 2014).Springenberg J. T., & Riedmiller M. (2014). Improving deep neural networks with probabilistic maxout units. In Proceedings of 3rd International Conference on Learning Representations (ICLR 2014).Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. M. S. diss., Department of Computer Science, University of Toronto.Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. M. S. diss., Department of Computer Science, University of Toronto.Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS 2012).Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS 2012).LeCun, Y., Bottou, L., Bengio, Y & Haffner, P. (1998). Gradient-based learning applied to document recognition. In Proceedings of the IEEE.LeCun, Y., Bottou, L., Bengio, Y & Haffner, P. (1998). Gradient-based learning applied to document recognition. In Proceedings of the IEEE.Ledoux, M., & Talagrand, M. (1991). Probability in banach spaces. Springer.Ledoux, M., & Talagrand, M. (1991). Probability in banach spaces. Springer.Lin, M., Chen, Q., & Yan S. (2014). Network in network. In Proceedings of 3rd International Conference on Learning Representations (ICLR 2014).Lin, M., Chen, Q., & Yan S. (2014). Network in network. In Proceedings of 3rd International Conference on Learning Representations (ICLR 2014).Mackay, D. C. (1995). Probable networks and plausible predictions: A review of practical Bayesian methods for supervised neural networks. In Bayesian Methods for Backpropagation Networks.Mackay, D. C. (1995). Probable networks and plausible predictions: A review of practical Bayesian methods for supervised neural networks. In Bayesian Methods for Backpropagation Networks.Scherer, D., Muller, A., & Behnke, S. (2010). Evaluation of pooling operations in convolutional architectures for object recognition. In Proceedings of 20th International Conference on Artificial Neural Networks (ICANN 2010).Scherer, D., Muller, A., & Behnke, S. (2010). Evaluation of pooling operations in convolutional architectures for object recognition. In Proceedings of 20th International Conference on Artificial Neural Networks (ICANN 2010).Srivastava, N., Hinton. G. E., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15, 1929-1958.Srivastava, N., Hinton. G. E., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15, 1929-1958.Vinod, N., & Hinton, G. E. (2010). Rectified linear units improve restricted Boltzmann machines. In Proceedings 27th of International Conference on Machine Learning (ICML 2010).Vinod, N., & Hinton, G. E. (2010). Rectified linear units improve restricted Boltzmann machines. In Proceedings 27th of International Conference on Machine Learning (ICML 2010).Wan, L., Zeiler, M. D., Zhang, S., LeCun, Y., & Fergus, R. (2013). Regularization of neural networks using DropConnect. In Proceedings of 30th International Conference on Machine Learning (ICML 2013).Wan, L., Zeiler, M. D., Zhang, S., LeCun, Y., & Fergus, R. (2013). Regularization of neural networks using DropConnect. In Proceedings of 30th International Conference on Machine Learning (ICML 2013).Warde, F. D., Goodfellow, I.J., Courville, A., & Bengio, Y. (2014). An empirical analysis of dropout in piecewise linear networks. In Proceedings of 3rd International Conference on Learning Rep­ resentations (ICLR 2014).Warde, F. D., Goodfellow, I.J., Courville, A., & Bengio, Y. (2014). An empirical analysis of dropout in piecewise linear networks. In Proceedings of 3rd International Conference on Learning Rep­ resentations (ICLR 2014).Wager, S., Wang, S., & Liang, P. (2013). Dropout training as adaptive regularization. In Advances in Neural Information Processing Systems (NIPS 2013).Wager, S., Wang, S., & Liang, P. (2013). Dropout training as adaptive regularization. In Advances in Neural Information Processing Systems (NIPS 2013).Zeiler, M. D., & Fergus R. (2013). Stochastic pooling for regularization of deep convolutional neural networks. In Proceedings of 2nd International Conference on Learning Representations (ICLR 2013).Zeiler, M. D., & Fergus R. (2013). Stochastic pooling for regularization of deep convolutional neural networks. In Proceedings of 2nd International Conference on Learning Representations (ICLR 2013).

The authors declare that there are no conflicts of interest present.