Computer vision system in mobile devices
Abstract
The article presents the results of designing a computer vision system using deep learning to control supervised systems. The results are based on the use of convolutional neural network and aimed at solving problems, both directly and indirectly related to the solution of environmental problems. The principle is described and the basic model of using computer vision technology in actual projects is shown: 1) social distancing norm compliance in COVID pandemic;
2) non-contact manipulator to control technical objects. The COVID-19 Social Distancing Project was developed as a means of automatic control of social distance in crowded places. The article describes the principle of operation of the detector, based on computer vision technology, and makes it possible to monitor compliance with the norm of physical distance. To work the detector requires a computer with the necessary software and the results of images from a video camera. The operation of the detector has been tested under real conditions; the test results are satisfactory. The error in determining the distance did not exceed 9 %. It is assumed that such a device can be used by various services, including sanitary services, to monitor the compliance of the population with the recommendations of the health care system regarding the rules of distancing. The «Contactless Manipulator» project was designed for remote control of objects when working in hard-to-reach and hazardous places (nuclear power plants, toxic waste processing), as well as for people with limited physical abilities. Three functions of “Noncontact manipulator” were implemented in the work – tracking of key points of hand, interception of computer mouse manipulator actions, and control of additional computer functions. The work of the manipulator was also tested in real conditions, the degree of reliability of the results is high. In the future, the manipulator is supposed to be used in a mobile hardware-software complex designed for image processing with subsequent data analysis. The expected fields of practical application of the complex are ecological monitoring, investigation of soil erosion processes, observation of vegetation changes, analysis of plant diseases, waste sorting, and human ecology. The resources of modern information technologies, implemented through the presented projects, demonstrate the possibility of replacing the production of some material devices with their virtual counterparts, safe in terms of ecology.
References
- Bishop CM. Pattern Recognition and Machine Learning. New York: Springer-Verlag; 2006. 738 p.
- Sense Time: Artificial Intelligence (AI) Software Provider [Internet; cited 2022 August 25]. Available from: www.sensetime.com.
- Tkachenko SV, Smirnova TV. Sovershenstvovaniye struktury iskusstvennykh neyronnykh setey na baze variatsionnoy optimizatsii [Improving of the structure of artificial neural networks based on variational optimization]. In: Sakharovskie chteniya 2020: ekologicheskie problemy XXI veka. Part 2. p. 322–326. DOI:10.46646/SAKH-2020-2-322-326. Russian.
- Rosenblatt F. Printsipy neyrodinamiki. Pertseptrony i teoriya mekhanizmov mozga [Principles of neurodynamics. Perceptrons and the theory of brain mechanisms]. Moscow: Mir; 1965. 302 p. Russian.
- Duch, J, Hazan E, Singer Y. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research. 2011;5:2121–2159.
- Popular Datasets Over Time [Internet; cited 2022 March 25]. Available from: www.kaggle.com/code.
- Brink H, Richards J, Feverolf M. Mashinnoye obucheniye [Machine learning]. Saint Petersburg: Piter; 2018. 336 p. Russian.
- Wang Ch, Bochkovskiy A., Liao M. Scaled-YOLOv4: Scaling Cross Stage Partial Network [Internet; cited 2022 August 25]. Available from: https://arxiv.org/abs/2011.08036v2.
- Boominathan L, Kruthiventi S, Babu R. CrowdNet: A Deep Convolutional Network for Dense Crowd Counting. Proceedings of the 2016 ACM on Multimedia Conference. Amsterdam, Netherlands. 2016: 640–644. DOI:10.1145/2964284.2967300
- Cao Z, Xu X, Biao Z, Zhou M, Li Q. Real-time gesture recognition based on feature recalibration network with multi-scale information. Neurocomputing. 2019;347:119–130.
- MediaPipe Python Framework [Internet; cited 2022 February 12] Available from: https://google.github.io/mediapipe/solutions/solutions.html
- Shakiryanov ED. Komp’yuternoye zreniye na Python [Computer vision in Python]. Moscow: Knowledge Lab; 2021. 163 p. Russian.
