Camera Image Processing on ESP32 Microcontroller with Help of Convolutional Neural Network

Authors

DOI:

https://doi.org/10.18372/1990-5548.72.16939

Keywords:

transfer learning, microcontrollers, image classification, ESP32

Abstract

This paper analyzes a common ESP32 microcontroller with a built-in camera for image classification tasks using a convolutional neural network. ESP32 is commonly used in IoT devices to read data and control sensors, so its computing power is not significant, which has a positive effect on the cost of the device. The prevalence of ultra-low power embedded devices such as ESP32 will allow the widespread use of artificial intelligence built-in IoT devices. The duration of photographing and photo processing is obtained in the paper, as this can be a bottleneck of the microcontroller, especially together with machine learning algorithms. Deployed convolutional neural network, pre-trained on another device, MobileNet architecture on microcontroller and proved that ESP32 capacity is sufficient for simultaneous operation of both the camera and convolutional neural network.

Author Biographies

Victor Sineglazov , National Aviation University, Kyiv

Doctor of Engineering Science. Professor. Head of the Department

Aviation Computer-Integrated Complexes Department

Faculty of Air Navigation Electronics and Telecommunications

Volodymyr Khotsyanovsky , National Aviation University, Kyiv

Post-graduate Student

Faculty of Air Navigation, Electronics and Telecommunications

References

D. Schweizer, M. Zehnder, H. Wache, H. Witschel, D. Zanatta and M. Rodriguez. Using Consumer Behavior Data to Reduce Energy Consumption in Smart Homes: Applying Machine Learning to Save Energy without Lowering Comfort of Inhabitants, 2015. https://doi.org/10.1109/ICMLA.2015.62

Liciotti, Daniele & Bernardini, Michele & Romeo, Luca & Frontoni, Emanuele. A Sequential Deep Learning Application for Recognising Human Activities in Smart Homes, Neurocomputing, 2019, pp. 396. https://doi.org/10.1016/j.neucom.2018.10.104

Keras documentation: MobileNet, MobileNetV2, and MobileNetV3. Keras: the Python deep learning API. https://keras.io/api/applications/mobilenet/

Jian Mao, Qixiao Lin, Jingdong Bian. Application of learning algorithms in smart home IoT system security, 2018. https://doi.org/10.3934/mfc.2018004

L. Y. Pratt, Discriminability-based transfer between neural networks. NIPS Conference: Advances in Neural Information Processing Systems, 1992.

STM32Cube.AI: Convert neural networks into optimized code for STM32. ST life.augmented Blog. https://blog.st.com/stm32cubeai-neural-networks/

L. Lai,, N. Suda, I. V. Chandra, CMSIS-NN: efficient neural network kernels for arm cortex-M CPUs. Comput. https://arxiv.org/abs/1801.06601, 2018

General vision, Presentation of the Curie Neurons on Arduino/Genuino101,https://www.general-vision.com/publications/PR_CurieNeuronsPresentation.pdf

Allan, A. Getting started with the NVIDIA jetson nano developer kit, 2019

M5-docs. https://docs.m5stack.com/en/unit/m5camera

Pseudostatic (random-access) memory (PSRAM) | JEDEC. https://www.jedec.org/standards-documents/dictionary/terms/pseudostatic-random-access-memory-psram, 2019

Micros() – arduino reference. Arduino - Home. https://www.arduino.cc/reference/en/language/functions/time/micros/

Stephen Johnson. Stephen Johnson on Digital Photography. O'Reilly. ISBN 0-596-52370-X , 2006

Contributors to Wikimedia projects. ImageNet - Wikipedia. Wikipedia, the free encyclopedia. https://en.wikipedia.org/wiki/ImageNet

Intro to machine learning with edge impulse - silicon labs. (2020). Silicon Labs. https://www.silabs.com/support/training/intro-machine-learning-with-edge-impulse/intro-machine-learning-with-edge-impulse-presentation

Tensorflow.(2022). models/research/slim/nets/ mobilenet at master tensorflow/models. GitHub.(2022),https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet

NetAdapt: Platform-aware neural network adaptation for mobile applications. (2018). arXiv.org. https://arxiv.org/abs/1804.03230

Downloads

Published

2022-11-20

How to Cite

Sineglazov , V., & Khotsyanovsky , V. (2022). Camera Image Processing on ESP32 Microcontroller with Help of Convolutional Neural Network. Electronics and Control Systems, 2(72), 26–31. https://doi.org/10.18372/1990-5548.72.16939

Issue

Vol. 2 No. 72 (2022)

Section

AUTOMATION AND COMPUTER-INTEGRATED TECHNOLOGIES

License

The scientific journal “Electronics and control systems” adheres to the principles of Open Access and provides free, immediate, and permanent access to all published materials without financial, technical, or legal barriers for readers.

All articles are published in Open Access under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Copyright

Authors who publish their works in the journal “Electronics and control systems”:

  • retain the copyright to their publications;

  • grant the journal the right of first publication of the article;

  • agree to the distribution of their materials under the CC BY 4.0 license;

  • have the right to reuse, archive, and distribute their works (including in institutional and subject repositories), provided that proper reference is made to the original publication in the journal.