KEYWORD SPOTTING SYSTEM WITH NANO 33 BLE SENSE USING EMBEDDED MACHINE LEARNING APPROACH
DOI:
https://doi.org/10.11113/jurnalteknologi.v85.18744Keywords:
Edge impulse, Keyword spotting, TinyML, MFCC, CNNAbstract
Due to the obvious advancement of artificial intelligence, keyword spotting has become a fast-growing technology that was first launched a few years ago by hidden Markov models. Keyword spotting is the technique of finding terms that have been pre-programmed into a machine learning model. However, because the keyword spotting system model will be installed on a small and resource-constrained device, it must be minimal in size. It is difficult to maintain accuracy and performance when minimizing the model size. We suggested in this paper to develop a TinyML model that responds to voice commands by detecting words that are utilized in a cascade architecture to start or control a program. The keyword detection machine learning model was built, trained, and tested using the edge impulse development platform. The technique follows the model-building workflow, which includes data collection, preprocessing, training, testing, and deployment. 'On,' 'Off,' noise, and unknown databases were obtained from the Google speech command database V1 and applied for training and testing. The MFCC was used to extract features and CNN was used to generate the model, which was then optimized and deployed on the microcontroller. The model's evaluation represents an accuracy of 84.51% based on the datasets. Finally, the KWS was successfully implemented and assessed on Arduino Nano 33 BLE Sense for two studies in terms of accuracy at three different times and by six different persons.
References
B. Qolomany et al. 2019. Leveraging Machine Learning and Big Data for Smart Buildings: A Comprehensive Survey. IEEE Access. 7: 90316-90356. Doi: 10.1109/ACCESS.2019.2926642.
C. Banbury et al. 2020. MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers. [Online]. Available: http://arxiv.org/abs/2010.11267.
A. D. I. A. Kadir, A. Al-Haiqi, and N. M. Din. 2021. A Dataset and TinyML Model for Coarse Age Classification Based on Voice Commands. 15th IEEE Malaysia Int. Conf. Commun. Emerg. Technol. IoT 5G, MICC 2021 - Proc. 75-80. Doi: 10.1109/MICC53484.2021.9642091.
V. Janapa Reddi et al. 2022. Widening Access to Applied Machine Learning with TinyML. Harvard Data Sci. Rev. 1-20. Doi: 10.1162/99608f92.762d171a.
S. Choi et al. 2019. Temporal Convolution for Real-time Keyword Spotting on Mobile Devices.Proc. Annu. Conf. Int. Speech Commun. Assoc. INTERSPEECH. 3372-3376. Doi: 10.21437/Interspeech.2019-1363.
Y. Zhang, N. Suda, L. Lai, and V. Chandra. 2017. Hello Edge: Keyword Spotting on Microcontrollers. 1-14. [Online]. Available: http://arxiv.org/abs/1711.07128.
A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash. 2015. Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications. IEEE Commun. Surv. Tutorials. 17(4): 2347-2376. Doi: 10.1109/COMST.2015.2444095.
F. García-Vázquez, H. A. Guerrero-Osuna, G. Ornelas-Vargas, R. Carrasco-Navarro, L. F. Luque-Vega, and E. Lopez-Neri. 2021. Design and Implementation of the e-switch for a Smart Home. Sensors. 21(11): 1-17. Doi: 10.3390/s21113811.
L. Filipe, R. S. Peres, and R. M. Tavares. 2021. Voice-Activated Smart Home Controller Using Machine Learning. IEEE Access. 9(May): 66852-66863. Doi: 10.1109/ACCESS.2021.3076750.
P. E. Novac, G. B. Hacene, A. Pegatoquet, B. Miramond, and V. Gripon. 2021. Quantization and Deployment of Deep Neural Networks On Microcontrollers. Sensors. 21(9): 1-32. Doi: 10.3390/s21092984.
P. M. Sørensen, B. Epp, and T. May. 2020. A Depthwise Separable Convolutional Neural Network for Keyword Spotting on an Embedded System. Eurasip J. Audio, Speech, Music Process. 2020(1). Doi: 10.1186/s13636-020-00176-2.
M. G. Ulkar and O. E. Okman. 2021. Ultra-Low Power Keyword Spotting at the Edge. [Online]. Available: http://arxiv.org/abs/2111.04988.
C. Shan, J. Zhang, Y. Wang, and L. Xie. 2018. Attention-based End-to-end Models for Small-footprint Keyword Spotting.Proc. Annu. Conf. Int. Speech Commun. Assoc. INTERSPEECH. 2018: 2037-2041. Doi: 10.21437/Interspeech.2018-1777.
C. De la Parra, A. Guntoro, and A. Kumar. 2020. Improving Approximate Neural Networks for perception Tasks Through Specialized Optimization. Futur. Gener. Comput. Syst. 113: 597-606. Doi: 10.1016/j.future.2020.07.031.
Y. Wei, Z. Gong, S. Yang, K. Ye, and Y. Wen. 2022. EdgeCRNN: An Edge-computing Oriented Model of Acoustic Feature Enhancement for Keyword Spotting. J. Ambient Intell. Humaniz. Comput. 13(3): 1525-1535. Doi: 10.1007/s12652-021-03022-1.
S. Yang, Z. Gong, K. Ye, Y. Wei, Z. Huang, and Z. Huang. 2020. EdgeRNN: A Compact Speech Recognition Network with Spatio-Temporal Features for Edge Computing. IEEE Access. 8: 81468-81478. Doi: 10.1109/ACCESS.2020.2990974.
J. S. P. Giraldo, V. Jain, and M. Verhelst. 2021. Efficient Execution of Temporal Convolutional Networks for Embedded Keyword Spotting. IEEE Trans. Very Large Scale Integr. Syst. 29(12): 2220-2228. Doi: 10.1109/TVLSI.2021.3120189.
P. Warden. 2018. Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition. [Online]. Available: http://arxiv.org/abs/1804.03209.
R. Tang and J. Lin. 2018. Deep Residual Learning for Small-Footprint Keyword Spotting Raphael Tang David R. Cheriton School of Computer Science University of Waterloo. 2018 IEEE Int. Conf. Acoust. Speech Signal Process. 5484-5488,
R. Tang and J. Lin. 2017. Honk: A PyTorch Reimplementation of Convolutional Neural Networks for Keyword Spotting. 1-3. [Online]. Available: http://arxiv.org/abs/1710.06554.
T. Mo, Y. Yu, M. Salameh, D. Niu, and S. Jui. 2020. Neural Architecture Search for Keyword Spotting. Proc. Annu. Conf. Int. Speech Commun. Assoc. INTERSPEECH. 1982-1986. Doi: 10.21437/Interspeech.2020-3132.
P. P. Ray. 2022. A Review on TinyML: State-of-the-art and prospects. J. King Saud Univ. - Comput. Inf. Sci. 34(4): 1595-1623. Doi: 10.1016/j.jksuci.2021.11.019.
Downloads
Published
Issue
Section
License
Copyright of articles that appear in Jurnal Teknologi belongs exclusively to Penerbit Universiti Teknologi Malaysia (Penerbit UTM Press). This copyright covers the rights to reproduce the article, including reprints, electronic reproductions, or any other reproductions of similar nature.
