“Machine Learning (ML) is an excellent tool for solving problems involving pattern recognition, where ML algorithms can turn messy raw data into usable signals. The basic process is to generate a model based on data, and then use the model to predict the output, so as to achieve the purpose of learning, reasoning and decision-making without human interaction. However, the need for high-performance computing resources limits many ML applications to the cloud.
Author: Doctor M
Machine Learning (ML) is an excellent tool for solving problems involving pattern recognition, where ML algorithms can turn messy raw data into usable signals. The basic process is to generate a model based on data, and then use the model to predict the output, so as to achieve the purpose of learning, reasoning and decision-making without human interaction. However, the need for high-performance computing resources limits many ML applications to the cloud. In other words, only cloud data center-level performance can meet the computing power requirements of ML. What is exciting for the industry is that with the continuous advancement of algorithm design and microprocessor architecture, it is becoming possible to run complex machine learning workloads on the smallest microcontrollers (MCUs).
Running machine learning models on embedded devices is often referred to as embedded machine learning. Machine learning in embedded devices has many benefits:
1. It eliminates the need to transfer and store data on cloud servers, thereby reducing the data and privacy leakage involved in transferring data.
2. It strengthens the protection of intellectual property, personal data and trade secrets.
3. The execution of the ML model can effectively avoid the need to transmit data to the cloud server, saving valuable bandwidth and network resources.
4. The use of embedded devices based on ML models is sustainable and has a much lower carbon footprint. Because the microcontrollers used in the device are all low energy consumption.
5. Embedded systems are more efficient than cloud-based systems, and ML models on edge devices can respond in real time.
TinyML: New Business Opportunities for MCU Manufacturers
The initial success of deep learning models was largely attributed to large servers with large amounts of memory and GPU clusters. While cloud-based deep learning has been very successful, it is not suitable for all situations, as many applications require on-device inference. Most of today’s AI applications are based on machine learning technology. If machine learning models can run smoothly on resource-constrained devices, it will surely open a technological door for many emerging applications, which is also the edge in recent years. A big reason why computing and embedded machine learning are getting hotter.
Embedded machine learning is a field of machine learning, and these models, known as Tiny Machine Learning (TinyML), are ideal for edge devices with limited memory and processing power, and with non-existent or limited internet connectivity. Now, TinyML has become a rapidly developing field in machine learning. Through the organic combination of hardware, algorithms and software, it can complete the analysis of sensor data with power of mW and below, and realize the process of embedding AI on small hardware.
While TinyML is a new concept, applying machine learning to smart devices is nothing new. For example, most smartphones have some kind of neural network, music recognition, and many camera modes (such as night vision and portrait modes) are examples of relying on embedded deep learning. These are where TinyML comes in and takes Edge AI a step further.
The Edge AI chipset brings AI inside countless devices, including mobile devices, cars, smart speakers, and wireless cameras. However, these devices are often unable to fully utilize all the data generated due to the difficulty in supporting high computing performance and high data throughput. The advent of TinyML made it possible to run machine learning models on MCUs. These MCUs are generally inexpensive and small in size, with built-in low-power memory (SRAM) of several hundred kilobytes and several megabytes of storage space, low power consumption, and a wide range of applications. The primary goal of TinyML chipsets is to address cost and energy efficiency. They enable data analytics performance on hardware with low power consumption, low processing power and small memory through software designed for small inference workloads, a technology that has the potential to completely Transform the future of IoT.
Today, there are more than 250 billion active Internet of Things (IoT) devices worldwide, with an expected annual growth rate of 20%. These devices collect huge amounts of data every day, and processing this data in the cloud presents considerable challenges. Now, TinyML promises to bridge the gap between edge hardware and device intelligence. McKinsey researchers predict that the IoT industry will have a potential economic impact of $4-11 trillion by 2025, with manufacturing being the largest vertical at $1.2-3.7 trillion.
In its new white paper, TinyML: The Next Big Opportunity in Tech, market consultancy ABI Research predicts that the number of IoT connections will nearly triple to 23.6 billion between 2021 and 2026. Every new connection represents an opportunity to leverage AI and machine learning, and TinyML technology will be key to capturing this opportunity for businesses. As a result, ABI expects shipments of TinyML devices to increase from 15.2 million in 2020 to 2.5 billion in 2030.
Famous factories gather on TinyML track
Since the birth of TinyML, the innovation market has been hot, and many products have attracted great attention. For example: industrial AI smart camera based on NVIDIA Jetson Xavier NX, the industry’s first industrial smart camera launched by Adlink, the camera is based on NVIDIA’s Jetson Xavier NX, with high performance, small size, and efficiency about the previous generation Ten times the product, is a compact, reliable, and powerful Edge AI application that opens the door to AI innovation in manufacturing, logistics, healthcare, agriculture, and many other business sectors.
TinyML focuses on optimizing machine learning workloads so that they can run on low-power microcontrollers. The proliferation of TinyML will lead to the expansion of Edge AI beyond traditional key markets, with more end users benefiting from smart connected sensors and IoT devices based on sound waves, temperature, pressure, vibration and other data sources. Today, TinyML is at the intersection of machine learning and embedded IoT, with the potential to disrupt many industries. The potential applications of TinyML are almost limitless, such as: industrial robots that can predict when service is needed, sensors that can monitor crops for the presence of harmful insects, in-store shelves that can ask for restocking when stocks dwindle, and the ability to maintain privacy while maintaining privacy. Medical monitors that track vital signs.
Audio analysis, pattern recognition, and speech human-machine interfaces are the areas where TinyML is most used today. NXP’s EdgeReady MCU-based 3D face recognition solution leverages the i.MX RT117F crossover MCU to help developers quickly add 3D face recognition and advanced liveness detection to their products, even outdoors The device also works well under lighting conditions. The solution’s 3D liveness detection capabilities also identify and defend against fraud using photos or 3D models, using only a high-performance 3D structured light camera module (SLM) and an optional low-cost CMOS sensor-based RGB camera. Expensive, power-hungry, Linux-based MPU.
The i.MX RT1170 used in the solution is a cross-border MCU. It uses a Cortex-M7 core with a main frequency of 1GHz and an Arm Cortex-M4 with a main frequency of 400MHz. It has excellent computing power, multiple media functions and real-time Function. Face recognition and liveness detection can be performed completely offline on the i.MX RT117F MCU, eliminating the need for the cloud, which not only eliminates latency issues, but also effectively protects consumer privacy.
Figure 1: i.MX RT117F 3D face recognition hardware block diagram (Source: NXP)
Vision, motion and gesture recognition are also important application areas for TinyML. ST’s AI solutions are primarily based on the STM32 portfolio, with pre-trained neural networks embedded developers can port, optimize and validate on any Cortex M4, M33 and M7 based STM32. STM32CubeMX is a graphical tool that makes it very easy to configure STM32 microcontrollers and microprocessors through a step-by-step process, as well as generate corresponding initialization C code for Arm Cortex-M cores or a specific Linux device tree for Arm Cortex-A cores .
STM32Cube.AI is an AI extension package of STM32CubeMX, based on which designers can develop their own AI products more efficiently. FP-AI-VISION1 belongs to a functional package (FP) of STM32Cube and contains computer vision application examples based on convolutional neural network (CNN).
Currently, FP-AI-VISION1 includes three CNN-based image classification application examples:
・ Food recognition applications running on color (RGB 24-bit) frame images;
・ Person presence detection application running on color (RGB 24-bit) frame images;
・ Person presence detection application running on grayscale (8-bit) frame images.
Now, the TinyML computer vision solution provided by ST can identify 18 common foods, and can also implement human presence detection, or count the number of people in the scene based on object detection models.
Figure 2: The execution flow of the food recognition model (Image source: ST)
With the expansion of the IoT market, the amount of data at the edge has grown rapidly, and AIoT powered by TinyML has emerged. According to analysis by Markets and Markets, the AIoT market size in 2019 was approximately US$5.1 billion and is expected to grow to US$16.2 billion by 2024, with a compound annual growth rate (CAGR) of 26%. The main role of AIoT is to empower connected devices with machine learning capabilities to perform complex intelligent operations.
The goal of ModusToolbox ML, launched by Infineon in June 2021, is to enable the company’s PSoC MCU with deep learning capabilities. ModusToolbox ML is a new feature based on ModusToolbox software that provides developers with the middleware, software libraries and specialized tools needed for deep learning-based ML models. ML seamlessly integrates with existing software frameworks in ModusToolbox, making it easy to integrate into secure AIoT systems. ModusToolbox ML allows developers to deploy directly to PSoC MCU using their preferred deep learning framework such as TensorFlow. In addition, ML helps engineers optimize models for embedded platforms, reduce platform complexity, and provide capabilities with performance verification based on test data.
Figure 3: Infineon PSoC6 MCU internal architecture (Source: Infineon)
To help developers quickly add local intelligence to their IoT designs, Infineon chose to partner with SensiML. SensiML, a subsidiary of QuickLogic, brings cutting-edge software to the market to enable AI in ultra-low-power IoT devices, and the company’s flagship SensiML Analytics Toolkit provides an end-to-end development platform covering data acquisition, tags, algorithms and firmware Automatically generated and tested. SensiML’s “Analytics Toolkit” Edge AI development software, now available with Infineon ModusToolbox, provides developers with a quick and easy way to record data from Infineon XENSIV sensors, create complex AI/ML-based models, and Run custom applications on PSoC6 MCU.
Growing TinyML Ecosystem
Founded in 2019, the TinyML community is a community of researchers and industry engineers working to bring ML capabilities to microcontroller devices. TinyML consists of machine learning architectures, techniques, tools, and methodologies capable of performing analysis on various sensing modalities (visual, audio, motion, chemical, and others) on low-power target devices, mostly battery-operated devices. Evgeni Gousev, one of TinyML’s founders, believes: “We are in the midst of a digital transformation revolution, and TinyML performs on-device machine intelligence and analytics at low cost, combined with inherent privacy features, providing great energy savings.”
TinyML will be pervasive in many industries, and it will impact almost every industry including: retail, healthcare, transportation, health, agriculture, fitness, and manufacturing. At the same time, industry players quickly recognized the value of TinyML and moved quickly to create a supportive ecosystem.
Arm is a strong supporter of TinyML and a leader in TinyML technology. With more than 180 billion Arm-based chips shipped, its IP, tools and more than 1,100 software partners have built billions of tiny smart IoT devices.
Today, Arm® Cortex®-M series MCUs have become the most widely used platform for TinyML, they can perform real-time computations quickly and efficiently, are inexpensive, have high reliability, respond quickly, and consume very little power. The Cortex-M55 processor is Arm’s most AI-capable Cortex-M processor, offering enhanced, power-efficient DSP and ML performance. The Ethos-U55 NPU is a new ML processor, called the microNPU, specifically designed to accelerate ML inference in area-constrained embedded and IoT devices. The Ethos-U55, combined with the AI-enabled Cortex-M55 processor, enables a 480x improvement in ML performance over existing Cortex-M-based systems.
In fact, in early 2021, the Raspberry Pi released its first microcontroller board, one of the most affordable development boards on the market at just $4. Called the Raspberry Pi Pico, it’s based on the RP2040 MCU with a powerful dual-core Cortex-M0+ processor inside, capable of running TensorFlow Lite Micro, and soon we’ll see various TinyML use cases for the board.
TinyML is a lifesaver for decision makers drowning in massive amounts of data, making the most of data at the edge so people can get the right information faster. In addition, TinyML also improves on widespread privacy concerns by processing data on the device and transmitting only critical information.
Next, we will see a new world with trillions of smart devices powered by TinyML technology that sense, analyze and act autonomously, and will create a healthier and more sustainable environment for us.