Edge AI on embedded devices: what is possible in 2026
Hephaestus Technology7 min read

Edge AI is machine learning inference that runs on the device itself, such as a sensor, camera or appliance, instead of sending data to the cloud. With edge AI embedded in microcontrollers in 2026, small models for keyword spotting, vibration and anomaly detection, gesture recognition and simple image classification run on low-cost, low-power microcontrollers. Larger vision and language models still need more capable processors, an attached neural accelerator or the cloud.
Key takeaways
- Edge AI means running a trained model on the device, which cuts latency, bandwidth and privacy risk.
- Microcontroller runtimes such as LiteRT for Microcontrollers fit in tens of kilobytes; Google states the core runtime fits in 16 KB on an Arm Cortex-M3.
- Neural accelerators beside microcontroller cores, such as Arm's Ethos-U55, make heavier models practical on low-power chips.
- Training still happens off the device; the device runs inference only, and model updates need a secure firmware update path.
- Start with the smallest model that meets accuracy targets and measure power, memory and latency on real hardware.
What is edge AI?
Edge AI is the practice of running machine learning inference locally on an embedded device rather than on a remote server. The model is trained on a computer or in the cloud, then compressed and deployed to the device, which uses it to classify or predict from its own sensor data. When this happens on very small microcontrollers, it is often called TinyML.
Three benefits drive adoption. Decisions are faster because no round trip to the cloud is needed. Bandwidth and connectivity costs fall because the device sends results, not raw data. And privacy improves because audio, images or personal measurements can stay on the device.
How does edge AI on embedded devices work?
The workflow has five stages, and only the last runs on the device.
- Collect data from the real sensors in real conditions, labelled for the task.
- Train a model on a computer or in the cloud using a standard framework.
- Optimise the model for the target, typically by quantising weights to 8-bit integers and pruning layers.
- Convert and compile it for an embedded runtime.
- Deploy and run inference on the device, alongside the normal firmware.
Google's LiteRT for Microcontrollers is a widely used runtime for the last step. Google states that the core runtime "just fits in 16 KB on an Arm Cortex M3", that it needs no operating system, standard C or C++ libraries or dynamic memory allocation, and that it has been tested heavily on Arm Cortex-M processors and ported to ESP32. Its documented limits matter too: it supports only a subset of TensorFlow operations and a limited set of devices, and on-device training is not supported.
Tool platforms aim to shorten the path. Edge Impulse's EON Compiler turns trained models into C++ source and, according to Edge Impulse, uses 25 to 65% less RAM and 10 to 35% less flash than LiteRT for the same model. Treat vendor comparisons as a starting point and benchmark your own model.
What can edge AI embedded in a microcontroller do?
The realistic boundary in 2026 is set by memory, compute and power. The table groups common tasks by the class of hardware they typically need. It is indicative only; your model size and accuracy target decide where you land.
| Task | Typical hardware class | Notes |
|---|---|---|
| Vibration or current anomaly detection | Small microcontroller | Low data rate, small models |
| Keyword spotting, wake words | Microcontroller with DSP or vector extensions | Always-on audio at low power |
| Gesture or activity recognition | Microcontroller | Accelerometer and gyroscope data |
| Simple image classification or person detection | Microcontroller with vector extensions or a microNPU | Low resolution, low frame rate |
| Object detection on live video | Microcontroller with a dedicated NPU, or an application processor | Memory for frames is the limit |
| Large language or generative models | Application processor with accelerator, or cloud | Beyond typical microcontrollers |
Chip makers publish their own capability claims. Espressif's ESP32-S3 adds vector instructions to accelerate neural network and signal processing workloads and has 512 KB of internal SRAM. Arm says its Cortex-M55 delivers up to 15 times the ML performance of existing Cortex-M processors, and that pairing it with the Ethos-U55 microNPU gives a 480 times uplift over existing Cortex-M based systems, with up to 90% energy reduction for AI applications. These figures come without a published benchmark, so read them as direction, not as a promise for your workload.
FPGAs are another route for always-on tasks. Lattice reports key-phrase detection on its iCE40 UltraPlus consuming less than 1 mW of average power. Our FPGA vs microcontroller vs ASIC guide explains when that route makes sense.
What does edge AI cost to add to a product?
There is no standard price, because the cost is mostly engineering time rather than silicon. The main cost items are:
- Data collection and labelling, which is often the largest effort and is specific to your sensors.
- Model development and optimisation to meet memory, latency and accuracy targets together.
- Hardware headroom: a chip with more memory or an accelerator raises unit cost on every device.
- Validation across devices, environments and edge cases, repeated after each model update.
- Tooling: open-source runtimes are free to use, while commercial platforms may charge by plan.
A useful rule is to prove that a model meets accuracy targets on a development kit before choosing the production chip, so that you size hardware from evidence.
What are the common pitfalls?
Most edge AI projects fail on data and integration, not on the model architecture.
- Training on lab data only. Models trained on clean recordings often fail in noisy, real environments. Collect data from the target hardware where it will be used.
- Ignoring memory for everything else. Firmware, radio stacks and buffers compete with the model for RAM.
- No update path. Models need improving after launch. Plan secure over-the-air updates, which connected products in the EU need anyway under the radio cybersecurity rules and, from 11 December 2027, the Cyber Resilience Act's main obligations.
- Measuring accuracy but not power. Always-on inference can dominate battery life; measure current in each mode.
- Overlooking regulation. Most simple embedded models will not be high-risk systems, but some uses, such as safety components or biometric functions, can bring obligations under the EU AI Act. Our guide does the EU AI Act apply to my small business? helps you check.
How should you start an edge AI project?
Start narrow and measure everything on real hardware.
- Define one task with a clear accuracy, latency and power target.
- Record real data with the target sensors in the target environment.
- Train a small baseline model and test it on a development kit.
- Measure RAM, flash, latency and current, then choose production hardware with headroom.
- Build the firmware update path before launch.
Edge AI is one feature of a wider product; our guide on how to build an IoT product shows where it fits in the development process.
Frequently asked questions
What is the difference between edge AI and cloud AI?
Edge AI runs the model on the device, while cloud AI sends data to remote servers for processing. Edge AI gives lower latency, lower bandwidth use and better privacy, while cloud AI can run much larger models. Many products combine both, with the device filtering events and the cloud handling heavier analysis.
Can a microcontroller run a neural network?
Yes, for small models. Google states that the LiteRT for Microcontrollers core runtime fits in 16 KB on an Arm Cortex-M3 and runs many basic models. Tasks such as keyword spotting, anomaly detection and gesture recognition are well within reach of modern microcontrollers.
Can embedded devices train models on the device?
Generally not with mainstream microcontroller runtimes. LiteRT for Microcontrollers documents that on-device training is not supported. Models are trained elsewhere and deployed to the device through firmware updates.
What is TinyML?
TinyML is a common term for machine learning on very small, low-power devices such as microcontrollers. It is a subset of edge AI focused on devices with kilobytes to a few megabytes of memory and milliwatt power budgets.
Do I need a neural processing unit for edge AI?
Not always. Many audio, motion and sensor tasks run on a standard microcontroller or one with vector extensions. An NPU becomes worthwhile for vision, higher frame rates or tight power budgets where the processor alone is too slow or too power-hungry.
Exploring an edge AI feature for your device? Book a free 30-minute call with our embedded team.
Cover photo by ThisIsEngineering on Pexels.