Two years ago, if you wanted to run a neural network at the edge, you were choosing between a cloud call, a high-end Raspberry Pi struggling through inference in software, or a dedicated AI accelerator that cost as much as your entire device budget. In 2026, that calculus has changed. Neural Processing Units are appearing in mainstream IoT SoCs at price points where they’re a default rather than a premium. That changes how you approach device selection for any project that involves on-device intelligence.

This is genuinely a 2026 inflection point. The ETSI and industry analysts tracking the embedded market have noted the shift: NPU availability is increasingly a baseline specification rather than a differentiator. Which means the question has moved from “can we afford an NPU?” to “which NPU actually fits our workload?”

What an NPU Gives You

A Neural Processing Unit is a processor optimised for the matrix multiplication operations that dominate neural network inference. Where a CPU is general-purpose and a GPU is optimised for parallel floating-point operations, an NPU’s architecture is specifically designed for the patterns in modern ML models: convolutional layers, attention mechanisms, activation functions.

The practical benefits for edge IoT:

Latency: An NPU can run a vision model inference in milliseconds that would take hundreds of milliseconds on an equivalent CPU. For industrial anomaly detection or safety applications that need real-time response, that gap matters.

Power efficiency: Running inference on an NPU uses a fraction of the power of the same inference on a CPU or GPU. For battery-powered devices, this is the difference between daily charging and a week between charges.

Cost per inference: With cloud inference, you pay per call plus network costs. With on-device NPU inference, it’s a fixed hardware cost amortised across the device lifetime. At scale, the economics flip decisively toward on-device.

The Current SoC Landscape

Qualcomm QCS6490 and QCS8550 (industrial IoT range): Qualcomm’s IoT-specific chips include the Hexagon NPU, capable of 15-45 TOPS depending on the variant. These are finding their way into smart cameras, factory inspection systems, and retail analytics devices. The associated SDK (Qualcomm AI Engine Direct) is mature and supports ONNX and TFLite model formats. Not the cheapest option but very well-documented.

MediaTek Genio 700/1200: MediaTek’s industrial-focused Genio range includes an AI Processing Unit (APU) targeting 4-6 TOPS. More affordable than Qualcomm equivalents, good for lightweight vision tasks (object detection, face recognition, defect classification). Popular in Asian manufacturing deployments.

Rockchip RK3588: A popular choice in the edge computing community, with a 6 TOPS NPU. The Radxa ROCK 5B and similar boards make this accessible for prototyping. The toolchain (RKNN Toolkit) has improved significantly over the past 18 months and supports quantised models efficiently.

NXP i.MX 9 series: NXP’s safety-and-automotive heritage makes the i.MX 9 range appealing for industrial and medical deployments where reliability certifications matter. The NPU (Ethos-U65 in some variants) is designed for inference at the sensor edge rather than complex models.

Nordic Semiconductor nRF9151 (ultra-low-power): At the opposite end: Nordic’s latest cellular IoT chip includes a modest ML accelerator capable of running keyword detection and basic anomaly detection models while running on coin cell batteries. Not for computer vision, but sufficient for audio event detection and simple sensor anomaly classification.

How to Evaluate for Your Workload

The TOPS (Tera Operations Per Second) number is useful but not the whole story. Here’s what to actually evaluate:

Model format support: Can the NPU run your model without painful conversion? Most NPUs have their own compiled format (Qualcomm’s QNN, Rockchip’s RKNN, etc.). Check that the vendor’s SDK converts from ONNX or TFLite without significant accuracy degradation.

Quantisation requirements: NPUs typically require INT8 or INT16 quantised models rather than FP32. Quantisation can degrade accuracy, especially for smaller models. Test your specific model quantised before committing to hardware.

Memory bandwidth: Large models that don’t fit in the NPU’s local memory will spill to DRAM, which is much slower. Check the NPU’s local SRAM size against your model’s working memory requirements.

SDK maturity and community: A chip with a poorly documented SDK will cost you development time. Qualcomm and Rockchip have the most developed developer communities in the edge AI space. NXP has strong documentation for industrial applications.

Power envelope: TOPS per watt matters more than raw TOPS for battery-powered designs. Get numbers from vendors rather than relying on peak-performance spec sheets.

A Practical Selection Starting Point

For industrial computer vision (defect detection, object counting, safety compliance): Qualcomm QCS or Rockchip RK3588 are the standard choices. Budget significantly affects which.

For smart building and facility management (occupancy counting, access control): MediaTek Genio or Rockchip cover most use cases at lower cost.

For battery-powered sensor nodes with lightweight classification: Nordic nRF9151 or similar ultra-low-power MCU with NPU is the right tier.

For regulated environments (medical devices, safety-critical industrial): NXP i.MX 9 with its functional safety heritage is worth the premium and complexity.

The hardware decision is the easy part compared to the model development and MLOps pipeline for updating models on deployed devices. If you’re evaluating NPU-capable SoCs, also spend time evaluating how you’ll update the inference models once devices are in the field. That operational question is what catches most teams out.