What does Edge AI mean?

Edge AI refers to running AI models directly on the device where the data is generated, instead of in a remote data center. A camera, sensor, or machine controller evaluates the data itself and passes on only the result. The difference from cloud processing is that the raw data never leaves the device and no permanent network connection is required.

For a model to run on limited hardware, it is shrunk beforehand. Quantization reduces the numerical precision of the weights from 32 to 8 bits, and pruning removes connections that contribute little. Distillation transfers the behavior of a large model onto a small one. The result runs through runtime environments such as ONNX Runtime or TensorFlow Lite, often on a dedicated accelerator built into the device. Training itself still happens centrally.

Edge AI pays off wherever large volumes of data are constantly generated but only a small result is actually needed. Typical uses include counting vehicles in road traffic, visual inspection on the production line, and speech recognition in devices without an internet connection. Where models change frequently or need heavy computing power, central operation remains the better choice.

The advantage over transmitting all raw data lies in the volume that needs to travel over the network. Instead of a continuous video stream, only a handful of numbers per hour go out, which means even a narrow mobile connection is enough. If it drops, evaluation on the device simply continues.

Edge AI requires a plan for updates, because models on distributed devices age and every device needs a traceable version record. If devices are deployed in public space, their location, purpose, and retention period belong in the record of processing activities, just like any other form of processing.