Architecture
Edge inference: when it is justified
Comparing computation next to the camera with a centralized cluster by cost, latency and operational load.
Processing near the camera removes network load and reduces latency. The price is a distributed fleet of devices that has to be updated and monitored.
A centralized cluster is simpler to operate: one place to update models, predictable GPU utilization, shared monitoring. But it requires a stable channel and is bounded by network throughput.
A practical rule: if the stream is needed only to produce an event and video is not stored, edge processing wins. If an archive and incident review are required, the stream goes to the centre anyway.
A hybrid scheme also works: light filtering at the edge, and the heavy model in the centre only for frames that passed the first threshold.
All articles