Edge AI: A Mind That Works Where the Data Is Born

A car travelling at 100 kilometres an hour covers about 2.8 metres in a tenth of a second. In optical fibre, light travels at about 200,000 kilometres a second, roughly 5 microseconds per kilometre, so a round trip costs about 10 milliseconds per thousand kilometres before any queuing or processing is counted. That arithmetic explains why engineers care about moving intelligence to where data is born.

What is the 'edge'?

The edge is the outermost end of the network: the phone in your hand, the camera on the street corner, the sensor in the factory, the watch on your wrist. In their survey (2019), Zhou and colleagues define 'edge intelligence' as bringing AI to these extremities, because billions of mobile and Internet-of-Things devices generate their data there. Instead of sending data on a trip to a data centre and waiting for the decision to return, inference runs on the device or near it. Processing is thus distributed rather than concentrated.

How did the giant shrink?

Running on a small device was not possible when deep networks were hungry for memory and power. One of the main changes was model compression. The 'Deep Compression' paper (Han, Mao and Dally, 2015) combined pruning, quantisation and coding, cutting the AlexNet network from 240 to 6.9 megabytes (35-fold) and VGG-16 from 552 to 11.3 megabytes (49-fold) with no loss of accuracy according to the researchers, along with gains in speed and energy efficiency. These are results on two particular networks of that period, so they should not be generalised to every model, least of all to today's large language models, which we explained in our article 'Large Language Models'.

The idea worth noting is that bigger is not always more suitable: a small, well-tuned model for one specific task may do what is needed where it stands, with no need for a giant summoned from afar.

Three motives

Speed. When response is measured in fractions of a second, the road to the cloud becomes an obstacle that better networks cannot erase, because light sets an upper limit on what can be reached. This matters for applications such as autonomous driving and industrial robots, where delay means risk.

Privacy. When a device's data is analysed on the device itself, there is no need to send it, so protection lies in the design rather than in a contractual promise. Federated-learning research has shown that devices can learn locally and share only updates: in the McMahan et al. paper (2016) the number of communication rounds fell by between 10 and 100 times compared with conventional synchronised training, and in an application to a phone keyboard (Hard et al., 2018) a next-word prediction model was trained on devices without sending texts to servers, and recorded better recall than central training in their experiment. It remains true that this makes privacy easier to build in, not that it guarantees privacy on its own.

Independence from connectivity. A remote field, an offshore platform and an underground mine are places where a permanent network cannot be assumed. A device that carries its mind with it does not stop when the thread is cut, and it saves bandwidth by sending the summary rather than the raw data.

Limits not to be forgotten

The edge is no universal remedy. A compressed model may give up some accuracy, and in each case the loss should be measured. Managing thousands of devices and updating their models is a complex logistical matter. And the more devices are spread about, the wider the attack surface: a device carrying a model may fall into the hands of someone who tampers with it. For this reason the edge is not presented as a rival to the cloud; the likelier outcome is that they cooperate, with the edge taking the immediate, time-sensitive decisions and the cloud taking heavy training, pooling of lessons and model updates.

In our own country

The Kingdom's Personal Data Protection Law sets conditions on transferring personal data outside it, among them a permitted purpose, an adequate level of protection with the importer, and adherence to data minimisation (according to the explanation by King & Spalding of the transfer regulation issued in August 2024). Processing data locally on the device lessens the need for such transfer in the first place, without relieving the organisation of its other obligations. Any body considering edge AI does well to ask: what do we truly need to collect, and would the summary suffice?

The choice here is not between two technologies but between two designs: one that gathers everything and then secures it, and one that gathers only what is needed. The latter is closer to what we count as honesty in guarding what we have been entrusted with.