Deepfakes: When Visual Evidence Becomes Doubtful

In January 2024 the British engineering firm Arup reported to Hong Kong police that an employee in its office there had transferred about US$25.6 million (HK$200 million) in 15 payments. It began with a message he distrusted; he then joined a video call in which his chief financial officer and other colleagues appeared, and he was reassured. The faces and voices were all fabricated. The firm itself said its internal systems had not been compromised: it was not the machines that were deceived, but the eye and the ear.

What do we mean by a deepfake?

It is the full generation of a voice, image or clip by neural networks that learn a particular person's features and tone until the copy is hard to tell from the original. It differs from traditional photo editing because the machine here produces the content rather than cutting and pasting it. The US Federal Trade Commission (FTC) warns that a fraudster may need only a short audio clip posted online to clone the voice of one of your relatives, then call you in distress and ask for money. For more on cyber fraud, see the article "AI and Cybersecurity"; here we stay with the question of trust.

The first victim is an individual, not a head of state

It is often assumed that the most dangerous fakery is political. Yet the 2019 Deeptrace report, one of the earliest surveys, counted 14,678 fake videos, 96% of them fabricated indecent material targeting women without their consent. Treat that figure with caution: the counting window was only two months in 2019, and things have changed since. But it signals that the first to suffer are ordinary individuals, and that protecting honour and reputation is among the objectives of the Sharia that make this area deserving of protection.

The liar's dividend

In 2019 the legal scholars Bobby Chesney and Danielle Citron published a study in the California Law Review that coined the "liar's dividend": the more people learn that audio and video can be faked, the easier it becomes for a wrongdoer to deny a genuine recording by calling it fabricated. Certainty about falsehood is not the only thing lost; certainty about the truth goes with it.

Falsehood also travels fast. A study by researchers at the Massachusetts Institute of Technology (MIT) analysed about 126,000 news cascades on Twitter between 2006 and 2017 and found that false stories were about 70% more likely to be retweeted than true ones, and that humans, not automated accounts, were mainly responsible for spreading them. The study concerns false news in general, not deepfakes, but it shows the nature of the audience a fabricated clip will reach.

Detection is not enough

So long as forger and detector learn from each other, automated detection alone is unlikely to settle the race. Efforts therefore point in two complementary directions. The first is technical: the guidelines of the Saudi Data and Artificial Intelligence Authority (SDAIA) call for labelling synthetic content with a visible watermark and for adopting content-provenance standards. The second is regulatory and legal: explicit consent from the owner of the image or voice, penalties proportionate to the offence, and reporting channels the guidelines list, including the "Kollona Amn" app and the Ministry of Interior's cybercrime unit. Such laws need a balance that does not stifle legitimate expression, and specialists draw that line.

What you can do

• Hang up and call back on a number you know belongs to the person, as the FTC advises; do not trust the voice alone.
• Ask for a second channel before any money transfer, even if the requester is "your manager" or "your son", and agree with your family or team on a question nobody else knows.
• Do not re-share a shocking clip before checking its source; truthfulness is a trust in relaying words just as it is in speaking them.
• If you are a victim, keep the evidence and report to the competent authorities and the platform.

The aim is not to doubt everything: a society that trusts nothing is paralysed, just as one that trusts everything is deceived. The aim is a trust that asks first: who passed this on, from where, and can I confirm it another way?