minbta.blogg.se

For detection tools to be effective at stopping malicious deepfakes, they would need to be widely adopted by the social media platforms and messaging apps – but no social media platform currently has deepfake detection in their media upload pipelines, and implementing detection on messaging apps like WhatsApp or Telegram would require monitoring users’ conversations, a significant change to these services’ current privacy-focused model.Īnother is how reliable these security measures would be. One is how developers roll them out in the first place. While developers’ readiness to address misuse of their apps is promising, deploying these security features poses several challenges. As the technology becomes more powerful and pre-training less restrictive, developers might see a competitive advantage in opening up their apps to user-uploaded content in an “off-rails” approach.

In order to quickly generate high-quality face-swaps with one or a few user images, apps “pre-train” their generative models on a number of popular movie scenes, such as the twins from The Shining, or Sean Bean’s “one does not simply walk into Mordor” meme from The Lord of the Rings. But these restrictions are often the outcome of technological limitations rather than a deliberate security choice. Many deepfake apps address these concerns by being “on rails”, or restricted: users can only swap faces into a selection of scenes from pre-approved films or shows.

Both examples point to a worrying future where deepfake apps could create harmful fakes on a massive scale, threatening anyone whose images are online. Although the video wasn't realistic, similar scenarios in the future may be more convincing. Over 100,000 of deepfake images of women and minors were shared on Telegram channels counting over 100,000 members.įears that deepfake apps could fuel the problem of political disinformation and deceptive content online were also sparked in April 2020, when Donald Trump retweeted a crudely manipulated video of Joe Biden lolling his tongue and twitching his eyebrows.

One of this article’s co-authors recently discovered a “deepfake pornography bot” on the messaging app Telegram, which allowed users to upload pictures of clothed women and “strip” them by generating their deepfake nude images. Glimpses of this misuse are already visible.