Deepfakes: What’s behind the AI-manipulated videos

6 min readApr 2, 2021

Some see in Deepfakes a danger to our democracy. Others see in it creative tools. The truth is somewhere in between.

For a long time now, we have been critically scrutinizing texts and images. In times of Photoshop and Apps the truth can be manipulated too easily. Simple videos and sound recordings, on the other hand, are still considered forgery-proof by many people. Manipulation is too expensive and time-consuming. But this image is likely to change fundamentally in the coming years! The reason for this is called Deepfakes.

Deepfakes are fake videos based on artificial intelligence. The technology behind it is frighteningly simple, because self-learning algorithms do most of the work. This method is called Deep Learning. All that is needed is as many shots of the people to be exchanged as possible and enough time to let the algorithm learn.

Famous Deepfakes

Very little IT knowledge is sufficient for creating deepfakes. Social networks like Snapchat enable their users to use increasingly sophisticated “faceswap” tools that follow the same basic principles. For those who want to go one step further, there are publicly available tools on the Internet, such as DeepFace Lab, which enable professional results with just a few hours of learning-by-doing.

The majority of deepfakes, however, are not harmless teenage party jokes. It is estimated that 95 percent of manipulated videos are porn, and they are still close to their origins, because the name “DeepFakes” comes from an anonymous Reddit user who used this method to put celebrities in porn for the first time at the end of 2017.

Before that, Deep Fakes were merely a popular method used by the film industry to replace dead actors. This happened in 1994 when Forrest Gump meets US President John F. Kennedy. Or in 2015 in the action movie Fast & Furious 7 with the actor Paul Walker, who died in a car accident while producing it.

How-To: Create Deepfakes yourself

Step 1: Select tools

With DeepFace Lab and Faceswap, the two most popular Deepfake programs are open source and freely available on the Internet. Just watch a few tutorials on YouTube and then choose your favorite. We will use DeepFace Lab for the How-To. But the steps should be similar.

If you want to go one step further, you can manipulate the voice with tools like Descript.

Step 2: Prepare training data

Now select the video that’s to be manipulated later. As many facial features of the actors as possible are needed.

If you want to see yourself in the final Deepfake, you should now create your own 20-minute video with as many facial features as possible in HD quality. If another person is to be used, many high quality video clips of the person are required. It is best to create close-ups with a clear field of vision and no other persons.

The software divides the old as well as the new video into lots of image files. The faces are then extracted from these files. This selection of thousands of images is now cleaned up to filter out blurred, faulty or inappropriate ones and to achieve better results.

Step 3: Train Model

It’s time! The Generative Adversarial Network (GAN), that is a group of algorithms, now has all the information it needs to start its learning process. This process is called Deep Learning and requires a lot of memory. With bad hardware you might want to google “Google Colab”. You can use 12h Google’s hardware.

For deep learning, the GAN creates a model for machine learning that mimics the distribution of data. The more iterations the software has for learning, the better results are achieved. However, different information is circulating on the Internet: For example, sometimes at least 40,000 to 60,000 iterations are recommended and elsewhere more than 150,000 iterations are recommended. These learning loops will take several hours to days, depending on the hardware.

Step 4: Postprocessing

Once the GAN has gone through all the learning loops, the video is almost finished. The AI overlays the new video with the old one. In the settings you can then adjust the color, contrast and other settings to refine the result.

In addition, you can save the files in several layers separately from each other, so that you can edit them perfectly in a professional video software afterwards.

The media dilemma

A look at our current society shows how dangerous this technology can become.

Fake news and alternative facts have found a firm place in our vocabulary. Although the media in democracies were originally established as a counterweight and controlling organ of politics, more and more people see them merely as their tools. This image is becoming more and more popular thanks to the support of public figures — above all US President Donald Trump.

Ironically, two of the main reasons for this trend can be found in the multiplicity of media and their democratization. In our society, everyone is allowed to express their opinion publicly. Through social media, every voice has the potential to reach the world. The certainty that professionally prepared content was reserved for the “real and neutral” media is no longer available due to our technological development. Nowadays, anyone can produce high-quality content with a smartphone and computer. The distinction between false news, seriously researched content and (political) opinion making is becoming more and more complicated.

Furthermore, most people are trapped in their filter bubbles. Social networks as well as other websites algorithmically try to suggest the most interesting content to their users. If the professionally prepared false reports from fringe groups are now forwarded unfiltered while all information outside the bubble is withheld, the world view of those affected is extremely distorted. Each person lives his or her own truth. According to a study by MIT, these false reports subsequently spread six times faster than the truth.

Detect Deepfakes: Easy today. More difficult in the future.

The good news first: many of the deepfakes can still be seen with the naked eye today. The transitions between fake and real video are often not clean. The light and the colors sometimes do not match. In addition, peoples eyes usually have no eyelid blink and appear empty.

According to Mike Schroepfer, CTO of Facebook, Deepfakes do not yet pose a serious threat because of this. However, they are getting better and better, because with enough time and iterations for the algorithm to learn, the criteria mentioned at the beginning become obsolete. Take a look at the deepfake examples at the beginning of the article. They seem frighteningly real.

That’s why companies, universities and even governments with artificial intelligence use the foundation behind Deepfakes as an antidote to the very same thing. In early 2020, Facebook, Microsoft and Amazon Web Services called data scientists around the world to take part in the Deepfake Detection Challenge. On the data science platform Kaggle, 2,265 teams of experts had just under a month to write algorithms that could detect Deepfakes. The winning algorithm by engineer Selim Seferbekov detected between 65 and 82 percent of the deepfakes. In addition, startups such as Depptrace, but also universities such as the TU Munich with FaceForensics++, are focusing on detection.

Shaping the future positively

In the years to come, the responsibility for filtering and clearly marking false reports, manipulated photos, but also deepfakes, will largely lie with social networks and providers. Politicians will establish clear rules so that their citizens will not be faced with the impossible task of having to detect manipulations themselves. Looking around, we already fail in it today.

At the same time, we will see if we can steer the technology in a positive direction. Once again it is up to us and all creative people…

Cool Stuff

A good example of how to use Deepfakes in a positive and creative way is the Dalí Museum in Florida. By means of a Deepfake it has brought the Spanish painter Salvador Dalí back to life.

Sources

Deepfakes: What’s behind the AI-manipulated videos

Famous Deepfakes

How-To: Create Deepfakes yourself

The media dilemma

Detect Deepfakes: Easy today. More difficult in the future.

Shaping the future positively

Cool Stuff

Sign up to discover human stories that deepen your understanding of the world.

Free

Membership

Written by Thilo Schinke

No responses yet