Original article can be found here (source): Artificial Intelligence on Medium
MakeItTalk: Speaker-Aware Talking Head Animation
This is super cool research → and the result is fabulous it is really cool and amazing. (only from audio and a single image we are able to create a full-length video)
With the character talking in a natural manner → this is crazy → and the demo is crazy.
And the secrete is the effective disentanglement of facial landmarks → here are able to generate real-looking videos.
Learning a good manifold will be a huge part when it comes to economical innovations → even for automation there are many ways to create new data from the learned manifold.
This is not new → in computer graphics → there has been a lot of innovation made, such as generating video from audio.
A lot of research even → reading lips is also possible → and thanks to all the innovation made in style transfer → this can be used in different methods as well.
And adversarial loss is used to improve realism → this is one great way in animation or computer graphics to use this kind of training method to make the output more realistic.
And this model → is made up of using multiple models → LSTM as well as CNN and more → this shows how powerful NN can be → each architecture does different things. (they are making manual connections, imagine if this was all automatic).
Quite a lot of advanced methods are used → to make the animation smoother and cooler → this is so cool!
And impressive how fat this technology can go → when it comes to creating animations → creating data → for industries that are making a huge amount of money using these creations.
With different models used → also need to use different loss functions to make this work → quite a complex process of creating these technologies → but we can see how powerful they can be.
Very impressive results → smooth like a baby’s butt. (even able to make natural images when there is head movement in the video).
The huge improvements are → very impressive!
Basically their method is the best.