How I smoothly injected a Mona Lisa talking-head into the full portrait

Original article was published by Mohammed El komy on Deep Learning on Medium


In the pipeline demo above, the portrait under without smoothing title shows how badly was the taking-head replaced and the discontinuities
(on borders) make it look very trivial.

On the other hand on the right of the demo, the refined/smoothed portrait under Smoothed title looks much better.

So how has been then smoothing done?

on the left of the demo, you should see 3 animated talking-heads the one on the top is the smallest, most focused, and has better animation, alternatively on the bottom is the largest, least focused, and poor animation (look at the lips).

So the idea is to merge all of those animated talking-heads together into the full portrait and gradually outside in, as we said we have 3 generated
talking-heads with different sizes and quality of animation and as in the weighting-scheme mask shown below you should see Red, Green, Blue gradual weighting.

Looking from the outside into the inside at the weighting-scheme mask shown below.

  1. The Red boundary starts with black (weight=0) into pure red (weight=1)
  2. The Green boundary mixes with red (weight=0) until the pure green (weight=1) at the end.
  3. The blue boundary mixes with green (weight=0) until the pure blue (weight=1) in the center.
Weighting-scheme mask gradually outside in, It looks blurry and I know it hurts (due to smoothness).

And this is also shown in the pipeline demo above under the title Boundary Details.

You may notice the diagonal discontinuity, this is true and I think it’s similar to Sobel derivative operator which doesn’t highlight changes in intensity in a diagonal direction.

That’s it for today πŸ™‚

Thanks for your time πŸ™‚

I used gifski open-source tool for generating high-quality gifs 🎊🎊