Neural networks learn to decorate a black and white video

A couple of weeks ago on Geektimes published material that the neural network learned to paint a black and white photo. Neural networks have long been able to much, not only to work with photos. But this is probably one of the most revealing demonstrations networking opportunities. Even more indicative is the "customization of image" AI DeepDream from Google, although the resulting images are more psychedelic ..., let's say.

Now the neural network learned to paint in black and white movies. This was announced by mar_cnu. The appropriate tool for colorization of black and white was his efforts to create a film. And although the result can not be called perfect, everything is pretty good.

As starting material it chose old French movie trailer of 1964 year. Here's the snippet:



Developer explained his choice by the fact that the trailer has everything you need - the jungle, river, people, houses, roads. That's what happened after the video processing:



As you can see, the result is not ideal, as described above. But one of the reasons for this may be the source video itself. It is not clear, as all films of the time, often on the "Picture" all fused, so it is logical that there are difficulties in the program.

This was done in all the following:

1. We found the original video;
2. The source is converted with the help of this service here;
3. Run the script

for file in video / *
do
echo "$ file"
th colorize.lua "$ file" «color - $ {file% .jpg} .png»
done

4. ffmpeg -i «color-frame% 04d.png» -c: v libx264 colorvideo.mp4
5. ffmpeg -i blackandwhitevideo.webm audio.mp3
6. 6) ffmpeg -i colorvideo.mp4 -i audio.mp3 -c: v libx264 -c: a copy -strict

For the service, as correctly noted users, the most difficult task is to implement a smooth transition between the individual pictures. If you put the video on pause, often able to "catch" the frame with almost perfect colors. But a smooth transition between frames - it is a separate challenge.

Source: geektimes.ru/post/275086/