Why have a headache from 3D / Part 3 entanglement angles

S3D: No pain IS gain i>




This is the third article in a series of "Why 3D from a headache." In the first part of it was about the problem of cinema equipment in the second - an overview of the problems of films and, finally, here and below, will be dealt with specific problems. Let's start with the most simple to understand, and one of the most painful for the perception - entangled angles, when the right eye image is fed to the left and to the left - to the right. It's hard to find words to convey the feeling experienced by the corresponding parts of the brain when they see this picture ... But we still try. )

The roots of the problems


When the left eye image is fed to the right and vice versa, we see an image, it is impossible in reality as something that should be seen as a more intimate and vice versa. The Ones. objects as if turned inside out. Typically, this problem occurs, an error mounting. It is very important to understand the main thing: to fix this specific problem - elementary, an order of magnitude easier than most of the other errors, which were discussed in the second part. This error is not corrected exactly because if your brain is already trained to poor stereo, he quite successfully compensates for the problem: people tend to see the picture even with elements of the "right" of three-dimensionality, but subjectively more "flat" than it should be. The Ones. our brain strives to recognize the "impossible" image and with sufficient training to poor stereo scenes such notices.

To be clear provisions of the human brain tell you about the experiments that in the XIX century spent in the University of California George Stratton. He had many interesting experiences, including pretty hard when the image is due to the nozzle with mirrors, which are fixed on the head, completely reverses:


Experiments were carried out for several days, and in the early days even the simplest tasks, such as to take any thing or go into the next room, not touching furniture, caused serious difficulties. But on the third day the situation has improved: "While walking in the narrow gap between the furniture I needed much less attention than hitherto. When I was writing, I could easily watch your hands, without experiencing discomfort. " On the fifth day Stratton was able to walk around the house and so on. D. Interestingly, when he took the unit on the eighth day, it took only two hours to the visual system "adapted back" to normal perception. At the same time, when re-equipped device time to adapt to the "wrong" image is also dramatically reduced. These interesting experiments have been many, including later in the XX century.

Chicken Pfister prism, deflecting the light entering into her eyes. H6> Also, similar experiments were carried out later with various animals - with chickens / hens, kittens / cats, monkeys, and so on. D. An interesting conclusion of these is that the human brain is much more adaptive than the brains of animals and man is capable of much more successfully adapt to the "impossible" image, even in comparison with monkeys.

Conclusion:
B> Professionals trained on a bad stereo, no longer experience severe discomfort from it, which leads to the deletion of the easily correctable errors in the mass releases of films. If a person shows poor stereo, then gradually decreases to experience discomfort, especially if the "train" binocular vision since childhood. Perhaps this explains the large number of "terrible" by the standards of Europeoids stereo, which is now mass produced and shown in China. As a result, China's domestic market, for example, most ready for new autostereoscopic devices, the image quality of which the first stage is bound to be imperfect. B>
Algorithmic complexity
From the point of view of computer vision problem of determining the angle of entangled very nontrivial and interesting. It is interesting in the first place so that it is a very simple statement and, in contrast to many other areas - elementary to prepare a test sample. You just take some stereofilm, doing short simple script, which in every other scene reverses the angles and get an excellent test base of 1000 scenes for debugging their own algorithms.

And then the fun begins. If a person, especially to poor stereo nenatrenirovannogo determine confused angles is quite simple, then the program is not a trivial task. The man knows how things should look, and that is the closest, and that further, as it has "Psychometric" about what forms of furniture, which form the house, and so on. D. The program has no such knowledge and despite regular (over the last 40 years) triumphant about the next breakthrough in science, recognition of objects and, as a consequence, for example, robot vision still leave much to be desired. It should be understood that in order, to correctly identify entangled angles, robots must conditional on information from one camera to determine which is closer, and then what better than a man with two eyes
To solve various features can be used, for example, that the bottom is likely to be objects closer than the top of that foreground objects are generally convex, etc. But the problem is that all these features are extremely unreliable, and, ultimately, on our test sample of 1,000 scenes such methods will be wrong too often.

A more reliable approach is to analyze the motion. In modern films are practically no scenes without motion, with virtually any motion, we can determine with respect to m. N. regions "opening and closing" where one object occludes the other, and hence closer to a object.





In scientific language problem of determining the depth of objects on 2D video called "depth ordering" , and is easy to see by following the link in this field is being quite a lot of research. Looking ahead, I must say that their results still leave much to be desired. Of course, in the article, you will often see beautiful pictures, but if you look at the independent tests themselves or try to drive algorithms are available, for example, in the source code for MATLAB, you will disappointed with the quality of their work and the mass of errors.

Next, you need to solve a simpler problem definition of the apparent depth of the object on the stereo, or speaking in scientific language - the task of building disparity maps. Algorithms for constructing improved disparity maps for about 30 years, but there is also still a lot of problems.



For example, if you go to this link , you will see a comparative analysis of algorithms for constructing about 160 cards disparities created over the past 15 years . Sort the plate and look at the results of the best algorithms. That rough edges? Pieces of objects or even objects lose? What you say! ) And then find relevant articles for the best algorithms and see how long they work. One frame. Now count how many years (or a supercomputer) is required to handle these algorithms with the entire movie. Who are too lazy, there in the worst case operating hours per frame, and as a result, decades of secondary computers to process the entire movie. Exactly this explains the fact that even just work on the definition of irregular angles today is not so much.

They were, as you might imagine, the bad news. Where is the good? Patience, gentlemen. Some good news. Firstly, to say that the scene confused perspectives we generally do not need all the objects in the scene. Suffice it to some one object (or sometimes even one boundary) reliably ranked their depth and out of traffic, and from the disparity map. This is a serious simplification, although we still need to build and the one and another card for the entire frame. Second, in recent years, increases exponentially the speed of the GPU and many video processing algorithms, which until recently were too slow for practical use, acquire a second life. Or first, if we assume that such is not the use of the articles and the use of practical problems in real life. And finally, thirdly, there are ways to effectively accelerate existing algorithms, if we are talking about the processing of the entire movie.

I must say that over the past 5 years, under the guidance of the author was made two serious attempts to solve this problem, and a second attempt is possible (with reservations) considered successful in the sense that built practically on the speed of the algorithm, which is an acceptable number of errors makes it possible to find Entangled angles in real films.

Also worth noting is that in the process of working on the algorithm began to be a scene in which individual objects (usually special effects) were imposed on the "wrong" depth, for example, deeper objects to which they overlap. From the point of view of the opening and closing regions in time and depth is also an impossible situation. It is impossible from the viewpoint of the brain. We see that the object is closer, but binocular vision tells us that it is farther away. This is a very unpleasant situation, the diagnosis is complicated by the fact that a large part of the scene is the "right", and only a single object is incorrect. Nevertheless, it is necessary to detect such things. Known, for example, the case where one low-budget Hollywood film in general, all the special effects imposed confusing angles. He went to the theaters (the poor audience!), But in 3D Blu-ray it was never released (maybe not to disgrace, and maybe just realized that additional charges will not be).

Conclusion:
B> As strange as it sounds to those who are not interested in computer vision, but the problem of determining the angle of entangled today is not completely solved. The main difficulty in determining the safe boundaries of objects on the motion and depth, as well as in the processing of scenes where there is no movement. However, for various reasons, even now an opportunity to create the first practical algorithms of this kind. Practical - is at an acceptable rate allowing to process the entire movie. B>
How often confused perspectives in life
And now - the most interesting results of the analysis of 105 films for scenes with reversed angles (Channel Mismatch, abbreviated CM). Due to the speed of our algorithm we have the opportunity to check the many tens of movies. It's still hard, but it was possible! In fact, our team had to check all the films that went on Blu-ray, which is specified in the budget IMDB.com, ie all more or less decent movies. Following results were obtained:

Release year
Number of movies
Movies with CM
Movies with CM, percentage
Scenes with CM
1954-2000
6
1
16, 7%
2
2001-2009
15
4
26, 7%
11
2010
17
7
41, 2%
32
2011
24
8
33, 3%
17
2012
20
2
10, 0%
2
2013
21
1
4, 8%
1
2014
2
0
0, 0%
0
or the same graphical view:


It is clearly seen that the conditional 20 films released before 2010 entangled angles met respectively 16% and 26% of the films. In 2010, immediately after the release of Avatar - has been a surge of 41% 17 films contain scenes with reversed angles. In 2011, 24 have only 33%, but in 2012 of 21 - only 4.8%. The Ones. defect rate in this area was sharply reduced, and the data is quite statistically significant. Statistics for 2014 are still too small to draw conclusions, but most likely it can be expected that the percentage will be reduced even marriage. Anticipating questions, once again, I note that checked releases issued on Blu-ray, which usually come with a fairly noticeable delay also runs themselves, despite the radical acceleration metrics still take considerable time, so we get the data with a delay. Nevertheless, the observed trend is encouraging.

Also by the author over the past three years to carry out technical analysis stereofilms "Moscow International Film Festival 3D-stereo".




The Ones.

The Ones.


The Ones.