The first photo might reveal a spot on the rabbit’s left leg. Suppose, once again, that one photo is taken from the left side of a rabbit and another photo is taken from the right side. Virtual correspondence offers a way to carry things further.
Structure from motion works when two images have points in common, because that means a triangle can always be drawn connecting the cameras to the common point, and depth information can thereby be gleaned from that. “We want to incorporate human knowledge and reasoning into our existing 3D algorithms” Ma says, the same reasoning that enabled him to look at his fingertips and conjure up fingernails on the other side - the side he could not see. That was the inspiration for the notion of virtual correspondence, which Ma has subsequently pursued with his advisor, Antonio Torralba, an EECS professor and investigator at the Computer Science and Artificial Intelligence Laboratory, along with Anqi Joyce Yang and Raquel Urtasun of the University of Toronto and Shenlong Wang of the University of Illinois. While looking at the palm of his hand and his fingertips in particular, it occurred to him that he could clearly picture his fingernails, even though they were not visible to him. MIT was closed at the time due to the pandemic, and Ma was home in Taiwan, relaxing on the couch. If the input images have very different viewpoints - and hence contain few, if any, points in common - he adds, “the system may fail.”ĭuring summer 2020, Ma came up with a novel way of doing things that could greatly expand the reach of structure from motion. So long as we can observe the same point, or points, across different images, we can use existing algorithms to determine the relative positions between cameras.” But the approach only works if the two images have a large overlap. And if enough common points are identified, it might be possible to obtain a detailed sense of the object’s (or “rabbit’s”) overall shape.Ĭonsiderable progress has been made with this technique, comments Wei-Chiu Ma, a PhD student in MIT’s Department of Electrical Engineering and Computer Science (EECS), “and people are now matching pixels with greater and greater accuracy. Knowing the distance between the cameras and the way they were oriented, one could then triangulate to work out the distance to a selected point on the rabbit. A researcher could go from there to determine the “poses” of the two cameras - the positions where the photos were taken from and the direction each camera was facing. The first step would be to find points or pixels on the rabbit’s surface that both images share.
Suppose you have two pictures of an object - a sculpted figure of a rabbit, for instance - one taken from the left side of the figure and the other from the right. Triangulation is a key element of structure from motion. This same basic idea - of triangulation or parallax views - has been exploited by astronomers for centuries to calculate the distance to faraway stars. Knowing the angles in the triangle and the distance between the eyes, it’s possible to determine the distance to that point using elementary geometry - although the human visual system, of course, can make rough judgments about distance without having to go through arduous trigonometric calculations.
#Multi shape collage frame plus
A triangle can be formed whose sides consist of the line segment connecting the two eyes, plus the line segments connecting each eye to a common point on the object in question. Because our eyes are separated from each other, they each offer slightly different views of an object.
The standard approach, called “structure from motion,” is modeled on a key aspect of human vision.
0 kommentar(er)
