A method of creating a 3D model of a physical object includes adaptively and iteratively generating a number disparity maps from image data representing a plurality of images of the physical object iteratively captured by a plurality of cameras having electrically adjustable focal lengths by varying at least one of the focal lengths of the plurality of cameras and a distance of the physical object from the plurality of cameras during capture of the images until one of the disparity maps is determined to have a least a threshold level of disparity, and converting the one of the disparity maps into the 3D model.

A method of creating a 3D model of a physical object includes adaptively and iteratively generating a number disparity maps from image data representing a plurality of images of the physical object iteratively captured by a plurality of cameras having electrically adjustable focal lengths by varying at least one of the focal lengths of the plurality of cameras and a distance of the physical object from the plurality of cameras during capture of the images until one of the disparity maps is determined to have a least a threshold level of disparity, and converting the one of the disparity maps into the 3D model.

A system and method for determining a location for a surgical jig in a surgical procedure includes providing a mixed reality headset, a 3D spatial mapping camera, an infrared or stereotactic camera, and a computer system configured to transfer data to and from the mixed reality headset and the 3D spatial mapping camera. The system and method also include attaching a jig to a bone, mapping the bone and jig using the 3D spatial mapping camera, and then identifying a location for the surgical procedure using the computer system. Then the system and method use the mixed reality headset to provide a visualization of the location for the surgical procedure.

An apparatus including a stereo camera and a processor. The stereo camera may comprise a first capture device and a second capture device in a vertical orientation. The first capture device may be configured to generate first pixel data and the second capture device may be configured to generate second pixel data. The processor may be configured to receive the first pixel data and the second pixel data, generate a vertical disparity image in response to the first pixel data and the second pixel data, generate a virtual horizontal disparity image in response to the first pixel data and the vertical disparity image and detect objects by analyzing the vertical disparity image and the virtual horizontal disparity image. An analysis of the virtual horizontal disparity image may enable the processor to detect the objects not detected in the vertical disparity image alone.

An optical passive stereo assembly for generating a three-dimensional image, the optical assembly comprising: two image capturing devices each mounted within the optical passive stereo assembly in a skewed position to the other with respect to the horizontal plane; a processor configured to: process data retrieved from a plurality of pixels comprised within images captured by the two image capturing devices, and generate a point cloud, being a set of data points in a 3D space retrieved from the tilted coordinate system (X′, Y′, Z′); apply a 2D rotation in the X-Y plane to the point cloud, thereby converting the coordinate system of the resulting point cloud to a conventional cartesian coordinate system (X, Y, Z), to enable generating the three-dimensional image.

An optical passive stereo assembly for generating a three-dimensional image, the optical assembly comprising: two image capturing devices each mounted within the optical passive stereo assembly in a skewed position to the other with respect to the horizontal plane; a processor configured to: process data retrieved from a plurality of pixels comprised within images captured by the two image capturing devices, and generate a point cloud, being a set of data points in a 3D space retrieved from the tilted coordinate system (X′, Y′, Z′); apply a 2D rotation in the X-Y plane to the point cloud, thereby converting the coordinate system of the resulting point cloud to a conventional cartesian coordinate system (X, Y, Z), to enable generating the three-dimensional image.

A captured scene captured of a live action scene while a display wall is positioned to be part of the live action scene may be processed. To perform the processing, stereoscopic image data of the live action scene having a live actor and the display wall displaying a rendering of a precursor image is received. Further, precursor metadata for the precursor image displayed on the display wall and display wall metadata for the display wall is determined. An image matte is accessed, where the image matte indicates a first portion associated with the live actor and a second portion associated with the precursor image on the display wall in the live action scene. Thereafter, background pixels for the precursor image on the display wall in the stereoscopic image data is moved to generate stereo-displaced pixels using the precursor metadata, the display wall metadata, and/or the image matte.

A captured scene captured of a live action scene while a display wall is positioned to be part of the live action scene may be processed. To perform the processing, stereoscopic image data of the live action scene having a live actor and the display wall displaying a rendering of a precursor image is received. Further, precursor metadata for the precursor image displayed on the display wall and display wall metadata for the display wall is determined. An image matte is accessed, where the image matte indicates a first portion associated with the live actor and a second portion associated with the precursor image on the display wall in the live action scene. Thereafter, background pixels for the precursor image on the display wall in the stereoscopic image data is moved to generate stereo-displaced pixels using the precursor metadata, the display wall metadata, and/or the image matte.

An optical passive stereo assembly for generating a three-dimensional image, the optical assembly comprising: two image capturing devices each mounted within the optical passive stereo assembly in a skewed position to the other with respect to the horizontal plane; a processor configured to: process data retrieved from a plurality of pixels comprised within images captured by the two image capturing devices, and generate a point cloud, being a set of data points in a 3D space retrieved from the tilted coordinate system (X′, Y′, Z′); apply a 2D rotation in the X-Y plane to the point cloud, thereby converting the coordinate system of the resulting point cloud to a conventional cartesian coordinate system (X, Y, Z), to enable generating the three-dimensional image.

An optical passive stereo assembly for generating a three-dimensional image, the optical assembly comprising: two image capturing devices each mounted within the optical passive stereo assembly in a skewed position to the other with respect to the horizontal plane; a processor configured to: process data retrieved from a plurality of pixels comprised within images captured by the two image capturing devices, and generate a point cloud, being a set of data points in a 3D space retrieved from the tilted coordinate system (X′, Y′, Z′); apply a 2D rotation in the X-Y plane to the point cloud, thereby converting the coordinate system of the resulting point cloud to a conventional cartesian coordinate system (X, Y, Z), to enable generating the three-dimensional image.