A long-baseline and long depth-range stereo vision system is provided that is suitable for use in non-rigid assemblies where relative motion between two or more cameras of the system does not degrade estimates of a depth map. The stereo vision system may include a processor that tracks camera parameters as a function of time to rectify images from the cameras even during fast and slow perturbations to camera positions. Factory calibration of the system is not needed, and manual calibration during regular operation is not needed, thus simplifying manufacturing of the system.

A 3D facial reconstruction system includes a main color range camera, a plurality of auxiliary color cameras, a processor and a memory. The main color range camera is arranged at a front angle of a reference user to capture a main color image and a main depth map of the reference user. The plurality of auxiliary color cameras are arranged at a plurality of side angles of the reference user to capture a plurality of auxiliary color images of the reference user. The processor executes instructions stored in the memory to generate a 3D front angle image according to the main color image and the main depth map, generate 3D side angle images according to the 3D front angle image and the plurality of auxiliary color images, and train an artificial neural network model according to a training image, the 3D front angle image and 3D side angle images.

A 3D facial reconstruction system includes a main color range camera, a plurality of auxiliary color cameras, a processor and a memory. The main color range camera is arranged at a front angle of a reference user to capture a main color image and a main depth map of the reference user. The plurality of auxiliary color cameras are arranged at a plurality of side angles of the reference user to capture a plurality of auxiliary color images of the reference user. The processor executes instructions stored in the memory to generate a 3D front angle image according to the main color image and the main depth map, generate 3D side angle images according to the 3D front angle image and the plurality of auxiliary color images, and train an artificial neural network model according to a training image, the 3D front angle image and 3D side angle images.

A method for monitoring the environment of a robot including at least one iteration of a detection phase including the following steps:—acquisition, at a measurement instant, of an image of the depth of the environment, referred to as a measurement image, by the at least one 3D camera,—readjustment of the reference and measurement images, and—detection of a change relating to an object in the environment of the robot by comparing the reference and measurement images. A device that implements such a method and a robot equipped with such a device is also provided.

A method for monitoring the environment of a robot including at least one iteration of a detection phase including the following steps:—acquisition, at a measurement instant, of an image of the depth of the environment, referred to as a measurement image, by the at least one 3D camera,—readjustment of the reference and measurement images, and—detection of a change relating to an object in the environment of the robot by comparing the reference and measurement images. A device that implements such a method and a robot equipped with such a device is also provided.

Implementations described herein relate to a computer-implemented method that includes capturing image data using one or more cameras, wherein the image data includes a primary image and associated depth values. The method further includes encoding the image data in an image format. The encoded image data includes the primary image encoded in the image format and image metadata that includes a device element that includes a profile element indicative of an image type and a first camera element, wherein the first camera element includes an image element and a depth map based on the depth values. The method further includes, after the encoding, storing the image data in a file container based on the image format. The method further includes causing the primary image to be displayed.

Implementations described herein relate to a computer-implemented method that includes capturing image data using one or more cameras, wherein the image data includes a primary image and associated depth values. The method further includes encoding the image data in an image format. The encoded image data includes the primary image encoded in the image format and image metadata that includes a device element that includes a profile element indicative of an image type and a first camera element, wherein the first camera element includes an image element and a depth map based on the depth values. The method further includes, after the encoding, storing the image data in a file container based on the image format. The method further includes causing the primary image to be displayed.

A wearable display system for a cross reality (XR) system may have a dynamic vision sensor (DVS) camera and a color camera. At least one of the cameras may be a plenoptic camera. The wearable display system may dynamically restrict processing of image data from either or both cameras based on detected conditions and XR function being performed. For tracking an object, image information may be processed for patches of a field of view of either or both cameras corresponding to the object. The object may be tracked based on asynchronously acquired events indicating changes within the patches. Stereoscopic or other types of image information may be used when event-based object tacking yields an inadequate quality metric. The tracked object may be a user's hand or a stationary object in the physical world, enabling calculation of the pose of the wearable display system and of the wearer's head.

A wearable display system including multiple cameras and a processor is disclosed. A greyscale camera and a color camera can be arranged to provide a central view field associated with both cameras and a peripheral view field associated with one of the two cameras. One or more of the two cameras may be a plenoptic camera. The wearable display system may acquire light field information using the at least one plenoptic camera and create a world model using the first light field information and first depth information stereoscopically determined from images acquired by the greyscale camera and the color camera. The wearable display system can track head pose using the at least one plenoptic camera and the world model. The wearable display system can track objects in the central view field and the peripheral view fields using the one or two plenoptic cameras, when the objects satisfy a depth criterion.

A depth camera assembly (DCA) determines depth information. The DCA projects a dynamic structured light pattern into a local area and captures images including a portion of the dynamic structured light pattern. The DCA determines regions of interest in which it may be beneficial to increase or decrease an amount of texture added to the region of interest using the dynamic structured light pattern. For example, the DCA may identify the regions of interest based on contrast values calculated using a contrast algorithm, or based on the parameters received from a mapping server including a virtual model of the local area. The DCA may selectively increase or decrease an amount of texture added by the dynamic structured light pattern in portions of the local area. By selectively controlling portions of the dynamic structured light pattern, the DCA may decrease power consumption and/or increase the accuracy of depth sensing measurements.