AR elements are occluded in video image frames. A depth map is determined for an image frame of a video received from a video capture device. An AR graphical element for overlaying over the image frame is received. An element distance for AR graphical elements relative to a position of a user of the video capture device (e.g., the geographic position of the video capture device) is also received. Based on the depth map for the image frame, a pixel distance is determined for each pixel in the image frame. The pixel distances of the pixels in the image frame are compared to the element distance, and in response to a pixel distance for a given pixel being less than the element distance, the pixel of the image frame is displayed rather than a corresponding pixel of the AR graphical element.

AR elements are occluded in video image frames. A depth map is determined for an image frame of a video received from a video capture device. An AR graphical element for overlaying over the image frame is received. An element distance for AR graphical elements relative to a position of a user of the video capture device (e.g., the geographic position of the video capture device) is also received. Based on the depth map for the image frame, a pixel distance is determined for each pixel in the image frame. The pixel distances of the pixels in the image frame are compared to the element distance, and in response to a pixel distance for a given pixel being less than the element distance, the pixel of the image frame is displayed rather than a corresponding pixel of the AR graphical element.

Imaging methods aid a user in understanding relative depths of objects or topography within a field of view. In one method, image data is gathered using 3D image sensors, while illumination of an area of interest is alternated with projection of a pattern onto the area of interest. The captured image data is used to display a 3D true color image; a monochrome grid on surfaces of the area of interest; a 3D true color image with a monochrome grid on surfaces of the area of interest; a 2D or 3D false color image where different colors identify different depths of regions of the area of interest; and/or a 2D or 3D true color image of an area of interest with features in the area of interest located a first depth relative to the image sensors highlighted with a common color. In another method, 3D image data is used to generate a 2D image that highlights features at a common depth using the same color.

An imaging system comprises two cameras and a processor. A first camera comprises a first grid of pixels configured to detect light in a broadband channel and a plurality of clusters of color filters sparsely arranged over the first grid of pixels. Each color filter passes light in a color channel to the grid of pixels. A second camera comprises a second grid of pixels configured to detect light in a broadband channel. The processor is configured to generate an aggregate image by combining a first image captured by the first camera and a second image captured by the second camera. The processor is also configured to add color values to the aggregate image based on color data captured by the first camera with the clusters of color filters. Additionally, the processor is configured to present the aggregate image with added color values on an electronic display.

Endoscopic imaging systems are disclosed herein. In some embodiments, an endoscopic imaging system can include an endoscope having a distal tip with a distal face configured to face a scene, such as a portion of a body cavity of a patient. The endoscope can further include first cameras for capturing first image data of the scene, a projector for projecting a light pattern into the scene, and second cameras for capturing second image data of the scene including the light pattern. A processor can be communicatively coupled to the first and second cameras for receiving the first and second image data. The processor can process the first and second image data to generate an image of the scene at the perspective of a virtual camera, and can vary the perspective, the aperture, the focus plane, and/or another parameter of the virtual camera without requiring the endoscope to be physically moved.

A optical system for measuring cervical dilation. The device of the subject invention can be a mono-view and/or stereo-view and can include computer enhancement features to provide two-dimensional (2D) and/or three-dimensional (3D) imaging, depth sensing, viewing, and/or measurement capability.

Geiger mode avalanche photo diodes are solid state photodetectors that are able to detect single photons. Such Geiger mode avalanche photo diodes are also referred to as single-photon avalanche diodes (SPADs). An array of SPADs can be used as a single detector element in an active sensing system, but camera/sensor systems based on SPAD arrays have at least two shortcomings due to ambient light. First, solar background light can hamper the ability to accurately determine depth. Second, ambient light impacts the reflectivity precision because of challenges differentiating between reflected light and ambient light. Embodiments enable active sensors that remove the ambient signal from a sensor's optical input. Other embodiments produce sensor intensity values that have higher precision than typical SPAD array devices. Further embodiments produce sensor depth values that have higher precision than typical SPAD array devices.

Geiger mode avalanche photo diodes are solid state photodetectors that are able to detect single photons. Such Geiger mode avalanche photo diodes are also referred to as single-photon avalanche diodes (SPADs). An array of SPADs can be used as a single detector element in an active sensing system, but camera/sensor systems based on SPAD arrays have at least two shortcomings due to ambient light. First, solar background light can hamper the ability to accurately determine depth. Second, ambient light impacts the reflectivity precision because of challenges differentiating between reflected light and ambient light. Embodiments enable active sensors that remove the ambient signal from a sensor's optical input. Other embodiments produce sensor intensity values that have higher precision than typical SPAD array devices. Further embodiments produce sensor depth values that have higher precision than typical SPAD array devices.

Methods and apparatus for supporting the capture of images of surfaces of an environment visible from a default viewing position and capturing images of surfaces not visible from the default viewing position, e.g., occluded surfaces, are described. Occluded and non-occluded image portions are packed into one or more frames and communicated to a playback device for use as textures which can be applied to a model of the environment where the images were captured. An environmental model includes a model of surfaces which are occluded from view from a default viewing position but which may be viewed is the user shifts the user's viewing location. Occluded image content can be incorporated directly into a frame that also includes non-occluded image data or sent in frames of a separate, e.g., auxiliary content stream that is multiplexed with the main content stream which communicates image data corresponding to non-occluded environmental portions.

Methods and apparatus for supporting the capture of images of surfaces of an environment visible from a default viewing position and capturing images of surfaces not visible from the default viewing position, e.g., occluded surfaces, are described. Occluded and non-occluded image portions are packed into one or more frames and communicated to a playback device for use as textures which can be applied to a model of the environment where the images were captured. An environmental model includes a model of surfaces which are occluded from view from a default viewing position but which may be viewed is the user shifts the user's viewing location. Occluded image content can be incorporated directly into a frame that also includes non-occluded image data or sent in frames of a separate, e.g., auxiliary content stream that is multiplexed with the main content stream which communicates image data corresponding to non-occluded environmental portions.