In an example embodiment, an image processing device includes a pixel array including pixels two-dimensionally arranged and configured to capture an image, each of the pixels including a plurality of photoelectric conversion elements and an image data processing circuit configured to generate image data from pixel signals output from the pixels. The image processing device further includes a color data processing circuit configured to extract color data from the image data and output extracted color data. The image processing device further includes a depth data extraction circuit configured to extract depth data from the image data and output extracted depth data. The image processing device further includes an output control circuit configured to control the output of the color data and the depth data.

In an example embodiment, an image processing device includes a pixel array including pixels two-dimensionally arranged and configured to capture an image, each of the pixels including a plurality of photoelectric conversion elements and an image data processing circuit configured to generate image data from pixel signals output from the pixels. The image processing device further includes a color data processing circuit configured to extract color data from the image data and output extracted color data. The image processing device further includes a depth data extraction circuit configured to extract depth data from the image data and output extracted depth data. The image processing device further includes an output control circuit configured to control the output of the color data and the depth data.

An image processing device acquires at least two first images and down-samples the at least two first images to obtain at least two second images, where a first resolution of the at least two first images is higher than a second resolution of the at least two second images. By using the at least two first images and the at least two second images, the image processing device respectively determines a first depth map corresponding to the at least two first images under a limit of a first disparity threshold, and a second depth map corresponding to the at least two second images under a limit of a second disparity threshold, where the second disparity threshold is greater than the first disparity threshold. The image processing device then combines the determined first depth map with the second depth map to generate a combined depth map.

An image processing device acquires at least two first images and down-samples the at least two first images to obtain at least two second images, where a first resolution of the at least two first images is higher than a second resolution of the at least two second images. By using the at least two first images and the at least two second images, the image processing device respectively determines a first depth map corresponding to the at least two first images under a limit of a first disparity threshold, and a second depth map corresponding to the at least two second images under a limit of a second disparity threshold, where the second disparity threshold is greater than the first disparity threshold. The image processing device then combines the determined first depth map with the second depth map to generate a combined depth map.

In an example embodiment, an image processing device includes a pixel array including pixels two-dimensionally arranged and configured to capture an image, each of the pixels including a plurality of photoelectric conversion elements and an image data processing circuit configured to generate image data from pixel signals output from the pixels. The image processing device further includes a color data processing circuit configured to extract color data from the image data and output extracted color data. The image processing device further includes a depth data extraction circuit configured to extract depth data from the image data and output extracted depth data. The image processing device further includes an output control circuit configured to control the output of the color data and the depth data.

In an example embodiment, an image processing device includes a pixel array including pixels two-dimensionally arranged and configured to capture an image, each of the pixels including a plurality of photoelectric conversion elements and an image data processing circuit configured to generate image data from pixel signals output from the pixels. The image processing device further includes a color data processing circuit configured to extract color data from the image data and output extracted color data. The image processing device further includes a depth data extraction circuit configured to extract depth data from the image data and output extracted depth data. The image processing device further includes an output control circuit configured to control the output of the color data and the depth data.

An apparatus includes an imaging element for obtaining a plurality of parallax images by receiving a plurality of light flux that passes through different areas of a pupil of an optical system by different pixels, and a processor for performing image processing on an image that is obtained by using the imaging element. The processor includes a first obtaining unit that obtains first shape information based on a plurality of photometric stereo images that are shot by using the imaging element in light source conditions that include at least three different light source conditions, a second obtaining unit that obtains second shape information by using the plurality of parallax images, and a third obtaining unit that obtains third shape information by using the first shape information and the second shape information.

A method and system for processing camera images is presented. The system receives a first depth map generated based on information sensed by a first type of depth-sensing camera, and receives a second depth map generated based on information sensed by a second type of depth-sensing camera. The first depth map includes a first set of pixels that indicate a first set of respective depth values. The second depth map includes a second set of pixels that indicate a second set of respective depth values. The system identifies a third set of pixels of the first depth map that correspond to the second set of pixels of the second depth map, identifies one or more empty pixels from the third set of pixels, and updates the first depth map by assigning to each empty pixel a respective depth value based on the second depth map.

A camera according to an embodiment of the present invention comprises: a light-emitting module for configured to output output light according to a set control mode; a light-receiving module configured to receive input light corresponding to the output light according to the control mode; and a control module configured to detect at least one of presence of a subject and a distance from the subject on the basis of the input light, reset the control mode according to a detection result, control an output of the light-emitting module and an input of the light-receiving module according to the reset control mode, and generate a depth map for the subject on the basis of the input light which is input according to the reset control mode.

A camera according to an embodiment of the present invention comprises: a light-emitting module for configured to output output light according to a set control mode; a light-receiving module configured to receive input light corresponding to the output light according to the control mode; and a control module configured to detect at least one of presence of a subject and a distance from the subject on the basis of the input light, reset the control mode according to a detection result, control an output of the light-emitting module and an input of the light-receiving module according to the reset control mode, and generate a depth map for the subject on the basis of the input light which is input according to the reset control mode.