In an example embodiment, an image processing device includes a pixel array including pixels two-dimensionally arranged and configured to capture an image, each of the pixels including a plurality of photoelectric conversion elements and an image data processing circuit configured to generate image data from pixel signals output from the pixels. The image processing device further includes a color data processing circuit configured to extract color data from the image data and output extracted color data. The image processing device further includes a depth data extraction circuit configured to extract depth data from the image data and output extracted depth data. The image processing device further includes an output control circuit configured to control the output of the color data and the depth data.

In an example embodiment, an image processing device includes a pixel array including pixels two-dimensionally arranged and configured to capture an image, each of the pixels including a plurality of photoelectric conversion elements and an image data processing circuit configured to generate image data from pixel signals output from the pixels. The image processing device further includes a color data processing circuit configured to extract color data from the image data and output extracted color data. The image processing device further includes a depth data extraction circuit configured to extract depth data from the image data and output extracted depth data. The image processing device further includes an output control circuit configured to control the output of the color data and the depth data.

A sensing data processing device is provided. The sensing data processing device may track keypoints within a full observable range of the sensing data processing device based on sensing data collected from a previous frame and a sensing data collected from a current frame through a reference sensor, the full observable range determined based on a combined field of view (FOV) of sensors, manage a new keypoint together with the keypoints in response to the new keypoint being detected, and perform localization of the device based on the managed keypoints.

A sensing data processing device is provided. The sensing data processing device may track keypoints within a full observable range of the sensing data processing device based on sensing data collected from a previous frame and a sensing data collected from a current frame through a reference sensor, the full observable range determined based on a combined field of view (FOV) of sensors, manage a new keypoint together with the keypoints in response to the new keypoint being detected, and perform localization of the device based on the managed keypoints.

Exterior lighting on vehicles and buildings is typically used to illuminate scenes for better vision. The same exterior lighting can be used as part of an active sensor at discrete times during the sensor's active imaging cycles. In embodiments, an intelligent control module enables emitter output in accordance with the imaging system during active imaging cycles and enables emitter output in accordance with the non-imaging lighting control unit during non-imaging cycles. Embodiments of intelligent control modules can be used in security applications, in Automatic Driving Alert Systems and Autonomous Control Systems for commercial and passenger vehicles, and in low-altitude aircraft applications.

Exterior lighting on vehicles and buildings is typically used to illuminate scenes for better vision. The same exterior lighting can be used as part of an active sensor at discrete times during the sensor's active imaging cycles. In embodiments, an intelligent control module enables emitter output in accordance with the imaging system during active imaging cycles and enables emitter output in accordance with the non-imaging lighting control unit during non-imaging cycles. Embodiments of intelligent control modules can be used in security applications, in Automatic Driving Alert Systems and Autonomous Control Systems for commercial and passenger vehicles, and in low-altitude aircraft applications.

Disclosed are an electronic device generating a multi-plane image of an arbitrary viewpoint and an operating method thereof. The electronic device includes a memory that stores a plurality of images, and at least one processor that executes a machine learning-based multi-plane image generating module generating an image of an arbitrary viewpoint from the plurality of images stored in the memory. When the multi-plane image generating module is executed, the processor selects at least two images from the plurality of images, by performing geometric calibration on the plurality of images, calculates a plane sweep volume including at least two layer images, performs an operation of a deep neural network to generate a plurality of multi-plane images having probability values, and generates the image of the arbitrary viewpoint by composing the plurality of multi-plane images and stores the image of the arbitrary viewpoint in the memory.

Systems and methods for determining disparity between two images are disclosed. Such systems and methods include obtaining a first raw pixel image of a scene from a first viewpoint, obtaining a second raw pixel image of the scene from a second viewpoint (e.g., separate from the first viewpoint in a camera baseline direction such as horizontal or vertical), modifying the first and second raw pixel images using component-separated correction to create respective first and second corrected pixel images maintaining pixel scene correspondence in the camera baseline direction from between the first and second raw pixel images to between the first and second corrected pixel images, determining pixel pairs from corresponding pixels between the first and second corrected pixel images in the camera baseline direction, and determining disparity correspondence for each of the determined pixel pairs from pixel locations in the first and second raw pixel images corresponding to respective pixel locations of the pixel pairs in the first and second corrected pixel images.

Systems and methods for determining disparity between two images are disclosed. Such systems and methods include obtaining a first raw pixel image of a scene from a first viewpoint, obtaining a second raw pixel image of the scene from a second viewpoint (e.g., separate from the first viewpoint in a camera baseline direction such as horizontal or vertical), modifying the first and second raw pixel images using component-separated correction to create respective first and second corrected pixel images maintaining pixel scene correspondence in the camera baseline direction from between the first and second raw pixel images to between the first and second corrected pixel images, determining pixel pairs from corresponding pixels between the first and second corrected pixel images in the camera baseline direction, and determining disparity correspondence for each of the determined pixel pairs from pixel locations in the first and second raw pixel images corresponding to respective pixel locations of the pixel pairs in the first and second corrected pixel images.

An image capturing apparatus including a lens and a processing unit, wherein the lens includes a first region through which a first light ray passes and a second region through which a second light ray passes, wherein the first region and the second region are arranged in a predetermined direction, and wherein the processing unit sets a component of the predetermined direction as a degree of freedom in a first relative positional relationship between a predetermined position in the first region and a predetermined position in the second region is employed.