This invention relates to a sensor and sensor platform, for an autonomous system. The sensor and its platform sense, perform signal or data processing, and make the decision locally at the point of sensing. More specifically, the sensor along with its platform simulates the human-like or human capacity to make decisions by combing the data from several sensors that detect different data sets, and combine them in a series of data processes that allows autonomous decisions to be made. Additionally, the sensor platform combines multiple sensors in one metasensor with the functionality of multiple sensors placed on a common carrier or platform.

Provided is an image display device including a display element configured to modulate light to form an image; a light transfer unit configured to transmit the image formed by the display element to eyes of an observer, the light transfer unit comprising a focusing member; and a driver configured to drive the display element to change a position of the display element, the driver including: a first actuator comprising a deformable wire; a housing that fixes a first end and a second end of the deformable wire; and an operating structure through which the deformable wire is routed, wherein the operating structure is configured move the display element in a first direction in conjunction with a length variation of the deformable wire, and the operating structure includes an elastic structure configured to prevent tilt with respect to an axis in the first direction.

Aspects of the disclosure relate to an apparatus including multiple image sensors sharing one or more optical paths for imaging. An example method includes identifying whether a device including a first aperture, a first image sensor, a second image sensor, and an optical element is to be in a first device mode or a second device mode. The method also includes controlling the optical element based on the identified device mode. The optical element directs light from the first aperture to the first image sensor in a first optical element mode. Light from the first aperture is directed to the second image sensor when the optical element is in the second optical element mode.

A camera tracking system for computer assisted navigation during surgery operatively determines a first pose of a second extended-reality (XR) headset relative to stereo tracking cameras located on a first XR headset based on first tracking information from the stereo tracking cameras. The camera tracking system determines a second pose of eyes of a user wearing the second XR headset relative to the stereo tracking cameras located on the first XR headset based on second tracking information from the stereo tracking cameras. The camera tracking system also calibrates an eye-to-display relationship defining pose of the eyes of the user wearing the second XR headset to a display device of the second XR headset based on the determined first and second poses. The camera tracking system also controls where symbols are displayed on the display device of the second XR headset based on the eye-to-display relationship.

One or more stereoscopic images are generated based on a single monoscopic image that may be obtained from a camera sensor. Each stereoscopic image includes a first digital image and a second digital image that, when viewed using any suitable stereoscopic viewing technique, result in a user or software program receiving a three-dimensional effect with respect to the elements included in the stereoscopic images. The monoscopic image may depict a geographic setting of a particular geographic location and the resulting stereoscopic image may provide a three-dimensional (3D) rendering of the geographic setting. Use of the stereoscopic to image helps a system obtain more accurate detection and ranging capabilities. The stereoscopic image may be any configuration of the first digital image (monoscopic) and the second digital image (monoscopic) that together may generate a 3D effect as perceived by a viewer or software program.

In various examples, systems and methods are disclosed that detect hazards on a roadway by identifying discontinuities between pixels on a depth map. For example, two synchronized stereo cameras mounted on an ego-machine may generate images that may be used extract depth or disparity information. Because a hazard's height may cause an occlusion of the driving surface behind the hazard from a perspective of a camera(s), a discontinuity in disparity values may indicate the presence of a hazard. For example, the system may analyze pairs of pixels on the depth map and, when the system determines that a disparity between a pair of pixels satisfies a disparity threshold, the system may identify the pixel nearest the ego-machine as a hazard pixel.

In various examples, systems and methods are disclosed that detect hazards on a roadway by identifying discontinuities between pixels on a depth map. For example, two synchronized stereo cameras mounted on an ego-machine may generate images that may be used extract depth or disparity information. Because a hazard's height may cause an occlusion of the driving surface behind the hazard from a perspective of a camera(s), a discontinuity in disparity values may indicate the presence of a hazard. For example, the system may analyze pairs of pixels on the depth map and, when the system determines that a disparity between a pair of pixels satisfies a disparity threshold, the system may identify the pixel nearest the ego-machine as a hazard pixel.

Using the same image sensor to capture both a two-dimensional (2D) image of a three-dimensional (3D) object and 3D depth measurements for the object. A laser point-scans the surface of the object with light spots, which are detected by a pixel array in the image sensor to generate the 3D depth profile of the object using triangulation. Each row of pixels in the pixel array forms an epipolar line of the corresponding laser scan line. Timestamping provides a correspondence between the pixel location of a captured light spot and the respective scan angle of the laser to remove any ambiguity in triangulation. An Analog-to-Digital Converter (ADC) in the image sensor generates a multi-bit output in the 2D mode and a binary output in the 3D mode to generate timestamps. Strong ambient light is rejected by switching the image sensor to a 3D logarithmic mode from a 3D linear mode.

Stereoscopy in which at least one image of a scene is recorded from a first viewing angle, and at least one image of the scene is recorded from a second viewing angle. The scene is recorded multiple times from the first viewing angle. A first combination image is obtained from the various images recorded from the first viewing angle, said combination image according to the stipulation of a comparison algorithm having smaller differences in relation to at least one image also recorded from the second viewing angle or smaller differences in relation to a second combination image obtained from the images from the second viewing angle than each individual image recorded from the first viewing angle. An image of the scene with the depth information is obtained from the first combination image and at least one image from the second viewing angle or the second combination image.

Stereoscopy in which at least one image of a scene is recorded from a first viewing angle, and at least one image of the scene is recorded from a second viewing angle. The scene is recorded multiple times from the first viewing angle. A first combination image is obtained from the various images recorded from the first viewing angle, said combination image according to the stipulation of a comparison algorithm having smaller differences in relation to at least one image also recorded from the second viewing angle or smaller differences in relation to a second combination image obtained from the images from the second viewing angle than each individual image recorded from the first viewing angle. An image of the scene with the depth information is obtained from the first combination image and at least one image from the second viewing angle or the second combination image.