Highly efficient three-dimensional image acquisition method based on multi-mode composite encoding and epipolar constraint

Abstract

A highly efficient three-dimensional image acquisition method based on multi-mode composite encoding and epipolar constraint, respectively using a fast imaging mode or a high-precision imaging mode, wherein in the fast imaging mode, two phase maps having different frequencies are obtained by four stripe gratings, and a high-frequency absolute phase is obtained by means of the epipolar constraint and a left-right consistency check, and the three-dimensional image is obtained by means of a mapping relationship between the phase and three-dimensional coordinates; and in the high precision imaging mode, two phases having different frequencies are obtained by means of N+2 stripe gratings, a low-frequency absolute phase is obtained by the epipolar constraint, and the unwrapping of a high-frequency phase is assisted by means of the low-frequency absolute phase, so as to obtain the high-frequency absolute phase, and finally, the three-dimensional image is obtained by the mapping relationship between the phase and the three-dimensional coordinates. In this way, the imaging efficiency is ensured, and the imaging precision is improved.

Claims

1. A highly efficient three-dimensional image acquisition method based on multi-mode composite encoding and epipolar constraint, comprising the step of: using a fast imaging mode, comprising: obtaining two phase maps having different frequencies by four stripe gratings; obtaining a high-frequency absolute phase by the epipolar constraint and a left-right consistency test; and obtaining a three-dimensional image by a mapping relationship between the high frequency absolute phase and three-dimensional coordinates; or using a high-precision imaging mode, comprising: obtaining two phases having different frequencies by N+2 stripe gratings, wherein the N+2 stripe gratings are two low frequency sinusoidal stripe gratings plus N high frequency sinusoidal stripe gratings, and N is greater than or equal to 3; obtaining a low-frequency absolute phase by the epipolar constraint, and assisting an unwrapping of a high-frequency phase by the low-frequency absolute phase; obtaining the high-frequency absolute phase; and obtaining the three-dimensional image by the mapping relationship between the high-frequency absolute phase and the three-dimensional coordinates.

2. The method according to claim 1, wherein the step of using the fast imaging mode comprises the steps of: 1) performing a calibration of an imaging system; 2) generating, projecting and collecting four dual-frequency grating fringes; 3) analyzing the grating fringes collected by a left camera and a right camera of the imaging system respectively so as to obtain a set of high frequency phases and a set of low frequency phases; 4) searching for each point on a high frequency phase map of the left camera by using the epipolar constraint, said each point to be searched for being corresponding points of an original point in space, and removing some error points with depth constraint; 5) a projecting remaining spatial corresponding points onto high-frequency and low-frequency phase diagrams of the right camera, and determining a final corresponding point by a phase difference between original points and the corresponding points, so as to obtain the high-frequency absolute phase; and 6) acquiring the three-dimensional image according to the high-frequency absolute phase, so as to realize efficient and precise acquisition of three-dimensional images of dynamic scenes.

3. The method according to claim 2, wherein the imaging system comprises a computer, the left camera the right camera, and a projector, the left camera, the right camera and the projector are respectively connected to the computer through data lines, and the projector is connected with the left camera and the right camera through trigger lines, and wherein the imaging system is calibrated so as to obtain calibration parameters of the left camera, the right camera and the projector in world coordinate system, and the calibration parameters comprise a scaling parameter, a translation parameter, a rotation parameter and a distortion parameter between a pixel coordinate system and the world coordinate system.

4. The method according to claim 2, wherein in the step 2), the four dual-frequency grating fringes generated by the computer through MatLab are two sinusoidal stripe gratings and two triangular wave stripe gratings, and the stripe gratings are as follows:
I.sub.1(x,y)=A(x,y)+B(x,y)sin [F.sub.H(2x/W1)]
I.sub.2(x,y)=A(x,y)+B(x,y)cos [F.sub.H(2x/W1)]
I.sub.3(x,y)=A(x,y)+B(x,y)tri[(2F.sub.Lx/W1)]
I.sub.4(x,y)=A(x,y)B(x,y)tri[(2F.sub.Lx/W1)] where I.sub.i(x,y) represents an intensity of the grating fringe at pixel coordinates (x,y) of the generated image, i=1, 2, 3, 4, representing the i-th grating fringe image, A is an image DC component, B is an amplitude, and tri is a triangular wave function with threshold interval [1, 1], F.sub.H, F.sub.L is respectively number of fringe periods included in I.sub.1, I.sub.2 and I.sub.3, I.sub.4, W is a pixel width of a entire grating fringe image, A=B=127.5, values for F.sub.H, F.sub.L are respectively 64 and 9 and a range of values for x is 0 to W1, wherein the gratings are synchronously captured by the left camera and the right camera after being projected by the projector, wherein the grating fringes collected by the left camera are as follows:
I.sub.1.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)sin .sub.H(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
I.sub.2.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos .sub.H(x.sup.c,y.sup.c)]+(x,y).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
where
I.sub.3.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c).sub.L(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
I.sub.4.sup.c(x.sup.cy.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)B(x.sup.c,y.sup.c).sub.L(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c) I.sub.i.sup.c (x.sup.c,y.sup.c) is a grating fringe image actually captured by the left camera, i=1, 2, 3, 4,(x.sup.c,y.sup.c) is pixel coordinates of the image captured by the camera, and is a surface reflectance of a measured object, .sub.1 is a reflected ambient light, .sub.2 is an ambient light directly into the camera, .sub.H (x.sup.c,y.sup.c) is a phase included in the grating fringe diagram I.sub.1.sup.c and the grating fringe diagram I.sub.2.sup.c, .sub.L(x.sup.c,y.sup.c) is the phase included in the grating fringe diagram I.sub.3.sup.c and the grating fringe diagram I.sub.4.sup.c, wherein assuming
A.sup.c=(x.sup.c,y.sup.c)A(x.sup.c,y.sup.c)+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c),B.sup.c=(x.sup.c,y.sup.c),B(x.sup.c,y.sup.c), and omitting (x.sup.c,y.sup.c), the above four equations can be rewritten as:
I.sub.1.sup.c=A.sup.c+B.sup.c sin .sub.H
I.sub.2.sup.c=A.sup.c+B.sup.c cos .sub.H,
and
I.sub.3.sup.c=A.sup.c+B.sup.c.sub.L
I.sub.4.sup.c=A.sup.cB.sup.c.sub.L wherein an acquisition process of the right camera is the same as an acquisition process of the left camera.

5. The method according to claim 2, wherein the grating fringes collected by the left camera and the right camera respectively are analyzed so as to obtain a set of high frequency phases and a set of low frequency phases, where the phases of the left camera is as follows: $_{H} = \tan^{- 1} \frac{2 I_{1}^{c} - I_{3}^{c} - I_{4}^{c}}{2 I_{2}^{c} - I_{3}^{c} - I_{4}^{c}}$ $_{L} = \frac{I_{3}^{c} - I_{4}^{c}}{2 B^{c}}, B^{c} = \frac{\sqrt{{(2 I_{1}^{c} - I_{3}^{c} - I_{4}^{c})}^{2} + {(2 I_{2}^{c} - I_{3}^{c} - I_{4}^{c})}^{2}}}{2}$ where .sub.H is high-frequency wrapped phases and .sub.L is low-frequency wrapping phases, and the same steps as those of the left camera can be used to determine that the phases corresponding to the stripe gratings collected by the right camera are .sub.H,.sub.L.

6. The method according to claim 2, wherein in the step 4), for any point p on .sub.H, being the original point, its cycle order k has F.sub.H possibilities, and its absolute phase
.sub.H=.sub.H+2k,k[0,F.sub.H1] has F.sub.H different values and from the following formula: $x_{p} = \frac{_{H} W}{2 F_{H}}$ $Z_{p} = M_{Z} + \frac{N_{Z}}{C_{Z} x_{p} + 1},$ wherein p corresponds to F.sub.H points in a three-dimensional space, being F.sub.H corresponding points, wherein M.sub.Z, N.sub.Z and C.sub.Z are derived from the calibration parameters obtained in the step 1), wherein a key to 3D imaging is to identify a only correct corresponding point among the F.sub.H corresponding points, and wherein an effective range of left camera, right camera and the projector is preset as (Z.sub.min, Z.sub.max), all k which make Z exceeds the preset range and its corresponding spatial points will be confirmed as error points and be eliminated; then a correct cycle order k of p and a range of its spatial corresponding points will be reduced to F.sub.H, where F.sub.H<<F.sub.H.

7. The method according to claim 2, wherein the step 5) further comprises the steps of: projecting residuary spatial points in (Z.sub.min, Z.sub.max) in the step 4) onto an imaging surface of the right camera, thus obtaining F.sub.H two-dimensional corresponding points on the imaging surface of right camera; excluding two-dimensional corresponding points whose .sub.diff exceed a threshold 0.5 rad by .sub.diff .sub.H (p).sub.H(p), where p is the corresponding point of p on the right camera, .sub.diff is a difference between the original point and the corresponding points in wrapping phases .sub.H and .sub.H, a range of correct corresponding points is reduced from F.sub.H to F.sub.H; and selecting a corresponding point that makes .sub.L(p).sub.L(p) the smallest among the residuary F.sub.H corresponding points, and considering the point to be a correct corresponding point; then a cycle order k corresponding to the point is a correct cycle order, whereby the unique .sub.H of the original point is confirmed.

8. The method according to claim 2, wherein the step 6) further comprises the step of: obtaining the coordinates of the three-dimensional image by combining the absolute phase .sub.H obtained in the step 5) with the following formula: $x_{p} = \frac{_{H} W}{2 N_{L}}$ $Z_{p} = M_{Z} + \frac{N_{Z}}{C_{Z} x_{p} + 1}$ $X_{p} = E_{X} Z_{p} + F_{X}$ $Y_{p} = E_{Y} Z_{p} + F_{Y}$ where E.sub.X, F.sub.X, E.sub.Y, F.sub.Y are obtained from the calibration parameters in the step 1), X.sub.p, Y.sub.p, Z.sub.p are three-dimensional coordinates of the measured object, whereby the three-dimensional a of the measured object can be obtained.

9. The method according to claim 1, wherein the step of using the high-precision imaging mode comprises the following steps: a) performing a calibration of an imaging system; b) generating, projecting, and acquiring N+2 double-frequency grating stripe patterns, where N is greater than or equal to 3; c) analyzing the grating fringes collected by a left camera and a right camera of the imaging system so as to obtain respectively a set of high frequency phases and a set of low frequency phases; d) searching for each point on high frequency phase maps of the left camera by using the epipolar constraint, said each point to be searched for being corresponding points of an original point in space, and removing some error points with depth restraint; e) projecting remaining spatial corresponding points onto low-frequency phase diagrams of the right camera, and determining a final corresponding point by a phase difference between the original points and the corresponding points, so as to obtain a high-frequency absolute phase through a low-frequency absolute phase; and f) acquiring a three-dimensional image according to the absolute phase, so as to realize efficient and precise acquisition of three-dimensional images of static scenes.

10. The method according to claim 9, wherein in the step b), N+2 the N+2 stripe gratings are two low-frequency sinusoidal stripe gratings plus N high-frequency frequency sinusoidal stripe gratings, where N3, and when N=3, the stripe gratings are as follows:
J.sub.1(x,y)=A(x,y)+B(x,y)cos [N.sub.H(2x/W1)]
J.sub.2(x,y)=A(x,y)+B(x,y)cos [N.sub.H(2x/W1)+2/3]
J.sub.3(x,y)=A(x,y)+B(x,y)cos [N.sub.H(2x/W1)+4/3]
J.sub.4(x,y)=A(x,y)+B(x,y)sin [N.sub.L(2x/W1)]
J.sub.5(x,y)=A(x,y)+B(x,y)cos [N.sub.L(2x/W1)] where J.sub.i(x,y) represents an intensity of the grating fringe at pixel coordinates (x,y) of the generated image, i=1, 2, 3, 4, 5,representing the i-th grating fringe image, and A is a DC component of the image, B is an amplitude, N.sub.H, N.sub.L are a number of fringe periods included in J.sub.1J.sub.3 and J.sub.4, J.sub.5 respectively, W is a pixel width of the entire grating fringe image, A=B=127.5, and values for N.sub.H, N.sub.L are respectively 128 and 8, and a range of values for x is 0 to W1, wherein the gratings are synchronously collected by the e camera and the right camera after being projected by the projector, wherein the grating fringes collected by the left camera are as follows:
J.sub.1.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos .sub.H(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
J.sub.2.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos(.sub.H(x.sup.c,y.sup.c)+2/3)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
J.sub.3.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos(.sub.H(x.sup.c,y.sup.c)+4/3)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
J.sub.4.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)sin .sub.L(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
J.sub.5.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos .sub.L(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c) where J.sub.i.sup.c(x.sup.c,y.sup.c) is a grating fringe image actually captured by the left camera, i=1, 2, 3, 4, 5 (x.sup.c,y.sup.c), is pixel coordinates of the image captured by the camera, and is a surface reflectance of a measured object, .sub.1 is a reflected ambient light, .sub.2 is an ambient light directly into the camera, .sub.H (x.sup.c,y.sup.c) is a phase included in the grating fringe map I.sub.1.sup.c and the grating fringe map I.sub.3.sup.c, .sub.L(x.sup.c,y.sup.c) is a phase included in the grating fringe diagram I.sub.4.sup.c the grating fringe diagram I.sub.5.sup.c, wherein assuming
A.sup.c=(x.sup.c,y.sup.c)A(x.sup.c,y.sup.c)+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c),B.sup.c=(x.sup.c,y.sup.c)B(x.sup.c,y.sup.c), and omitting (x.sup.c,y.sup.c), the above equations can be rewritten as:
J.sub.1.sup.c=A.sup.c+B.sup.c cos .sub.H
J.sub.2.sup.c=A.sup.c+B.sup.c cos(.sub.H+2/3)
J.sub.3.sup.c=A.sup.c+B.sup.c cos(.sub.H+4/3),
and
J.sub.4.sup.c=A.sup.c+B.sup.c sin .sub.L
J.sub.5.sup.c=A.sup.c+B.sup.c cos .sub.L wherein an acquisition process of the right camera is the same as an acquisition process of the left camera.

11. The method according to claim 9, wherein in the step c), the grating fringes collected by left camera and right camera respectively are analyzed so as to obtain a set of high frequency phases and a set of low frequency phases, where the phase of the left camera is as follows: $_{H} = \arctan \frac{\sqrt{3} (J_{1}^{c} - J_{3}^{c})}{(2 J_{2}^{c} - J_{1}^{c} - J_{3}^{c})}$ $_{L} = \arctan \frac{J_{4}^{c} - A^{c}}{J_{5}^{c} - A^{c}}, A^{c} = \frac{J_{1}^{c} + J_{2}^{c} + J_{3}^{c}}{3}$ where .sub.H is a high-frequency wrapping phase and .sub.L is a low-frequency wrapping phase, and the same steps as those of the left camera can be used to determine that the phases corresponding to the stripe gratings collected by the right camera are .sub.H,.sub.L.

12. The method according to claim 9, wherein in the step d), for any point q on .sub.L, being the original point, its cycle order k has N.sub.L possibilities, and its absolute phase
.sub.L=.sub.L+2l,l[0,N.sub.L1] has N.sub.L different values and from the following formula: $x_{q} = \frac{_{L} W}{2 N_{L}}$ $Z_{q} = M_{Z} + \frac{N_{Z}}{C_{Z} x_{q} + 1},$ wherein q corresponds to N.sub.L points in a three-dimensional space, being corresponding points, where M.sub.Z, N.sub.Z and C.sub.Z are derived from the calibration parameters obtained in the step a), wherein a key to 3D imaging is to identify an only correct corresponding point among N.sub.L corresponding points, and wherein an effective range of the left camera, the right camera and the projector is preset as (Z.sub.min, Z.sub.max), all l which make Z.sub.q exceeds the preset range and its corresponding spatial points will be confirmed as error points and be eliminated; and then a range of correct cycle order q and its spatial corresponding points will be reduced to N.sub.L, where N.sub.L<<N.sub.L.

13. The method according to claim 9, wherein the step e) further comprises the steps of: projecting residuary spatial points in (Z.sub.min, Z.sub.max) in step d) onto an imaging surface of the right camera, and obtaining N.sub.L two-dimensional corresponding points on the imaging surface of the right camera; confirming an unique .sub.L of the original point, wherein since N.sub.L<<N.sub.L, a corresponding point making .sub.L(p).sub.L(p) the smallest can be selected from N.sub.L corresponding points, and considering that the point is a correct corresponding point, then the corresponding cycle order l of the correct point is a correct cycle order .sub.L; and obtaining the final high frequency absolute phase .sub.H by the following equation: $l = round (\frac{N_{H}_{L} / N_{L} -_{H}}{2})$ $_{H} =_{H} + 2 l .$

14. The method according to claim 9, wherein the step f) further comprises reconstructing the three-dimensional image according to the absolute phase by: obtaining the coordinates of the three-dimensional image by combining the absolute phase .sub.H obtained in the step e) with the following formula: $x_{q} = \frac{_{H} W}{2 N_{L}}$ $Z_{q} = M_{Z} + \frac{N_{Z}}{C_{Z} x_{q} + 1}$ $X_{q} = E_{X} Z_{q} + F_{X}$ $Y_{q} = E_{Y} Z_{q} + F_{Y}$ where E.sub.X,F.sub.X,E.sub.Y,F.sub.Y are obtained from the calibration parameters in the step a), X.sub.q, Y.sub.q, Z.sub.q are three-dimensional coordinates of the measured object, thereby obtaining the three-dimensional image of the measured object.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

(1) FIG. 1 is the flow diagram of the steps of a fast imaging mode.

(2) FIG. 2 is the test result of the fast imaging mode, and FIG. 2(a) and FIG. 2(b) are respectively the three-dimensional image of a dynamic object acquired by the method and the prior art, and FIG. 2(c) and FIG. 2(f) are partial details of FIG. 2(a), FIG. 2(d) and FIG. 2(g) are partial details of FIG. 2(b), and FIG. 2(e) and FIG. 2(h) are the comparison of partial details.

(3) FIG. 3 is the flow diagram of the steps of a high precision imaging mode.

(4) FIG. 4 is the test result of a high-precision imaging mode. FIG. 4(a) shows the result of the three-dimensional imaging of a standard ceramic plate, and FIG. 4(b) is a detail display of a line in FIG. 4(a).

DESCRIPTION OF THE PREFERRED EMBODIMENT

(5) Combining FIG. 1 with FIG. 3, the present invention is based on a highly efficient three-dimensional image acquisition method based on multi-mode composite encoding and the epipolar constraint, respectively using a fast imaging mode or a high-precision imaging mode; the two imaging modes share the same imaging system and the same imaging system calibration results and use the epipolar constraint combing with the auxiliary phase information as the core technology of phase unwrapping. Compared with a traditional multi-frequency time phase unwrapping algorithm, the two imaging modes are greatly improved in efficiency; and compared with a traditional spatial phase unwrapping algorithm, the two imaging modes are greatly improved in efficiency and accuracy. The steps of the fast imaging mode are as follows:

(6) step one, imaging system calibration;

(7) step two, generating, projecting and collecting four dual-frequency grating fringes;

(8) step three, analyzing the grating fringes collected by left camera and right camera respectively so as to obtain a set of high frequency phases and a set of low frequency phases;

(9) step four, searching for each point on the high frequency phase maps of the left camera by using the epipolar constraint, that is, the corresponding point of the original point in space, and removing some error points by the depth constraint;

(10) step five, projecting the remaining spatial corresponding points onto the high-frequency and low-frequency phase maps of the right camera, and determining the final corresponding point by the phase difference between the original points and the corresponding points, so as to obtain a high-frequency absolute phase;
step six, acquiring a three-dimensional image according to the absolute phase, so as to realize efficient and precise acquisition of three-dimensional images of dynamic scenes
the high-precision imaging mode comprises the following steps:
step one, imaging system calibration;
step two, generating, projecting, and acquiring N+2 double-frequency grating stripe patterns;
step three, analyzing the grating fringes collected by left camera and right camera respectively so as to obtain a set of high frequency phases and a set of low frequency phases;
step four, using the epipolar constraint to search for each point on the high frequency phase maps of the left camera, that is, the corresponding point of the original point in space, and removing some error points by the depth constraint;
step five, the remaining space corresponding points are projected to the low-frequency phase map of the right camera, projecting the remaining spatial corresponding points onto the low-frequency phase maps of the right camera, and determining the final corresponding point by the phase difference between the original points and the corresponding points, so as to obtain a high-frequency absolute phase through a low-frequency absolute phase;
step six, acquiring a three-dimensional image according to the absolute phase, so as to realize efficient and precise acquisition of three-dimensional images of static scenes.

(11) The process of the two imaging modes is described in detail below.

(12) The flow diagram of the steps of the fast imaging mode of the present invention is shown in FIG. 1, and the specific implementation steps are as follows:

(13) step one, imaging system is calibrated.

(14) The imaging system comprises a computer, a left camera, a right camera, and a projector, wherein the left camera, the right camera and the projector are respectively connected to the computer through data lines, and the projector is connected with the left camera and the right camera through trigger lines; After the imaging system is built, the calibration method mentioned in A flexible new technique for camera calibration. by Z. Zhang (Z Zhang, a flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence. 22 (11), 1330-1334 (2000).) is used for the imaging system calibration so as to obtain calibration parameters of the left camera, the right camera and the projector in a world coordinate system, wherein the calibration parameters comprise a scaling parameter, a translation parameter, a rotation parameter and a distortion parameter between the pixel coordinate system and the world coordinate system.

(15) step two, four dual-frequency grating fringes are generated, projected and collected. the four dual-frequency grating fringes generated by the computer through MatLab are two sinusoidal stripe gratings and two triangular wave stripe gratings, and the stripe gratings are as follows:
I.sub.1(x,y)=A(x,y)+B(x,y)sin [F.sub.H(2x/W1)]
I.sub.2(x,y)=A(x,y)+B(x,y)cos [F.sub.H(2x/W1)]
I.sub.3(x,y)=A(x,y)+B(x,y)tri[(2F.sub.Lx/W1)]
I.sub.4(x,y)=A(x,y)B(x,y)tri[(2F.sub.Lx/W1)]
where I.sub.i(x,y) represents the intensity of the grating fringe at the pixel coordinates (x,y) of the generated image, i=1, 2, 3, 4 representing the i-th grating fringe image, A is the image DC component, B is the amplitude, and tri is the triangular wave function with threshold interval [1, 1], F.sub.H, F.sub.L is the number of fringe periods included in I.sub.1, I.sub.2 and I.sub.3, I.sub.4. W is the pixel width of the entire grating fringe image, A=B=127.5, the values for F, are respectively 64 and 9, and the range of values for x is 0 to W1; the gratings are synchronously collected by the left camera and the right camera after being projected by the projector. For the sake of simplicity, only the left camera is analyzed here. The analysis process of the right camera is the same as that of the left camera. The grating fringes collected by the left camera are as follows:
I.sub.1.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)sin .sub.H(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
I.sub.2.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos .sub.H(x.sup.c,y.sup.c)]+(x,y).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
I.sub.3x(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c).sub.L(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
I.sub.4(x.sup.cy.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)B(x.sup.c,y.sup.c).sub.L(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
where I.sub.1.sup.c(x.sup.c,y.sup.c) is the grating fringe image actually captured by the left camera, i=1, 2, 3, 4. (x.sup.c,y.sup.c) is the pixel coordinates of the image captured by the camera, and is the surface reflectance of the measured object, .sub.1 is the reflected ambient light, .sub.2 is the ambient light directly into the camera, .sub.H(x.sup.c,y.sup.c) is the phase included in the grating fringe diagram I.sub.1.sup.c and the grating fringe diagram I.sub.2.sup.c, .sub.L (x.sup.c,y.sup.c) is the phase included in the grating fringe diagram I.sub.3.sup.c and the grating fringe diagram I.sub.4.sup.c; assuming
A.sup.c=(x.sup.c,y.sup.c)A(x.sup.c,y.sup.c)+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c),B.sup.c=(x.sup.c,y.sup.c)B(x.sup.c,y.sup.c),
and leaving out (x.sup.c,y.sup.c) the above four equations can be reduced to:
I.sub.1.sup.c=A.sup.c+B.sup.c sin .sub.H
I.sub.2.sup.c=A.sup.c+B.sup.c cos .sub.H
I.sub.3.sup.c=A.sup.c+B.sup.c.sub.L
I.sub.4.sup.c=A.sup.cB.sup.c.sub.L

(16) The acquisition process of the right camera is the same as the acquisition process of the left camera.

(17) Step three, the grating fringes collected are analyzed by the left camera and the right camera to obtain respectively a set of high-frequency phases and a set of low-frequency phases.

(18) According to the image acquired by the left camera in step two, two sets of phases are obtained as follows:

(19) $_{H} = \tan^{- 1} \frac{2 I_{1}^{c} - I_{3}^{c} - I_{4}^{c}}{2 I_{2}^{c} - I_{3}^{c} - I_{4}^{c}}$ $_{L} = \frac{I_{3}^{c} - I_{4}^{c}}{2 B^{c}}, B^{c} = \frac{\sqrt{{(2 I_{1}^{c} - I_{3}^{c} - I_{4}^{c})}^{2} + {(2 I_{2}^{c} - I_{3}^{c} - I_{4}^{c})}^{2}}}{2}$
where .sub.H is a high-frequency wrapped phase and .sub.L is a low-frequency wrapped phase, and the same steps as those of the left camera can be used to determine that the phases corresponding to the stripe gratings collected by the right camera are .sub.H,.sub.L.
Step four, the epipolar constraint is used to search for each point on the high frequency phase maps of the left camera, that is, the corresponding point of the original point in space, and removing some error points by the depth constraint.

(20) For any point p on .sub.H, that is, the original point, its cycle order k has F.sub.H possibilities, which means that its absolute phase
.sub.H=.sub.H+2k,k[0,F.sub.H1]
has F.sub.H different values and from the following formula:

(21) $x_{p} = \frac{_{H} W}{2 F_{H}}$ $Z_{p} = M_{Z} + \frac{N_{Z}}{C_{Z} x_{p} + 1}$

(22) It can be seen that p corresponds to F.sub.H points in the three-dimensional space, that is, F.sub.H corresponding points, wherein M.sub.Z, N.sub.Z and C.sub.Z are derived from the calibration parameters obtained in step one (K. Liu, Y. Wang, et al Dual-Frequency pattern scheme for high-speed 3-D shape measurement. Optics express. 18(5), 5229-5244 (2010).). There is at most one correct corresponding point in the F.sub.H corresponding points. The key to 3D imaging is to identify the only correct corresponding point among the F.sub.H corresponding points. Considering the limited measurement space range in the actual 3D imaging system, the effective range of the left camera, the right camera and the projector is preset as (Z.sub.min, Z.sub.max), such as Z.sub.min=200 mm, Z.sub.max=200 mm, all k which make Z exceeds this preset range and its corresponding spatial points will be confirmed as error points and be eliminated: after this step is processed, the correct cycle order k of p and the range of its spatial corresponding points will be reduced to F.sub.H, where F.sub.H<<F.sub.H.

(23) Step five, the residuary spatial corresponding points are projected onto the high frequency and low frequency phase maps of the right camera, and the final corresponding point is determined by the phase difference between the original point and the corresponding point, thus obtaining the high frequency absolute phase.

(24) First of all, the residuary spatial points in (Z.sub.min, Z.sub.max) of step four are projected onto the imaging surface of the right camera, and F.sub.H two-dimensional corresponding points on the imaging surface of the right camera are obtained. In fact, the wrapped phase .sub.H,.sub.L of the correct corresponding point p and the wrapped phase .sub.H, .sub.L of the original point should be very close, so that, the two-dimensional corresponding points whose .sub.diff exceed the threshold of 0.5 rad are further excluded by the formula
.sub.diff=.sub.H(p).sub.H(p)
(the rad is the radian unit and the threshold size is a prior determination) where p is the corresponding point of p on the right camera, .sub.diff is the difference between the original point and the corresponding point in the wrapped phases .sub.H and .sub.H, the range of the correct corresponding points is reduced from F.sub.H to F.sub.H; finally, selecting the corresponding point that makes .sub.L(p).sub.L(p) the smallest among the residuary F.sub.H corresponding points, and considering the point to be the correct corresponding point; then the cycle order k corresponding to the point is the correct cycle order and thus the unique .sub.H of the original point is confirmed.
Step six, acquiring a three-dimensional image according to the absolute phase.

(25) The three-dimensional image coordinates are obtained by combining the absolute phase .sub.H obtained in step five with the following formula:

(26) $x_{p} = \frac{_{H} W}{2 N_{L}}$ $Z_{p} = M_{Z} + \frac{N_{Z}}{C_{Z} x_{p} + 1}$ $X_{p} = E_{X} Z_{p} + F_{X}$ $Y_{p} = E_{Y} Z_{p} + F_{Y}$
Where E.sub.X, F.sub.X, E.sub.Y, F.sub.Y are obtained from the calibration parameters in step one (K. Liu, Y. Wang, et al Dual-frequency pattern scheme for high-speed 3-D shape measurement. Optics express. 18 (5), 5229-5244 (2010).), X.sub.P, Y.sub.P, Z.sub.p are the three-dimensional coordinates of the measured object, and thus the three-dimensional image of the measured object can be obtained. It can be seen through the above steps that the fast imaging mode of the present invention utilizes four grating fringes to obtain a three-dimensional image of the measured object, and compared with the prior art, the combination of the four composite coding grating fringes ensures the high efficiency of three-dimensional image measurement; at the same time, the introduction of the epipolar constraint in the binocular vision enables the technique to obtain high-precision phases with up to 64 fringe periods by using four composite grating stripes, and the high precision of the three-dimensional image is ensured and finally, since the high-frequency absolute phase is directly solved by the epipolar constraint, it does not rely on the low-frequency absolute phase, which avoids the inaccuracy of the three-dimensional image caused by the difference in modulation degree between the two frequency stripe gratings.

(27) In order to test the effect of a highly efficient three-dimensional image acquisition method based on multi-mode composite encoding and the epipolar constraint, the present invention gives two sets of test results. FIG. 2(a) and FIG. 2(b) are respectively the three-dimensional image of a dynamic object acquired by the method and the prior art, and FIG. 2(c) and FIG. 2(f) are partial details of FIG. 2(a), FIG. 2(d) and FIG. 2(g) are partial details of FIG. 2(b), and FIG. 2(e) and FIG. 2(h) are the comparison of partial details.

(28) It can be seen from these experimental results that the fast imaging mode of the present invention retains more detailed measurement results than the prior art, and the imaging precision is greatly improved while ensuring high-efficiency three-dimensional imaging.

(29) The flow diagram of the steps of the high-precision imaging mode of the present invention is shown in FIG. 3, and the specific implementation steps are as follows:

(30) Step one, imaging system is calibrated

(31) The imaging system comprises a computer, a left camera, a right camera, and a projector, wherein the left camera, the right camera and the projector are respectively connected to the computer through data lines, and the projector is connected with the left camera and the right camera through trigger lines; After the imaging system is built, the calibration method mentioned in A flexible new technique for camera calibration. by Z. Zhang (Z. Zhang, a flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence. 22 (11), 1330-1334 (2000).) is used for the imaging system calibration so as to obtain calibration parameters of the left camera, the right camera and the projector in a world coordinate system.

(32) Step two, N+2 double-frequency grating stripe patterns are generated, projected, and acquired.

(33) N+2 stripe gratings are two low frequency sinusoidal stripe gratings plus N high frequency sinusoidal stripe gratings, where N3, for the sake of simplicity, taking N=3 as an example, the stripe gratings are as follows:
J.sub.1(x,y)=A(x,y)+B(x,y)cos [N.sub.H(2x/W1)]
J.sub.2(x,y)=A(x,y)+B(x,y)cos [N.sub.H(2x/W1)+2/3]
J.sub.3(x,y)=A(x,y)+B(x,y)cos [N.sub.H(2x/W1)+/43]
J.sub.4(x,y)=A(x,y)+B(x,y)sin [N.sub.L(2x/W1)]
J.sub.5(x,y)=A(x,y)+B(x,y)cos [N.sub.L(2x/W1)]
where J.sub.i(x,y) represents the intensity of the grating fringe at the pixel coordinates (x,y) of the generated image, i=1, 2, 3, 4, 5, representing the i-th grating fringe image, and A is the DC component of the image, B is the amplitude, N.sub.H, N.sub.L are the number of fringe periods included in J.sub.1J.sub.3 and J.sub.4, J.sub.5 respectively, W is the pixel width of the entire grating fringe image, A=B=127.5, and the values for N.sub.H, N.sub.L are respectively 128 and 8, and the range of values for x is 0 to W1; the gratings are synchronously collected by the left camera and the right camera after being projected by the projector. For the sake of simplicity, only the left camera is analyzed here. The analysis process of the right camera is the same as that of the left camera. The grating fringes collected by the left camera are as follows:
J.sub.1.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos .sub.H(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
J.sub.2.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos(.sub.H(x.sup.c,y.sup.c)+2/3)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
J.sub.3.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos(.sub.H(x.sup.c,y.sup.c)+4/3)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
J.sub.4.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)sin .sub.I(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
J.sub.5.sup.c(x.sup.c,y.sup.c)=(x.sup.c,y.sup.c)[A(x.sup.c,y.sup.c)+B(x.sup.c,y.sup.c)cos .sub.I(x.sup.c,y.sup.c)]+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c)
where J.sub.1.sup.c(x.sup.c,y.sup.c) is a grating fringe image actually captured by the left camera, i=1, 2, 3, 4, 5, (x.sup.c,y.sup.c) is the pixel coordinates of the image captured by the camera, and is the surface reflectance of the measured object, .sub.1 the reflected ambient light, .sub.2 is the ambient light directly into the camera, .sub.H (x.sup.c,y.sup.c) is the phase included in the grating fringe diagram I.sub.1.sup.c and the grating fringe diagram I.sub.x.sup.3, .sub.L (x.sup.c,y.sup.c) is the phase included in the grating fringe diagram I.sub.4.sup.c and the grating fringe diagram I.sub.5.sup.c; assuming
A=(x.sup.c,y.sup.c)A(x.sup.c,y.sup.c)+(x.sup.c,y.sup.c).sub.1(x.sup.c,y.sup.c)+.sub.2(x.sup.c,y.sup.c),B.sup.c=(x.sup.c,y.sup.c)B(x.sup.c,y.sup.c),
and leaving out (x.sup.c,y.sup.c), the above four equations can be reduced to:
J.sub.1.sup.c=A.sup.c+B.sup.c cos .sub.H
J.sub.2.sup.c=A.sup.c+B.sup.c cos(.sub.H+2/3)
J.sub.3.sup.c=A.sup.c+B.sup.c cos(.sub.H+4/3)
J.sub.4.sup.c=A.sup.c+B.sup.3 sin .sub.I
J.sub.5.sup.c=A.sup.c+B.sup.c cos .sub.I

(34) The acquisition process of the right camera is the same as that of the above left camera.

(35) Step three, the grating fringes collected by left camera and right camera respectively are analyzed so as to obtain a set of high frequency phases and a set of low frequency phases, where the phase of the left camera is as follows:

(36) $_{H} = \arctan \frac{\sqrt{3} (J_{1}^{c} - J_{3}^{c})}{(2 J_{2}^{c} - J_{1}^{c} - J_{3}^{c})}$ $_{L} = \arctan \frac{J_{4}^{c} - A^{c}}{J_{5}^{c} - A^{c}}, A^{c} = \frac{J_{1}^{c} + J_{2}^{c} + J_{3}^{c}}{3}$
Where .sub.H is a high-frequency wrapped phase and .sub.L is a low-frequency wrapped phase, and the same steps as those of the left camera can be used to determine that the phases corresponding to the stripe gratings collected by the right camera are .sub.H,.sub.L.
Step four, using the epipolar constraint to search for the corresponding spatial point of each point (original point) on the high frequency phase map of the left camera, and eliminating some error points by the depth constraint.

(37) For any point q on .sub.L, that is, the original point, its cycle order k has N.sub.L possibilities, which means that its absolute phase
.sub.L=.sub.L+2,l[0,N.sub.L1]
has N.sub.L different values and from the following formula:

(38) $x_{q} = \frac{_{L} W}{2 N_{L}}$ $Z_{q} = M_{Z} + \frac{N_{Z}}{C_{Z} x_{q} + 1}$

(39) It can be seen that q corresponds to N.sub.L points in the three-dimensional space, that is, corresponding points, wherein M.sub.Z, N.sub.Z and C.sub.Z are derived from the calibration parameters obtained in step one (K. Liu. Y. Wang, et al Dual-frequency pattern Scheme for high-speed 3-D shape measurement. Optics express. 18(5). 229-5244 (2010).); the key to 3D imaging is to identify the only correct corresponding point among the N.sub.L corresponding points. Considering the limited measurement space range in the actual 3D imaging system, the effective range of the left camera, the right camera and the projector is preset as (Z.sub.min, Z.sub.max), such as Z.sub.min=200 mm, Z.sub.min=200 mm, all l which make Z.sub.q exceeds this preset range and its corresponding spatial points will be confirmed as error points and be eliminated; after this step, the correct cycle level of q and the range of its spatial corresponding points will be reduced to N.sub.L, where N.sub.L<<N.sub.L.

(40) Step 5, the residuary spatial corresponding points are projected onto the low frequency phase map of the right camera, and the final corresponding point is determined by the phase difference between the original point and the corresponding point, and the high frequency absolute phase is obtained by means of the low frequency absolute phase.

(41) First of all, the residuary spatial points in (Z.sub.min, Z.sub.max) of step four are projected onto the imaging surface of the right camera, and F.sub.H two-dimensional corresponding points on the imaging surface of the right camera are obtained.

(42) In fact, the wrapped phase .sub.L the correct corresponding point q and the wrapped phase .sub.L of the original point should be very close, since N.sub.L<<N.sub.L, the corresponding point making .sub.L (p).sub.L(p) the smallest can be selected from N.sub.L corresponding points, and considering that the point is a correct corresponding point, then the corresponding, cycle level l of the correct point is the correct cycle level, so that the unique .sub.L of the original point is confirmed.

(43) Finally, the final high frequency absolute phase .sub.H is obtained by the following equation

(44) $l = round (\frac{N_{H}_{L} / N_{L} -_{H}}{2})$ $_{H} =_{H} + 2 l$
Step six, a three-dimensional image is acquired according to the absolute phase.

(45) The coordinates of the three-dimensional image are obtained by combining the absolute phase .sub.H obtained in step five with the following formula:

(46) $x_{q} = \frac{_{H} W}{2 N_{L}}$ $Z_{q} = M_{Z} + \frac{N_{Z}}{C_{Z} x_{q} + 1}$ $X_{q} = E_{X} Z_{q} + F_{X}$ $Y_{q} = E_{Y} Z_{q} + F_{Y}$
where E.sub.X, F.sub.X, E.sub.Y, F.sub.Y are obtained from the calibration parameters in step one (K. Liu, Y. Wang, et al Dual-frequency pattern scheme for high-speed 3-D shape measurement. Optics express. 18 (5), 5229-5244 (2010).), X.sub.q, Y.sub.q, Z.sub.q are the three-dimensional coordinates of the measured object, and thus obtaining the three-dimensional image of the measured object.

(47) From the above four steps, it can be seen that the high-precision imaging mode of the invention acquires the three-dimensional image of the measured object through the N+2 grating fringes, compared with the prior art, the introduction of the epipolar constraint enables the 8-cycle low-frequency absolute phase to be directly obtained through the two low-frequency grating fringes, which greatly avoids the redundancy in the prior art that utilizes multi-frame grating fringes (usually much larger than 2) to obtain low-frequency absolute phase, thus improving measurement efficiency. On the other hand, the coding pattern of N 128 fringe periods ensures that the precision of the finally acquired three-dimensional image is not lower than the precision of the prior art.

(48) In order to test the effect of a highly efficient three-dimensional image acquisition method based on multi-mode composite encoding and the epipolar constraint, one set of experimental results are presented.

(49) FIG. 4(a) shows the result of the three-dimensional imaging of a static object in a high-precision imaging mode, FIG. 4(b) shows the details of a line in FIG. 4(a), and From FIG. 4, it can be seen the high-precision imaging mode of the invention can achieve imaging precision of the static object up to 50 um.

Highly efficient three-dimensional image acquisition method based on multi-mode composite encoding and epipolar constraint

Assignee

Inventors

Cpc classification

Classification Explorer

G01B11/2527

PHYSICS

Classification Explorer

G01B11/2545

PHYSICS

Classification Explorer

H04N13/239

ELECTRICITY

Classification Explorer

H04N13/246

ELECTRICITY

Classification Explorer

H04N2013/0081

ELECTRICITY

Classification Explorer

H04N23/667

ELECTRICITY

Classification Explorer

G01B11/2536

PHYSICS

International classification

Classification Explorer

H04N13/246

ELECTRICITY

Classification Explorer

H04N5/232

ELECTRICITY

Classification Explorer

G01B11/25

PHYSICS

Classification Explorer

H04N13/239

ELECTRICITY

Abstract

Claims

Description