Acoustic positioning system and method for smartphone and wearable device, and terminal
11686809 · 2023-06-27
Assignee
Inventors
Cpc classification
G01S5/0264
PHYSICS
G01S5/02585
PHYSICS
G01S5/22
PHYSICS
International classification
Abstract
The present disclosure belongs to the technical field of acoustic positioning, and discloses an acoustic positioning system and method for a smartphone and a wearable device, and a terminal. A ranging signal is transmitted by virtue of a base station network, and specific space signals of which the frequencies are 12 kHz to 21 kHz are designed; the ranging signal is received and decoded by virtue of a user terminal, distances from base stations to the user terminal are estimated according to the first arrival signals, and the position of a user is estimated according to a plurality of distances measured on the position of the user. The present disclosure provides the acoustic positioning system (APS) for the smartphone and the wearable device, which is a technology for precise ranging based on acoustic waves.
Claims
1. An acoustic positioning method for a smartphone and a wearable device, wherein the acoustic positioning method for the smartphone and the wearable device comprises: transmitting a ranging signal by virtue of a base station network, receiving and decoding the ranging signal by virtue of a user terminal, estimating distances from base stations to the user terminal according to first arrival signals, and estimating the position of a user according to a plurality of distances measured on the position of the user; wherein the acoustic positioning method for the smartphone and the wearable device comprises the following steps: step 1, designing Chirp signals of which the frequencies are 12 kHz to 21 kHz; step 2, performing precise ranging based on acoustic waves; and step 3, performing positioning based on acoustic ranging and an inertial sensor; wherein performing positioning based on acoustic ranging and an inertial sensor in the step 3 further comprises: by applying a sliding window program to the measurement of an IMU of which the step length is 10 Hz, obtaining the output rate, which is 20 Hz, of a learned velocity vector, wherein a displacement vector is estimated by multiplying a time interval of two continuous periods t.sub.1 and t.sub.2 by the velocity vector and is expressed as follows:
{tilde over (P)}.sub.k=F.sub.kP.sub.k-1F.sub.k.sup.T+Q.sub.k
K.sub.k={tilde over (P)}.sub.kH.sub.k.sup.T(H.sub.k{tilde over (P)}.sub.kH.sub.k.sup.T+R.sub.k).sup.−1
X.sub.k={tilde over (X)}.sub.k+K.sub.k(Z.sub.k−{tilde over (Z)}.sub.k)
P.sub.k=(I−K.sub.kH.sub.k){tilde over (P)}.sub.k, wherein F.sub.k=∂h/∂X.sub.k, H.sub.k=∂h/∂X.sub.k, X.sub.k is state estimation, {tilde over (P)}.sub.k is a predicated covariance matrix, P.sub.k is an updated covariance matrix, K.sub.k is a Kalman gain, and I is a unit vector.
2. The acoustic positioning method for the smartphone and the wearable device of claim 1, wherein the Chirp signals designed in the step 1 are expressed as:
3. The acoustic positioning method for the smartphone and the wearable device of claim 1, wherein performing precise ranging based on acoustic waves in the step 2 comprises: adopting a 50 ms Chirp period for the Chirp signals; generating the given ranging signal s(t) in the smartphone, and performing Fourier transformation on a corresponding signal x(t) received by a microphone to obtain frequency domain descriptions of signals S(f) and X(f); and deducing a generalized cross spectrum G(f) as:
G(f)=E[S(f)X(f).sup.H] wherein E[*] is a mathematical expectation operator, f is a signal frequency, and H is hermitian transformation; adopting a phase transformation and weighting solution to obtain unit gains of all frequency components, and retaining actual delay information;
Ĝ(f)=G(f)/|G(f)| performing inverse Fourier transformation to obtain the following relevant function R(τ): wherein τ is the delay of the relevant function, and the delay Δt of the ranging signal s(t) is obtained by the following formula:
d=Δt-c wherein c is a sound velocity which is 343 m/s; and after a distance among at least three base stations, estimating the position of the user terminal by applying a least square trilateration algorithm or synthesis filtering algorithm.
4. The acoustic positioning method for the smartphone and the wearable device of claim 1, wherein performing positioning based on acoustic ranging and an inertial sensor in the step 3 comprises: estimating the position of the user terminal by integrating an acoustic range and a measured value of the inertial sensor, wherein a data driving part and a model driving part are comprised in data processing for estimating the position of the user, the data driving part estimates a velocity vector of the user based on a deep learning method, and the model driving part integrates the velocity vector and ranging measurement to estimate the position, heading and velocity of the user by using an extended Kalman filter; (1) the data driving part estimates the velocity vector v=[v.sub.x, v.sub.y, v.sub.z].sup.T according to measured values of an accelerometer and a gyroscope sensor on the smartphone or a label, which is a data driving method based on the deep learning method; (2) the model driving part integrates the velocity vector and precise ranging based on acoustic waves to estimate the position, heading and velocity of the user by using the extended Kalman filter.
5. An information data processing terminal, wherein the information data processing terminal is for achieving the acoustic positioning system for the smartphone and the wearable device of claim 4.
6. An acoustic positioning system for a smartphone and a wearable device, to which the acoustic positioning method for the smartphone and the wearable device of claim 1 is applied, wherein the acoustic positioning system for the smartphone and the wearable device comprises a base station network for transmitting a ranging signal, specifically designed space signals, and a user terminal used for receiving and decoding the ranging signal and estimating the position of a user; wherein the base station network is a group of signal transmitters with each consisting of a loudspeaker, a clock synchronization module, a Bluetooth low energy module, and a micro-controller unit, and is used for broadcasting the specifically designed signals in the air, and base station clocks in the network are synchronized with a 433 MHz radio frequency; the space signals are a group of linear frequency modulation signals of which the frequencies are 12 kHz to 21 kHz; the user terminal is used for receiving and decoding the ranging signal, estimating distances from base stations to the user terminal according to first arrival signals, and estimating the position of the user according to a plurality of distances measured on the position of the user; and the user terminal executes a synthetic signal processing program and is used for positioning a smartphone of a person or a label of an object or vehicle and developing a dedicated chip for a label-based user terminal.
Description
BRIEF DESCRIPTION OF DRAWINGS
(1)
(2)
(3)
(4)
(5)
(6)
(7)
(8)
(9)
DETAILED DESCRIPTION OF THE EMBODIMENTS
(10) For the problems existing in the prior art, the present disclosure provides an acoustic positioning system and method for a smartphone and a wearable device, and a terminal, and the present disclosure will be described in detail below with reference to the accompanying drawings.
(11) First, explanation embodiments: in order to make the skilled in the art know about the specific implementation of the present disclosure sufficiently, this part shows the explanation embodiments for describing technical solutions of the claims.
(12) As shown in
(13) S101, Chirp signals of which the frequencies are 12 kHz to 21 kHz are designed;
(14) S102, precise ranging based on acoustic waves is performed; and
(15) S103, positioning based on acoustic ranging and an inertial sensor is performed.
(16) As shown in
(17) The base station network is a group of signal transmitters with each consisting of a loudspeaker, a clock synchronization module, a Bluetooth low energy module, and a micro-controller unit, and is used for broadcasting the specifically designed signals in the air, and base station clocks in the network are synchronized with a 433 MHz radio frequency;
(18) the space signals are a group of linear frequency modulation signals of which the frequencies are 12 kHz to 21 kHz;
(19) the user terminal is used for receiving and decoding the ranging signal, estimating distances from base stations to the user terminal according to first arrival signals, and estimating the position of the user according to a plurality of distances measured on the position of the user; and
(20) the user terminal executes a synthetic signal processing program and is used for positioning a smartphone of a person or a label of an object or vehicle and developing a dedicated chip for a label-based user terminal.
(21) The technical solution of the present disclosure will be further described below in combination with specific embodiments.
(22) 1. System Overview
(23) An advanced architecture of the acoustic positioning system is shown as
(24) The base station network is a group of signal transmitters broadcasting the specifically designed signals in the air. Base station clocks in the network are synchronized with a 433 MHz radio frequency. A master clock is preserved by the whole network, all other clocks are synchronized with the master clock, with the precision being lower than 100 thousand seconds, which is high enough for the APS because a sound velocity is about 343 m/s. It means that a synchronization error is smaller than 3.4 cm.
(25) The space signals are a group of linear frequency modulation (LFM) signals. The LFM signals are also called Chirp signals capable of better resisting to a Doppler effect. As the propagation velocity of acoustic waves is lower, it is an important issue existing in acoustic positioning. In order to reduce influences of environmental noise and generate no audible sound when the ranging signal is transmitted, the Chirp signals of which the frequencies are 12 kHz to 21 kHz are selected in the present disclosure. All smartphones support this frequency band, and therefore, the APS may be directly used for smartphones with Android or Apple systems without changing any hardware of the smartphones. Due to such a characteristic, this technology can be easily applied to the mass market.
(26) The specifically designed Chirp signals may be expressed as:
(27)
(28) wherein A(t) is an amplitude of an acoustic wave, T is a period of each Chirp signal, and f.sub.0 and f.sub.e are respectively an initial frequency and a cutoff frequency. In view of an indoor fading channel, a received signal may be expressed as:
x(t)=s(t)*h(t)=Σ.sub.i=0.sup.L-1(t−τ.sub.1)+N.sub.i(t)
(29) wherein α.sub.i, τ.sub.i and (t) are respectively a channel fading coefficient, propagation delay and random noise of an i.sup.th propagation path, h(t) is a channel impulse response (CIR) in an indoor environment, T.sub.1 is the delay of an i.sup.th component, and τ.sub.0 is the delay of a first line-of-sight (LOS) component.
(30) The user terminal has the functions of receiving and decoding the ranging signal, estimating distances from base stations to the user terminal according to first arrival signals, and estimating the position of a user according to a plurality of distances measured on the position of the user. The received signals may be disturbed by environmental noise, a wall reflected signal, a glass window and the like, and therefore, a synthetic signal processing program is executed in the user terminal to extract first arrival signals of distance signals. The user terminal may be used for positioning a smartphone of a person or labels of an object and a vehicle. A microphone built in the smartphone may be used for receiving an acoustic ranging signal, and therefore, it is unnecessary to change any hardware of a smartphone with an Android or iOS operation system. For a label-based user terminal, a dedicated chip has been developed.
(31) 2. Precise Ranging Based on Acoustic Waves
(32) The Chirp signal expressed in the formula (1) may be generated by using a loudspeaker of a base station and broadcast in the air. The Chirp signal is an impulse signal having different feature codes. The period of the Chirp signal of the APS is about 50 ms which is shorter than an audio signal for oral communication of human beings. The Chirp signal is regularly broadcast by the base station and is received by the user terminal. The user terminal may be a smartphone or a label of an embedded chip and is used for specially processing the ranging signal transmitted by the base station.
(33) The microphone built in the smartphone may generally record a frequency band 20 Hz to 23 kHz. In order to utilize the existing microphone in the smartphone, the ranging signal in the present disclosure is designed within this frequency band. The frequency of an audio signal of the sound of a person is generally lower than 15 kHz; in order to reduce disturbance of the signal, the ranging signal is adjusted to the frequency band 12 kHz to 21 kHz which is also applicable to the above-mentioned ranging signal, and therefore, it is unnecessary to change any hardware in the smartphone. The period of the Chirp signal is 50 ms.
(34) Generalized cross correlation (GCC) is a typical delay estimation method, and delay may be converted into a geometrical distance from the user terminal to the base station. The given ranging signal s(t) defined in the formula (1) may be generated in the smartphone, and Fourier transformation is performed on a corresponding signal x(t) received by the microphone to obtain frequency domain descriptions of signals S(f) and X(f). A generalized cross spectrum G(f) may be deduced as:
G(f)=E[S(f)X(f).sup.H
(35) wherein E[*] is a mathematical expectation operator, f is a signal frequency, and H is hermitian transformation. In order to achieve a more steady solution, adopting a phase transformation (PHAT) and weighting solution to obtain unit gains of all frequency components, that is, actual delay information is retained.
Ĝ(f)=G(f)/|G(f)|
(36) Inverse Fourier transformation (IFT) is performed to obtain the following relevant function R(τ):
R(τ)=F.sup.−1(Ĝ(f))
(37) wherein τ is the delay of the relevant function, and the delay Δt of the ranging signal s(t) may be obtained by the following formula:
(38)
(39) wherein β is a threshold factor, and β.Math.max[|R(τ)↑] represents an estimated threshold.
(40) After the delay Δt is obtained, a distance d from the user terminal to a corresponding base station sending the signal may be easily calculated;
d=Δt.Math.c
(41) wherein c is a sound velocity which is about 343 m/s. After a distance among at least three base stations is obtained, the position of the user terminal may be estimated by applying a least square trilateration algorithm or synthesis filtering algorithm, so that measured results of other sensors such as the inertial sensor built in the smartphone are prevented from being integrated.
(42) 3. Positioning Based on Acoustic Ranging and Inertial Sensor
(43) The position of the user terminal is estimated by integrating an acoustic range and a measured value of the inertial sensor, a data processing framework for estimating the position of the user mainly consists of two parts, as shown in
(44) The data driving part estimates the velocity vector v=[v.sub.x, v.sub.y, v.sub.z].sup.T according to measured values of an accelerometer and a gyroscope sensor on the smartphone or the label, which is a data driving method based on the deep learning method;
(45) The model driving part integrates the velocity vector and precise ranging based on acoustic waves to estimate the position, heading and velocity of the user by using the extended Kalman filter (EKF).
(46) The inertial sensor plays a vital role in positioning the smartphone and the wearable device due to the advantages in the aspects of low energy and high data rate. However, there is a drift error in the measurement of a low-cost inertial sensor such as a accelerometer, a gyroscope, and a magnetometer, when dual integration is performed on an accelerated velocity to obtain a displacement, the error may be increased infinitely. A walking track of a pedestrian is estimated by adopting the deep learning method and utilizing the measurement of the inertial sensor as an input. It is a data driving method for pedestrian dead reckoning (PDR), which is completely different from the model driving method in which step-by-step detection is used.
(47) In the solution, used is a ResNet-18 network which is a convolutional neural network with 18-layer depth and a training network usually used for image recognition. A netversion for training more than 1000 thousands of images may be loaded from an ImageNet database. The framework is adopted to train a PDR network, as shown in
(48)
(49)
(50) The learned velocity vector is a relative vector. The geometrical shape of the walking track obtained by utilizing the velocity vector learned by virtue of the PDR network may be kept, but no any absolute horizontal directions are shown. Therefore, it is still necessary to collect some measured values such as distances and azimuths from absolute coordinates such as common east, north, up coordinates in automobile navigation.
(51) For an outdoor environment, a GNSS technology is usually used for positioning the smartphone or the wearable device and is a very mature technology, and therefore, the APS is more applicable to the indoor environment. Although the APS is also applicable to the outdoor environment, the APS is not the solution having the highest cost benefit in the outdoor environment. Indoor positioning usually occurs on the plane of a horizontal floor, and therefore, the position of the user is estimated by adopting a 2D method as the algorithm in the present disclosure.
(52) As shown in
X.sub.k=[e.sub.kn.sub.kθ.sub.k].sup.T
(53) wherein [e.sub.k,n.sub.k] are respectively horizontal coordinates in east and north directions within an epoch k. In fact, θ.sub.k includes a rotation angle between the coordinates G and L and an accumulated drift error of the gyroscope. A state transition equation of a system is described as follows:
(54)
(55) wherein {tilde over (X)}.sub.k is a coordinator for performing prediction by using the state transition equation; W.sub.k is process noise subject to normal distribution N(0, Q.sub.k), and Q.sub.k is a covariance matrix of the process noise. Δx.sub.k and Δy.sub.k are displacement components in the local coordinates L from a period k-1 to a period k obtained by a data-driven PDR network.
(56) In the APS, clocks among all the signal transmitters are synchronized by using a 433 MHz radio frequency, and a synchronization error which is smaller than 0.1 ms is achieved, which is enough for the solution. However, it is impossible to synchronize the clock of the smartphone and the clocks of the signal transmitters. An observable known as a time difference of arrival (TDoA), which is used as a delay difference of two different transmitters at the same period, is utilized to eliminate the clock error of the smartphone. If the ranging signal may be received from the plurality of signal transmitters, a plurality of TDoA observation data may be used. An observation equation Z.sub.k may be expressed as:
(57)
(58) wherein h(*) is an observed transition matrix, {tilde over (Z)}.sub.k is a vector of a predicted observed value calculated by using a predicted state vector {tilde over (X)}.sub.k, St is a coordinate vector of an i.sup.th transmitter in the coordinates G, V.sub.k is an error vector of Z.sub.k, and Z.sub.k is subject to normal distribution N(0, R.sub.k), wherein R.sub.k is a covariance matrix of Z.sub.k, m is the number of the transmitters available on the current position of the user, and c is a sound velocity at the temperature T. The EKF may be updated as:
{tilde over (P)}.sub.k=F.sub.kP.sub.k-1,F.sub.k.sup.T+Q.sub.k
K.sub.k={tilde over (P)}.sub.kH.sub.k.sup.T(H.sub.k{tilde over (P)}.sub.kH.sub.k.sup.T+R.sub.k).sup.−1
X.sub.k={tilde over (X)}.sub.k+K.sub.k(Z.sub.k−{tilde over (Z)}.sub.k)
P.sub.k=(I−K.sub.kH.sub.k){tilde over (P)}.sub.k
(59) wherein F.sub.k=∂f/∂X.sub.k, H.sub.k=∂h/∂X.sub.k, X.sub.k is state estimation, {tilde over (P)}.sub.k is a predicated covariance matrix, P.sub.k is an updated covariance matrix, K.sub.k is a Kalman gain, and I is a unit vector.
(60) Second, application embodiments: in order to prove creativity and technical values of the technical solution of the present disclosure, this part shows the application embodiments for applying the technical solution in the claims to specific products or relevant technologies.
Application Example 1
(61) An indoor visitor guiding system for exhibition hall 1 of a certain international exhibition center: the certain international exhibition center is located in a certain small town, wherein the overall hall of the exhibition hall 1 is in the size of 47.7*106.2*19 m and has the total occupied area greater than 5000 m.sup.2. The scenario is oriented to meet the demand for positioning the visitors during actual exhibition, acoustic waves are taken as cores, and an analog-digital fused positioning solution is adopted in applications. A high-precision acoustic positioning source is covered in the venue, and thus, verification and index test on the technical solution of the present disclosure in an actual application scenario are completed. As shown in
Application Example 2
(62) A system for epidemic control assisted by positioning four dining halls for teachers and students in a university in a certain city: in order to embody the advanced effect of an analog-digital fused positioning technology on the monitoring level of epidemic control, the certain university is cooperated, and the system for epidemic control assisted by acoustic positioning is tried out in the four dining halls for the teachers and the students in the university. The system is a subsystem of a large health management system in the university and aims at meeting the demand that basic health information and flow information of personnel in the overall university are mastered in real time to efficiently trace various crowds for epidemic control so as to avoid large-scale viral infection under the condition that the normal teaching in the university gets back to normal. The acoustic system has clear functional demands in two aspects of personnel density monitoring and epidemic control source tracing.
(63) 1) Personnel Density Monitoring
(64) Activity information of personnel in the university is acquired based on a real-time positioning signal, and the number and density of the personnel may be displayed in real time by data analysis and presentation (a thermodynamic diagram) of a gathering condition of the personnel in all areas of the university, including, but not limited to life supporting facilities such as an office building, an apartment building, a teaching building, a dining room, a library, and a gymnasium.
(65) 2) Epidemic Control Source Tracing
(66) The record and inquiry of behavior trajectories of the personnel in the overall university are achieved by real-time positioning service. Activity trajectories of the personnel are dynamically presented in combination with a campus map and positioning data, and thus, the activity trajectories of the personnel may be rapidly found. Meanwhile, based on the positioning data, a contact distance between every two of the personnel and a contact duration are analyzed to establish a close contact model, and thus, close contacts with different time ranges such as 14 days and 21 days may be rapidly inquired. In the present project, an audio signal positioning method is introduced to make up defects of an indoor positioning technology in this aspect, thereby providing the technical assurance for establishing a complete in-campus position service network for the university and guaranteeing the safety of the teachers and the students.
(67) By trial popularization, it is proven that the audio signal positioning technology has the characteristics of accuracy, stability, reliability, compatibility, and durability when indoor positioning data is provided.
Application Example 3
(68) A positioning and navigation system for a large railway station in a certain city: by a pseudo-satellite signal provided by an acoustic base station mounted in an area of the railway station, on a square outside the railway station and in an underground parking, indoor and outdoor integrated seamless navigation is provided for the public, and meanwhile, position service of production and security personnel is provided for a management platform.
(69) A user uses a special navigation application APP integrated with an indoor engine, and a GNSS satellite signal is abutted when the personnel are located in an outdoor area; and after the personnel enter indoors, the APP automatically determines to enter an indoor environment, a positioning signal is switched from the GNSS satellite signal to an audio signal, and thus, positioning service is provided continuously. Positional information may be loaded to a management cloud service platform via a WiFi or 4G signal, and thus, functions such as device or personnel distribution perception, risk pre-warning, dynamic patrol exhibition, video monitoring linkage, and emergency direction management are provided for a manager.
Application Example 4
(70) A positioning system for personnel/devices in a cargo airport in a certain city: a positioning result is shown for a user by adopting an acoustic positioning technology and using an analog-digital fused positioning algorithm and a high-precision map, and thus, indoor and outdoor integrated personnel/device positioning service is provided. Positioning of personnel, vehicles, and capital devices in the airport and main indoor areas such as relevant loading/unloading areas and workshops is achieved, and functions mainly include real-time positioning of personnel, vehicles, and capitals, electronic fence alarming, one-button alarming, trajectory replaying and the like, so that the safety management work is converted from labor management and process management into wisdom management, the safety management effect is greatly improved, and a high application value is achieved.
Application Example 5
(71) An visitor guiding and positioning system for a 5G exhibition hall of a mobile company: the mobile company popularizes how 5G will change the life style of people and improve the life quality to the public and also holds an exhibition with the theme “5G+future” in a wing of the company to promote the latest application results of 5G technologies in various industries. By using an acoustic analog-digital fused positioning system, all devices in the exhibition hall may acquire positions themselves in real time and provide the positions to the visitors; and when the devices induce that the visitors appear near, introduction videos or voices are automatically played to achieve the purpose of linking “person looking for hall, and hall missing person”.
(72) Third, evidences for relevant results in embodiments: some positive benefits are achieved in research and development or use process in the embodiment of the present disclosure, compared with the prior art, the embodiment has great advantages in deed, and the following contents will be described in combination with data, charts and the like in an experimental process.
(73) In order to evaluate the measurement precision of the TDoA of the APS, as shown in
TDoA=(Δt.sub.1+Δt.sub.c)−(Δt.sub.2+Δt.sub.c)=Δt.sub.1−Δt.sub.2
Δd=c.Math.TDoA
(74) In addition, a true value Δd of a ranging difference is precisely measured by using a laser interferometer. Then, an error of Δd may be calculated by comparing the measured value of the smartphone with the true value measured by the laser interferometer. Four experiments are performed along the orbit, and for each experiment, measurement is performed every other two meters along the orbit. Table 1 shows error statistics for the ranging difference.
(75) TABLE-US-00001 TABLE 1 Error Statistics for Ranging Difference Mean Minimum Maximum Mean square value value value error Error (m) 0.02 0.44 0.16 0.20
(76) After the ranging difference performance is evaluated, performance evaluation for a positioning solution is performed in the exhibition hall 1 of the certain international exhibition center, as shown in
(77) As shown in
(78) TABLE-US-00002 TABLE 2 Static Positioning Error of APS Mean Type of Minimum Maximum Mean square phone value value value error MIUI 10 0.06 0.76 0.38 0.42 Honor 8 0.08 0.77 0.32 0.37 Huawei 0.02 0.75 0.37 0.42 P9 Huawei 0.05 1.23 0.48 0.56 P40 Mean 0.05 0.88 0.38 0.44 value
(79) TABLE-US-00003 TABLE 3 Error Statistics for Dynamic Test Mean Type of Minimum Maximum Mean square phone value value value error MIUI 10 0.00 2 25 0.48 0.63 Honor 8 0.00 2.48 0.46 0.61 Huawei 0.00 1.96 0.84 0.55 P9 Huawei 0.00 3.31 0.53 0.70 P40 Mean 0.00 2.50 0.58 0.62 value