II. HUMAN MOTION ANALYSIS

Based on observations of pedestrians with a 24 GHz Pulse-Doppler radar, a six point model representing feet, knees, and two points of the upper body is developed in [Reference Fölster, Rohling and Ritter3]. Thereby the movements of the legs are approximated by sinusoidal oscillations in the direction of motion superimposed to the motion of the center of gravity (CoG). Swinging arms are neglected. A more detailed approach for pedestrian modeling to estimate human walking parameters is presented in [Reference Dorp and Groen4]. Parameters, such as cycle frequency, cycle length or cycle phase are estimated and visualized with a human walking scene in virtual reality. In [Reference Ghaleb, Vignaud and Nicolas5], high resolution 12.5 GHz inverse synthetic aperture radar (ISAR) is used for Doppler measurements of pedestrians. Human motion simulation and measurements are compared to define the constraints of image processing radar. Thereby two-dimensional radar images were simulated from motion data partially obtained from [6] with respect to seven distinctive points (sternum, hands, knees, and toes). However, simulations of pedestrian accident scenarios are very rarely addressed in literature. Most existing target models underlay simplifications which could affect the derivation of classification features. Further on, observations of pedestrians in literature are often based on measurement principles, like synthetic aperture radar (SAR) or ultra wideband radar (UWB), that do not suitably match automotive requirements in terms of sensor size or available bandwidth.

In this work, a simple but well suited measurement setup is used to develop a pedestrian target model. Motion sequences of test persons, tagged at the feet (FR/FL), knees (KR/KL), hands (HR/HL), elbows (ER/EL), right shoulder, right hip, and head (H), were recorded cinematically with a frame rate of up to 240 fps to obtain the space-time information of different body parts. Velocity-time functions have been derived through numerical differentiation and low pass filtering. Figure 1(a) displays the measured space versus time functions and Fig. 1(b) the speed versus time functions of each body part. Because of the linear uniform movement of the head, its function is considered to be representative for the pedestrian's CoG. Closer look at the speed-time-functions shows that they do not follow an ideal sinusoidal shape, as simple models in literature propose [Reference Fölster, Rohling and Ritter3]. This must be kept in mind when pedestrians shall be realistically represented during system assessment by articulating dummies, capable of arm and leg movement. Different lengths of arms and legs result in different velocity amplitudes. Furthermore it is obvious from the records that the body parts are accelerated several times during one gait cycle. Thereby the feet reach the highest spread in the velocity domain of approximately three times the mean value. Caused by the rolling motion of the feet, their velocities reach zero only for a distinct amount of time while other body parts are in motion over the whole gait cycle. Caused by the cinematic measurement method the positions of the left body parts could not be measured continuously. Nevertheless the measured data are sufficient enough to prove the assumption that the motion patterns of left and right extremities are identical with exception of a phase shift of pi, as can be seen in the space time functions given in Fig. 1(a).

Fig. 1.

Space-Time-Function of a walking pedestrian (a) and Speed-Time-Functions of a walking (b), strolling, (c) and jogging (d) pedestrian (FR/FL: foot right/left, KR/KL: knee right/left, HR/HL: hand right/left, ER/EL: elbow right/left, H: head).

For further investigations on the influence of the walking speed to the maximum velocity, step frequency and step length measurements of different paces of a person were conducted. Figures 1(c) and 1(d) show the speed time functions of strolling and jogging motion. Characteristic values of the different speed-time-functions are listed in Table 1. Thereby the step frequencies and step lengths are given in relation to the motion of single legs. Given the minor difference between the peak velocity during walking and jogging, the existence of an upper border for the peak velocity can be assumed. For faster paces the increased mean value is achieved rather by a broader shape of the curve than by the increase of the peak velocity. Furthermore, in case of faster movement, sinusoidal approximation of the velocity functions is nearly justifiable. The results gained with the presented approach are quite comparable with the findings of more sophisticated measurement methods of [Reference Ghaleb, Vignaud and Nicolas5, 6] and therefore highly suitable for simulation model development.

Table 1.

Characteristic values of different paces.

III. SIMULATION ENVIRONMENT

A) Sensor model

A sensor model featuring chirp sequence modulation provides the advantage of unambiguous range and velocity determination. Thereby K frequency modulated continuous wave signals are consecutively transmitted. The reflected and down converted signal of a single reflecting point with radial velocity v _r and radial distance d _r is described by

(1) $$\eqalign{s_{IF} (t) & = e^{2\pi j\left( \left[ 2f_c \cdot d_r \right]/c_0 \right)}\cr &\quad \cdot \sum\limits_{k = 0}^{K - 1} e^{2\pi j\left[ \left\{ \left( 2f_c \cdot v_r \cdot T_{RRI} \cdot k \right)/c_0 \right\} + \left( \left\{ \left( 2f_c \cdot v_r \right)/c_0 \right\} + \left\{ \left( 2B \cdot d_r \right)/T \cdot c_0 \right\} \right) \cdot t \right]}\cr &\quad\cdot rect\left( \displaystyle{t - k \cdot T_{RRI} \over T} \right),}$$

with single-chirp duration T, bandwidth B, carrier frequency f _c , chirp repetition interval T _RRI , and the chirp number k of a chirp sequence of length K−1 [Reference Andres, Menzel, Bloecher and Dickmann7]. Consequently, the radar target model of the pedestrian must provide the radial range and velocity of all considered reflection points for each time step of a given scenario. Characteristic values of the used modulation are given in Table 2. These values ensure results with high resolution and excellent target separability. With regards to state of the art update rates of automotive radar sensors a dead time was implemented between the chirp sequences for taking into account the processing time of the sensor. Of course the maximum detection range of 76 m is not suitable for all automotive radar applications but it is considered as sufficient for pedestrian detection at vehicle velocities of up to 100 km/h.

Table 2.

Modulation characteristics.

B) Pedestrian target model

To determine analytical velocity versus time and space versus time functions of each reflection point of the pedestrian target model, the measured data of Fig. 1 were processed by a Fast Fourier Transform (FFT). Each velocity function can be represented by the sum of a small amount of oscillations depending on a set of complex Fourier coefficients c _k .

(2) $$v(t) = \displaystyle{{c_{mean}} \over {c_0}} \cdot \sum\limits_{k = - \infty} ^\infty {c_k e^{j2\pi kf_0 t}}.$$

Thereby f ₀ denotes the sample frequency of the camera which was used for the cinematic measurements described in Section II. To compensate slight measurement uncertainties the steady components are rescaled to a common mean value c _mean , corresponding to v _mean of Table 1. Integration of (2) results in space-time representation

(3) $$\eqalign{s(t) & = c_{mean} \cdot t + \sum\limits_{k = - \infty} ^\infty {\displaystyle{{c_{mean}} \over {c_0}} \displaystyle{{c_k} \over {j2\pi kf_0}} (e^{j2\pi kf_0 t} - 1) + s_0}, \cr & \quad k \in {\rm Z}^{\ast},}$$

of the different body parts. Thereby the integration constant was replaced by the position offsets (s ₀) between the CoG and each body part at the beginning of the gait cycle. Given (2) and (3) and the parameter set of c _k enables the simulation of a multiple point pedestrian motion model for any desired time length. In the last step the functions were transformed into a radial coordinate system of the sensor model and included in the Matlab-based simulation environment. By providing adjustable parameters such as starting point, direction angle, vehicle velocity, and simulation duration, this environment enables the simulation of radar signal responses of most relevant accident scenarios without the need to run dangerous or life-threatening real-world tests.

To provide realistic radar signal responses the different reflection points of the human body feature different Radar cross-sections (RCS). The signal levels are approximated by simply combining the mean RCS value of a pedestrian in the 76 GHz band that is –6.6 dBsm [Reference Schubert, Kunert, Menzel, Fortuny-Guasch and Chareau8] with the partial distribution of the body surfaces according to [Reference Livingston and Lee9]. The RCS used for the different body parts are listed in Table 3. Additionally a factor to model the free space loss was implemented.

Table 3.

RCS values for simulation of reflection points.

Measurements with a radar sensor in the 76 GHz band were performed in an anechoic chamber to verify the developed model. Thereby the same modulation parameters, as provided in Table 2, were used. Figure 2 shows the simulated (a) and measured (b) Range-Doppler (RD) response of a radial walking pedestrian developed over one 10 ms long chirp sequence recorded at the point in gait cycle at which the speed of 1 ft reached its maximum of approximately 4 m/s (e.g. at time instant of 0.35 s in Fig. 1(b)). Besides ground clutter presence and minor RCS differences, both RD responses show very similar distributions of velocities. The nine points used for simulation are just enough to sufficiently approximate the main reflection points of a pedestrian's radar signal response. Especially the velocity spread, which is needed to develop and test classification algorithms, is comparable. However, to generate an accurate radar backscatter signal of a pedestrian with a high level of detail the use of a model with more reflection points or even a simulated body with continuous surface, comparable with real pedestrians, is recommended.

Fig. 2.

Simulated (a) and measured (b) RD response of a walking pedestrian.

IV. ACCIDENT SCENARIO SIMULATION

A closer look on the distribution of accident scenarios shows, that pedestrian accidents in real world happen not exclusively in scenarios proposed by Euro NCAP tests [Reference Wisch, Seiniger, Pastor, Edwards, Visvikis and Reeves10]. A real world interpretation of an exemplarily taken test of the NCAP protocol is illustrated in Fig. 3(a). Furthermore, a significant amount of accidents with killed or seriously injured pedestrians occurs in additional scenarios like: “Along the carriageway on a straight road without obstruction” (Fig. 3(b)). A system capable to indicate the presence of pedestrians under adverse visibility conditions, like rain, fog or darkness in combination with wet roads and related reflecting backlight, may be a good opportunity for radar-based pedestrian classification to enter the market.

Fig. 3.

Accident scenarios: (a) crossing pedestrian in a simplified urban area and (b) along a carriageway during darkness.

An automatic triggered pedestrian warning enables the driver to adapt his driving behavior faster and consequently helps to avoid potential safety-critical situations. For PCMS capable of automatic interventions the classification algorithms increase system robustness and decrease the false intervention rate.

In state of the art automotive radar signal processing, where high resolution leads also to a high amount of backscattering points, several detected reflections are clustered to a lower amount of hypothesized objects. State variables, such as position, speed or additional properties of these objects are then further processed by appropriate tracking algorithms like a Kalman-Filter. Additionally such a formed group of reflections can also be used for classification of the objects by analyzing the expansion in speed or range inside the cluster. Classification results will be more reliable if more reflections per object can be separated by the radar. To separate reflections three domains are available for radar signal processing: range, velocity or angular domain. The minimum requirement for separation of at least two reflections of the same domain scattered by one real object is that the radar's resolution cell is smaller than half of the objects expansion in the specific domain. To determine separability requirements for pedestrian classification in range, velocity, and angle, pedestrian expansions were analyzed by simulation of the trajectories of pedestrian body parts in different significant accident scenarios. Figure 4 exemplarily illustrates the curves of the velocities and angles over time to collision (TTC) of the simplified urban crossing scenario introduced in Fig. 3(a).

Fig. 4.

Trajectories of radial velocities and angles of different body parts versus TTC of a crossing pedestrian scenario with the aim to obtain the necessary separation requirements.

The expansion in velocity domain can be obtained to a range of 0.15–0.45 m/s and the expansion in angular domain to less than 0.5° at 3 s and to ca. 2.5° at 1 s before impact. Based on human geometry the minimum expansion in range is simply approximated to 0.5 m. Given those values and a closer consideration of limitations in bandwidth and sensor size, the conclusion that the velocity domain seems to be the most suitable to determine separation requirements can be drawn, which are achievable by state of the art and near future radar systems.

For demonstration of the classification potential of high resolution radar, Fig. 5 provides simulated RD responses cumulated over several successive chirp sequences for different scenarios. A comparison between Figs 5(a) and 5(b) shows the principle possibility to discriminate a longitudinal walking pedestrian from a static target next to the road by evaluating the velocity distribution – even several seconds before a possible collision may occur. Nevertheless the velocity spread should not be the one and only feature for robust classification algorithms. A lot of additional effects are present in real urban scenarios like the presence of multiple pedestrians within the same resolution cell or multipath and interference issues. These effects could lead to velocity spreads similar to those of pedestrians. To overcome these issues high resolution radar enables to distinguish between several scattering points of one physical object. Additional features like the estimation of multidimensional object size or further statistical analysis should be derived after grouping of detections to develop robust pedestrian classification with low false alarm rate.

Fig. 5.

RD responses of several measurement cycles cumulated over time for different scenarios: (a) approaching stationary target with 80 km/h and initial distance of 76 m; (b) approaching longitudinally walking pedestrian with 80 km/h and initial distance of 76 m; (c) approaching crossing pedestrian on collision course with v _veh  = 50 km/h.

Especially scenarios with lateral moving pedestrians and high vehicle velocities, as illustrated in Fig. 3(a), are challenging for radar-based pedestrian classification, because of smaller angles and corresponding smaller contributions to the radial velocities. Figure 5(c) shows the RD responses cumulated over time of such a scenario. Even if some differences can be recognized by comparing the shapes of Figs 5(a) and 5(c), a clear separation of different reflection points by their velocities is not reliably possible. Nevertheless a wide spread of the detected velocities around their mean values can be still observed over time. This is caused by the different RCS of the body parts and the fact that radar backscatter always responds the superposition of non-resolvable reflection points. Evaluation of the velocity's variance over time is also appropriate for classification of pedestrians in case of poor separability. This effect can also be seen in the distribution of the blue dots of Fig. 8.

V. INCREASING VELOCITY RESOLUTION BY SIGNAL EXTRAPOLATION

Improvement of the velocity separation capability of a microwave radar requires in general extending the measurement time and consequently decreasing the cycle rate of the sensor. Linear prediction based signal extrapolation may offer solutions for this dilemma to some extent. The algorithm used for this section was adapted from [Reference Kauppinen and Roth11] and its application to real measured data of a radial walking pedestrian is additionally illustrated in Fig. 6. With Linear Prediction theory previously unknown samples of a signal are generated from a few known samples by the use of the following equation and the corresponding linear prediction finite impulse response (FIR) filter

(4) $$x_n = \sum\limits_{i = 1}^M {h_i \cdot x_{n - 1}}, \quad P(z) = \sum\limits_{i = 1}^M {h_i \cdot z^{ - 1}},$$

where x _n is the new sample predicted from the previous samples x _n−1 and the prediction filter coefficients h _i [Reference Kauppinen and Roth11].

Fig. 6.

Overview of the algorithm used for extrapolation applied to real measured data of a radial walking pedestrian.

The step-by-step application of (4) allows the prediction of an unlimited amount of new samples. In the straightforward approach h _i are determined by application of (4) to a known section of the signal and the minimization of the prediction error. Because the exact analytical solution only exists for noiseless theoretical signals for real radar signals another approach should be used [Reference Kauppinen and Roth11]. The coefficients can be determined by autoregressive modeling algorithms like the Burg algorithm and relation

(5) $$h = [h_1, \ldots, h_M ] = [ - a_1, \ldots, - a_p ],$$

where [a ₀, …, a _p ], a ₀ = 1 are the p + 1 coefficients of an autoregressive model of the order p determined from the original signal. The step by step application of (4) can also be implemented with the infinite impulse response (IIR) synthesis filter

(6) $$\eqalign{H(z) & = \displaystyle{1 \over {1 - P(z)}} = \displaystyle{1 \over {1 - \sum\nolimits_{i = 1}^M {h_i \cdot x_{n - 1}}}} \cr & = \displaystyle{1 \over {\sum\nolimits_{i = 0}^p {a_i \cdot x_{n - 1}}}}},$$

fed with zero input vector of the length of the amount of samples to be predicted [Reference Kauppinen and Roth11]. The needed initial conditions for this filter are either obtained by feeding the linear prediction FIR filter (4) with all samples of the original signal or by assuming the prediction of the last p−1 originally known samples by the IIR extrapolation filter.

As displayed in Fig. 6 the presented extrapolation algorithm can be applied to the Doppler signal which, according to chirp sequence signal processing, is formed by the complex spectral peaks of a specific range cell of successive chirps. A frequency analysis of this signal gives the spectrum of velocities for each range. Additional processing of Constant False Alarm Rate (CFAR) based noise threshold estimation and peak detection further leads to detected range velocity pairs, indicated by pink rhombuses in Fig. 6. Extrapolation increases the separability, which, for constant sample rate, directly depends on the number of samples. The resulting separability of the presented two-staged algorithm consisting of linear prediction based signal extrapolation and subsequent FFT spectral analysis could maximally be as good as the worst of each stage. In best case the separability of the FFT can be expected to be the double of the frequency resolution, whereas the separability of the AR spectra nonlinearly depends on signal power (SNR) and model order [Reference Marple12]. Figure 7 compares the separation capabilities of autoregressive spectral analysis with the classical Fourier spectrum separability, without application of a window to the time signal.

Fig. 7.

SNR and model order dependency of AR model spectral separability in comparision with seperability of FFTs with 512 or 1024 points and no windowing applied.

For proper increase of separability the AR model order should be chosen between the intersections of the dashed lines, illustrating the separability of the FFT for length of the original and extrapolated signal, with the lines of the SNR dependent separability of the AR spectral analysis. For highest possible separability enhancement the model order should be chosen according to the intersection with the red dashed line, e.g. for targets with 20 dB SNR an AR model order of 60 would be appropriate in case 512 original samples should be extrapolated to 1024 samples.

To validate the obtained gain in resolution and separability, Fig. 8 shows the detections over time (blue dots) determined by classical radar signal processing of 512 samples long simulated signals in comparison with the detections determined by processing simulated signals extrapolated to 1024 samples (red rhombuses) of the lateral walking scenario presented in Fig. 3(a). Doubling the number of samples increases the spectral velocity separation from 0.3 to 0.15 m/s and enables separation especially of the legs and feet from the body, at least during the times of gait cycle when the feet reaches maximum velocity. A further comparison of the detections with the velocity functions of the underlying pedestrian motion model shows that the additional detections determined by an analysis of the extrapolated signal are mainly based on the pedestrian's model functions than on random effects or noise. To prevent possible disturbing influence the same CFAR algorithm was used for both analyses.

Fig. 8.

Detections based on simulated signal (blue dots) and based on the extended simulated signal (red rhombuses) versus time of the crossing pedestrian scenario in comparison with the model based velocity functions.

The improved separation capability compared with classical radar detection algorithms increases the number of detectable range Doppler pairs within one physical object. Adequate grouping of these pairs to hypothesized objects enables estimation of additional features for object classification.

For additional validation the algorithm was applied to real data measured from a radial walking pedestrian. The processed RD responses of the original and extrapolated baseband signals of 50 consecutive measurement cycles cumulated over time are given in Fig. 9. Thereby in sum about 400 reflections have been detected from the original signal whereas more than 500 could be detected after extrapolation. Additionally the narrower peaks and the corresponding increased separability in the spectrogram of the extrapolated signal can be seen in the circled areas. The peak widths especially in the velocity domain in the spectrum of the extrapolated signal are smaller compared with the spectrum of the original signal. This is caused by the application of signal windowing and FFTs of larger sizes with respect to the processing of the original signal.

Fig. 9.

RD responses of a radial walking pedestrian cumulated over 50 consecutive chirp sequences without signal extrapolation (left) and with application of extrapolation (right).

In theory all relevant high resolution target information of the signal have already been included in the AR parameters or in the initial conditions and filter coefficients of the synthesis filter, respectively. A method to obtain signal parameters directly from the AR parameters is expected to achieve an even better separability because no separability deteriorating windowing function needs to be applied in contrast to Fourier analysis. For example, the frequencies of the original signal can be obtained directly from the roots of the characteristic polynomial of the FIR prediction filter. Unfortunately thereby the phase and amplitude information are lost in the first place and need to be recovered additionally. An advantage of the proposed method is that, besides enlargement, the classical radar signal processing chain consisting of FFT, CFAR, and peak picking algorithm can be maintained. Furthermore, the challenge of sensitive estimation of the ideal order of the autoregressive model is avoided because the proposed method seems to be more robust than the direct method against common known effects like spurious spectral peaks and line splitting caused by over fitting the model order. Another observed advantage is the fact that the synthesis filter only extrapolates predictable signal components. Because noise is not predictable doubling the samples halves the noise power in the spectrum of the extrapolated signal. This can also directly be recognized from the color of the noise levels of the spectrograms of Fig. 9. This effect supports especially the detection of smaller targets, whose amplitudes without extrapolation would be close beneath the CFAR threshold.