A comparative analysis of three monocular passive ranging methods on real infrared sequences

Boban P. Bondžulić, Srđan T. Mitrović, Žarko P. Barbarić, Milenko S. Andrić

This is a postprint of an article which appeared as:
, , & : A comparative analysis of three monocular passive ranging methods on real infrared sequences, Journal of Electrical Engineering 64 (5), pp. 305-310, , DOI: 10.2478/jee-2013-0044

Abstract

Three monocular passive ranging methods are analyzed and tested on the real infrared sequences. The first method exploits scale changes of an object in successive frames, while other two use Beer-Lambert’s Law. Ranging methods are evaluated by comparing with simultaneously obtained reference data at the test site. Research is addressed on scenarios where multiple sensor views or active measurements are not possible. The results show that these methods for range estimation can provide the fidelity required for object tracking. Maximum values of relative distance estimation errors in near-ideal conditions are less than 8%.

1 Introduction

Passive ranging is of special interest for wide range of applications, such as video surveillance and security, speed control, air traffic control, obstacle detection, robotics. Passive sensors are widely used in military systems for target tracking, missile guidance or weapon fire control, where the use of active ranging sensors is disadvantageous over the passive, since they can be quickly detected and destroyed by homing missiles.

Optical flow and stereo vision are the most common techniques used to passively estimate distance to an object. Both methods are relied on the geometrical principle of triangulation. In optical flow, the baseline is created due to the sensor motion, whereas in stereo the distance between cameras (baseline) is fixed [1]. Number of used sensors varies from one in optical flow method, through two for single baseline approach, to three or more for multiple baseline method [2] and methods exploring the network of passive sensors [3].

This research is focused on scenarios where only one passive imaging sensor is available. An overview of the existing literature devoted to this topic suggests three main approaches. One of them exploits surface changes of an object in successive frames. Object’s surface changes can be result of sensor movement [4,5], object movement [6,7] or both [8]. This approach require additional data: initial distance, dimensions of targets, or trajectory travelled by the sensor. The second group of methods is relied on Beer-Lambert's Law and atmospheric propagation model. Monocular passive ranging method for tracking emissive targets [9] is based on atmospheric oxygen absorption in near-infrared spectrum, while spectral attenuation of two oxygen absorption bands in the near-infrared and visible spectrum is suggested in [10]. The study [8] examines fusion of the object surface measurement and atmospheric propagation model based approaches. The third group of approaches is relied on camera focus information. The relative distance between a moving object and the sensor is estimated from image defocus data in [11], where an estimation of the background together with the object image is needed, as a replacement for two images required by traditional depth from defocus algorithms [12].

The main objective of this research was to test, on real infrared (IR) sequences, the efficiency of a three methods for monocular passive ranging: method based on image size measurement, method based on intensity measurement and method based on contrast measurement. Relevant literature does not include many reports on testing above described methods on real life scenarios, such that this paper is deemed to be a modest contribution to the important field of passive distance estimation. With this objective in mind, the rest of paper is structured as follows: Section 2 discusses the passive ranging methods and relevant theory. Section 3 addresses the application of algorithms on real data, the dataset collection, the passive ranging data extraction, and the analysis of obtained results. Section 4 concludes the paper and is followed by references.

2 Passive ranging methods

Three passive ranging methods using image size, intensity and contrast measurements from one sensor are analyzed. No prior knowledge about the sensor or about the shape, size or any other features of the target is assumed.

2.1 Method based on image size measurement

By measuring the change in apparent target size, it is possible to estimate the range from the sensor to the target. It is well known that size of an incoming target grows while approaching the sensor, and principles of projective geometry can be used for calculations [7]. The apparent size of target also depends on viewing aspect changes, but if a straight line motion is assumed, the distance to the target at the frame \(k\) is given by:
\begin{equation} R_A(k) = R_0 \sqrt {\frac{{A_0}}{{A(k)}}} \end{equation}
where \(A(k)\) is object size (area) in the current frame, and initial distance-\(R_0\) and the target size in initial frame-\(A_0\) are known. Target size values \(A_0\) and \(A(k)\) are obtained through video image processing.

2.2 Method based on intensity measurement

The intensity or gray level \(I\) in the image is a function of scene radiance attenuated by transmission through the atmosphere, and characteristics of the image sensor. In its simplest form this statement becomes:
\begin{equation} I = GL\tau = GL e^{ - \sigma R} \end{equation}
where \(G\) is the sensor transfer function, \(L\) is the scene radiance, \(\sigma\) is the atmosphere extinction coefficient, \(R\) is optical path length through the atmosphere (distance), and transmittance of the atmosphere \(\tau\) is described by Beer-Lambert's Law. If constant radiance of the target is supposed, for two successive frames from (2) follows:
\begin{equation} R_I(k) = R_I(k-1) + \frac{1}{\sigma }\ln \frac{{I(k-1) }}{{I(k) }} \end{equation}
where \(R_I(k-1)\) and \(R_I(k)\) are target to sensor distances, or ranges, and \(I(k-1)\) and \(I(k)\) are average gray levels of the object in two successive frames. Equation (3) can also be written as:
\begin{equation} R_I(k) = R_0 + \frac{1}{\sigma }\ln \frac{{I_0 }}{{I(k) }} \end{equation}
where \(R_0\) and \(I_0\) are initial distance and object average gray level in the initial frame, respectively.

2.2 Method based on contrast measurement

The image contrast is given by:
\begin{equation} C = \frac{{I_T - I_B }}{{I_B }} \end{equation}
where \(I_T\) is the average gray level value of target, and \(I_B\) is the average gray level value of background. As is well known, image contrast is scene contrast reduced by the transmittance of atmosphere:
\begin{equation} C = KC_{SC} e^{-\sigma R} \end{equation}
where \(K\) is the sensor contribution, and \(C_{SC}\) is scene contrast, given by:
\begin{equation} C_{SC}=\frac{L_T-L_B}{L_B} \end{equation}
where \(L_T\) is the radiance of target, and \(L_B\) is the radiance of its background.
If constant contrast in the scene is supposed, for two successive frames from (6) can be derived:
\begin{equation} R_C(k) = R_C(k-1) + \frac{1}{\sigma }\ln \frac{{C(k-1) }}{{C(k) }} \end{equation}
where \(R_C(k)\) and \(R_C(k-1)\) are object to sensor distances and \(C(k)\) and \(C(k-1)\) are the target contrasts in two successive frames. It can be shown that (8) can be written as follows:
\begin{equation} R_C(k) = R_0 + \frac{1}{\sigma }\ln \frac{{C_0 }}{{C(k) }} \end{equation}
where \(C_0\) is the target contrast in the initial frame.
A great influence on the veracity of distance estimation in (4) and (9) has the atmospheric extinction coefficient \(\sigma\) [13]. Assessment of \(\sigma\) can be made based on atmospheric model, such as: LOWTRAN, MODTRAN or HITRAN. These computer programs require knowledge of a large number of input parameters for a valid estimation of atmospheric transmissivity. The second method commonly used in the visible range is based on an assessment of optical visibility, but it can not be used for thermal windows. Moreover, the problem of uncertainty of this parameter can be solved by introducing the novel algorithms, such as unbiased estimation coupled with the extended Kalman filter, suggested in [14].

3 The application of algorithms on real data

3.1 Dataset collection

Previously described methods were applied to real infrared image sequences, which were recorded by IR camera ($3-5 \mu$m). The sequences were generated using the Dual Observer Passive Ranging System (DOPRS) that is designed to track an airborne target. System uses two thermal cameras and estimates distance by triangulation method. In this research sequences from one camera are used, while the distance obtained from the DOPRS is used as the reference distance in analysis and comparison of results. The reference distances to the target in the analyzed video sequences were determined with an absolute error of less than five meters.
All the analyzed sequences contain a single airborne target in the vicinity of the center of the frame, of which the maneuver in relation to the acquisition sensor differs in each sequence. Sequences were recorded in various weather conditions and with different airplane types.

3.2 Passive ranging data extraction

Three important steps for each method are performed on sequences: detection and segmentation - finding the object and extracting it from the background; feature extraction - obtaining description of object and/or background; and finally combining the features with a known atmosphere extinction coefficient and initial range into the target distance estimate.
A gray level threshold for the detection and segmentation of the target is determined by the method of Tsai [15]. For distance estimation by contrast and intensity methods is necessary to know the value of the parameter $\sigma$. It can be expressed as function of object range, and contrast or intensity from (4) and (9). For the purposes of this study parameter $\sigma$ was estimated using the test sequences recorded immediately before the analyzed. Test sequences and distance estimation in 100 frames from DOPRS are used, together with related image contrast and intensity measures.

3.3 Analysis of results

Three algorithms for passive distance estimation were applied on the four typical infrared image scenarios in order of their verification and analysis.

IR sequence A

In this scenario, the target is moving smoothly toward the sensor. Figure 1 shows the first, hundredth and last frame of the analyzed sequence. The sky in the background is clear, without clouds. Contours of the airplane: the wings and tail are clearly visible in all frames. In the Figure 1 can be seen the difference in the intensity of the target gray level in the three frames shown.
Target in sequence A frames
Fig. 1. Target in the first, the 100th and the last frames in sequence A.

The estimated distances to the target are shown in Figure 2, where $R_A$ is distance obtained on the basis of changes in the target size, the $R_I$ is calculated based on changes in the targets intensity, the $R_C$ is estimated from changes in target contrast from the background, and $R$ is reference range from the DOPRS. All methods perform well and are stable.
The estimated distances to the target in sequence A
Fig. 2. The estimated distances to the target in sequence A.

The average relative error is calculated by:
\begin{equation} \overline{Err_{m}}\ =\frac{1}{N}\sum_{k=1}^{N}\frac{\left|R(k)-R_{m}(k)\right|}{R(k)}, \ \ m\in \left\{A,I,C\right\} \label{MSE01} \end{equation}
where $N$ is the total number of frames, and the index $m$ refers to the applied method: the size-$A$, the intensity-$I$ and the contrast-$C$. For the analyzed sequence the following results were obtained: the average relative error of distance estimation using the method of target size changes $\overline{Err_{A}}=$2.38%, using the intensity method $\overline{Err_{I}}=$1.97% and the method of contrast $\overline{Err_{C}}=$1.88%. Maximum values of relative errors are less than 8%.

IR sequence B

In this scenario, the target is going away from the sensor. Figure 3 shows the first, hundredth and the last frame. The target is in the black square that represents the tracking window of the DOPRS. The sky in the background is clear. The target is at relatively large distance, and its contours are not clearly visible.
Target in sequence B frames
Fig. 3. Target in the first, the 100th and the last frames in sequence B.

Figure 4 shows the estimated distances to the target using the three algorithms. From the figure one can see that the estimated distance $R_A$ is significantly different than the reference distance $R$. The average relative error is $\overline{Err_{A}}=$12.41%, while it’s maximum value even $Err_{A\max}=$30.78%. Distance estimation error of the other two algorithms is about the same as in the previous scenario, $\overline{Err_{I}}=$1.60%, $\overline{Err_{C}}=$1.86%.
The estimated distances to the target in sequence B
Fig. 4. The estimated distances to the target in sequence B.

Large error in $R_A$ estimation is a consequence of poor target size estimation. Figure 5 shows the target size, which vary from 3 to 10 pixels.
Number of pixels representing the target in sequence B
Fig. 5. Number of pixels representing the target in sequence B.

IR sequence C

Background of the target (sky) fluctuates significantly as can be seen in Figure 6, which shows the first, 160th and last frames of the sequence. The target is uniformly approaching to the sensor, while a cloudy sky in the background is replacing by the clear and overcast again.
Target in sequence C frames
Fig. 6. Target in the first, the 100th and the last frames in sequence C.

The estimated distances in this sequence are shown in Figure 7, and average error of distances estimations are illustrated in Figure 8. Figure 8 shows a good comparison between range estimations and the reference data at the beginning and the end of sequence, with significant deviations from 100 to 170 frames. This can be explained by an influence of large background changes to the object contrast.
The estimated distances to the target in sequence C
Fig. 7. The estimated distances to the target in sequence C.
Range estimation relative error in sequence C
Fig. 8. Range estimation relative error in sequence C.

IR sequence D

Figure 9 shows three frames of target that is moving towards the sensor at relatively short distances. From Figure 9 it can be seen that background gray level is uniformly changing while the target is of a high intensity.
Target in sequence D frames
Fig. 9. Target in the first, the 100th and the last frames in sequence D.

In Figure 10 the estimated distances are shown. The figure shows that the distance $R_A$ sufficiently assess in relation to the reference distance $R$ with the average distance estimation error at $\overline{Err_{A}}=$3.64%. Significant deviation of distance $R_I$ compared to $R$ can be explained by the high intensity of target gray level, due to it proximity.
The estimated distances to the target in sequence D
Fig. 10. The estimated distances to the target in sequence D.

The average gray level of the target’s pixels is relatively high (94%) and it is more or less constant, which can be explained by saturation of the sensor. The effect of sensor saturation were compensated by the changes of the background gray levels, and the distance estimation $R_C$ is significantly better than the $R_I$.

4 Conclusion

Three monocular passive ranging methods were discussed and applied on four real sequences recorded by an IR sensor. It is shown that these methods offer useful, but not ideal range estimations. Method based on image size measurement requires an extended target (represented with multiple pixels) for acceptable results, although in case of point target (large distances) it generates poor estimations. On the other hand, methods relied on Beer-Lambert's Law give satisfactory results for point target range estimation. When target is close to the sensor, saturation effect arises and deviates contrast and intensity based estimations. Background fluctuation additionally affects the ranging. Results obtained and analyzed suggest the use of a hybrid approach with image size, intensity and contrast features all taken into account.

References

  1. Y. Barniv: Error analysis of combined optical-flow and stereo passive ranging, IEEE Transactions on Aerospace and Electronic Systems, 1992, 28(4), pp. 978-989
  2. R.J. Pieper, A.W. Cooper, and G. Pelegris: Passive range estimation using dual-baseline triangulation. Optical Engineering, 1996, 35, pp. 685
  3. M. Ito, S. Tsujimichi, and Y. Kosuge: Tracking a three-dimensional moving target with distributed passive sensors using extended Kalman filter. Electronics and Communications in Japan (Part I: Communications), 2001, 84(7), pp. 74-85
  4. J.Z. Xu, Z.L. Wang, Y.H. Zhao, and X.J. Guo: A distance estimation algorithm based on infrared imaging guided. International Conference on Machine Learning and Cybernetics, 2009, Vol. 4, pp. 2343-2346
  5. R. Rao, and S. Lee S. 'A video processing approach for distance estimation', Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Toulouse, France, July 2006, Vol. III, pp. 1192-1195
  6. D.R. Van Rheeden: Passive range estimation using image size measurements. US Patent 5,867,256. February, 1999.
  7. C. Raju, S. Zabuawala, S. Krishna, and J. Yadegar: 'A hybrid system for information fusion with application to passive ranging', Proc. of Int. Conf. on Image Processing, Computer Vision and Pattern Recognition, Las Vegas, June 2007, pp. 402-406
  8. M. De Visser, P.B.W. Schwering, J.F. de Groot, and E.A. Hendriks: Passive range estimation using dual-baseline triangulation. Optical Engineering, 2006, 45, pp. 026402
  9. J.R. Anderson, M.R. Hawks, K.C. Gross, and G.P. Perram: 'Flight test of an imaging O (Xb) monocular passive ranging instrument', Proceedings of SPIE, Vol. 802005-12, 2011.
  10. R.A. Vincent, M.R. Hawks: 'Passive ranging of dynamic rocket plumes using infrared and visible oxygen attenuation', Proceedings of SPIE, Vol. 80520D-16, 2011.
  11. P. Gil, S. Lafuente, S. Maldonado, and FJ Acevedo: Distance estimation from image defocus for video surveillance systems, Electronics Letters, 2004, 40(17), pp. 1047-1049
  12. G. Witus and S. Hunt: 'Monocular visual ranging', Proceedings of SPIE, Vol. 696204-1, 2008.
  13. Yang De-gui and Li Xiang: Passive ranging based on IR radiation characteristics. In Vasyl Morozhenko (editor) Infrared Radiation, InTech 2012, pp. 199-214.
  14. G.D. Dikic, and Z.M. Djurovic: 'Unbiased estimation of atmosphere attenuation coefficient', Electrical Engineering, 2007, 89, pp. 343-347
  15. W.-H Tsai: 'Moment-preserving thresolding: A new approach',Computer Vision, Graphics, and Image Processing, 1985, 29(3), pp. 377-393

This work was supported by Ministry of Education and Science of the Republic of Serbia (grants TR-32023 and III-47029).