The Statistical Approach for Overcoming the Sensor Saturation Effect in Passive Ranging

Srđan T. Mitrović, Boban P. Bondžulić, Milenko S. Andrić, Žarko P. Barbarić

This is a postprint of an article which appeared as:

Srđan T. Mitrović, Boban P. Bondžulić, Milenko S. Andrić & Žarko P. Barbarić: The Statistical Approach for Overcoming the Sensor Saturation Effect in Passive Ranging, Elektronika ir elektrotechnika-Electronics and Electrical Engineering 20 (2), pp. 52-57, 2014, DOI: 10.5755/j01.eee.20.2.3788

Abstract

Gray level intensity distribution on thermal infrared images is estimated in this research. General Pareto distribution describes grey level distribution better than other considered statistics. Error of passive ranging distance estimation based on intensity method is too large at short distance, because grey level of imaging sensor is saturated. Suggested modification has a great application in compensation of the effect of sensor saturation. Experiment on real saturated infrared sequence demonstrates that distance estimation error of suggested approach increases around three times slower compared to the conventional intensity based approach.

1 Introduction

Optical flow and stereo vision are the most common techniques used to passively estimate distance to an object. Both methods are relied on the geometrical principle of triangulation. In optical flow, the baseline is created due to the sensor motion, whereas in stereo the distance between cameras (baseline) is fixed [1]. Number of used sensors varies from one in optical flow method, through two for single baseline approach, to three or more for multiple baseline method and methods exploring the network of passive sensors. This research is focused on scenarios where only one passive imaging sensor is available.
A few different approaches for passive ranging using a single image sensor are known. The methods presented in [2] and [3] utilize size changes of a target in the image sequence to compute distance to the target. The approach for tracking emissive targets by a monocular passive sensor presented in [4] is based on atmospheric oxygen absorption in near-infrared spectrum, since research [5] utilizes spectral attenuation of two oxygen absorption bands in the visible and near-infrared spectrums for distance to target estimation.
In a recent research [6] two new passive ranging methods based on intensity and contrast measurements are proposed and compared with method based on object size measurement in [7]. It is shown in [6] that error of distance estimation based on contrast method is less than that of the produced by size changes based method. Moreover, intensity based method produces even better results than contrast method. Real life application of distance estimation based on intensity measurements, using one [6] or more [8] passive sensors, is limited by characteristics of used sensors. The effect of sensor saturation has a significant influence on distance estimation accuracy at relatively short distances [6]- [9]. After extracting the target’s pixels from the background, level of pixel intensity is bounded with threshold from one side. Maximal pixel intensity that can be measured causes another limit, known as sensor saturation.
Target tracking approach with two passive infrared sensors suggested in [8] cannot be used in cases where the distance between the sensors and the target is less than the value determined by the maximum output power of the sensors (saturation limit). It is well known that influence of sensor saturation effect increases as distance from the target to the sensor decreases. Research [9] overcomes saturation from atmospheric propagation model estimation by it fusion with object surface measurements and target motion analysis.
The main goal of this research is enhancement of distance estimation based on intensity measurements when sensor operates in saturation conditions. Relevant literature does not include many reports on the phenomenon of saturation using single sensor, such that this paper is deemed to be a modest contribution to the important field of passive ranging.
Since the distance estimation is based on target’s intensity measurements, it is essential to determine intensity in both cases, normal conditions and sensors saturation. The first case is solved in [6], and this research addresses the second, more complicated case. It is expected that the mean target’s intensity in the saturation can be estimated more correctly with suggested method based on object’s statistics knowledge and relevant measurements in saturation than by conventional methods based on measurements.
Another important topic this research is deal with, is to find distribution fitting the target’s statistic the best, and estimate their parameters in saturation. Using the Quantile-Quantile (QQ) plots of the real infrared video sequence of a target, which is moving smoothly toward the sensor, the same statistical distribution of the target’s pixels is verified before sensor saturation occurred. Eight commonly used statistical distributions are considered. Coefficient of linear correlation between real data quantiles and appropriate theoretical distribution quantiles on the QQ-plot is used as quality criterion for distribution fitting, and it is found that General Pareto (GP) distribution satisfied established criterion with the highest rank.
The GP distribution parameters can be determined by various methods [10], [11]. This research suggests a procedure for the estimation of GP distribution parameters, and a estimation of the average of the target’s grey level in saturation. It is shown that, relying on target statistics in normal operation conditions and relevant measurements in saturation, significant improvement of distance estimation can be achieved. The quality of suggested method is tested on real infrared sequence, and a relative error of distance estimation in saturation is up to ten percent smaller than error of standard intensity approach.
The rest of paper is organized as follows. Section 2 describes the posed problem. Section 3 is dedicated to the QQ-plot analysis of the intensities of the target’s pixels on a real infrared video sequence. Section 4 deals with the choice of a distribution fitting target statistics. The procedure for the estimation of GP distribution parameters; estimation of the average of the target’s grey level in saturation and their verification through the experiment on real infrared sequence are described in Section 5.

2 Sensor saturation problem

The application of intensity based method on the real infrared sequence is studied. The sequence is recorded using the Dual Observer Passive Ranging System (DOPRS) that is designed for tracking a single airborne target. The system utilizes two thermal cameras and calculates distance by triangulation method.
In this research, sequence from one camera is used, while the distance obtained from the DOPRS is used as the reference distance in analysis and comparison of results. The reference distance to the target in the analyzed image sequence is determined with an absolute error of less than five meters. Figure 1.a shows the first, Fig. 1.b the 150^th and Fig. 1.c the 350^th frames of the analyzed infrared sequence. Scene intensity in infrared image is represented on grayscale image among the 256 levels of gray. Target in Fig. 1.c has a significant amount of white pixels, indicating the saturation of the sensor.

frames in real sequence — **Fig. 1_I** Target in: a) the first, b) the 150^th and c) the 350^th frames in IR sequence.

The original approach suggested in [6] does not produce acceptable distance estimation when target is close to the sensor as a consequence of target intensity saturation. Probability of the target pixels intensity in the ith frame is defined as:

\begin{equation} \mathbf{p}_{(i)} = \frac{{\mathbf{G}_{(i)} }}{{\sum_{k=0}^{255}\mathbf{G}_{(i)}(k) }},\,\,\,i=1,2,\ldots,N \end{equation}

where G_(i) denotes histogram of the target’s pixels intensity at the i^th frame over the 256 bins and N is the total number of frames in the sequence. The threshold-θ for the detection and segmentation of the object in all frames is determined by the method of Tsai [12]. The results of detection and segmentation in first, 150^th and 350^th are shown in Fig. 1.d, Fig. 1.e and Fig. 1.f, respectively.

Figure 2 shows probability of target pixel intensity (1) as a function of the intensity level in the first frame of the analyzed sequence. It may be noted that the vector p₍₁₎ is equal to zero when the pixel intensity is less than 69, which is the threshold value for the current frame (θ₍₁₎=69). Figure 3 presents the results of the applying (1) on the 350^th frame.

frame in real sequence — **Fig. 2.** Probability p₍₁₎ of the target pixels intensity of the first frame.

In addition to p₍₃₅₀₎ values are zero when pixel intensity is less than 136 (θ₍₃₅₀₎=136), a high value of probability can be noted at the upper limit of the range, as a result of saturation of the sensor. The result of applying (1) to the whole sequence of images is a probability matrix:

\begin{equation} \mathbf{P_T}_{(i)} =\left[\mathbf{p}_{(1)}^T,\mathbf{p}_{(2)}^T,\ldots,\mathbf{p}_{(N-1)}^T,\mathbf{p}_{(N)}^T \right] \end{equation}

which means that the matrix P_T size is 256×N. The matrix P_T is illustrated in Fig. 4, and it can be noted that threshold parameter increases from the first frame (θ₍₁₎=69) to the 350^th frame (θ₍₃₅₀₎=136), and high probabilities in matrix P_T (bright pixels in Fig. 4) move from the lower limit of the range (Fig. 2) to the upper limit of the range (Fig. 3). Saturation becomes dominant after the 300^th frame, meaning that intensity of more than 10 percent of the target’s pixel (white color on p_(i)=255) is measured as upper sensor limit.

3 QQ-plot sequence analysis

It is assumed that the mean target’s intensity in the saturation can be estimated on the basis of object’s statistics. As one target is in the sequence, it is expected that its intensity has the same distribution function in the entire sequence, since the parameters of this distribution can change from frame to frame. 199 QQ-plots from analyzed IR sequence of the target pixels intensities from the frame (i) versus intensities from the frame (i-1) are shown in Fig. 5, where i takes values from 2 to 200. The short review of the QQ-plot technique is presented in [13].

It can be seen that most of the quantiles are close to the straight line of constant slope (45°) throwing the coordinate origin, indicating the same distribution of target intensity over the sequence [13]. Deviations from this line imply that the distribution parameters are changing with time of sequence, as expected with regard to the target’s intensity levels shown in Fig. 4.

4 Target’s intensity distribution consideration

It is assumed that the distribution of the intensity levels of the target can be described using one of the following distributions:

Weibull distribution,
Birnbaum-Saunders (BS) distribution,
Gaussian distribution,
gamma distribution,

Nakagami distribution,
lognormal distribution,
inverse Gaussian distribution, and
general Pareto (GP) distribution [10], [11].

The real data are fitted on the above distributions using maximum likelihood method, and as a result two parameters, describing appropriate distribution, are obtained for each sequence frame. With the aim of selecting which of the distributions best fits the real data, the following procedure is performed:

QQ-plots of the analyzed theoretical distribution against the real data is done, for first N=200 frames in the video sequence.
Linear correlation coefficients r_i (i=1,2,...,N ) of points in QQ-plot are calculated, for all N=200 frames.
Mean value – $\bar{r}$_1:N of all N=200 linear correlation coefficients for analyzed distributions is calculated.

Results of analyses are presented in Table 1.

Table 1. Distribution Fitting Results.

Distribution	$\bar{r}$_1:N	Rank
Weibull	0.95177	8
Birnbaum-Saunders (BS)	0.97580	2
Gaussian	0.95205	7
gamma	0.97261	5
Nakagami	0.96645	6
lognormal	0.97485	4
inverse Gaussian	0.97579	3
general Pareto (GP)	0.99371	1

On the basis of the given results it can be concluded that GP distribution the most closely describes the target’s intensity for established criterion: mean value of linear correlation coefficients for analyzed two hundred frames is the largest: $\bar{r}$_1:N = 0.9937. As examples, Figs. 6. and 7. show a QQ-plots of real data quantiles from one frame versus theoretically quantiles of GP (the first ranked) and BS (the second ranked) distributions, respectively, where straight lines represent a perfect match to distributions. The QQ-plot in Fig. 6. approximately matches the straight line, while in Fig. 7. deviations from the line are significant, which is an additional confirmation of the correctness of the distribution function choice.

5 Experimental results

Having previously established that the distribution of target’s pixel intensity has a GP distribution, the distance to the target in saturation will indirectly be estimated on the basis of estimations of the shape parameter k and scale parameter σ of the GP distribution (Appendix), and the measured values of the threshold θ. It is assumed that the sensor saturation occurs when more than 20% of the target’s pixels have a maximum pixel intensity level, which in the analyzed sequence occurs after 289 frames. The GP distribution parameters k and can be determined by various methods [10], [11], depending on their range. The range of estimated parameters is: k ∈ [-1.4,-0.06], σ ∈ [31, 180], and θ ∈ [68, 139]. Since the relevant literature utilizes two different notations for the cumulative general Pareto distribution (GPD) function (sign of the shape parameter), notation used through this research is given in the Appendix.
In [6] to estimate the distance to the target, average of the target’s gray level (intensity) is used. In order to preserve information about the intensity mean it is more convenient to use the method of moments (MOM) instead of the maximum likelihood method to estimate parameters of the GP distribution. Figure 8 shows estimation of shape parameter $\hat{k}$ to the frame when sensor enters the saturation (gray circles). Parameter $\hat{k}$ data are fitted as a linear polynomial function based on estimations before the sensor saturation arises, and as a result, prediction is obtained (full line), with the determination coefficient $r^2_{\hat{k}}$ = 0.79.

Estimations of scale parameter $\hat{\sigma}$ before sensor saturation are shown with gray circles in Fig. 9. Parameter $\hat{\sigma}$ data are fitted as a quadratic polynomial function based on estimations before the saturation occurred, and prediction $\hat{\sigma}^P$ of scale parameter is obtained as a result (full line), with the determination coefficient $r^2_{\hat{\sigma}}$ = 0.96.
According to [6], distance to the target in frame is:

\begin{equation} D_{(i)} = D_{(i-1)} + \frac{1}{\zeta }\ln \frac{{I_{(i-1)} }}{{I_{(i)} }} \end{equation}

where D_(i) and D_(i-1) are distances from the object to the sensor, and I_(i) and I_(i-1) are the average gray levels of the object on two successive frames. Initial range D₍₁₎ and a reliable estimate of extinction coefficient ζ are required. For this experiment, coefficient ζ is estimated on the test sequence preceding the analyzed, as suggested in [6], while initial distance D₍₁₎ is taken from DOPRS. Intensity of the target is presented in Fig 10. Gray line denotes intensity measured directly from the sequence denoted with , while black line identifies intensity estimation based on target statistics (Appendix):

\begin{equation} \hat{I}_{(i)}=\frac{\sigma(i)}{1-k(i)}+\theta_{i}, \left\{ \begin{matrix} \sigma_{(i)}=\hat{\sigma}_{(i)}, & k_{(i)}=\hat{k}_{(i)}, & i \lt i_{sat}\\ \sigma_{(i)}=\hat{\sigma}_{(i)}^P, & k_{(i)}=\hat{k}_{(i)}^P, & i\ge i_{sat} \end{matrix} \right. \end{equation}

where i_sat represents ordinal number of the frame at the time when saturation is detected.

It can be noted that before the saturation occurred, estimated intensity Î had a good fit to the measured intensity $\bar{x}$, since the estimation of Î is performed on the basis of mean and variance of a real signal. In saturation Î and $\bar{x}$ diverge, as expected. Distances to the target in the analyzed sequence are calculated based on (3) and intensities Î and $\bar{x}$, and results with a true distance from DOPRS are shown in Fig. 11. Distance D₍₁₎ from the DOPRS is used for initialization of (3) in both calculations.
Relative errors of distance estimation obtained by original approach [6] and suggested modification are shown in Fig.12. Significant improvement of distance estimation is clearly observable in saturated region (right side in Fig.12). Distance estimation error of GP based approach increases around three times slower compared to the conventional intensity based approach. This confirms the assumption that the distance to the target in the saturation can be estimated more accurately based on determined object’s statistics and relevant measurements in saturation than by conventional method [6] based on raw measurements.

Estimated distances — **Fig. 11.** True and estimated distances.

6 Conclusions

In authors’ recent research [6], two new passive ranging methods based on intensity and contrast measurements are proposed and compared with the method based on object size measurements. It is shown that error of distance estimation based on contrast method is less than that of the produced by the size changes based method. Moreover, intensity based method produces even better results than contrast method. Error of passive ranging distance estimation based on intensity method is unacceptably large at short distances, since gray level of acquisition sensor is saturated.
This research suggests the extension of the intensity method for passive ranging using a single camera operating in normal and saturation conditions, enabling the significantly better distance estimation at short ranges. Estimation of the target’s mean value of gray level in non ideal conditions (sensor saturation) is based both on evaluation of object statistics and image measurements, instead of using the only image measurements in standard intensity based approaches. It is found that general Pareto distribution is fitting real target intensity best, compared to other distributions analyzed in the paper. A simple algorithm for GP parameters estimation and prediction is suggested and experimentally confirmed.
Experiment on real saturated infrared sequence verifies that a relative error of distance estimation is up to ten percent smaller and increases around three times slower compared to error of the conventional intensity based approach. Although the distance estimation error increases with intensity saturation time, the new approach enables additional time for target tracking, depending on its speed and size as well as of used image sensor characteristics.

References

T. Surgailis, A. Valinevicius, V. Markevicius, D. Navikas, and D. Andriukaitis: Avoiding forward car collision using stereo vision system, Electronics and Electrical Engineering 18(8), pp. 37–40, 2012.
C. Raju, S. Zabuawala, S. Krishna, and J. Yadegar, “A hybrid system for information fusion with application to passive ranging,” in Proc. of Int. Conf. on Image Processing, Computer Vision and Pattern Recognition, 2007, pp. 402–406.
D.R. Van Rheeden, “Passive range estimation using image size measurements,” February 2 1999. US Patent 5,867,256.
J.R. Anderson, M.R. Hawks, K.C. Gross, and G.P. Perram, “Flight test of an imaging O (Xb) monocular passive ranginginstrument,” in Proc. of SPIE 8020, pp 802005–12, 2011.
R.A. Vincent and M.R. Hawks, “Passive ranging of dynamic rocket plumes using infrared and visible oxygen attenuation,” in Proc. of SPIE, vol. 8052, pp 80520D–16, 2011.
Ž. P. Barbarić, B. P. Bondžulić, and S. T. Mitrović: Passive ranging using image intensity and contrast measurements,” Electronics Letters 48(18), pp. 1122–1123, 2012.
B.P. Bondžulić, S.T. Mitrović, Ž.P. Barbarić and M.S. Andrić: A comparative analysis of three monocular passive ranging methods on real infrared sequences, Journal of Electrical Engineering 64(5), pp. 305-310, 2013.
Ž. M. Đurović, B. D. Kovačević, and G. D. Dikić: Target tracking with two passive infrared non-imaging sensors, Signal Processing, IET 3(3), pp. 177–188, 2009.
M. De Visser, P.B.W. Schwering, J.F. de Groot, and E.A. Hendriks: Passive ranging using an infrared search and track sensor, Optical Engineering 45, p 026402, 2006.
P. de Zea Bermudez and S. Kotz, Parameter estimation of the generalized Pareto distribution-Part I, Journal of Statistical Planning and Inference 140(6), pp. 1353–1373, 2010.
P. de Zea Bermudez and S. Kotz, Parameter estimation of the generalized Pareto distribution-Part II, Journal of Statistical Planning and Inference 140(6), pp. 1374–1388, 2010.
W.-H. Tsai: Moment-preserving thresolding: A new approach, Computer Vision, Graphics, and Image Processing 29(3), pp. 377–393, 1985.
G. S. Kvaščev, Ž. M. Đurović, and B. D. Kovačević: Adaptive recursive M-robust system parameter identification using the QQ-plot approach, IET Control Theory and Applications 5(4), pp. 579–593, 2009.

Appendix

Let X be a random variable. The cumulative GPD function with location, shape, and scale parameters θ(θ ∈ ℝ), k(k ∈ ℝ) and σ(σ > 0), respectively, is defined as:

\begin{equation} F(x|\theta,k,\sigma)=1-\left(1+k \frac{x-\theta}{\sigma} \right)^{-\frac{1}{k}}, \,\,\,\,k\neq 0 \end{equation}

and probability density function for the three parameters generalized Pareto distribution is:

\begin{equation} f(x|\theta,k,\sigma)=\frac{1}{\sigma} \left(1+k \frac{x-\theta}{\sigma} \right)^{-\frac{1}{k+1}} \end{equation}

In applications when threshold parameter θ is known, its subtraction from signal X allows the use of two parameters GPD(θ=0), then (5) becomes:

\begin{equation} F(x|k,\sigma)=1-\left(1+k \frac{x}{\sigma} \right)^{-\frac{1}{k}}, \,\,\,\,k\neq 0 \end{equation}

and (6) transforms to:

\begin{equation} f(x|k,\sigma)=\frac{1}{\sigma} \left(1+k \frac{x}{\sigma} \right)^{-\frac{1}{k+1}} \end{equation}

The mean value and the variance of GPD(k, σ) have the following expressions:

\begin{equation} E(X)=\frac{\sigma}{1-k} \end{equation}

where k < 1.

\begin{equation} Var(X)=\frac{\sigma^2}{(1-k)^2(1-2k)} \end{equation}

where k < 0.5, while the mean value of three parameters GPD(θ,k,σ) is:

\begin{equation} E(X)=\frac{\sigma}{1-k}+\theta \end{equation}

where k < 1. The time-honored and direct MOM is widely used for estimating the parameters of the two-parameters GP distribution [10]. The MOM estimates of parameters and are obtained from expressions for the mean (9) and the variance (10), as follows:

\begin{equation} \hat{k}=\frac{1}{2}\left(1-\frac{\bar{x}^2}{s^2} \right) \end{equation}

\begin{equation} \hat{\sigma}=\frac{\bar{x}}{2}\left(\frac{\bar{x}^2}{s^2} +1 \right) \end{equation}

provided k < 0.5, where $\bar{x}$ and s² stand for the sample mean and the sample variance, respectively [10].