360° Virtual-World Imaging

Kelvin Chew Tiong Meng

B.Eng. Final Year Project 2000/2001

The University of Birmingham

School of Electronic & Electrical Engineering

---

1. Imaging Theory:

 

 

STAGE 1:

2 images sharing the same scene can be related by a Projective transformation. [Wolberg'90][Foley'94][Heckbert'89]. This is calculated with the corresponding control points selected by the user.

 

The general form of a projective transformation is a rational linear mapping:

&

 

 

In homogeneous coordinates, this is given algebraically as:

I' = M.I

 

 

or in matrix notation:

 

 

 

 

STAGE 2:

After solving for the initial motion parameters, local image registration was performed using the Levenberg-Marquardt Iterative non-linear Minimsation Alogorithm. This method minimises the squared intensity errors between images iteratively which fine tunes the motion parameters.

The complete registration algorithm as given in [Szeliski’96] consists of the following steps:

1. For each pixel at location (xi, yi):

(a). compute its corresponding position in the other image (x’i, y’i) using equations:

& ;

(b). compute the error in intensity between the corresponding pixels, ei with equation:

;

(c). compute the intensity gradient (¶ I’/¶ x’, ¶ I’/¶ y’) using bilinear intensity interpolation on I’;

(d). compute the partial derivative of ei with respect to mk using equation:

;

(e). add the pixel’s contribution to the Hessian matrix, A and Intensity gradient vector, b with components:

& ;

 

2. Solve the system of equations,and update the transform matrix estimate

m(t+1) = m(t) + D m.

3. Check that the error, E has decreased. If not, increment l by a factor of 10 as described in [Press’92] and compute a new D m.

4. Continue iterating until either the error is below a threshold or a fixed number of steps have been completed.

 

 

STAGE 3:

 

With the motion parameters, the observed image was warped onto the projection of the reference image. Bilinear interpolation was used to resample all mapped pixels where its intensity is the average of four neighbouring pixels.

 

As shown in [Wolberg'90], 2 pass bilinear interpolation was used.

 

 

1. In the first pass along the horizontal direction, we compute:

 

2. These two results are then combined in the second vertical pass to obtain the final interpolated value:

 

 

 

 

STAGE 4:

To eliminate the seams for a smooth transition between images, a weighted average splining technique was used for this purpose [Burt'83]. Pixels at the center of the image contributes more than pixels at the edges in a linearly decreasing fashion. This can be shown mathematically where F(i) is the splined image:

 

 

 

 

 

 

All implementations were done in MATLAB 5.0 with the Image Processing Toolbox v2.2.

Click here to see the results.

 

 

 

 

References:

 

[Burt'83] P.J.Burt E.H.Adelson. A Multiresolution Spline with Application to Image Mosaics. ACM Trans. On Graphics, 2(4):217-236, Oct 1983.

 

[Foley'94] J.D.Foley (et al.). Introduction to Computer Graphics. Addison-Wesley Publishing Company Inc, 1994.

 

[Heckbert'89] P.S.Heckbert. Fundamentals of Texture Mapping and Image Warping. Master's Thesis, Dept. of Electrical Engineering and Computer Science, University of California, Berkeley, June 1989.

 

[Szeliski'94] R.Szeliski. Image Mosaicing for Tele-Reality Applications. Technical Report CRL 94/2, Digital Equipment Corp. Cambridge Research Lab, May 1994.

 

[Szeliski'96] R.Szeliski. Video Mosaics for Virtual Environments. IEEE Computer Graphics & Applications, Vol.16, No.2, March 1996.

 

[Wolberg'90] G.Wolberg. Digital Image Warping. IEEE Computer Society Press, 1990.

 

Back to FYP Main