Computational Vision, Fall 1999
Class 11, Wednesday October 6
Geometry of Image Formation

Review from last class

• Introduction to rigid, affine and perspective transformations in two and three dimensions.
  • We will finish this discussion today.
• We reviewed the class design for Spatial Objects in TargetJr.

Today

• Finish the discussion of transformation models.
• Introduce the geometry of image formation.

The geometry of image formation

• Camera modeling from a geometric perspective.
• Fundamental question: ``What is the image location of the projection of a point in the world?''
• Answering this leads forces us to examine the algebra and geometry of perspective, orthographic and pseudo-perspective projection.
• Next week we will discuss the formation of light intensity values.

Reading suggestions

• The best discussion is in Chapter 3 of Faugeras (on reserve). Skip sections 3.3.1.2 and 3.4.1.3 unless you are prepared to read Chapter 2 thoroughly. In reading Faugeras, be aware that points are described using the notation ${\bf M}$ and ${\bf m}$ and matrices are described using ${\bf P}$ and ${\bf p}$. This is exactly the opposite of intuition.
• Another option is to pick your way through Chapter 12 of Jain, Kasturi and Schunck. There is a lot more in this chapter than we are going to cover.

Perspective Projection

• The perspective model uses a simplified lens model where the diameter of the lens is infinitesimally small. This means that all light rays pass through the lens and strike the image plane without bending, and every point in the image is sharply focused.
• The perspective model is formed by drawing a 3-D coordinate system:

  
  
  
  

  
  

  
  
  
  

  • The image plane is the plane on which the light is recorded. For now, think of it an inverted Cartesian plane.
  • The center of projection is the point through which all line rays pass. It is (0,0,0)^T in the coordinate system shown in the figure.
  • The optical axis is the line normal to the image plane and through the center of projection. It is the Z-axis in this figure.
  • The piercing point is the intersection of the optical axis and the image plane. We can write its coordinates in both the 3D world and the 2D image plane.

• The perspective projection of a point, ${\bf P} = (X, Y, Z)^T$, in the world is found by similar triangles to be

  $${\bf p} = (x,y)^T = \left( \frac{f X}{Z}, \frac{f Y}{Z} \right)^T = \frac{f}{Z} (X, Y)^T$$

  Note the use of capital letter for world coordinates and small letters for image coordinates. This will be our convention.
• The perspective model is often drawn with the image plane in front of the center of projection and the image coordinate system not inverted.

  
  
  
  

  
  

  
  
  
  

Pseudo-Perspective and Orthographic Projections

• Suppose all objects appearing in the image are at approximately the same depth, i.e. the depth of any point is

  $$Z \in [ Z_0 - \Delta Z, Z_0 + \Delta Z]$$

  where $Z_0 \gg \Delta Z$.
• Then,

  $$(x,y)^T \approx \frac{f}{Z_0} (X,Y) = m (X, Y)$$

  where m = f/Z₀ is the magnification. This is the pseudo-perspective model.
• Letting $m \rightarrow 1$ gives

  (x, y) = ( X, Y)

  which is the model for orthographic projection.

Perspective vs. Orthographic Projection

• Perspective projection is the more accurate but more complex model.
• Pseudo-perspective and orthographic projections are useful models for techniques recovering orientations of surfaces that are relatively far from the camera.
• In both orthographic and perspective projections, an image location ${\bf p}$ is the projection of a point that sits along an infinite ray (line) in the world.
  • The location of this point can not be determined from a single image, but the equation of the line can.

Matrix Description of Perspective Projection

• Switching to homogeneous coordinates, the perspective projection may be written in matrix form:

   

  $$\widetilde{{\bf p}} = \begin{pmatrix} f & 0 & 0 & 0 \\ 0 & f & 0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix} \widetilde{{\bf P}}$$ (1)

• Dividing through by the last term in $\widetilde{{\bf p}}$ retrieves the image (affine) coordinates.

Exercises

1.

Find the line of points in the world that projects to image location ${\bf p}$. Consider both orthographic and perspective projections.

2.

Write a matrix to describe pseudo-perspective projection.

Intrinsic Camera Parameters

• We need to generalize the assumptions made about the image plane in deriving the perspective model.
• In switching from world to image coordinates:
  • The dimensions (units) have changed. We need to know the new units.
  • The origin of the coordinate system has changed.
  • Sometimes even the angle between the x and y axes in a camera is not 90 degrees!
• More specifically, we need to know the pixel dimensions k_x and k_y and the position (x₀, y₀)^T of the piercing point in the image array. These are the ``intrinsic'' camera parameters.

  
  
  

  
  

  
  
  

• If $\widetilde{{\bf p}} = (x,y,1)^T$ is the coordinate vector of a point in the image array coordinate system and if $\widetilde{{\bf p}}' = (x',y',1)^T$ is the coordinates of the same point in the idealized coordinate system used thus far,

  $$\widetilde{{\bf p}} = \begin{pmatrix} k_x & 0 & x_0 \\ 0 & -k_y & y_0 \\ 0 & 0 & 1 \end{pmatrix} \widetilde{{\bf p}}'.$$

• Using this, the complete perspective matrix becomes
  $$\begin{align} \begin{pmatrix} k_x & 0 & x_0 \\ 0 & -k_y & y_0 \\ 0 & 0 & 1 \en... ...x_0 & 0 \\ 0 & -\alpha_y & y_0 & 0 \\ 0 & 0 & 1 & 0 \end{pmatrix}\end{align}$$
  
• $\alpha_x = f k_x$ and $\alpha_y = f k_y$ are scaling parameters. Their individual components can not be determined with the model we are using here.

Extrinsic Camera Parameters

• Thus far we have assumed the points in the world have been described in a coordinate system centered at the camera.
• There is no way to know this coordinate system because the origin is inside the camera lens and axes depend on the internals of the camera.
• We generally have to describe points in terms of a known coordinate system in the world and then transform this coordinate system.
• This transformation from the known coordinate system to the initially unknown camera centered coordinate system is a rigid transformation.
• Hence, if $\widetilde{{\bf P}}$ is the homogeneous coordinate vector for a point in the world, its corresponding point in the image is

  
  $$\begin{align} \widetilde{{\bf p}} &= \begin{pmatrix} \alpha_x & 0 & x_0 & 0 \\ ... ...trix} \widetilde{{\bf P}} \\ &= {\bf K} \widetilde{{\bf P}} \nonumber\end{align}$$
  

• ${\bf K}$ is a 3x4 matrix called the camera matrix. As described, ${\bf K}$ has 10 degrees of freedom.
• This can be written out as

  $${\bf K} = \begin{pmatrix} {\bf k}_1^T & k_{14} \\ {\bf k}... ...23} & k_{24} \\ k_{31} & k_{32} & k_{33} & k_{34}\end{pmatrix}$$

• What's called the ``affine camera'' is obtained by assigning ${\bf k}_3 = {\bf 0}$, yielding  

  $${\bf K}_a = \begin{pmatrix} {\bf k}_1^T & k_{14} \\ {\bf ... ..._{21} & k_{22} & k_{23} & k_{24} \\ 0 & 0 & 0 & 1\end{pmatrix}$$ (2)

Camera Calibration

• The knowns are a set of measured (e.g. with a ruler!) point coordinates in the world $\{ \widetilde{{\bf P}}_i \}$ and a set containing the corresponding points in image coordinates $\{ \widetilde{{\bf p}}_i \}$.
• The unknowns are the parameters of ${\bf K}$.
• Since $\widetilde{{\bf p}}_i = {\bf K} \widetilde{{\bf P}}_i$, each pair of points gives two constraints on K. A minimum of 6 points are necessary.
• A least-squares (either linear or non-linear) fit yields the parameters of ${\bf K}$. The intrinsic and extrinsic parameters can be obtained from these.
  • We will not go into the details of this now.

Homework

The due date for this will be announced in class.

1.

Show that a line in the world projects onto a line in the image under perspective projection. You may use the original (simplified) camera matrix (Equation 1) that includes neither intrinsic nor extrinsic parameters.

(a)

10 points Prove this algebraically. Start with the parametric equation for a line in three-dimensions. Project this equation into an image. Then eliminate the parameter s from the resulting equations.

(b)

5 points Under what conditions is the projected line actually a single point!

(c)

5 points, extra credit Prove the claim using a geometry argument. In other words, without using equations, reason about why a line in the world must project onto a line on the image plane.

2.

5 points Use your result from 1a to show that parallel lines in the world DO NOT necessarily project to parallel lines in an image.

3.

5 points Show that using an affine camera (Equation 4), parallel lines in the world DO project to parallel lines in an image.