The pinhole camera model is a simple, yet accurate model for most cameras [21], and it is defined by the following elements:
The retinal plane R, onto which the image is formed;
The optical center C;
The focal length f, equal to the distance between R and C;
The perspective projection operator.
The model is shown in Figure2.1. The imagemof a pointMis the intersection of the line< CM >with the planeR. The operator that performs the projection is called perspective projection. Other elements derived from the pinhole camera model are:
2.1 Pinhole Camera Model
The focal plane F, which is parallel to R and contains C;
The optical axis, which is the perpendicular from C toR;
The principal point c, which is the intersection of the optical axis withR.
y
Figure 2.1: Pinhole camera model.
The points on F do not have an image, because the lines < CM >, M ∈ F, are parallel to the retinal plane, or undefined, if M is in C.
Let (C, x, y, z) be the coordinate system of the camera and (c, u, v) the coordi-nate system of the retinal plane, with the axisxparallel withuand, respectively, the axis yparallel with v, as shown in the figure. Let a generic point M (not be-longing toF) have the coordinates (X, Y, Z), and its image mcoordinates (U, V), then:
equation (2.1) can be rewritten, using homogeneous coordinates:
2.1 Pinhole Camera Model
The camera coordinate system is said to be normalized, if the retinal plane is situated at unit distance from the optical center (the plane z = 1). A camera with the retinal plane atz = 1 is a normalized camera. The image on the camera with the retinal plane z =−f is the image on the normalized camera, reversed and scaled by f.
The perspective projection operatorP from (2.3) is further extended, to com-pensate that:
The origin of the image coordinate system does not coincide with that of the retinal coordinate system;
The axis units of the image system coordinate system may differ.
Letku and kv be the axes scales of the image coordinate, andU0 and V0 the coordinates of c, in image coordinate system. Then, the camera projection P, decomposed into a perspective projection PN on the retinal plane of the nor-malized camera, and a homography transformation K to image coordinates (see Appendix A for 2D homographies), is:
P = K · PN V0 do not depend on the position and orientation of the camera in space, and are called intrinsic parameters of the camera. The homography K transforms the distance units of the camera space to units of image space (pixels). If the measure units for x, y, z, andf, are expressed in meters, and,u and v in pixels, then αu and αv are expressed in pixels, and ku, kv in pixel/meter;U0 and V0 are expressed in pixels.
The camera model described by the projection operator with the matrixP, as defined above, is that of an ideal (distortion-free) pinhole camera. Real cameras present aberrations, due to imperfections of the lens. The most usual aberration
2.1 Pinhole Camera Model
is the radial distortion, due to the imperfections of the curvature of the lens.
Other distortions are possible, such as the tangential distortion, caused by the misalignment between the centers of curvature of the surfaces of the lens.
To support nonlinear lens distortion, the pinhole camera model is extended in (2.5), with first and second order radial and tangential distortion coefficients, k1 and k2, respectively p1 and p2, with (U0, V0), the principal point, assumed to be the center of distortion:
Up to now we considered only one 3D coordinate system, the one associated with the camera. In general, the objects are given in a different coordinate system (world, or object coordinate system), so, in addition to the projection described above, we have to transform the points from world to camera coordinate system.
This is a 3D rigid body transform, defined by the 3×3 rotation matrixRand the 3×1 translation vector T, which transforms the world coordinate (Xw, Yw, Zw) to camera coordinate (X, Y, Z):
The parametersRandT from (2.6) are theextrinsic parameters of the camera.
Although, in (2.6), they are represented by 12 parameters, the extrinsic part has 6 degrees of freedom, 3 for rotation and 3 for translation.
We update the projection matrix P to model the overall transformation from world coordinates to image coordinates, assuming the distortion-free pinhole
cam-2.1 Pinhole Camera Model
The 3×4 projection matrix P is called camera matrix and it describes the transformation from 3D world coordinates to 2D image coordinates (in homoge-neous coordinates). In (2.7), P is decomposed in a projection operator for the normalized camera and anisotropic 2D scaling. The distortion-free pinhole cam-era model described above has 10 degrees of freedom: 6 corresponding to the rigid-body transform and 4 intrinsic to the camera. The matrix K is called the camera calibration matrix. The distortion correction follows equations (2.5).
The task of resectioning is to find the perspective projection matrix P; P may be eventually decomposed into K and R|T; we denote by R|T the 3×4 projection matrix of a normalized camera. A general camera matrix P has 11 degrees of freedom (since, in homogeneous coordinates, the projection is invariant with respect to scale of the matrix, one degree of freedom is lost). Decomposition into K and R|T is achieved by factorizing the 3×3 left sub-matrix of P, using RQ decomposition, into the product of an upper-triangular matrix (K) and an orthogonal matrix (R). In the degenerated case when the left 3×3 sub-matrix is singular, RQ decomposition is not possible, and P does not correspond to a real camera. Otherwise, K is also invertible. The translation vector T is obtained by forward multiplying the last column of P with K−1.
Since the 2D homography achieved by the multiplication withK is invariant with respect to the scale of the matrix, the matrix K is normalized so that the last element is 1 (the element on the third row and third column). If P can be factorized into K and R|T, the last element of K will not be 0 (otherwise, K would be singular), so the normalization is possible. After the decomposition of P, the calibration matrix K has 5 degrees of freedom. In the model discussed