The Pinhole Camera

The Pinhole Camera

The pinhole camera is the simplest geometric model of how a camera forms an image. All light entering the camera passes through a single point, the optical center, before reaching a flat image plane. Every 3D point in the world projects to a 2D point on this plane along a straight ray through the optical center.

Three coordinate frames are used:

• World frame: $X_w=[x_w,y_w,z_w,1{]}^T$ — positions of objects in the scene.
• Camera frame: $X_c=[x_c,y_c,z_c{]}^T$ — origin at the optical center, $z$-axis pointing forward along the optical axis, $x$ to the right, $y$ down (or up, depending on convention).
• Image (pixel) frame: $(u,v)$ — indexed from a corner of the sensor.

The map world → camera → image composes three operations:

1. Extrinsics (rigid transformation): $X_c=R{X}_w+T$, where $R\in SO(3)$ is the camera’s orientation and $T\in {\mathbb{R}}^3$ its position.
2. Perspective projection onto the image plane at $z=f$:
   $x_i=f\frac{x_c}{z_c},y_i=f\frac{y_c}{z_c}.$
3. Intrinsics (pixelation): multiply by the intrinsic matrix
   $K=\left[\begin{array}{ccc}{f}_x& 0& o_x\\ 0& f_y& o_y\\ 0& 0& 1\end{array}\right],$
   where $f_x,f_y$ are focal lengths in pixels and $(o_x,o_y)$ is the principal point (pixel where the optical axis meets the image).

In homogeneous coordinates, the composite projection is
$s\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=K[R\mid T]\left[\begin{array}{c}{x}_w\\ y_w\\ z_w\\ 1\end{array}\right],$
where $s=z_c$ is the projective scale factor. Dividing the homogeneous result by $s$ gives the pixel coordinates.

The Pinhole Inversion front of the optical center, and the image is inverted (flipped horizontally and vertically). Real cameras place the sensor behind the pinhole, which is geometrically equivalent but produces an uninverted image.

In pure pinhole geometry the image plane lies in

Intuition

A camera takes a 3D world and flattens it onto a 2D image. The pinhole model makes this flattening exact and geometric: each pixel corresponds to a single ray through the optical center, and the scene point that produced the pixel is somewhere along that ray. The division by $z_c$ in the projection formula is the single most important feature of the model — it is perspective foreshortening, the reason distant objects look smaller, and the reason a single image cannot tell you how far away anything is (the ray alone is known; the depth along the ray is lost).

The intrinsic and extrinsic decomposition is what makes the model composable. Extrinsics tell you how the camera is placed in the world; intrinsics tell you how the camera turns metric-units rays into pixel-units measurements. Calibration, stereo, and multi-view reconstruction all rest on this split: extrinsics change as the camera moves, intrinsics stay fixed.

Properties

Projection

(Perspective division) Depth scaling is the hallmark of projection: $x_i=f{x}_c/z_c$.

The division by $z_c$ encodes that a point twice as far away projects to half the image-space offset. Translated into images, this is why two parallel rails appear to converge to a vanishing point — the ratio $x_c/z_c$ tends to a limit as $z_c\to \infty$.

(Loss of depth) A single image determines the ray through each pixel but not the depth along it.

Given a pixel $(u,v)$ and known intrinsics $K$, one can recover the ray $K^{-1}(u,v,1{)}^T$ in camera coordinates, but the scene point could lie anywhere along that ray. Recovering depth requires a second view (stereo), motion (SfM), or active sensing (structured light, LiDAR).

(Homogeneous projection matrix) The full $3\times 4$ projection matrix is $P=K[R\mid T]$.

In homogeneous coordinates the projection reads $\tilde{x}=P\tilde{X}$ as a linear map. This linearity is what makes projective multi-view geometry tractable: epipolar constraints, triangulation, and bundle adjustment all rely on working in homogeneous coordinates so that the nonlinear “divide by $z_c$” disappears from the estimator and returns only at the final de-homogenization.

Intrinsic Parameters

(Focal length) $f_x$ and $f_y$ are the focal lengths measured in pixels.

In physical units $f$ is a distance (millimeters). Converting to pixels requires multiplying by the sensor-pixel density: $f_x=f/s_x$ where $s_x$ is the pixel pitch along $x$. Non-square pixels produce $f_x\ne f_y$; for most modern sensors these are equal to within measurement precision.

(Principal point) $(o_x,o_y)$ is the pixel location where the optical axis pierces the image.

Ideally the principal point is at the image center, but manufacturing tolerances shift it by a few pixels. Calibration recovers $o_x,o_y$ along with $f_x,f_y$.

(Skew) The off-diagonal entry of $K$ encodes sensor axis non-orthogonality and is almost always zero for modern sensors.

Some texts write $K$ with a skew parameter $s$ in the $(1,2)$ position. It accounts for the angle between $x$ and $y$ sensor axes deviating from 90° and is negligible for CMOS/CCD sensors.

Extrinsic Parameters

(Rigid body transform) Extrinsics $(R,T)\in SE(3)$ describe the camera’s pose.

$R$ rotates the world frame into the camera frame; $T$ translates the world origin into the camera frame. Together they specify where the camera is and where it is looking. When multiple cameras share a scene, their relative pose $(R_{12},T_{12})=(R_2{R}_1^T,T_2-R_2{R}_1^T{T}_1)$ is the input to epipolar geometry.

(Calibration recovers intrinsics) A standard Zhang calibration recovers $K$, $R$, $T$ from images of a known planar target.

Several views of a checkerboard of known dimensions give enough correspondences to fit $K$ via the DLT, then refine $K$ jointly with per-view $(R_i,T_i)$ by minimizing reprojection error. OpenCV’s calibrateCamera is the standard implementation.

Limitations

(No lens distortion) Real lenses deviate from the pinhole model at wide fields of view.

Radial distortion (barrel and pincushion) bends straight lines near the image edges; a standard correction is the Brown-Conrady model
$x_d=x(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)+\text{tangential terms},$
whose coefficients are fit alongside $K$ during calibration. After undistortion the pinhole model holds to sub-pixel accuracy.

(No finite aperture) The pinhole model assumes an infinitesimal aperture, so everything is in focus.

Real cameras have finite apertures and depth-of-field effects (blur away from the focal plane). For small apertures the pinhole model is an excellent approximation; for large apertures (shallow DoF) the thin-lens model is needed.

Examples

Pixel Coordinates from a World Point

A camera at world origin with $R=I$, $T=0$, $f_x=f_y=500$ px, $(o_x,o_y)=(320,240)$. Where does the world point $X_w=(1,0.5,5)$ project?

Solution $X_c=X_w=(1,0.5,5)$, so $z_c=5$.

With identity extrinsics,

Projection: $x_i=f_x\cdot 1/5=100$, $y_i=f_y\cdot 0.5/5=50$.

Pixel: $u=x_i+o_x=420$, $v=y_i+o_y=290$.

Equivalently, $s(u,v,1{)}^T=K(X_c)=(500\cdot 1+320\cdot 5,500\cdot 0.5+240\cdot 5,5)=(2100,1450,5)$, and dividing by $s=5$ gives $(420,290,1)$.

Ray from a Pixel

Given a pixel $(u,v)=(420,290)$ and the same intrinsics, what ray in camera coordinates does it correspond to?

Solution $K^{-1}(u,v,1{)}^T={\left(\frac{u-o_x}{f_x},\frac{v-o_y}{f_y},1\right)}^T=(0.2,0.1,1{)}^T$.

The ray is $\{t(0.2,0.1,1):t>0\}$. Depth is unknown from a single pixel — any scene point along this ray would project to the same pixel.

See Also

• 5.2. Stereo Geometry — the two-camera setup that recovers the lost depth
• 5.3. Epipolar Geometry — the two-view generalization of the pinhole projection
• 3.4. Homographies — a special case relating two pinhole views of a plane
• SfM — multi-view extension of pinhole-projection estimation
• 4. Visual Servoing — where the pinhole projection is differentiated to obtain the image Jacobian