Cameras, lenses and sensors : Cameras, lenses and sensors Marc Pollefeys
COMP 256
Cameras, lenses and sensors : Camera Models
Pinhole Perspective Projection
Affine Projection
Camera with Lenses
Sensing
The Human Eye
Reading: Chapter 1. Cameras, lenses and sensors
Slide3 : Images are two-dimensional patterns of brightness values. They are formed by the projection of 3D objects. Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of
Naval Personnel. Reprinted by Dover Publications, Inc., 1969.
Slide4 : Animal eye:
a looonnng time ago. Pinhole perspective projection: Brunelleschi, XVth Century.
Camera obscura: XVIth Century. Photographic camera:
Niepce, 1816.
Distant objects appear smaller : Distant objects appear smaller
Parallel lines meet : Parallel lines meet vanishing point
Vanishing points : Vanishing points VPL VPR H VP1 VP2 VP3 To different directions
correspond different vanishing points
Geometric properties of projection : Geometric properties of projection Points go to points
Lines go to lines
Planes go to whole image
or half-plane
Polygons go to polygons
Degenerate cases:
line through focal point yields point
plane through focal point yields line
Slide9 : Pinhole Perspective Equation
Slide10 : Affine projection models:
Weak perspective projection is the magnification. When the scene relief is small compared its distance from the
Camera, m can be taken constant: weak perspective projection.
Slide11 : Affine projection models:
Orthographic projection When the camera is at a
(roughly constant) distance
from the scene, take m=1.
Slide12 : Planar pinhole
perspective Orthographic
projection Spherical pinhole
perspective
Limits for pinhole cameras : Limits for pinhole cameras
Slide14 : Camera obscura + lens
Slide15 : Lenses Snell’s law
n1 sin a1 = n2 sin a2 Descartes’ law
Slide16 : Paraxial (or first-order) optics Snell’s law:
n1 sin a1 = n2 sin a2 Small angles:
n1 a1 n2a2
Slide17 : Thin Lenses spherical lens surfaces; incoming light parallel to axis; thickness << radii; same refractive index on both sides
Slide18 : Thin Lenses http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html
Slide19 : Thick Lens
Slide20 : The depth-of-field
Slide21 : The depth-of-field
Slide22 : The depth-of-field decreases with d, increases with Z0 strike a balance between incoming light and
sharp depth range
Slide23 : Deviations from the lens model 3 assumptions : 1. all rays from a point are focused onto 1 image point 2. all image points in a single plane 3. magnification is constant
deviations from this ideal are aberrations
Slide24 : Aberrations chromatic : refractive index function of
wavelength 2 types : 1. geometrical 2. chromatic geometrical : small for paraxial rays study through 3rd order optics
Slide25 : Geometrical aberrations spherical aberration
astigmatism
distortion
coma aberrations are reduced by combining lenses
Slide26 : Spherical aberration rays parallel to the axis do not converge
outer portions of the lens yield smaller
focal lenghts
Astigmatism : Astigmatism Different focal length for inclined rays
Distortion : Distortion magnification/focal length different
for different angles of inclination
Can be corrected! (if parameters are know) pincushion
(tele-photo) barrel
(wide-angle)
Coma : Coma point off the axis depicted as comet shaped blob
Slide30 : Chromatic aberration rays of different wavelengths focused
in different planes
cannot be removed completely
sometimes achromatization is achieved for
more than 2 wavelengths
Slide31 : Lens materials reference wavelengths :
F = 486.13nm
d = 587.56nm
C = 656.28nm
lens characteristics :
1. refractive index nd
2. Abbe number Vd= (nd - 1) / (nF - nC)
typically, both should be high
allows small components with sufficient refraction
notation : e.g. glass BK7(517642)
nd = 1.517 and Vd= 64.2
Slide33 : Vignetting Figure from http://www.vanwalree.com/optics/vignetting.html
Slide34 : Photographs
(Niepce,
“La Table Servie,” 1822) Milestones:
Daguerreotypes (1839)
Photographic Film (Eastman,1889)
Cinema (Lumière Brothers,1895)
Color Photography
(Lumière Brothers, 1908)
Television
(Baird, Farnsworth, Zworykin, 1920s) CCD Devices (1970)
more recently CMOS Collection Harlingue-Viollet.
Slide35 : Cameras we consider 2 types : 1. CCD 2. CMOS
Slide36 : CCD separate photo sensor at regular positions
no scanning
charge-coupled devices (CCDs)
area CCDs and linear CCDs
2 area architectures :
interline transfer and frame transfer photosensitive storage
The CCD camera : The CCD camera
CMOS : CMOS Same sensor elements as CCD
Each photo sensor has its own amplifier
More noise (reduced by subtracting ‘black’ image)
Lower sensitivity (lower fill rate)
Uses standard CMOS technology
Allows to put other components on chip
‘Smart’ pixels
CCD vs. CMOS : CCD vs. CMOS Mature technology
Specific technology
High production cost
High power consumption
Higher fill rate
Blooming
Sequential readout Recent technology
Standard IC technology
Cheap
Low power
Less sensitive
Per pixel amplification
Random pixel access
Smart pixels
On chip integration with other components
Color cameras : Color cameras We consider 3 concepts:
Prism (with 3 sensors)
Filter mosaic
Filter wheel
… and X3
Prism color camera : Prism color camera Separate light in 3 beams using dichroic prism
Requires 3 sensors & precise alignment
Good color separation
Prism color camera : Prism color camera
Filter mosaic : Filter mosaic Coat filter directly on sensor Demosaicing (obtain full colour & full resolution image)
Filter wheel : Filter wheel Rotate multiple filters in front of lens
Allows more than 3 colour bands Only suitable for static scenes
Prism vs. mosaic vs. wheel : Prism vs. mosaic vs. wheel Wheel
1
Good
Average
Low
Motion
3 or more approach
# sensors
Separation
Cost
Framerate
Artefacts
Bands
Prism
3
High
High
High
Low
3
High-end
cameras
Mosaic
1
Average
Low
High
Aliasing
3
Low-end
cameras
Scientific applications
new color CMOS sensor Foveon’s X3 : new color CMOS sensor Foveon’s X3 better image quality smarter pixels
Slide47 : The Human Eye Helmoltz’s
Schematic
Eye Reproduced by permission, the American Society of Photogrammetry and
Remote Sensing. A.L. Nowicki, “Stereoscopy.” Manual of Photogrammetry,
Thompson, Radlinski, and Speert (eds.), third edition, 1966.
Slide48 : The distribution of rods and cones across the retina Reprinted from Foundations of Vision, by B. Wandell, Sinauer
Associates, Inc., (1995). 1995 Sinauer Associates, Inc. Cones in the
fovea Rods and cones in
the periphery Reprinted from Foundations of Vision, by B. Wandell, Sinauer
Associates, Inc., (1995). 1995 Sinauer Associates, Inc.
Geometric camera model : Geometric camera model (Man Drawing a Lute, woodcut, 1525, Albrecht Dürer) perspective projection
Slide50 : Models for camera projection the pinhole model revisited : center of the lens = center of projection
notice the virtual image plane
this is called perspective projection
Slide51 : Perspective projection origin lies at the center of projection
the Zc axis coincides with the optical axis
Xc-axis to image rows, Yc-axis to columns Yc Zc Xc v u
Slide52 : Pseudo-orthographic projection If Z is constant x= kX and y = kY,
where k=f/Z i.e. orthographic projection + a scaling Good approximation if ƒ/Z ± constant, i.e. if objects
are small compared to their distance from the camera
Slide53 : Pictoral comparison Pseudo -
orthographic Perspective
Slide54 : Projection matrices the perspective projection model is incomplete :
what if : 1. 3D coordinates are specified in a
world coordinate frame 2. Image coordinates are expressed as
row and column numbers We will not consider additional refinements,
such as radial distortions,...
Slide55 :
Slide56 : (x0, y0) the pixel coordinates of the principal point
fx the number of pixels per unit length horizontally
fy the number of pixels per unit length vertically
s indicates the skew ; typically s = 0 NB7 : fully calibrated means internally and
externally calibrated Projection matrices Image coordinates are to be expressed as
pixel coordinates with : NB1: often only integer pixel coordinates matter NB2 : ky/kx is called the aspect ratio NB3 : kx,ky,s,x0 and y0 are called internal camera
parameters NB4 : when they are known, the camera is
internally calibrated NB5 : vector C and matrix R SO (3) are the
external camera parameters NB6 : when these are known, the camera is
externally calibrated
Slide57 : Projection matrices Exploiting homogeneous coordinates :
Slide58 : Projection matrices We define yielding for some non-zero
Next class Radiometry: lights and surfaces : Next class Radiometry: lights and surfaces