METU EE 584 Lecture Notes by A.

Aydin ALATAN © 2012

EE 584
MACHINE VISION
Introduction
Relation with other areas
Image Formation & Sensing
Projections
Brightness
Lenses
Image Sensing

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Introduction
Vision is the most powerful sense
Vision is the most complicated sense

The purpose of a general

machine/computer/robot vision
system is to produce a symbolic
description of what is being imaged.
2

1
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Machine Vision System

A typical machine vision system :
Imaging MACHINE
Device VISION

Scene Image Description

Illumination Application
Feedback

Machine vision should be based on complete understanding of image formation

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Relation to other fields :

4 important related fields :
Imaging (science on creating all kinds of images)
Image processing
Pattern recognition
Scene analysis

Image Pattern Scene

Processing Recognition Analysis
Input Output Feature Class Input Output
Vector ID
Image Image Description Description

None of them provides a solution to the problem

“developing symbolic descriptions from images”.

2
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Relation to other fields :

Machine vision vs Computer vision:
Two terms can be used interchangeably
Machine vision more constraints on the
environment and focus on (industrial) applications
Computer vision more generic in terms of
content and applications

This course is about

fundamentals of vision
research

Image from http://en.wikipedia.org/wiki/Computer_vision 5

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Vision and Graphics

Images Vision Model

Graphics

Inverse problems: analysis and

synthesis.

slide from Computer Vision Lecture Notes Trevor Darrell 6

3
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Image Formation
Projection of 3-D world onto 2-D
image plane

Two crucial questions :

What determines the position of a 3-D
object point on the 2-D image plane?
What determines the brightness of a 3-D
object point on the 2-D image plane?

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Image Formation

slide from Computer Vision Lecture Notes Trevor Darrell 8

4
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Camera Models
Pinhole camera model

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Camera Models
Pinhole camera model

X Y
x= f ,y= f
Z Z

10

5
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Camera Models
In pin-hole camera, distant objects are observed smaller

C’
B’

11

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Camera Models
In pin-hole camera, parallel lines meet

12

6
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Perspective Projection
X Y
x= f ,y= f
Z Z
Pin-hole camera model is called perspective
projection

It is also possible to make approximations

to perspective projection
Affine : Scene points are planar
Weak-perspective : Scene is approximated by a
plane and assumed to be far away from camera
Orthographic : Scene is approximated to be
planar and far away from camera and camera
distance does not change
13

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Affine Projection
If all scene points are on a plane

X Y f
x=−f , y=−f ⇒m≡ ⇒ x = −m X , y = −m Y
Z0 Z0 Z0

14

7
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Weak-perspective Projection
Now, assume all scene points are on a plane

Z0

X Y
x≅−f , y≅−f ⇒ x ≅ −m X , y ≅ −m Y
Z0 Z0
This assumption can only be justified, if scene depth range is small
compared to average distance from camera
∆z is small wrt Z0

15

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Orthographic Projection
If scene depth range is small wrt average depth and camera
distance remain at a constant distance (i.e. Z0 is contant)
Choose m=-1 x=X , y=Y

16

8
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Projections : Summary
Y

Perspective : (x,y,z)

x y
x′ = f , y ′ = f f 0
z z (x’,y’)

X
y
Orthographic : (x’,y’) (x,y,z)

x′ = x , y ′ = y

Perspective projection is a more realistic projection for (pin-hole) camera recordings

If depth range is small compared to average distance from the camera, orthographic is
also a good approximation
17

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Brightness :
Two different brightness concepts :
•Image brightness : irradiance
Light power per unit area falling on a (image) surface

•Scene brightness : radiance

Light power per unit area emitted into a solid angle
from a (object) surface

Image and scene brightness are proportional to each other

Pinhole camera needs non-zero diameter for enough light

18

9
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Lenses :
A pin-hole camera
needs light

19

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Lenses : Paraxial (or first-order) optics

α 1 = β1 + γ ≈
h h
+
d1 R

h h
α2 = γ − β2 ≈ −
R d2

h h h h
n1  +  = n2  − 
Snell’s law: Small angles:  d1 R   R d2 

n1 n2 n2 − n1
n1 sin α1 = n2 sin α2 n1 α1 ≈ n2α2 + =
d1 d 2 R
Note that the relation is independent of β 1 and β 2 all rays pass from P1, also pass P2.
slide from Computer Vision Lecture Notes by Marc Pollefeys 20

10
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Lenses : Thin Lenses

spherical lens surfaces; incoming light ± parallel to axis;
n1 n2 n2 − n1
+ = thickness << radii; same refractive index on both sides
d1 d 2 R

1 n n −1
+ =
Z Z* R Z* Z

n 1 1− n
+ =
Z* Z' R Z′ Z*

n n −1 1 n 1− n 1
= − = −
Z* R Z Z* R Z'

1 1 1 R
n −1 1− n 1 1
− = − − = and f =
R R Z Z'
z' z f 2(n − 1)
slide from Computer Vision Lecture Notes by Marc Pollefeys

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Lenses :
An ideal thin lens produces the same projection with a
pinhole camera, plus some finite amount of light.

1 1 1
+ =
z′ − z f

Once you focus for one

distance z, points on other
distances will be blurred.
z’ -z

22

11
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Lenses : Deviations from the lens model

3 assumptions :
1. all rays from a point are focused onto 1 image point
2. all image points in a single plane
3. magnification is constant

deviations from this ideal are aberrations

2 types of aberrations:
geometrical : small for paraxial rays
study through 3rd order optics

chromatic : refractive index function of

wavelength

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Lenses : Vignetting : Brightness drop in

image periphery for compound
lenses

Figure from http://www.vanwalree.com/optics/vignetting.html

12
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Camera Field of View

Angular measure of the portion of 3D space seen by
the camera

α = 2 arctan(d / 2F)

Image from http://en.wikipedia.org/wiki/Angle_of_view 25

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Image Sensing :
Light photons striking a suitable (vacuum or semi-
conductor device) surface generate electron-hole pairs
which are measured to determine the irradiance.

Quantum efficiency : ratio of electron flux to incident

photon flux & depends on energy (wavelength) of photon
Solid-state devices almost ideal for some wavelengths
Photographic films have poor quantum efficiency
26

13
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

Quantization of Image

Electrons should be measured/averaged at some

predefined regions on the image sensor -> Spatial
quantization
These regions can be square, rectangular or hexagonal
Each predefined region represents a pixel (picture
element) location and the quantized values are pixel
values (usually 0 to 255)

27

METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

The Human Eye

Reproduced by permission, the American Society of Photogrammetry and

Remote Sensing. A.L. Nowicki, “Stereoscopy.” Manual of Photogrammetry,
Thompson, Radlinski, and Speert (eds.), third edition, 1966.

Helmoltz’s
Schematic
Eye

28

14
 METU EE 584 Lecture Notes by A.Aydin ALATAN © 2012

The distribution
of rods and
cones across the
retina

Reprinted from Foundations of Vision, by B. Wandell, Sinauer

Associates, Inc., (1995).  1995 Sinauer Associates, Inc.

Cones in the Rods and cones in

3 types of cones that yield color perception
fovea the periphery

Reprinted from Foundations of Vision, by B. Wandell, Sinauer

Associates, Inc., (1995).  1995 Sinauer Associates, Inc.
29

15