INDIAN INSTITUTE OF TECHNOLOGY ROORKEE

LINEAR ALGEBRA FOR IMAGE PROCESSING

DA 201 – Applied Linear Algebra

Rohit Singh Nitwal

22125029
HOW DO THESE VISUALS SURROUND
OUR SCREENS ?

2
 .
Linear algebra is a power-full tool. It comprises of
algorithms/methods which helps a lot in the real world specially in
the study and manipulation of images. Images are one of the highly
used medium of communication in today’s digital and social world.

We all share thousands of images, memes etc. to our known ones. But when it comes
to sharing large image files we need to compress it such that the size of image got
reduced significantly keeping the image quality and content as good as possible.
Images are represented as 3 dimensional(2 for height and width and 1 for channel)
array/matrix of pixels, and we all know whenever matrix is coined linear algebra
appears automatically.

3
How computer perceive images

The most common and the simplest way to represent the image is in the form
of a matrix. Basically an image is a matrix of pixels.

Representing a part of the image as a matrix (Image credit: IIT, Madras, NPTEL Deep Learning for Computer Vision)

4
5
 What is image processing ?
Image processing is the process of transforming an image into a
digital form and performing certain operations to transform,
enhance, and extract some useful information from it.
It involves the use of algorithms, software, and
hardware to process digital images typically obtained from
sources like cameras, satellites, medical devices, or digital
sensors.
At the heart lies the Linear algebra that provides the essential
mathematical foundation for a wide range of image processing
techniques.
6
Digital image processing

• It is an essential preprocessing step in many applications, such as face recognition, object

detection, and image compression.
• Image processing is done to enhance an existing image or to take out important
information from it. This is important in several Deep Learning-based Computer Vision
applications, where such preprocessing can dramatically boost the performance of a
model. Manipulating images, for example, adding or removing objects to images, is
another application, especially in the entertainment industry.

7
• Linear algebra provides a powerful framework for representing and solving
mathematical problems in image processing. It's a versatile tool that can handle a
wide variety of tasks, from basic operations to more complex operations, with a high
degree of efficiency and elegance.

• Many fundamental geometric operations can be elegantly and efficiently performed

using linear algebra. These operations include rotation, translation, scaling,
reflections, dot and cross products, shearing, vector field transformations ,PCA,low
rank approximation etc.

• More complex operations, require a combination of mathematical and algorithmic

approaches, including linear algebra, calculus, statistics, and machine learning

8
 Convolution and Filtering:
Linear filters, which are widely used in
image processing for tasks like blurring,
sharpening, and edge detection, can be
expressed as convolution operations,
which are inherently linear
Principal Component Analysis
Low-Rank Approximations:
(PCA): PCA is a linear algebra Methods like singular value
technique used for dimensionality
reduction and feature extraction in decomposition (SVD) are used for
image analysis. It identifies the most approximating images with
significant directions of variation in an Geometric Transformations: reduced dimensions while
image dataset. Transformations like affine preserving essential features.
transformations (including
rotation, scaling, and
translation), which are
fundamental in image
Image Registration: Aligning registration and correction, are
and matching two images, linear operations .
Image Restoration: Techniques
which is vital in medical for removing noise and restoring
images often involve solving linear
imaging, remote sensing, and systems of equations, which can be
computer vision, often relies described using linear algebra.
on linear transformations.
Wavelet Transform: The
discrete wavelet transform (DWT)
is a linear operation used for image
compression and multi-resolution
analysis.

9
GEOMETRIC TRANSFORMATIONS
• LINEAR TRANSFORMATION

A Linear transformation meets the following three conditions

• T(v1 + v2) = T(v1) + T(v2)
• T(c v1) = c T(v1)
• T(0) = 0

• Sequential application of linear transformation is again a

linear transformation.
• Order of transformation is important as in case of matrix
multiplication.

10
In practice A linear transformation is used as a matrix –vector product.

11
• Zero vector is always mapped to zero vector on applying linear transformation
• Parallel lines remain parallel.
• Linear transformations preserve lines and ratios of lengths over each line.
• Linear transformations may not preserve angles and length.
• It is quite useful in performing following operations:
• Image rotations
• Image shear
• Image scaling
• Image flip

12
• AFFINE TRANSFORMATIONS
An affine transformation consists of two parts: a linear transformation and a
translation.

13
• Affine transformations may even change the origin.
• Affine transformations preserve lines and ratios of lengths over each line.
• Affine transformations may not preserve angles and length.
• Parallel lines remain parallel.

14
When we move from cartesian to homogenous
coordinates 2D image is moved to z=1 plane in
3D .
Embedding 2D image in 3D is useful to perform
linear transformation

Going back to cartesian

coordinates the x and y
homogenous coordinates after
transformation is divided by z
coordinate.
Projective transformations have 8
degrees of freedom
15
• https://yizhe-ang.github.io/matrix-explorable/

16
SVD

Matrix approximation with Core APIs | TensorFlow Core

17
Image feature extraction using SVD

Approximating image with less memory

18
Rank analysis

19
 Kernels CONVOLUTION

Convolution is a fundamental operation in image

processing that involves applying a filter or kernel
to an image.

Each pixel in the output image is a function of

the nearby pixels (including itself) in the input
image, the kernel is that function.

A kernel or convolutional matrix as a tiny matrix that is

used for blurring, sharpening, edge detection, and other
image processing functions.

https://en.wikipedia.org/wiki/Kernel_(image_processing)

20
 The emboss kernel givens
An outline kernel is used to highlight the illusion of depth by
The blur kernel large differences in pixel values. A emphasizing the differences
de-emphasizes pixel next to neighbor pixels with of pixels in a given
differences in close to the same intensity will direction. In this case, in a
adjacent pixel appear black in the new image while direction along a line from
values one next to neighbor pixels that the top left to the bottom
differ strongly will appear white. right.
21
Original Emboss Outline Blur
22
Convolution plays a
key role in
enhancing,
analyzing, and
transforming images
for various
applications,
including computer
vision, image
enhancement, and
deep learning.

23
PCA

PCA is mathematically defined as an

orthogonal linear transformation that
transforms the data to a new coordinate
system such that the greatest variance by
some projection of the data comes to lie on
the first coordinate (called the first principal
component), the second greatest variance
on the second coordinate, and so on. In
other words, we convert a set of
observations of possibly correlated variables
into a set of values of linearly uncorrelated
variables called principal components .

24
Step 1: Calculate Mean of each features

Step 2: Calculation of the covariance

matrix.

Step 3: Eigenvalues of the covariance

matrix

Step 4: Computation of the eigenvectors

Step 5: Computation of first principal

components

25
Image Compression Using PCA

PCA is mathematically defined as an

orthogonal linear transformation that
transforms the data to a new coordinate
system such that the greatest variance by
some projection of the data comes to lie
on the first coordinate (called the first
principal component), the second
greatest variance on the second
coordinate, and so on. In other words, we
convert a set of observations of possibly
correlated variables into a set of values of
linearly uncorrelated variables called
principal components

26
 • Image Inpainting:
• 3D Reconstruction:
Reconstructing a Filling in missing or
three-dimensional corrupted parts of
scene from a set of 2D an image with
images. It's applied in plausible content.
fields like computer Object Recognition and It's used for image
Image Compression:
vision, cultural Tracking: Detecting and
More advanced • restoration and
tracking objects within
heritage compression algorithms
images or video streams • editing.
preservation, like JPEG 2000 and HEVC
is a complex operation.
and virtual (H.265) involve complex
It's fundamental in
mathematical operations
reality. surveillance, robotics,
to reduce data size while
and autonomous
preserving image quality
vehicles

Image Stitching and Deep Learning-Based

Panorama Creation: Operations: Various
Merging multiple deep learning
images into a single techniques, including
panoramic image. CNNs and generative
adversarial networks
Used in creating (GANs), are employed
virtual tours, Google for tasks like image • Image Denoising
• Super-Resolution: Street View, and generation, style and Deblurring:
Increasing the landscape transfer, and image-to- Advanced methods
resolution of an image to photography. image translation.
incorporate statistical
reveal finer details. models and machine
Applications include medical learning to reduce
imaging, satellite imagery, noise and restore
and enhancing photographs. sharpness in images
27
CNN
• Deep learning has revolutionized the world of computer vision—In particular,
Convolutional Neural Networks (CNNs) were designed to process image data more
efficiently.
• CNN is a powerful algorithm for image processing. These algorithms are currently
the best algorithms we have for the automated processing of images.
• Since images contain a consistent pattern spanning several pixels, processing them one
pixel at a time—as MLPs do—is inefficient. This is why CNNs that process images in
patches or windows are now the de-facto choice for image processing tasks.

mage Processing: Techniques, Types, & Applications [2023] (v7labs.com)

28
LNCS 8692 - Learning a Deep Convolutional Network for Image Super-Resolution (springer.com)
29
The authors achieved a 3% boost in performance with this simple preprocessing procedure which
is a considerable enhancement, especially in a biomedical application where the accuracy of
diagnosis is crucial for AI systems.

30
• https://medium.com/analytics-vidhya/compressing-images-using-linear-algebra-
bdac64c5e7ef
• Kernels / Convolution / Image Filtering | Computer Vision (wordpress.com)

31