Scribd
Upload
37 views

Image Retrieval: Current Techniques, Promising Directions, and Open Issues

This document summarizes an presentation on image retrieval techniques. It begins with a brief history of image retrieval and the limitations of text-based approaches. It then introduces content-based image retrieval as an improved solution, extracting features like color, texture and shape from images. Current techniques for feature extraction and multidimensional indexing are described. Several current commercial image retrieval systems are outlined, along with some open issues still being researched. The presentation concludes that progress has been made but more work remains to close the gap between low-level features and high-level concepts.

Uploaded by

Rahul Bhardwaj
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
37 views

Image Retrieval: Current Techniques, Promising Directions, and Open Issues

This document summarizes an presentation on image retrieval techniques. It begins with a brief history of image retrieval and the limitations of text-based approaches. It then introduces content-based image retrieval as an improved solution, extracting features like color, texture and shape from images. Current techniques for feature extraction and multidimensional indexing are described. Several current commercial image retrieval systems are outlined, along with some open issues still being researched. The presentation concludes that progress has been made but more work remains to close the gap between low-level features and high-level concepts.

Uploaded by

Rahul Bhardwaj
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 41

Image Retrieval: Current

Techniques, Promising
Directions, and Open Issues
Yong Rui, Thomas Huang and Shih-Fu
Chang
Published in the Journal of Visual
Communication and Image
Representation.
Presented by: Deepak Bote

Presentation Outline

History of image retrieval Issues faced


Solution Content-based image retrieval
Feature extraction
Multidimensional indexing
Current Systems
Open issues
Conclusion

History of Image
Retrieval

Traditional text-based image search


engines

Manual annotation of images


Use text-based retrieval methods
Water lilies

E.g.
Flowers in a
pond
<Its biological
name>

Limitations of text-based
approach

Problem of image annotation

Problem of human perception

Large volumes of databases


Valid only for one language with image
retrieval this limitation should not exist
Subjectivity of human perception
Too much responsibility on the end-user

Problem of deeper (abstract) needs

Queries that cannot be described at all, but


tap into the visual features of images.

Outline

History of image retrieval Issues faced


Solution Content-based image retrieval
Feature extraction
Multidimensional indexing
Current Systems
Open issues
Conclusion

What is CBIR?

Images have rich content.


This content can be extracted as
various content features:

Mean color , Color Histogram etc

Take the responsibility of forming


the query away from the user.
Each image will now be described by
its own features.

CBIR A sample search


query

User wants to search for, say, many rose


images
He submits an existing rose picture as query.
He submits his own sketch of rose as query.

The system will extract image features for


this query.
It will compare these features with that of
other images in a database.
Relevant results will be displayed to the
user.

Sample Query

Sample CBIR
architecture

Outline

History of image retrieval Issues faced


Solution Content-based image retrieval
Feature extraction
Multidimensional indexing
Current Systems
Open issues
Conclusion

Feature Extraction

What are image features?


Primitive features
Mean color (RGB)
Color Histogram

Semantic features

General features

Color Layout, texture etc

Domain specific features

Face recognition, fingerprint matching


etc

Mean Color

Pixel Color Information: R, G, B


Mean component (R,G or B)=
Sum of that component for all pixels
Number of pixels

Pixel

Histogram

Frequency count of each individual


color
Most commonly used color feature
representation

Image

Corresponding
histogram

Color Layout

Need for Color Layout

How it works:

Global color features give too many false


positives
Divide whole image into sub-blocks
Extract features from each sub-block

Can we go one step further?


Divide into regions based on color feature
concentration
This process is called segmentation.

Example: Color layout

** Image adapted from Smith and Chang : Single Color Extraction

Texture

Texture innate property of all surfaces

Clouds, trees, bricks, hair etc

Refers to visual patterns of homogeneity


Does not result from presence of single color
Most accepted classification of textures
based on psychology studies Tamura
representation

Coarseness

Linelikeness

Contrast

Regularity

Directionality

Roughness

Segmentation issues

Considered as a difficult problem


Not reliable
Segments regions, but not objects
Different requirements from
segmentation:
Shape extraction: High Accuracy required
Layout features: Coarse segmentation
may be enough

Presentation Outline

History of image retrieval Issues faced


Solution Content-based image retrieval
Feature extraction
Multidimensional indexing
Current Systems
Open issues
Conclusion

Problem of high
dimensions
Mean Color = RGB = 3 dimensional
vector
Color Histogram = 256 dimensions
Effective storage and speedy retrieval
needed
Traditional data-structures not sufficient
R-trees, SR-Trees etc

2-dimensional space
Point A

D2

D1

3-dimensional space

Now, imagine

An N-dimensional
box!!
We want to conduct a
nearest neighbor
query.
R-trees are designed
for speedy retrieval
of results for such
purposes
Designed by
Guttmann in 1984

Presentation Outline

History of image retrieval Issues faced


Solution Content-based image retrieval
Feature extraction
Multidimensional indexing
Current Systems
Open issues
Conclusion

IBMs QBIC

QBIC Query by Image Content


First commercial CBIR system.
Model system influenced many others.
Uses color, texture, shape features
Text-based search can also be combined.
Uses R*-trees for indexing

QBIC Search by color

** Images courtesy : Yong Rao

QBIC Search by shape

** Images courtesy : Yong Rao

QBIC Query by sketch

** Images courtesy : Yong Rao

Virage

Developed by Virage inc.


Like QBIC, supports queries based
on color, layout, texture
Supports arbitrary combinations of
these features with weights attached
to each
This gives users more control over
the search process

VisualSEEk

Research prototype University of


Columbia
Mainly different because it considers
spatial relationships between objects.
Global features like mean color, color
histogram can give many false positives
Matching spatial relationships between
objects and visual features together
result in a powerful search.

ISearch my own system

ISearch my own system

ISearch my own system

Feature selection in
ISearch

Database Admin facility in


ISearch

Presentation Outline

History of image retrieval Issues faced


Solution Content-based image retrieval
Feature extraction
Multidimensional indexing
Current Systems
Open issues
Conclusion

Open issues

Gap between low level features and


high-level concepts
Human in the loop interactive
systems
Retrieval speed most research
prototypes can handle only a few
thousand images.
A reliable test-bed and measurement
criterion, please!

Presentation Outline

History of image retrieval Issues faced


Solution Content-based image retrieval
Feature extraction
Multidimensional indexing
Current Systems
Open issues
Conclusion

Conclusion

Satisfactory progress, but still

A long way to go!!

Acknowledgements

Dr. Padma Mundur


Mr. Yong Rao
Mr. Sumit Jain, Software Engineer,
KPIT Cummins, India
Mr. Ajay Joglekar, Software
Engineer, Veritas India.

References

Y. Rui, T. S. Huang, and S.-F. Chang,


Image retrieval: Current techniques,
promising directions, and open issues

S. Jain, A. Joglekar, and D. Bote,


ISearch: A Content-based Image
Retrieval (CBIR) Engine, as Bachelor
of Computer Engineering final year
thesis, Pune University, 2002

THANK YOU!!!
Questions?

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy