23UBCAN26 : Multimedia Systems

UNIT II:Images
Plan Approach - Organize Tools - Configure Computer Workspace -Making Still Images - Color - Image
File Formats. Sound: The Power of Sound -DigitalAudio-MidiAudio-Midivs.DigitalAudio-
MultimediaSystemSoundsAudio File Formats -Vaughan's Law of Multimedia Minimums - Adding Sound to
Multimedia Project.

BY

L.VARSHAA

AP/COMPUTER SCIENCE
 Multimediaon a computer screen is a composite ofelements: text, symbols, photograph-like bitmaps,
vector- drawn graphics, three-dimensional renderings, distinctive buttons to click, and windows of motion
videos.

Plan Approach:

 To get a handle on any multimedia project, you start with pencil, eraser, and paper.

 Outline your project and your graphic ideas first: make a flow-chart; storyboard the
project using stick figures; use three-by-five index cards and shuffle them until you get it
right.

Organize Tools:

 Most authoring systems provide the tools.

 We can create the graphic objects of multimedia (text, interactive buttons, vector-drawn
objects, and bitmaps) directly on our screen.

 If one of these tools is not inclu2d2ed, the authoring system usually offers a
mechanism for importing the object you need from another application.

Configure Computer Workspace:

 When developing multimedia, it is helpful to have more than one monitor to

provide lots of screen real estate (viewing area).
 In this way, we candisplay the full-screen working area of your project or
presentation and still have space to put your tools and other menus.
 During development there is a lot of cutting and pasting among windows and among
various applications, and with an extra monitor.

 we can open many windows at once and spread them out. Both Macintosh and
Windows operating systems support this extra hardware.


Making Still Images:

  Still images may be small or large, or even full screen
 Still images may be a single tree on a wintry hillside
 still images are generated by the computer in two ways:
(i) bitmaps (or paint graphics)
(ii) vector-drawn(or just plain ―drawn‖)
(i) Bitmaps:
o Bitmaps may also be called ―raster‖ images.
o Likewise, bitmap editors are sometimes called ―painting‖ programs.
o vector editors are sometimes called ―drawing‖ programs.
(ii) vector-drawn:
o Vector-drawn objects are used for lines, boxes, circles, polygons.
o Other graphic shapes that can be mathematically expressed in
angles, coordinates, and distances.
o A drawn object can be filled with color and patterns, and you can select it as
a single object.

File formats already use compression within the file itself—

23
for example, GIF, JPEG, and PNG.

Bit Depth Number of Available Binary Combinations for

• Colors Possible Describing a Color
1-bit 2 0, 1
2-bit 4 00, 01, 10, 11
4-bit 16 0000, 0001, 0011, 0111, 1111, 0010, 0100,
1000, 0110, 1100, 1010, 0101, 1110, 1101,
1001, 1011

Bitmap Sources:

Where do bitmaps come from? How are they made? You can do the following:

 Capture a bitmap from a photo or other artwork using a scanner to digitize

the image.

 Once made, a bitmap can be copied, altered, e-mailed, and otherwise used
in many creative ways.
  Grab a bitmap from an active computer screen with a screen capture program, and
then paste it into a paint program or your application.

Bitmap Software:

 Photoshop
 Adobe’s Illustrator
 CorelDRAW
 InDesign

Morphing:

 Morphing is another effect that can be used to manipulatestill images or to

create interesting and often bizarre animated transformations.

 Morphing allows you to smoothly blend two images so that one image seems
to melt into the next, often producing some amusing result.

24

Vector Drawing:

 Most multimedia authoring systems provide for use of vector-drawn objects such as
o lines,
o rectangles,
o ovals,
o polygons
complex drawings created from those objects, and text.
  Computer-aided design (CAD) programs have traditionally used vector-drawn
object systems for creating the highly complex and geometric renderings needed by
architects and engineers.

 Programs for 3-D animation also use vector-drawn graphics.

 The higher resolution of the printer, using a page description format such as
Portable Docu-ment Format (PDF).

How Vector Drawing Works:

A vector is a line that is described by the location of its two endpoints.

Vector drawing uses Cartesian coordinates where a pair of numbers describes a point in two-dimensional
space as the intersection of hori-zontal and vertical lines (the x and y axes).

In three-dimensional space, a third dimension depth is described by a z axis (x,y,z).

Simply, 25

<line x1="0" y1="0" x2="200" y2="100">

where x1 and y1 define the starting point (in the upper-left corner of the
viewing box) and x2 and y2 define the end point.

Add color like,

<rect x="0" y="0" width="200" height="100" fill="#FFFFFF" stroke="#FF0000"/>

3-D Drawing and Rendering:

 A great deal of information is needed to display a 3-D scene. Scenes consist of objects
that in turn contain many small elements such as blocks, cylinders, spheres, or cones.

 Objects and elements in 3-D space carry with them properties such as shape, color, texture,
shading, and location.
  A scene contains many different objects. Imagine a scene with a table, chairs, and
a background.

 To model an object that you want to place into your scene, you must start with a shape.

26
(A free-form objectcreated by extrusion and a wine flute created by lathing)

Extrude:
When weextrude a plane surface, its shape extends some distance, either perpendicular to
the shape’s outline or along a defined path.

Lathe:
When we lathe a shape, a profile of the shape is rotated around a defined axis (we can set
the direction) to create the 3-D object.
. Rendering:

Rendering is when the computer finally uses intricate algorithms to apply the effects you have specified on
the objects you have created.
Rendering an image requires great computing muscle and often takes many hours for a single image, and
you will feel the strength (or weakness) of your hardware.

(A scene rendered with four different methods of shading)

Shading: 27
Shading can usually be applied in several ways. As illustrated in above Figure, flat shading(b) is the fastest
for the computer to render and is most often used in preview mode.Gouraud shading (a)
, Phong shading (d), and ray tracing (c) take longer to render but provide photo-realistic images.

 Gouraud shading (a)

 flat shading (b)
 ray tracing (c)
 Phong shading (d)

Color:
Understanding Natural Light and Color:

 Color is a vital component of multimedia.

  Color is the frequency of a light wave within the narrow band of the
electromagnetic spectrum to which the human eye responds.

 The letters of the mnemonic ROY G. BIV, learned by many of us to remember the
colors of the rainbow, are the ascending frequencies of the visible light spectrum:
red, orange, yellow, green, blue, indigo, and violet.

Computerized Color:
 Additive Color
 Subtractive Color

Additive Color:
A color is created by combining colored light sources in three primary colors: red, green, and blue (RGB).
This is the process used for cathode ray tube (CRT), liquid crystal (LCD), and plasma displays.
Subtractive Color:
color is created by combining colored media such as paints or ink
Subtractive color is the process used to create color in printing.
28
The printed page is made up of tiny halftone dots of three primary colors: cyan, magenta, and yellow
(designated as CMY).

Red only (255,0,0) Red

Green only (0,255,0) Green
Blue only (0,0,255) Blue
Red and green (blue subtracted)
(255,255,0) Yellow
Red and blue (green subtracted)
(255,0,255) Magenta
Green and blue (red subtracted)
(0,255,255) Cyan
Red, green, and blue (255,255,255) White
None (0,0,0) Black
Computer Color Models:

 we specify a color by setting each amount of red, green, and blue to a value in
a range of 256 choices, from 0 to 255.

 Eight bits of memory are required to define those 256 possible choices, and that has
to be done for each of the three primary colors; a total of 24 bits of memory (8 + 8
+
 8 = 24) are therefore needed to describe the exact color, which is one of ―millions‖ (256

× 256 × 256 = 16,777,216).

 Two hexadecimal numbers, written in a scale of 16 numbers and letters in the range
―0123456789ABCDEF‖ represent the required 8 bits (16 × 16 = 256) needed
to specify the inten-sity of red, green, and blue.

 Thus, in HTML, you can specify pure green as #00FF00, where there is no red (first
pair is #00), there is maximum green (second pair is #FF), and there is no blue (last
pair is #00).

 The number sign (#) specifies the value as hexadecimal.

Color Palettes:
Red
255 (#FF) 255 (#FF) 255 (#FF) White (#FFFFFF)
255 (#FF) 255 (#FF) 0 (#00) Yellow (#FFFF00)
255 (#FF) 0 (#00) 255 (#FF) Magenta (#FF00FF)
0 (#00) 255 (#FF) 255 (#FF) Cyan (#00FFFF)
255 (#FF) 0 (#00) 0 (#00) Red (#FF0000)
0 (#00) 255 (#FF) 0 (#00) Green (#00FF00)
0 (#00) 0 (#00) 255 (#FF) Blue (#0000FF)
0 (#00) 0 (#00) 0 (#00) Black (#000000)

Palettes are mathematical tables that define the color of a pixel displayedon the screen.

The most common palettes are 1, 4, 8, 16, and 24 bits deep.

29
 Color Depth

1-bit Black and white (or any two colors)

4-bit 16 colors
8-bit 256 colors (good enough for color images)
16-bit Thousands of colors (65,536; excellent for color images)
24-bit More than 16 million colors (16,777,216; totally

GIF files using 256-color palettes are saved in a lossless format.

The PNG format also uses palettes 24-bits or 32 bits transparency and is lossless.

Dithering:

 Dithering is a process whereby the color value of each pixel is changed to the
closest matching color value in the target palette, using a mathematical
algorithm.

 Dithering software is usually3b0uilt into image-editing programs.

Image File Formats:

Most applications on any operating system can manage.

 JPEG
 GIF
 PNG
 TIFF
 JPEG, PNG, and GIF images are the most common bitmap formats used on the
Web and may be considered cross-platform, as all browsers will display them.

 TIFF Tagged Interchange File Format, was designed to be a universal

bitmapped image format and is also used extensively in desktop publishing
packages.
  IGS (or IGES, for Initial GraphicsExchange Standard) was developed by an
industry committee as abroader standard for transferring CAD drawings.

 PCX files were originally developed for use in Z -Soft MS-DOS paint packages.

Sound:

 Soundis perhaps the most luxurious element of multimedia.

 Itis meaningful ―speech‖ in any language, from a word to a squeal. It can provide
the listening pleasure of music, the amazing pronunciation of special effects.

The Power of Sound:

 Acoustics is the branch of physics that studies sound.

31
 Sound pres-sure levels (loudnes s or volume) are measured in decibels (dB).

 A logarithmic scale is also used for measuring the power of earth-quakes

1 10 100 1000 10000
 195 25–40 million Saturn rocket
170 100,000 Jet engine with afterburner

160 10,000 Turbojet engine at 7,000-pounds thrust

150 1,000 ALSETEX splinterless stun grenade
140 100 2 JBL2226 speakers pulling 2,400 watts inside

31

an automobile
130 10 75-piece orchestra, at fortissimo
120 1 Large chipping hammer
110 0.1 Riveting machine
100 0.01 Automobile on highway
90 0.001 Subway train; a shouting voice
80 0.0001 Inside a 1952 Corvette at 60 mph
70 0.00001 Voice conversation; freight train 100 feet away
60 0.000001 Large department store
50 0.0000001 Average residence or small business office
40 0.00000001 Residential areas of Chicago at night
30 0.000000001 Very soft whisper
20 0.0000000001 Sound studio
 (Typical Sound Levels in Decibels (dB) and Watts)

Digital Audio:

Digital audio is created when you represent the characteristics of a soundwave using numbers.

Digitized sound is sampled sound. Every nth fraction o f a second, a sample of sound is taken and stored
3 2
as digital information in bits andbytes.

The quality of this digital recording depends upon how often the samples are taken (sampling rate or
frequency, measured in kilohertz, or thousands of samples per second).

The three sampling rates most often used in multimedia are,

 44.1 kHz
 22.05 kHz
 11.025 kHz
Sample sizes are either 8 bits or 16 bits.

8-bit sample:

An 8-bit sample size provides 256 equal measurement units to describe the level and frequency of the sound
in that slice of time.

32
16- bit sample:

A 16-bit sample size, on the other hand, provides a staggering 65,536 equal units to describe the sound in
that same slice of time.

33

(Figure It is impossible to reconstruct the original

waveform if the sampling frequency is too low.)

The value of each sample is rounded off to the nearest integer (quantization), and if the
amplitude is greater than the intervals available, clipping of the top and bottom of the wave occurs.

Quantization:

Quantization can produce an unwanted background mocking noise, and clipping may severely
distort the sound.

33
 (Figure Examples of quantizing and clipping)

34
Making Digital Audio Files:

 Setting Proper Recording Levels

 Editing Digital Recordings

Setting Proper Recording Levels:

 Any good piece of digital audio recording and editing software will display
digital meters to let you know how loud your sound is.

 Watch the meters closely during recording analog meters that usually have a
0 setting somewhere in the middle and extend up into ranges like +5, +8, or even higher, digital meters
peak out.
 To avoid distortion, do not cross over this limit. If this happens, lower your
volume and try again. Try to keep peak levels between 3 and –10.

34
Editing Digital Recordings:

 Audacity is free open-source sound editing application for Windows, Macintosh,

and Linux (http://audacity.sourceforge.net).

 The basic sound editing operations

 Trimming
 Splicing and Assembly
 Volume Adjustments
 Format Conversion
 Resampling or Down sampling
 Fade-ins and Fade-outs
 Equalization
 Time Stretching
 Digital Signal Processing (DSP)
 Reversing Sounds
 Multiple Tracks
35

Trimming:

 Removing ―dead air‖ or blank space from the recording and any unnecessary extra
time off .

 Trimming is typically accomplished by dragging the mouse cursor over a

graphic representation of our recording and choosing a menu command such
as Cut, Clear, Erase, or Silence.

Splicing and Assembly:

 we will probably want to remove the extraneous noises that inevitably creep into
a recording.

35
  we may need to assemble longer recordings by cutting and pasting together
many shorter ones.

Volume Adjustments:

 It is best to use a sound editor to normalize the assembled audio file to a

particular level, say 80 percent to 90 percent of maximum (without clip-ping), or
about –16 dB.

Format Conversion:

 we have a Digital Rights Management (DRM) protected MP4 file downloaded from
the iTunes store and burn that file to an Audio CD track, the DRM data will be lost
because the Audio CD format does not provide for DRM data.

 The now-unprotected tune on the CD can then be ripped into a playable MP3 format.

Resampling or Down sampling:

36
 If you have recorded and edited yoursounds at 16-bit sampling rates but are using
lower rates and resolutions in your project, you must resample or downsample the file.

Fade-ins and Fade-outs:

 Fade in or fade out helps to smooth out the very beginning and the very end of
a sound file.

Equalization:

 It offer digital equalization (EQ) capabilities that allow we to modify a

recording’s frequency content.
Time Stretching:

 Let we alter the length (in time)of a sound file without changing its pitch. This feature
can be very useful.

36
Digital Signal Processing (DSP):

 It allow you to processthe signal with echo, multitap delay, chorus, flange, and
other spe-cial effects using digital signal processing (DSP) routines.

Reversing Sounds:

 Another simple manipulation is to reverse all or aportion of a digital audio recording.

 Sounds, particularly spoken dialog, can produce a strangeeffect when
played backward.

Multiple Tracks:

 Being able to edit and combine multiple tracks forsound effects, voice, music, etc.

MIDI Audio:
37
 MIDI (Musical Instrument Digital Interface) is a communications standard developed in
the early 1980s for electronic musical instruments andcomputers.

 MIDI provides a protocol for passing detailed descriptions of a musical score, such as
the notes, the sequences of notes, and the instrument that will play these notes.

 But MIDI data is not digitized sound; it is a shorthand representation of music stored
in numeric form.

 A MIDI keyboard is also useful for simplifying the creation of musical scores.

0 Acoustic grand piano 16 Hammond organ

1 Bright acoustic piano 17 Percussive organ
2 Electric grand piano 18 Rock organ
3 Honky-tonk piano 19 Church organ
4 Rhodes piano 20 Reed organ
5 Chorused piano 21 Accordion
6 Harpsichord 22 Harmonica

37
 7 Clarinet 23 Tango accordion
8 Celesta 24 Acoustic guitar (nylon)
9 Glockenspiel 25 Acoustic guitar (steel)
10 Music box 26 Electric guitar (jazz)
11 Vibraphone 27 Electric guitar (clean)
12 Marimba 28 Electric guitar (muted)
13 Xylophone 29 Overdriven guitar
14 Tubular bells 30 Distortion guitar
15 Dulcimer 31 Guitar harmonics

(General MIDI Instrument Sounds)

MIDI vs. Digital Audio:

 In contrast to MIDI data, digital audio data is the actual representation of a

sound, stored in the form of thousands of individual numbers (samples).

 The digital data represents the instantaneous amplitude (or loudness) of a sound
at discrete slices of time.

 MIDI data is to digital audio da3ta8 what vector or drawn graphics are to
bitmapped graphics. That is, MIDI data is device dependent; digital data is not.

Advantages of MIDI:

i. MIDI files are much more compact than digital audio files, and the size of a MIDI file
is completely independent of playback quality.

ii. MIDI files will be 200 to 1,000 times smaller than CD- quality digital audio files.
Because MIDI files are small, they don’t take up as much memory, disk space, or bandwidth.

Advantages of Digital Audio:

1. consistent play-back quality

2. more frequently
3. sound as good

38
Multimedia System Sounds:

 In Windows, system sounds are WAV files, and they be located in the
Windows\Media subdirectory.

 System event sounds include

 start.wav
 chimes.wav
 chord.wav
 ding.wav
 logoff.wav
 notify.wav
 recycle.wav
 tada. wav
and the Microsoft sound.wav that typically plays when Windows starts up.
39

 In OS X on a Macintosh, you can only change your system alert sound.

 Put your custom sound file (in AIF format) into ~/System/Library/Sounds,
then select it in the Sound preference pane.

39
Audio File Formats:

 There are many ways to store the bits and bytes that describe a sampled waveform
sound.

 The method used for consumer-grade music CDs is •Linear Pulse Code
Modulation (LPCM), often shortened to PCM. An audio CD provides up to 80
minutes of playing time.

 LPCM tracks from an audio CD are usually converted and stored on a

computer in uncompressed AIFF (Audio Interchange File Format) or wave format
(WAV) files when copied from the CD.

 The MP3 format was developed by the Moving Picture Experts Group
(MPEG) for storing consumer audio.

 It incorporates a ―lossy‖ compression algorithm to save space.

 MP4 is a format based on Apple’s QuickTime movie (.mov) ―con-

tainer‖ model and is similar to the MOV format, which stores various types of media,
particularly time-based streams such as audio40and video.

 The m4a extension is used when the file contains only audio data.

 M4p files contain only audio, but are encrypted for Digital Rights
Management (DRM).

 M4r files are used for ringtones on Apple’s iPhone.

 The AAC (Advanced Audio Coding) format, which is part of the MP4
model, was adopted by Apple’s iTunes store, and many music files are commercially
available in this format.

 Flash video files (FLV) contain both a video stream and an audio stream,
and the FLV format has been adopted by YouTube, Google, Yahoo, Reuters.com,
BBC.com, CNN.com, and other news pro-viders for Internet delivery of content.
Vaughan’s Law of Multimedia Minimums:

40
 There is an acceptable minimum level of adequacy that will
satisfy the audience, even when that level may not be the best
that technology, money, or time and effort can buy.

Adding Sound to Your Multimedia Project:

 Edit the sounds to fit your project.

 Test the sounds to be sure they are timed properly with the project’s images.

Space Considerations:

The following formula will help you estimate your storage needs.

41
If you are using two channels fo r stereo, double the result.
(sampling rate * bits per sample) / 8 = bytes per second

If you prefer to solve for kilobytes (KB), not bytes, then try:

sample rate * sample size / 8 * # seconds * 2 (if stereo) = file size in KB

For example, 60 seconds of stereo in Red Book

Audio: 44.1 * 16 / 8 * 60 * 2 = 10,584KB ≅ 10.59MB

Audio Recording:

Digital audio tape (DAT) systems provide a tape-based 44.1 kHz,16-bit record and playback capability.
  USB and flash memory recorders range in quality, some suitable for voice only,
some generating compressed MP3 files, and some recording in CD-quality stereo.

 Recordings can be directly downloaded as digital files using a USB cable or flash
memory card reader.

Audio CDs:
The method for digitally encoding the high-quality stereo of the consumer CD music market is an
international standard, called ISO 10149. This isalso known as the Red Book Audio standard

The digital audio sample size and sampling rate of Red Book Audio (16 bits at 44.1 kHz) allow accurate
reproduction of all the sounds that humans can hear.

Unlike DVDs, audio CDs do not contain information about artists, titles, or tracklists of songs.
But player software such as Apple iTunes and AOL Winamp will automatically link to a database on the
Internet when you insert a music CD.

Sound for Your Mobile:

42
When the mobile receives a notice that someone is calling, the unit’s software takes over and, depending on
the programmed options, plays the user’s choice of ringtone—either generated by internal MIDI software or
played from a stored sound file.

MP3 files will play on most mobiles.

Sound for the Internet:

The simplest way to embed a sound file in a web page is to call it from an inline HTML anchor:
<a href="mysound.wav"> Click here to play MySound! </a>

Streaming files are dependent upon connection speed.

Media players are designed to play files as soon as enough of the data is cached in your computer’s
buffer. Testing and Evaluation

 Putting everything together can be tough, but testing and evaluating what you’ve
done can be even tougher—especially if your project involves a complicated live presentation, or
if you’re shipping a commercial multi-media application.

 In the world of professional film and video production, sound is incorporated during
post-production, or a post-session, after all the film and video footage has been assembled.

Copyright Issues:

 Ownership rights are significant issues for multimedia.

 A number of software vendors have entered the multimedia mar-ketplace by

selling digitized clip sounds with an unlimited-use, royalty-free license.