Unit-04 Sorting

Ms. Atufaali Saiyed

Department of Information Technology,
Devang Patel Institute of Advance Technology & Research
How important is Sorting?

 Have u seen a Telephone bill or Mobile postpaid bill? Did u notice any kind
of sorting?
 Have u seen ur Bank statement/credit card bill for 1 month? How is it
organized?
 Have u registered for an email account? Did u notice any kind of sorting?
 How will u find out the largest file on ur Desktop? How will u find out the
most recently created file or the oldest file on ur Desktop? Do u see any kind of
sorting?
 Let’s say u r searching for the occurrence of a particular word in a big text
book. What will u do?
Example of Software
 MS Office
 Window
 Web Application like-
 Yahoo
 Gmail

 Identify the elements of sorting in these.

 Sorting
 To arrange a set of items in sequence.
 It is estimated that 25~50% of all computing power is used for sorting
activities.
 Possible reasons:
 Many applications require sorting;
 Many applications perform sorting when they don't have to;
 Many applications use inefficient sorting algorithms.
 Sorting Applications
 To prepare a list of student ID, names, and scores in a table (sorted by ID or name) for
easy checking.
 To prepare a list of scores before letter grade assignment.
 To produce a list of horses after a race (sorted by the finishing times) for payoff
calculation.
 To prepare an originally unsorted array for ordered binary searching.

 Other Practical application

 People by last name
 Countries by population
 Search engine results by relevance
Some Definitions
1. Internal Sort
 The data to be sorted is all stored in the computer’s main memory.
2. External Sort
 Some of the data to be sorted might be stored in some external, slower, device.
3. In Place Sort
 The amount of extra space required to sort the data is constant with the input size.
 Stability
 A STABLE sort preserves relative order of records with equal keys

Sorted on first key:

Sort file on second key:

Records with key value 3 are not in

order on first key!!
Sorting Algorithms
1. Selection Sort
2. Bubble Sort
3. Insertion Sort
4. Quick Sort
5. Merge Sort
6. Counting Sort
7. Radix Sort
8. Shell Sort
9. Tree Sort
10. Heap Sort
11. Topological Sort
 Sorting Classification
External
In memory sorting
sorting
Comparison sorting
Specialized Sorting
(N log N)
# of tape
O(N2) O(N log N) O(N) accesses
• Bubble Sort • Merge Sort • Bucket Sort • Simple
• Selection Sort • Quick Sort • Radix Sort External Merge
• Insertion Sort • Heap Sort Sort
• Shell Sort • Variations

9
 Comparison Sorting

Determine order through comparisons on the

input data
Selection Sort
 Selection sort is a simple sorting algorithm.

 The list is divided into two sublists,

 Sorted list at the left end and
 Unsorted list at the right end.
 Initially, sorted part is empty and the unsorted part is entire list.

 We find the smallest element from the unsorted sublist and swap it with the element at the
beginning of the unsorted data.

 Each time we move one element from the unsorted sublist to the sorted sublist, we say that we have
completed a sort pass.

 A list of n elements requires n-1 passes to completely rearrange the data.

 This algorithm is not suitable for large data sets  O(N2)

Selection Sort Example
Selection Sort Example
Selection Sort Example
Selection Sort Example
Selection Sort
 Idea:
 Find the smallest element in the array
 Exchange it with the element in the first position
 Find the second smallest element and exchange it
with the element in the second position
 Continue until the array is sorted

16
Selection Sort Algorithm
Selection Sort Algorithm
Alg.: SELECTION-SORT(A)
n ← length[A]
for j ← 1 to n - 1
do smallest ← j
for i ← j + 1 to n
do if A[i] < A[smallest]
then smallest ← i
exchange A[j] ↔ A[smallest]
Example-
Consider the following array of int values.
[23, 78, 45, 8, 32, 56]
Sort the array using selection sort
Sorted Unsorted

23 78 45 8 32 56 Original List

8 78 45 23 32 56 After pass 1

8 23 45 78 32 56 After pass 2

After pass 3
8 23 32 78 45 56

8 23 32 45 78 56 After pass 4

After pass 5
8 23 32 45 56 78
Analysis of Selection Sort
cost times
Alg.: SELECTION-SORT(A)
c1 1
n ← length[A]
c2 n
for j ← 1 to n - 1
do smallest ← j c3 n-1
n2/2
for i ← j + 1 to n c4 nj11 (n  j  1)
comparisons
do if A[i] < A[smallest] c5 
n 1
j 1
(n  j )
n
exchanges then smallest ← i c6 
n 1
j 1
(n  j )

exchange A[j] ↔ A[smallest] c7 n-1

21
 n 1 n 1 n 1
T (n)  c1  c2 n  c3 (n  1)  c4  (n  j  1)  c5   n  j   c6   n  j   c7 (n  1)  (n 2 )
j 1 j 1 j 2
 Selection sort runtime
 Running time for input size n:

n 1 n 1 n 1


j 1 i  j 1
j   (n  1  ( j  1)  1)
j 1
n 1
  (n 
j 1
j  1)

n 1 n 1 n 1
 n1   j  1
j 1 j 1 j 1

( n  1) n
n  2
n
2
 ( n 2 )
23
What will be the space complexity of Selection Sort
Algorithm?

Needs O(1) extra space

One for exchanging elements
Is selection sort a stable algorithm?

A sorting algorithm is said to be a stable algorithm if the

relative ordering of two equal elements in the input array is the
same in the sorted array.
How can we improve performance of selection sort
algorithm?
Bubble Sort
 Unlike selection sort, instead of finding the smallest element and performing the
swap, two elements are interchanged immediately upon discovering that are out of
order
 more specifically:
 scan the list, exchanging adjacent elements if they are not in relative order; this bubbles the
highest value to the top
 scan the list again, bubbling up the second highest value
 repeat until all elements have been placed in their proper order
 After the first pass, the largest element will be in the nth position
 On each successive pass, the elements with the next largest value will be placed in
position n-1, n-2, …..2 respectively
 This approached required at most n-1 passes, the complexity of bubble sort is O(n2)
Bubble Sort Example
Bubble Sort
Example
Bubble Sort
Alg.: BUBBLESORT(A)
for i  1 to length[A]
do for j  length[A] downto i + 1
do if A[j]
i
< A[j -1]
then exchange A[j]  A[j-1]

8 4 6 9 2 3 1
i=1 j
Bubble-Sort Running Time
Alg.: BUBBLESORT(A)
for i  1 to length[A] c1
do for j  length[A] downto i + 1 c2
Comparisons:  n2/2 do if A[j] < A[j -1] c3
Exchanges:  n2/2 then exchange A[j]  A[j-1]
c4
 Bubble sort runtime
 Running time (# comparisons) for input size n:
n 1 n 1 i n 1

 i 0 j 1
j   (n  1  i )
i 0
n 1 n 1 n 1
 n1  1   i
i 0 i 0 i 0

( n  1) n
 n n2

2
 ( n 2 )
 number of actual swaps performed depends on the data; out-of-order data performs many swaps

32
Repeat all the questions which we have discussed
for selection sort algorithm?
Insertion Sort
 In Insertion sort, after each iteration a element moves from unsorted portion to
sorted portion until all the elements are sorted in the list.

Steps for Insertion sort

1 Assume that first element in the list is in sorted portion of the list
and remaining all elements are in unsorted portion

2 Select first element from the unsorted list and insert that element
into the sorted list in order

3 Repeat the above steps until all the elements from the unsorted list
are moves to sorted list
Insertion Sort
Complexity of the Insertion Sort Algorithm

To sort a unsorted list with ‘n’ number of elements we need to

make
(1 + 2 + 3 + ….. + n-1) = (n (n-1) ) /2
Number of comparision in the worst case.

If the list is already sorted, then it requires ‘n’ number of

comparisons.
Insertion Sort Example
Insertion Sort Example
Insertion Sort Example
Insertion Sort Example
Insertion Sort Algorithm
Insertion Sort Algorithm
Alg.: INSERTION-SORT(A)
for j ← 2 to n
do key ← A[ j ]
Insert A[ j ] into the sorted sequence A[1 . . j -1]

i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
• Insertion sort – sorts the elements in place
Insertion Sort Algorithm
1. FOR i ← 1 TO length[A]
2. DO item ← A[i]
3. place ← i
4. WHILE place > 0 && A[place-1] > item
5. DO A[place] ← A[place-1]
6. place ← place − 1
7. A[place] ← item
 Insertion sort runtime
 worst case: reverse-ordered elements in array.
n 1
(n 1)n
 i  1  2  3  ...  (n 1)  2
i 1

 (n 2 )
 best case: array is in sorted ascending order.
n 1


 i  n  1  ( n )
i 1
 average case: each element is about halfway in order.
n 1
i 1 (n 1)n
 2  2 (1  2  3...  (n 1))  4
i 1

 (n 2 )
Advantages of Insertion Sort
 It is easy to implement and efficient to use on small sets of data.
 It can be efficiently implemented on data sets that are already substantially sorted.
 It performs better than algorithms like selection sort and bubble sort. Insertion
sort algorithm is simpler than shell sort, with only a small trade-off in efficiency. It
is over twice as fast as the bubble sort and almost 40 per cent faster than the
selection sort.
 It requires less memory space (only O(1) of additional memory space).
 It is said to be online, as it can sort a list as and when it receives new elements.
 Comparing sorts
 We've seen "simple" sorting algorithms so far, such as selection sort bubble sort and
insertion sort.
 They all use nested loops and perform approximately n2 comparisons
 They are relatively inefficient
 Sorting practice problem
Consider the following array of int values.

[22, 11, 34, -5, 3, 40, 9, 16, 6]

(a) Write the contents of the array after 3 passes of the outermost loop of bubble sort.

(b) Write the contents of the array after 5 passes of the outermost loop of insertion sort.

(c) Write the contents of the array after 4 passes of the outermost loop of selection sort.
What is the running time of the best and worst cases
of the following sorting algorithms?
Insertion Selection Sort Bubble Sort
Sort
Best Case O(n) O(n^2) O(n)

Worst Case O(n^2) O(n^2) O(n^2)

What is the worst case input for

Insertion Sort Reversed List

Insertion Sort
Bubble Sort Reversed list
Bubble Sort

Selection Sort
Selection Sort Doesn’t matter, all lists take the same
number of steps
Quick Sort
 Quick sort is a highly efficient sorting algorithm and is based on partitioning
of array of data into smaller arrays.
 Quick sort is divide & conquer algorithm.
 The basic idea of Quick Sort is as follow:
 Quick sort works by finding an element randomly, called the pivot, in the given
input array
 The array divide into 3 sub array such that
 The left sub array-> contain elements which are less than or equal to pivot element
 The middle sub array-> contain pivot
 The right sub array-> contain elements which are greater than or equal to pivot element
Now the 2 sub arrays, are sorted recursively.
Quick Sort
 There are many different version of Quick Sort that pick pivot element in
different ways.
 Pick first element as pivot.
 Pick last element as pivot.
 Pick a random element as pivot.
 Pick median as pivot.

This algorithm is quite efficient for large data sets.

Partition
 Partitioning places the pivot in its correct place position within the array.

 Arranging the array elements around the pivot p generates two smaller sorting problems.
 sort the left section of the array, and sort the right section of the array.
 when these two smaller sorting problems are solved recursively, our bigger sorting
problem is solved.
Quick Sort Algorithm
 Quick Sort Algorithm
Quicksort(A, p, r) Partition(A, p, r)
1: if p ≥ r then return x = A[r]
2: q = Partition(A, p, r) i←p−1
3: Quicksort(A, p, q − 1) for j ← p to r − 1 do
4: Quicksort(A, q + 1, r) if A[ j ] ≤ x then {
i←i+1
Exchange A[ i ] and A[ j ]
A[p..r] }
Exchange A[i + 1] and A[r]
5
return i + 1
A[p..q – 1] A[q+1..r]
Partition

5 5
 Example
p r
initially: 2 5 8 3 9 4 1 7 10 6 note: pivot (x) = 6
i j

first iteration ( j=1): 2 5 8 3 9 4 1 7 10 6

i j Partition(A, p, r)
next iteration ( j=2): 2 5 8 3 9 4 1 7 10 6 x:= A[r];
i j i:=p – 1;
next iteration ( j=3): 2 5 8 3 9 4 1 7 10 6 for j := p to r – 1 do
i j if A[j]  x then
next iteration ( j=4): 2 5 3 8 9 4 1 7 10 6 i := i + 1;
i j A[i]  A[j]
A[i + 1]  A[r];
return i + 1
 Example (Continued)
next iteration ( j=4): 2 5 3 8 9 4 1 7 10 6 Partition(A, p, r)
i j
next iteration ( j=5): 2 5 3 8 9 4 1 7 10 6 x:= A[r];
i j i:=p – 1;
next iteration( j=6): 2 5 3 4 9 8 1 7 10 6
i j
for j := p to r – 1 do
next iteration( j=7): 2 5 3 4 1 8 9 7 10 6 if A[j]  x then
i j i := i + 1;
next iteration( j=8): 2 5 3 4 1 8 9 7 10 6
i j A[i]  A[j]
next iteration( j=9): 2 5 3 4 1 8 9 7 10 6 A[i + 1]  A[r];
i j
return i + 1
after final swap: 2 5 3 4 1 6 9 7 10 8
i j
return 6
Partitioning
 Select the last element A[r] in the subarray A[p..r] as the pivot – the
element around which to partition.
 As the procedure executes, the array is partitioned into 3 (possibly empty)
regions.
1. A[p..i ] — All entries in this region are < pivot.
2. A[i+1..j – 1] — All entries in this region are > pivot.
3. A[r] = pivot.
 The above hold before each iteration of the for loop, and constitute a loop
invariant. (4 is not part of the loopi.)
 Partition the list into 2-sub list
2 5 3 4 1 6 9 7 10 8

Quicksort(A, p, r) q=6 Quicksort(A, p, r)

return value of
1: if p ≥ r then return i 1: if p ≥ r then return
2: q = Partition(A, p, r) 2: q = Partition(A, p, r)
3: Quicksort(A, p, q − 1) 3: Quicksort(A[2,5,3,4,1], 1, 5)
4: Quicksort(A, q + 1, r) 4: Quicksort(A[9,7,10,8], 6, 10)
 Correctness of Partition
 Use loop invariant.
 Initialization:
 Before first iteration
 A[p..i] and A[i+1..j – 1] are empty – Condition. 1 and 2 are satisfied (trivially).
 r is the index of the pivot
 Cond. 3 is satisfied. Partition(A, p, r)
x, i := A[r], p – 1;
 Maintenance:
for j := p to r – 1 do
 Case 1: A[j] > x if A[j]  x then
 Increment j only. i := i + 1;
 Loop Invariant is maintained. A[i]  A[j]
A[i + 1]  A[r];
return i + 1
 Correctness of Partition
Case 1:

p i j r
>x x

x >x
p i j r
x

x >x
 Correctness of Partition
 Case 2: A[j]  x  Increment j
 Increment i  Condition 2 is maintained.

 Swap A[i] and A[j]  A[r] is unaltered.

 Condition 1 is maintained.  Condition 3 is maintained.

p i j r
x x

x >x
p i j r
x

x >x
Correctness of Partition
 Termination:
 When the loop terminates, j = r, so all elements in A are partitioned into one of the
three cases:
 A[p..i]  pivot
 A[i+1..j – 1] > pivot
 A[r] = pivot

 The last two lines swap A[i+1] and A[r].

 Pivot moves from the end of the array to between the two subarrays.
 Thus, procedure partition correctly performs the divide step.
Complexity of Partition
 PartitionTime(n) is given by the number of iterations in the for loop.
 (n) : n = r – p + 1.

Partition(A, p, r)
x, i := A[r], p – 1;
for j := p to r – 1 do
if A[j]  x then
i := i + 1;
A[i]  A[j]
A[i + 1]  A[r];
return i + 1
 Quicksort Overview
 To sort a[left...right]:
1. if left < right:
1.1. Partition a[left...right] such that:
all a[left...p-1] are less than a[p], and
all a[p+1...right] are >= a[p]
1.2. Quicksort a[left...p-1]
1.3. Quicksort a[p+1...right]
2. Terminate
Partitioning in Quicksort
 A key step in the Quicksort algorithm is partitioning the array
 We choose some (any) number p in the array to use as a pivot
 We partition the array into three parts:

numbers less than p p numbers greater than or equal to p

Alternative Partitioning
 Choose an array value (say, the first) to use as the pivot
 Starting from the left end, find the first element that is greater than or equal to the
pivot
 Searching backward from the right end, find the first element that is less than the
pivot
 Interchange (swap) these two elements
 Repeat, searching from where we left off, until done
 Alternative Partitioning
 To partition a[left...right]:
1. Set pivot = a[left], l = left + 1, r = right;
2. while l < r, do
2.1. while l < right & a[l] < pivot , set l = l + 1
2.2. while r > left & a[r] >= pivot , set r = r - 1
2.3. if l < r, swap a[l] and a[r]
3. Set a[left] = a[r], a[r] = pivot
4. Terminate
 Example of partitioning
 choose pivot: 436924312189356

 search: 436924312189356

 swap: 433924312189656

 search: 433924312189656

 swap: 433124312989656
Continue..
 search: 433124312989656

 swap: 433122314989656

 search: 433122314989656

 swap with pivot: 133122344989656

Partition Implementation (Java)
static int Partition(int[] a, int left, int right) {
int p = a[left], l = left + 1, r = right;
while (l < r) {
while (l < right && a[l] < p) l++;
while (r > left && a[r] >= p) r--;
if (l < r) {
int temp = a[l]; a[l] = a[r]; a[r] = temp;
}
}
a[left] = a[r];
a[r] = p;
return r;
}
Quicksort Implementation (Java)
static void Quicksort(int[] array, int left, int right) {
if (left < right) {
int p = Partition(array, left, right);
Quicksort(array, left, p - 1);
Quicksort(array, p + 1, right);
}
}
 Analysis of quicksort—best case
 Suppose each partition operation divides the array almost exactly in half
 Then the depth of the recursion in log2n

 We note that
 Each partition is linear over its subarray
 All the partitions at one level cover the array
Partitioning at various levels
Best Case Analysis
 We cut the array size in half each time
 So the depth of the recursion in log2n
 At each level of the recursion, all the partitions at that level do work that is linear
in n
 O(log2n) * O(n) = O(n log2n)
 Hence in the best case, quicksort has time complexity O(n log2n)
 What about the worst case?
Worst case
 In the worst case, partitioning always divides the size n array into these three
parts:
 A length one part, containing the pivot itself
 A length zero part, and
 A length n-1 part, containing everything else
 We don’t recur on the zero-length part
 Recurring on the length n-1 part requires (in the worst case) recurring to depth
n-1
Worst case partitioning
Worst case for quicksort
 In the worst case, recursion may be n levels deep (for an array of size n)
 But the partitioning work done at each level is still n
 O(n) * O(n) = O(n2)
 So worst case for Quicksort is O(n2)
 When does this happen?
 There are many arrangements that could make this happen
 Here are two common cases:
 When the array is already sorted
 When the array is inversely sorted (sorted in the opposite order)
 Typical case for quicksort
 If the array is sorted to begin with, Quicksort is terrible: O(n2)
 It is possible to construct other bad cases
 However, Quicksort is usually O(n log2n)
 The constants are so good that Quicksort is generally the faster algorithm.
 Most real-world sorting is done by Quicksort.
Picking a better pivot
 Before, we picked the first element of the subarray to use as a pivot
 If the array is already sorted, this results in O(n2) behavior
 It’s no better if we pick the last element
 We could do an optimal quicksort (guaranteed O(n log n)) if we always picked a
pivot value that exactly cuts the array in half
 Such a value is called a median: half of the values in the array are larger, half are
smaller
Quicksort for Small Arrays
 For very small arrays (N<= 20), quicksort does not perform as well as insertion
sort
 A good cutoff range is N=10
 Switching to insertion sort for small arrays can save about 15% in the running time
Quick Sort Example

Solved by your own

 O(n2 ) time complexity

What if all keys equal?

 O(n2 ) time
complexity

What if all the keys are already sorted?

Randomly choosing the pivot overcomes the second
problem.
Merge Sort
 Merge sort is yet another sorting algorithm which works on the Divide and
Conquer design principal.
 It works by dividing the given arrays several sub-lists until each sub-list consists
of a single element
 Merging those sub-lists in the manner that results into a sorted list
 Procedure
 Divide the unsorted list into N sub-list, each containing 1 element
 Take adjacent pair of 2 singleton lists and merge them to form a list of 2 elements.
 Repeat the process till a single sorted list obtained.

Time complexity is O(n log n)

Merge Sort Algorithm
 Divide: If S has at least two elements, remove all the elements from S and put
them into two sequences, S1 and S2 , each containing about half of the elements of
S.
 Conquer: Sort sequences S1 and S2 using Merge Sort.
 Combine: Put back the elements into S by merging the sorted sequences S1 and S2
into one sorted sequence
MergeSort (Example) - 1
MergeSort (Example) - 2
MergeSort (Example) - 3
MergeSort (Example) - 4
MergeSort (Example) - 5
MergeSort (Example) - 6
MergeSort (Example) - 7
MergeSort (Example) - 8
MergeSort (Example) - 9
MergeSort (Example) - 10
MergeSort (Example) - 11
MergeSort (Example) - 12
MergeSort (Example) - 13
MergeSort (Example) - 14
MergeSort (Example) - 15
MergeSort (Example) - 16
MergeSort (Example) - 17
MergeSort (Example) - 18
MergeSort (Example) - 19
MergeSort (Example) - 20
MergeSort (Example) - 21
Merge Sort (Example) - 22
Merge Sort
Merge-Sort(A, p, r)
if p < r then
q(p+r)/2
Merge-Sort(A, p, q)
Merge-Sort(A, q+1, r)
Merge(A, p, q, r)

Merge(A, p, q, r)
Take the smallest of the two topmost elements of
sequences A[p..q] and A[q+1..r] and put into the
resulting sequence. Repeat this, until both sequences
are empty. Copy the resulting sequence into A[p..r].
Merge Sort
MergeSort(A, left, right) {
if (left < right) {
mid = floor((left + right) / 2);
MergeSort(A, left, mid);
MergeSort(A, mid+1, right);
Merge(A, left, mid, right);
}
}

// Merge() takes two sorted subarrays of A and

// merges them into a single sorted subarray of A
Analysis of Merge Sort
Statement Effort
MergeSort(A, left, right) { T(n)
if (left < right) { (1)
mid = floor((left + right) / 2); (1)
MergeSort(A, left, mid); T(n/2)
MergeSort(A, mid+1, right); T(n/2)
Merge(A, left, mid, right); (n)
}

So T(n) = (1) when n = 1, and

2T(n/2) + (n) when n > 1
Merge Sort Example
Merge Sort Example
Step-2: Select the left sub-list and split it.
Merge Sort Example
Merge algorithm
Merge Example
Merge Example..
Merge Sort Example
Exercise
 Show MergeSort() running on the array

A = {10, 5, 7, 6, 1, 4, 8, 3, 2, 9};
What is the running time in worst, best & average
cases of the following sorting algorithms?
Insertion Selection Sort Bubble Sort Quick Sort Merge Sort
Sort

Worst Case

Best Case

Average
Case
 Answer-

Insertion Selection Sort Bubble Sort Quick Sort Merge Sort

Sort

Worst Case O(n^2) O(n^2) O(n^2) O(n^2) O(n log n)

Best Case O(n) O(n^2) O(n) O(n log n) O(n log n)

Average O(n^2) O(n^2) O(n^2) O(n log n) O(n log n)

Case
Linear Sorts
 Algorithms that do not depend only on comparing whole keys to be
sorted.

 Counting sort
 Bucket sort
 Radix sort
Counting sort
 Assumptions:
 n records
 Each record contains keys and data
 All keys are in the range of 1 to k
 Space
 The unsorted list is stored in A, the sorted list will be stored in an additional array B
 Uses an additional array C of size k
Counting sort
 Main idea:
1) For each key value i, i = 1,…,k, count the number of times the keys occurs in the unsorted
input array A.
2) Store results in an auxiliary array, C
3) Use these counts to compute the offset. Offseti is used to calculate the location where the
record with key value i will be stored in the sorted output list B.
4) The offseti value has the location where the last keyi .

 When would you use counting sort?

 How much memory is needed?
Counting Sort Counting-Sort( A, B, k)
1. for i  1 to k
2. do C[i ]  0
Input: A [ 1 .. n ],
A[J]  {1,2, . . . , k } 3. for j  1 to length[A]
4. do C[A[ j ] ]  C[A[ j ] ] + 1
Output: B [ 1 .. n ], sorted
5. for i  2 to k
Uses C [ 1 .. k ],
auxiliary storage
6. do C[i ]  C[i ] +C[i -1]

7. for j  length[A] down 1

8. do B [ C[A[ j ] ] ]  A[ j ]
9. C[A[ j ] ] ]  C [A[ j ] ] -1
 1 2 3 4 5 6

A 4 1 3 4 3 4
Counting-Sort( A, B, k)
k = 4, length = 6 1. for i  1 to k
2. do C[i ]  0
C 0 0 0 0

after lines 1-2 3. for j  1 to length[A]

4. do C[A[ j ] ]  C[A[ j ] ] + 1
C 1 0 2 3
5. for i  2 to k
after lines 3-4 6. do C[i ]  C[i ] +C[i -1]
C 1 1 3 6

after lines 5-6

 1 2 3 4 5 6

A 4 1 3 4 3 4
7. for j  length[A] down 1
8. do B [ C[A[ j ] ] ]  A[ j ]
9. C[A[ j ] ] ]  C [A[ j ] ] -1

1 2 3 4 5 6

B
<-1-> <- - 3 - -> <- - - 4 - ->

C 1 1 3 6
Counting Sort Example
Analysis:
 O(k + n) time
 What if k = O(n)
 Requires k + n extra storage.
 This is a stable sort: It preserves the original order of equal keys.
 Clearly no good for sorting 32 bit values.
Radix Sort
 Radix sort algorithm different than other sorting algorithms that we talked.
 It does not use key comparisons to sort an array.
 It is used to sort integer and string both.
 It is also an extension of bucket sort algorithm.
 The radix sort :
 Treats each data item as a group of number or group of character string.
 First it groups data items according to their rightmost character, and put these groups
into order w.r.t. this rightmost character.
 Then, combine these groups.
 We, repeat these grouping and combining operations for all other character positions in
the data items from the rightmost to the leftmost character position.
 At the end, the sort operation will be completed.
Radix Sort
String of Names Integer
 Radix is 26 or 26 buckets(a,b,…z)  Radix is 10 or 10 buckets(0,1,…..9)
Example-
Sort the numbers given below using radix sort.
345, 654, 924, 123, 567, 472, 555, 808, 911
In the first pass, the numbers are sorted according
to the digit at ones place.
After this pass, the numbers are collected bucket by
bucket. The new list thus formed is used as an input
for the next pass. In the second pass, the numbers
are sorted according to the digit at the tens place.
In the third pass, the numbers are sorted according
to the digit at the hundreds place.

The numbers are collected bucket by bucket. The new list thus formed is the final sorted
result. After the third pass, the list can be given as
123, 345, 472, 555, 567, 654, 808, 911, 924
Example-
Sort the number given below using Radix Sort
189, 986, 205, 421, 97, 192, 535
 Radix Sort – Example
mom, dad, god, fat, bad, cat, mad, pat, bar, him original list

(dad,god,bad,mad) (mom,him) (bar) (fat,cat,pat) group strings by rightmost letter

dad,god,bad,mad,mom,him,bar,fat,cat,pat combine groups

(dad,bad,mad,bar,fat,cat,pat) (him) (god,mom) group strings by middle letter

dad,bad,mad,bar,fat,cat,pat,him,god,mom combine groups

(bad,bar) (cat) (dad) (fat) (god) (him) (mad,mom) (pat) group strings by first letter

bad,bar,cat,dad,fat,god,him,mad,mom,par combine groups (SORTED)

Radix Sort Exercise- sort it using radix sort

COW, DOG, SEA, RUG, ROW,

MOB, BOX, TAB, BAR, EAR, TAR,
DIG, BIG , TEA, NOW, FOX
Example
 Sorting a sequence of 4-bit integers

1001 0010 1001 1001 0001

0010 1110 1101 0001 0010

1101 1001 0001 0010 1001

0001 1101 0010 1101 1101

1110 0001 1110 1110 1110

 Radix Sort - Algorithm
radixSort(inout theArray:ItemArray, in n:integer, in d:integer)
// sort n d-digit integers in the array theArray
for (j=d down to 1) {
Initialize 10 groups to empty
Initialize a counter for each group to 0
for (i=0 through n-1) {
k = jth digit of theArray[i]
Place theArrar[i] at the end of group k
Increase kth counter by 1
}
Replace the items in theArray with all the items in group 0,
followed by all the items in group 1, and so on.
}
Radix Sort -- Analysis
 The radix sort algorithm requires 2*n*d moves to sort n strings of d characters
each.
 So, Radix Sort is O(n)

 Although the radix sort is O(n), it is not appropriate as a general-purpose sorting

algorithm.
 Its memory requirement is d * original size of data (because each group should be big enough to hold the
original data collection.)
 For example, we need 27 groups to sort string of uppercase letters.
 The radix sort is more appropriate for a linked list than an array. (we will not need the huge memory in this case)
Shell Sort
Improves on insertion sort

Compares elements far apart, then less far

apart, finally compares adjacent elements
(as an insertion sort).
 Shell Sort - example
Initial Segmenting Gap = 4

80 93 60 12 42 30 68 85 10

10 30 60 12 42 93 68 85 80
 Shell Sort - example (2)
Resegmenting Gap = 2

10 30 60 12 42 93 68 85 80

10 12 42 30 60 85 68 93 80
Shell Sort - example (3)
Resegmenting Gap = 1

10 12 42 30 60 85 68 93 80

10 12 30 42 60 68 80 85 93
Example-2
Sort the elements given below using shell sort.
63, 19, 7, 90, 81, 36, 54, 45, 72, 27, 22, 9, 41, 59, 33
Shell Sort Algorithm
 Analysis
Shellsort's worst-case performance using Hibbard's
increments is Θ(n3/2).
The average performance is thought to be about O(n 5/4)
The exact complexity of this algorithm is still being debated

for mid-sized data : nearly as well if not better than the

faster (n log n) sorts.

Animations:
http://www.sorting-algorithms.com/shell-sort
Tree Sort
 The properties of binary search tree is completely make use
of tree sort algorithm.
 The tree sort algorithm first builds a binary search tree
using the elements in an array which is to be sorted and
then does an in-order traversal so that the numbers are
retrieved in a sorted order.
Saving Binary Search Tree in a File
 An initially empty binary search tree after the insertion of 60, 20, 10, 40, 30, 50,
and 70
Saving Binary Search Tree in a File- Tree Sort
 A full tree saved in a file by using inorder traversal

Inorder traversal (left-root-right)

First, visit all the nodes in the left subtree
Then the root node
Visit all the nodes in the right subtree
Tree Sort
 Algorithm for tree sort
How to handle duplicates/equal keys in Binary
Search Tree?
 A Simple Solution is to allow same keys on right side (we could also choose left side).
For example consider insertion of keys 12, 10, 20, 9, 11, 10, 12, 12 in an empty Binary
Search Tree
Complexity
Worst complexity: n log(n) (balanced)
Average complexity: n log(n)
Advantages - Tree Sort
 The main advantage of tree sort algorithm is that we
can make changes very easily as in a linked list.
 Sorting in Tree sort algorithm is as fast as quick sort
algorithm.
Disadvantages - Tree Sort

 The worst case occur when the elements in an array is

already sorted.
 In worst case, the running time of tree sort algorithm is
O (n2)
Heap Sort
A sorting algorithm that works by first organizing
the data to be sorted into a special type of binary
tree called a heap.
Procedures on Heap
 Build Heap
 Heapify
 Heap Sort
Special Types of Trees
4
 Def: Full binary tree = A binary
tree in which each node is either a leaf 1 3
or has degree exactly 2. 2 16 9 10
14 8 7 12
Full binary tree

 Def: Complete binary tree = A 4

binary tree in which all leaves are on the
same level and all internal nodes have 1 3
degree 2. 2 16 9 10

Complete binary tree

Definitions
 Height of a node = the number of edges on the longest simple path from the node down
to a leaf
 Level of a node = the length of a path from the root to the node
 Height of tree = height of root node

4 Height of root = 3

1 3
Height of (2)= 1 2 16 9 10 Level of (10)= 2
14 8
The Heap Data Structure
 Def: A heap is a nearly complete binary tree with the following two properties:
 Structural property: all levels are full, except possibly the last one, which is filled
from left to right
 Order (heap) property: for any node x
Parent(x) ≥ x
From the heap property, it
follows that:
8 “The root is the
maximum/minimum
7 4 element of the heap!”
5 2

Heap
A heap is a binary tree that is filled in order
Array Representation of Heaps
 A heap can be stored as an array A.
 Root of tree is A[1]
 Left child of A[i] = A[2i]
 Right child of A[i] = A[2i + 1]
 Parent of A[i] = A[ i/2 ]
 Heapsize[A] ≤ length[A]

 The elements in the subarray

A[(n/2+1) .. n] are leaves
Heap Types
 Max-heaps (largest element at root), have the max-heap property:
 for all nodes i, excluding the root:
A[PARENT(i)] ≥ A[i]

 Min-heaps (smallest element at root), have the min-heap property:

 for all nodes i, excluding the root:
A[PARENT(i)] ≤ A[i]

162
Max Heap Example
19

12 16

1 4 7

19 12 16 1 4 7

Array A
Min heap example
1

4 16

7 12 19

1 4 16 7 12 19
Array A
Heapify
 Heapify picks the largest child key and compare it to the parent key. If parent
key is larger than heapify quits, otherwise it swaps the parent key with the
largest child key. So that the parent is now becomes larger than its children.
Heapify(A, i)
{
l  left(i)
r  right(i)
if l <= heapsize[A] and A[l] > A[i]
then largest l
else largest  i
if r <= heapsize[A] and A[r] > A[largest]
then largest  r
if largest != i
then swap A[i]  A[largest]
Heapify(A, largest)
}
Build Heap
 We can use the procedure 'Heapify' in a bottom-up fashion to convert an array
A[1 . . n] into a heap. Since the elements in the subarray A[n/2 +1 . . n] are all
leaves, the procedure BUILD_HEAP goes through the remaining nodes of the
tree and runs 'Heapify' on each one. The bottom-up order of processing node
guarantees that the subtree rooted at children are heap before 'Heapify' is run at
their parent.

Buildheap(A)
{
heapsize[A] length[A]
for i |length[A]/2 //down to 1
do Heapify(A, i)
}
Heap Sort Algorithm
 The heap sort algorithm starts by using procedure BUILD-HEAP to build a heap
on the input array A[1 . . n]. Since the maximum element of the array stored at the
root A[1], it can be put into its correct final position by exchanging it with A[n]
(the last element in A). If we now discard node n from the heap than the
remaining elements can be made into heap. Note that the new element at the root
may violate the heap property. All that is needed to restore the heap property.

Heapsort(A)
{
Buildheap(A)
for i  length[A] //down to 2
do swap A[1]  A[i]
heapsize[A]  heapsize[A] - 1
Heapify(A, 1)
}
Example: Convert the following array to a heap
16 4 7 1 12 19

Picture the array as a complete binary tree:

16

4 7

12 19
1 Step-1
Max- Heapify
 16 16

4 7 4 19
swap

12 19 1 12 7
1

16 19
swap

12 19 12 16
swap

4 7 1 4 7
1
Heap Sort
 The heapsort algorithm consists of two phases:
- build a heap from an arbitrary array
- use the heap to sort the data

 To sort the elements in the decreasing order, use a min heap

 To sort the elements in the increasing order, use a max heap
19

12 16

1 4 7
Example of Heap Sort
Take out biggest
19

12 16
Move the last element
to the root

1 4 7

Sorted:
Array A

12 16 1 4 7 19
 7
swap
HEAPIFY()
12 16

1 4

Sorted:
Array A

7 12 16 1 4 19
 16

12 7

1 4

Sorted:
Array A

16 12 7 1 4 19
 Take out biggest
16
Move the last element
to the root
12 7

1 4

Sorted:
Array A

12 7 1 4 16 19
 4

12 7

Sorted:
Array A

4 12 7 1 16 19
 swap 4

HEAPIFY()
12 7

Sorted:
Array A

4 12 7 1 16 19
 12

4 7

Sorted:
Array A

12 4 7 1 16 19
 Take out biggest
12
Move the last
element to the
root 4 7

Sorted:
Array A

4 7 1 12 16 19
 1
swap

4 7

Sorted:
Array A

1 4 7 12 16 19
 7

4 1

Sorted:
Array A

7 4 1 12 16 19
 Take out biggest
7
Move the last
element to the
4 1 root

Sorted:
Array A

1 4 7 12 16 19
 swap 1

HEAPIFY()
4

Sorted:
Array A

4 1 7 12 16 19
 Take out biggest
Move the last 4
element to the
root
1

Sorted:
Array A

1 4 7 12 16 19
 Take out biggest
1

Sorted:
Array A

1 4 7 12 16 19
 Sorted:

1 4 7 12 16 19
Time Analysis
 Build Heap Algorithm will run in O(n) time
 There are n-1 calls to Heapify each call requires O(log n)
time
 Heap sort program combine Build Heap program and
Heapify, therefore it has the running time of O(n log n)
time
 Total time complexity: O(n log n)
Comparison with Quick Sort and Merge Sort
 Quick sort is typically somewhat faster, due to better cache behavior and other factors,
but the worst-case running time for quick sort is O (n2), which is unacceptable for
large data sets and can be deliberately triggered given enough knowledge of the
implementation, creating a security risk.

 The quick sort algorithm also requires Ω (log n) extra storage space, making it not a
strictly in-place algorithm. This typically does not pose a problem except on the
smallest embedded systems, or on systems where memory allocation is highly
restricted. Constant space (in-place) variants of quick sort are possible to construct,
but are rarely used in practice due to their extra complexity.
Comparison with Quick Sort and Merge Sort (cont)
 Thus, because of the O(n log n) upper bound on heap sort’s running time and constant
upper bound on its auxiliary storage, embedded systems with real-time constraints or
systems concerned with security often use heap sort.

 Heap sort also competes with merge sort, which has the same time bounds, but
requires Ω(n) auxiliary space, whereas heap sort requires only a constant amount.
Heap sort also typically runs more quickly in practice. However, merge sort is simpler
to understand than heap sort, is a stable sort, parallelizes better, and can be easily
adapted to operate on linked lists and very large lists stored on slow-to-access media
such as disk storage or network attached storage. Heap sort shares none of these
benefits; in particular, it relies strongly on random access.
Possible Application
 When we want to know the task that carry the highest priority given a large
number of things to do

 Interval scheduling, when we have a lists of certain task with start and finish
times and we want to do as many tasks as possible

 Sorting a list of elements that needs and efficient sorting algorithm

Heapsort – Example
26 24 20 18 17 19 13 12 14 11
1 2 3 4 5 6 7 8 9 10

26

24 20

18 17 19 13

12 14 11
THANK YOU