Unit 11

Algorithms
Contents
• Deterministic and Non-deterministic Algorithm
• Divide and Conquer Algorithm
• Series and Parallel Algorithm
• Heuristic and Approximate Algorithm
Divide and Conquer Algorithm
• In divide and conquer approach, the problem in hand, is divided into smaller sub-
problems and then each problem is solved independently.
• When we keep on dividing the subproblems into even smaller sub-problems, we
may eventually reach a stage where no more division is possible.
• Those "atomic" smallest possible sub-problem (fractions) are solved.
• The solution of all sub-problems is finally merged in order to obtain the solution
of an original problem.
Divide and Conquer Algorithm
• Broadly, we can understand divide-and-conquer approach in a three-step process.
• Divide/Break
• This step involves breaking the problem into smaller sub-problems. Sub-problems should represent a part of
the original problem. This step generally takes a recursive approach to divide the problem until no sub-
problem is further divisible. At this stage, sub-problems become atomic in nature but still represent some part
of the actual problem.
• Conquer/Solve
• This step receives a lot of smaller sub-problems to be solved. Generally, at this level, the problems are
considered 'solved' on their own.
• Merge/Combine
• When the smaller sub-problems are solved, this stage recursively combines them until they formulate a
solution of the original problem. This algorithmic approach works recursively and conquer & merge steps
works so close that they appear as one.
• The following computer algorithms are based on divide-and-conquer programming approach −
• Merge Sort
• Quick Sort
• Binary Search
• Strassen's Matrix Multiplication
• Closest pair (points)
Series and Parallel Algorithm
• Sequential Algorithm − An algorithm in which some consecutive steps of
instructions are executed in a chronological order to solve a problem.
• Parallel Algorithm − The problem is divided into sub-problems and are executed
in parallel to get individual outputs. Later on, these individual outputs are
combined together to get the final desired output.
• It is not easy to divide a large problem into sub-problems.
• Sub-problems may have data dependency among them.
• Therefore, the processors have to communicate with each other to solve the
problem.
• It has been found that the time needed by the processors in communicating with
each other is more than the actual processing time.
• So, while designing a parallel algorithm, proper CPU utilization should be
considered to get an efficient algorithm.
Series and Parallel Algorithm
• Foster's design methodology. It is a four-stage design process whose input is the problem statement. The
four stages are
• Partitioning - The process of dividing the computation and the data into pieces.
• Communication - The process of determining how tasks will communicate with each other, distinguishing
between local communication and global communication.
• Agglomeration - The process of grouping tasks into larger tasks to improve performance or simplify
programming.
• Mapping - The process of assigning tasks to physical processors.
Heuristic and Approximate Algorithm
• A heuristic algorithm is one that is designed to solve a problem in a faster and more efficient fashion than
traditional methods by sacrificing optimality, accuracy, precision, or completeness for speed.
• Heuristic algorithms often times used to solve NP-complete problems, a class of decision problems.
• In these problems, there is no known efficient way to find a solution quickly and accurately although
solutions can be verified when given.
• Heuristics can produce a solution individually or be used to provide a good baseline and are supplemented
with optimization algorithms.
• Heuristic algorithms are most often employed when approximate solutions are sufficient and exact solutions
are necessarily computationally expensive.
• Example Algorithms
• Swarm Intelligence
• Tabu Search
• Simulated Annealing
• Genetic Algorithms
• Artificial Neural Networks
• Support Vector Machines
Heuristic and Approximate Algorithm
• Traveling Salesmen Problem
• A well-known example of a heuristic algorithm is used to solve the common Traveling
Salesmen Problem.
• The problem is as follows: given a list of cities and the distances between each city, what is
the shortest possible route that visits each city exactly once?
• A heuristic algorithm used to quickly solve this problem is the nearest neighbor (NN)
algorithm (also known as the Greedy Algorithm).
• Starting from a randomly chosen city, the algorithm finds the closest city.
• The remaining cities are analyzed again, and the closest city is found.
• These are the steps of the NN algorithm:
1. Start at a random vertex
2. Determine the shortest distance connecting the current
vertex and an unvisited vertex V
3. Make the current vertex the unvisited vertex V
4. Make V visited
5. Record the distance traveled
6. Terminate if no other unvisited vertices remain
7. Repeat step 2.
Heuristic and Approximate Algorithm
• Virus Scanning
• In virus scanning, an algorithm searches for key pieces of code associated with
particular kinds or viruses, reducing the number of files that need to be scanned.
• One of the benefits of heuristic virus scanning is that different viruses of the same
family can be detected without being known due to the common code markers.
• Searching and Sorting
• One of the most common uses of heuristic algorithms is in searching and sorting.
• As a search runs, it adjusts its working parameters to optimize speed, an important
characteristic in a search function.
• The algorithm discards current possibilities if they are worse than already found
solutions.
• Some forms of the heuristic methods can be detrimental to searching such as the
best-first search algorithm.
• It takes search results close to the goal and follows the new path even when it may
not continue to lead to the optimal search result.
Heuristic and Approximate Algorithm
• An approximation algorithm is a way of dealing with NP-completeness for an
optimization problem.
• This technique does not guarantee the best solution.
• The goal of the approximation algorithm is to come as close as possible to the
optimal solution in polynomial time.
• Such algorithms are called approximation algorithms or heuristic algorithms.
• Features of Approximation Algorithm :
• An approximation algorithm guarantees to run in polynomial time though it does not
guarantee the most effective solution.
• An approximation algorithm guarantees to seek out high accuracy and top quality
solution(say within 1% of optimum)
• Approximation algorithms are used to get an answer near the (optimal) solution of an
optimization problem in polynomial time
Heuristic and Approximate Algorithm
• Some examples of the Approximation algorithm :
• The Vertex Cover Problem –
• In the vertex cover problem, the optimization problem is to find the vertex cover with
the fewest vertices, and the approximation problem is to find the vertex cover with few
vertices.
• Travelling Salesman Problem –
• In the traveling salesperson problem, the optimization problem is to find the shortest
cycle, and the approximation problem is to find a short cycle.
• The Set Covering Problem –
• This is an optimization problem that models many problems that require resources to be
allocated. Here, a logarithmic approximation ratio is used.
• The Subset Sum Problem –
• In the Subset sum problem, the optimization problem is to find a subset of {x1,×2,×3…xn}
whose sum is as large as possible but not larger than the target value t.