Intractability
Intractability
I NTRACTABILITY
‣ introduction
‣ P vs. NP
Algorithms F O U R T H E D I T I O N
‣ poly-time reductions
‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
‣ dealing with intractability
https://algs4.cs.princeton.edu
2
I NTRACTABILITY
‣ introduction
‣ P vs. NP
Algorithms ‣ poly-time reductions
‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
‣ dealing with intractability
https://algs4.cs.princeton.edu
Fundamental questions
A Turing machine
4
A dif cult problem: integer factorization
Integer factorization. Given an integer x, find a nontrivial factor. or report that no such factor exists
not 1 nor x
5
fi
Another dif cult problem: boolean satis ability
Boolean satisfiability. Given a system of boolean equations, find a satisfying truth assignment.
Applications.
・Automatic verification systems for software.
・Mean field diluted spin glass model in physics.
・Electronic design automation (EDA) for hardware.
・…
6
fi
fi
Another dif cult problem: boolean satis ability
Boolean satisfiability. Given a system of boolean equations, find a satisfying truth assignment.
Ex.
¬ x1 or x2 or x3 = true
x1 or ¬ x2 or x3 = true
¬ x1 or ¬ x2 or ¬ x3 = true
¬ x1 or ¬ x2 or or x4 = true
¬ x2 or x3 or x4 = true
a SAT instance
7
fi
fi
How dif cult can it be?
quantity estimate
Q. Could galactic computer solve satisfiability instance with 1,000 variables using brute-force search?
A. Not even close: 21000 > 10300 >> 1079 ⋅ 1013 ⋅ 1017 = 10109.
9
Intractability: quiz 1
10
Some computational problems
¬ x2 or x3 = true x1 = false
SAT given a system of boolean equations,
¬ x1 or ¬ x2 or ¬ x3 = true x2 = true ?
(boolean satis ability) nd a satisfying assignment
x2 or ¬ x3 = true x3 = true
3 0
ST-CONN given a graph and two vertices,
0–3–2–4 depth- rst search
(graph connectivity) nd a path that connects them
1 2 4
⋮ ⋮ ⋮ ⋮ ⋮
11
fi
fi
fi
fi
fi
fi
Intractable problems
tractable intractable?
12
Intractable problems
13
I NTRACTABILITY
‣ introduction
‣ P vs. NP
Algorithms ‣ poly-time reductions
‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
‣ dealing with intractability
https://algs4.cs.princeton.edu
Search problems
Search problem. Computational problem for which you can check a solution in poly-time.
14 7 57395 25 89 67 64 1 292 7 1 93 70 7 72 1
instance I solution S
Poly-time checking algorithm. Check whether solution is a divisor of x. O(n 2) time via long division
Search problem. Computational problem for which you can check a solution in poly-time.
¬ x1 or x2 or x3 = true
x1 or ¬ x2 or x3 = true x1 = false
¬ x1 or ¬ x2 or ¬ x3 = true x2 = false
¬ x1 or ¬ x2 or or x4 = true x3 = true
¬ x2 or x3 or x4 = true x4 = true
instance I solution S
Poly-time checking algorithm. Plug values of solution into system of equations and check.
O(mn) time
16
NP
ST-CONN given a digraph and two vertices, check for existence of edges between
(graph connectivity) nd a path that connects them consecutive vertices in path
BLOCK-CHAIN given strings s and h, nd a string t whose compute the hash of the concatenation
(hash veri cation) concatenation with s hashes to h and compare with h
⋮ ⋮ ⋮
Significance. Problems that scientists, engineers, and programmers aspire to solve in practice.
17
fi
fi
fi
fi
fi
fi
fi
Intractability: quiz 2
D. Neither A nor B.
18
P
Definition. P is the class of all search problems that can be solved in poly-time. more accurately, FP
⋮ ⋮ ⋮
NP
intractable P = NP
P search problems
P ≠ NP P = NP
brute-force search may be poly-time algorithms for
the best we can do FACTOR, SAT, LONGEST-PATH, …
ordinary appreciation
Intuition. Checking a solution seems like it should be way easier than finding it.
22
fi
fi
fi
Princeton computer science building
23
Princeton computer science building (closeup)
1
0 1
0 0 0
1 1 0
1 1 0
0 0 1 0
1 1 1 1
0 1 1
0 0 0
1 1 0 0
1 0
1 1 1
char ASCII binary
P 80 1010000
= 61 0111101
N 78 1001110
P 80 1010000
? 63 0111111
24
I NTRACTABILITY
‣ introduction
‣ P vs. NP
Algorithms ‣ poly-time reductions
‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
‣ dealing with intractability
https://algs4.cs.princeton.edu
Bird s-eye view
26
’
Poly-time reduction
Algorithm
instance I solution to I
for Y
(of X)
Algorithm for X
Design algorithms. If X poly-time reduces to Y, and can solve Y efficiently, then can also solve X.
27
Poly-time reduction
Algorithm
instance I solution to I
for Y
(of X)
Algorithm for X
Establish intractability. If SAT poly-time reduces to Y, then Y is intractable. assuming SAT is intractable
Algorithm
instance I solution to I
for Y
(of X)
Algorithm for X
X reduces to SAT: X is no harder than SAT. (If I can solve SAT, then I can solve X.)
SAT reduces to X: X is no easier than SAT. (If I can solve X, then I can solve SAT.)
29
Integer linear programming
instance I solution S
instance I
(of SAT) Algorithm solution to I
for ILP
yi = 0 ⟹ xi = false
Post-processing:
yi = 1 ⟹ xi = true
32
Intractability: quiz 3
33
More poly-time reductions from SAT
SAT
PARTITION
34
I NTRACTABILITY
‣ introduction
‣ P vs. NP
Algorithms ‣ poly-time reductions
‣ NP-completeness
R OBERT S EDGEWICK | K EVIN W AYNE
‣ dealing with intractability
https://algs4.cs.princeton.edu
NP-completeness
Two worlds.
NP
NPC P = NP = NPC
P
P ≠ NP P = NP
36
Intractability: quiz 4
I. X is in NP.
II. If X can be solved in poly-time, then P = NP.
III. If X cannot be solved in poly-time, then P ≠ NP.
A. I only.
B. II only.
C. I and II only.
Key property. An NP-complete problem can be solved in poly-time if and only if P = NP.
37
Cook–Levin theorem
38
Implications of Cook–Levin theorem
EXACT-COVER HAMILTON-CYCLE
SUBSET-SUM TSP
PARTITION CLIQUE
EXACT-COVER HAMILTON-CYCLE
SUBSET-SUM TSP
PARTITION CLIQUE
EXACT-COVER HAMILTON-CYCLE
SUBSET-SUM TSP
PARTITION CLIQUE
44
Intractability: quiz 5
A program with which of these running times is most likely to be useful in practice?
A. 10226 n
poly-time
(but probably not useful in practice)
B. n226
n exponential time
C. 1.000000001 (but probably useful in practice)
Key point. Poly-time is not always a surrogate for useful in practice, some poly-time algorithms are slow;
though it tends to be true for the algorithms we encounter in the wild. some exponential-time algorithms are fast!
45
Identifying intractable problems
46
Identifying intractable problems
I “I
guess I'm nd
can't justa to dumb. algorithm.”
poly-time “I can't nd a poly-time algorithm, but neither can all these famous people.”
47
fi
fi
Approaches to dealing with intractability
48
Dealing with intractability: nd solutions to real-world problem instances
Observations.
・Worst-case inputs may not occur for practical problems.
・Instances that do occur in practice may be easier to solve.
・Reasonable approach: relax the condition of guaranteed poly-time.
Boolean satisfiability.
・Chaff solves real-world instances with 10,000+ variables.
・Princeton senior independent work (!) in 2000.
Traveling salesperson problem.
・Concorde routinely solves large real-world instances.
・85,900-city instance solved in 2006.
TSP solution for 13,509 US cities
Integer linear programming.
・CPLEX routinely solves large real-world instances.
・Routinely used in scientific and commercial applications.
49
fi
Dealing with intractability: approximation algorithms
MAX-CUT: given a graph G, find the cut with maximum number M of crossing edges.
Approximate version: find a large cut.
50
Dealing with intractability: approximation algorithms
3-SAT: given 3-variable equations on n boolean variables, find satisfying truth assignment.
Approximate version: find assignment that satisfies many equations.
51
Leveraging intractability: RSA cryptosystem
multiply
(easy)
factor
(di cult)
52
ffi
Leveraging intractability: guiding scienti c inquiry
53
fi
Summary
54
Credits
— John Nash
56
fi
fi