From Decidability to Complexity
Early computer science (Turing, Church) focused on Decidability: Can a problem be solved by a machine at all? However, as computers became real, a new question emerged: Can it be solved efficiently?
The Complexity Shift
In the 1960s and 70s, researchers like Hartmanis, Stearns, Cook, and Karp began classifying problems not just by "computability" but by the "resources" required (Time and Space).
- Turing Machines: Provided the foundation, but standard TMs were too slow for some tasks.
- The Question: What is the limit of "reasonable" computation?
The Cobham-Edmonds Thesis
Computational efficiency is often equated with Polynomial Time. A problem is considered "tractable" or "efficiently solvable" if it can be solved in \(O(n^k)\) time for some constant \(k\).
The Class P: Deterministic Efficiency
Formal Definition of P
The class P (Polynomial Time) is the set of all languages that are decidable in polynomial time on a deterministic single-tape Turing machine.
\[ P = \bigcup_{k \ge 1} \text{TIME}(n^k) \]A language \(L \in P\) if there exists a deterministic TM \(M\) such that for any input \(w\) of length \(n\):
- \(M\) decides \(L\).
- \(M\) runs in \(O(n^k)\) steps for some constant \(k\).
Properties of P
- Robustness: The class P is invariant across different models of deterministic computation (Multi-tape TMs, RAM machines, etc.). Changing the model only changes the polynomial degree.
- Closure: P is closed under complement, union, and intersection.
Examples in P
- Path Finding (Is there a path from \(s\) to \(t\) in a graph \(G\)?).
- Greatest Common Divisor (Euclidean Algorithm).
- Primality Testing (AKS Primality Test - proven to be in P in 2002).
The Class NP: Nondeterministic Power
Formal Definition of NP
The class NP (Nondeterministic Polynomial Time) is the set of all languages that are decidable in polynomial time on a nondeterministic Turing machine.
\[ NP = \bigcup_{k \ge 1} \text{NTIME}(n^k) \]A language \(L \in NP\) if there exists a nondeterministic TM \(N\) such that:
- \(N\) decides \(L\).
- For every \(w \in L\), there exists at least one computation path of length \(O(n^k)\) that results in acceptance.
- For every \(w \notin L\), all possible computation paths reject.
NTM as "Guessing"
Think of an NTM as having two stages:
- Nondeterministic stage: The machine "guesses" a potential solution (e.g., a path, a certificate).
- Deterministic stage: The machine "verifies" if the guess is correct in polynomial time.
The Prover-Verifier Gap
The Verifier Definition
A language \(L\) is in NP if and only if it has a Polynomial-Time Verifier.
A verifier \(V\) for \(L\) is a deterministic TM where:
\[ L = \{ w \mid \exists c, V(w, c) \text{ accepts} \} \]- \(w\) is the input string.
- \(c\) is a certificate (or witness) whose length is polynomial in \(|w|\).
- \(V\) must run in time polynomial in \(|w|\).
The Role of the Prover
We imagine two entities:
- The Prover: An all-powerful entity that provides the certificate \(c\). It doesn't care how long it takes to find \(c\).
- The Verifier: A resource-constrained entity (Polynomial time) that checks if \(c\) is a valid proof for \(w\).
The Gap: It is much easier to verify a solution than it is to find one. This is the heart of the P vs NP mystery.
NP Examples and NTM Designs
1. Boolean Satisfiability (SAT)
Problem: Given a boolean formula \(\phi\) with variables \(x_1, \dots, x_m\), is there an assignment of truth values to variables that makes \(\phi\) evaluate to TRUE?
NTM Design Strategy:
- Guessing Phase:
- The NTM scans the input string to determine the number of variables \(m\).
- It nondeterministically writes a string \(s \in \{0, 1\}^m\) on the tape. Each bit represents a truth assignment (0 for False, 1 for True).
- This step takes \(O(m)\) time.
- Verification Phase:
- The machine evaluates the formula \(\phi\) by substituting the guessed values from \(s\) into the variables.
- Evaluation of a boolean formula can be done in time polynomial in the length of the formula (e.g., using a stack or recursive parsing).
- Acceptance:
- If the evaluation results in TRUE (1), the NTM enters the accept state.
- If the evaluation results in FALSE (0), this branch of computation rejects.
Since the guess is polynomial in size and verification is polynomial in time, SAT \(\in\) NP.
2. CLIQUE Problem
Problem: Given a graph \(G=(V,E)\) and an integer \(k\), does \(G\) contain a clique (complete subgraph) of size \(k\)?
NTM Design Strategy:
- Guessing Phase:
- The NTM nondeterministically selects a subset of vertices \(V' \subseteq V\).
- It can do this by scanning the list of vertices and for each vertex, nondeterministically choosing to mark it or not.
- This takes \(O(|V|)\) time.
- Verification Phase:
- Step 1: Count the size of \(V'\). If \(|V'| \neq k\), reject.
- Step 2: Check if \(V'\) forms a clique. Loop through every pair of distinct vertices \(u, v \in V'\).
- For each pair, scan the edge list \(E\) to verify that \((u,v) \in E\).
- Checking all pairs takes \(O(k^2 \cdot |E|)\) time, which is polynomial.
- Acceptance:
- If the subset size is correct AND all edges exist, accept.
3. SUBSET-SUM
Problem: Given a set of integers \(S = \{x_1, \dots, x_n\}\) and a target \(t\), is there a subset \(S' \subseteq S\) that sums to exactly \(t\)?
NTM Design Strategy:
- Guessing Phase:
- The NTM scans the input set \(S\).
- For each number \(x_i\), it nondeterministically chooses to either "select" it or "ignore" it.
- Effectively, it guesses a binary vector of length \(n\). Time: \(O(n)\).
- Verification Phase:
- The machine computes the sum of all selected numbers.
- It adds them up one by one. Addition of standard binary integers is efficient.
- Finally, it compares the computed sum to the target \(t\).
- Acceptance:
- If the sum equals \(t\), accept. Otherwise, reject.