The Challenge: Recognizing $\{a^n b^n c^n\}$
Problem Statement
We want to design a Turing Machine that decides the language:
\[ L = \{ a^n b^n c^n \mid n \ge 1 \} \]Where \(\Sigma = \{a, b, c\}\). An input string is in \(L\) if and only if it consists of \(n\) 'a's, followed by \(n\) 'b's, followed by \(n\) 'c's.
Time Complexity Definition
Let \(M\) be a deterministic Turing machine that halts on all inputs.
The time complexity (or running time) of \(M\) is the function \(f: \mathbb{N} \rightarrow \mathbb{N}\), where \(f(n)\) is the maximum number of steps that \(M\) uses on any input of length \(n\).
If \(f(n)\) is the running time of \(M\), we say that \(M\) runs in time \(f(n)\) and that \(M\) is an \(O(f(n))\) time Turing machine.
Why this Example?
This language is a classic example of something that is Context-Sensitive but not Context-Free. While a Pushdown Automaton (PDA) can recognize \(\{a^n b^n\}\) using its stack, it cannot handle three balanced distinct groups like \(\{a^n b^n c^n\}\).
Moreover, implementing this on different variants of Turing Machines reveals striking differences in efficiency, highlighting the power of additional tapes and heads.
Standard Single-Tape Turing Machine
Design Strategy
On a single tape, we must repeatedly scan back and forth across the input to match symbols one by one.
- Mark first unmarked 'a': Scan right to find the first unmarked \(a\). Mark it (e.g., replace with \(X\)).
- Match with 'b': Scan right past any \(a\)'s and marked \(b\)'s to find the first unmarked \(b\). Mark it (replace with \(Y\)). If no \(b\) is found before a \(c\) or blank, reject.
- Match with 'c': Scan right past any \(b\)'s and marked \(c\)'s to find the first unmarked \(c\). Mark it (replace with \(Z\)). If no \(c\) is found before blank, reject.
- Return to start: Scan left all the way to the beginning of the tape.
- Repeat: Repeat steps 1-4 until no unmarked \(a\)'s are left.
- Verify: Ensure no extra unmarked \(b\)'s or \(c\)'s remain. If clean, accept.
Complexity Analysis
Let \(n\) be the length of the input string \(w\).
- In each complete round, we mark one \(a\), one \(b\), and one \(c\).
- Therefore, there are exactly \(n/3\) rounds (if \(w \in L\)).
- Round 1: Scan requires moving across the string ~ \(n\) steps and returning ~ \(n\) steps. Total \(\approx 2n\) steps.
- Round \(i\): Still requires scanning across the relevant portion of the tape. Even as symbols are marked, we traverse them. Cost is proportional to \(n\).
- Total Time: Sum of costs for \(n/3\) rounds.
Thus, a single-tape TM decides \(L\) in \(O(n^2)\) time.
Two-Tape Turing Machine
Design Strategy
With an extra tape, we can copy information to avoid re-scanning.
- Copy 'a's: Scan Tape 1. Copy all \(a\)'s onto Tape 2. When a \(b\) is encountered on Tape 1, stop copying.
- Rewind Tape 2: Move Tape 2 head back to the beginning of the copied \(a\)'s.
- Match 'b's: Continue scanning \(b\)'s on Tape 1 while simultaneously scanning \(a\)'s on Tape 2. For every \(b\) on Tape 1, move right on Tape 2. If Tape 1 sees a \(c\) or Tape 2 runs out of \(a\)'s prematurely (or vice versa), reject.
- Rewind Tape 2 again: Move Tape 2 head back to the beginning of the \(a\)'s. (Alternatively, if we can read backwards, just scan left).
- Match 'c's: Continue scanning \(c\)'s on Tape 1 while re-scanning \(a\)'s on Tape 2. Verify counts match.
- Verify Completion: Ensure both tapes finish reading their respective segments simultaneously.
Complexity Analysis
Let's count the steps for input of length \(n\):
- Phase 1 (Copy \(a\)'s): Scan \(a\)'s on Tape 1. \(O(n)\) steps.
- Phase 2 (Match \(b\)'s): Scan \(b\)'s on Tape 1 and \(a\)'s on Tape 2. \(O(n)\) steps. Tape 2 rewind takes \(O(n)\).
- Phase 3 (Match \(c\)'s): Scan \(c\)'s on Tape 1 and \(a\)'s on Tape 2. \(O(n)\) steps. Tape 2 rewind takes \(O(n)\).
We never re-scan the input on Tape 1. We verify each segment essentially once. The rewind operations are linear.
\[ \text{Total Time} = O(n) + O(n) + O(n) = O(n) \]The 2-tape TM is significantly faster, running in linear time.
Two-Head Turing Machine
Design Strategy
A 2-head machine has one tape, but two independent heads that can move separately.
- Position Heads: Keep Head 1 at the start. Move Head 2 forward to find the first \(b\). This validates the \(a^n\) block and effectively "measures" \(n\) by the distance between Head 1 and Head 2.
- Match 'a' vs 'b': Move both Head 1 and Head 2 right simultaneously step-by-step.
- Head 1 reads \(a\), Head 2 reads \(b\).
- Continue until Head 1 hits the first \(b\) (where Head 2 started).
- At this point, Head 2 must have hit the first \(c\) (or something else if malformed).
- Match 'b' vs 'c': Now Head 1 is at the start of \(b\)'s, and Head 2 is at the start of
\(c\)'s. Move both right simultaneously.
- Head 1 reads \(b\), Head 2 reads \(c\).
- Verify they hit the boundary (first \(c\) for Head 1, blank for Head 2) simultaneously.
Complexity Analysis
Analysis of head movements:
- Find first 'b': Head 2 scans forward. \(O(n)\) steps.
- Compare 'a' and 'b': Both heads move forward \(n/3\) steps. \(O(n)\).
- Compare 'b' and 'c': Both heads move forward \(n/3\) steps. \(O(n)\).
We perform a constant number of linear sweeps across the input.
\[ \text{Total Time} = O(n) \]Unlike the single-tape machine, we don't need to zag-zag back to the beginning for every symbol. The second head acts as a "finger" to keep our place.
Model Comparison Summary
| Model | Time Complexity for \(L\) | Space Used | Mechanism |
|---|---|---|---|
| Single-Tape TM | \(O(n^2)\) | \(O(n)\) | Repeated back-and-forth scanning (Zig-zag) |
| Two-Tape TM | \(O(n)\) | \(O(n)\) | Copy and compare in parallel |
| Two-Head TM | \(O(n)\) | \(O(n)\) | Simultaneous scanning of different segments |
For any Turing Machine to decide a non-trivial language property that depends on the entire input, it must read all \(n\) symbols at least once. Thus, the time complexity can never be less than \(O(n)\), regardless of the number of tapes or heads.
Theorem (Hartmanis & Stearns, 1965)
If a language is decidable in \(T(n)\) time on a \(k\)-tape Turing machine, it is decidable in \(O(T(n)^2)\) time on a single-tape Turing machine.
Our example confirms this relationship: \(O(n)\) on 2-tape vs \(O(n^2)\) on single-tape.
Advanced Complexity Examples
Example 1: \(L = \{ a^n b^{n^2} \mid n \ge 1 \}\)
The Problem: Checking if the number of \(b\)'s is the square of the number of \(a\)'s.
- Single-Tape TM: \(O(N^{1.5})\)
- We iterate through each \(a\) (total \(n\) times).
- For the \(k\)-th \(a\), we mark \(2k-1\) \(b\)'s (since \(n^2 = \sum_{k=1}^n (2k-1)\)).
- Input length \(N \approx n^2\). So \(n \approx \sqrt{N}\).
- In each of the \(n\) passes, we scan the full tape of length \(N\).
- Total time \(\approx n \times N \approx \sqrt{N} \times N = N^{1.5}\).
- Two-Tape TM: \(O(N)\)
- Copy \(a^n\) to Tape 2.
- Use Tape 2 to generate \(n^2\) markers on Tape 3 or compare directly by repeating the scan of Tape 2 \(n\) times while scanning Tape 1.
- Wait! Standard multiplication approach: To scan \(n^2\) items on Tape 1, we can loop through Tape 2 (\(n\) items) exactly \(n\) times (using a third counter or destructive read/restore).
- The time taken is proportional to the number of steps, which is exactly \(N\) (the length of the input).
Example 2: \(L = \{ ww \mid w \in \{0,1\}^* \}\)
The Problem: Checking if the first half of the string is identical to the second half.
| Model | Complexity | Reason |
|---|---|---|
| Single-Tape | \(O(N^2)\) | Must zigzag between the corresponding characters of the two halves. Each character match costs \(O(N)\) movement. |
| Two-Tape | \(O(N)\) | Copy the first half to Tape 2. Rewind. Compare Tape 1 and Tape 2 in a single linear pass. |
Example 3: \(L = \{ a^{2^n} \mid n \ge 0 \}\)
The Problem: Checking if the length of the string is a power of 2.
- Single-Tape TM: \(O(N \log N)\)
- Pass 1: Scan left-to-right, marking every second \(a\). Reject if odd remainder. (Cost \(N\)).
- Pass 2: Return to start and repeat on remaining unmarked \(a\)'s (length \(N/2\)). (Cost \(N/2\) scans, but we traverse the whole tape of length \(N\) to find them). So cost is \(N\).
- Total: We perform \(\log N\) passes, each taking \(O(N)\) time to traverse the tape. Total \(O(N \log N)\).
- Two-Tape TM: \(O(N)\)
- Write \(1\) on Tape 2. Repeatedly double the number of \(1\)'s ($1 \to 2 \to 4 \dots$) until the length matches or exceeds the input on Tape 1.
- Copying/doubling \(k\) items takes \(O(k)\). Total sum \(1 + 2 + 4 + \dots + N = 2N = O(N)\).