Strings and Alphabets
Fundamental Definitions
Def 1Alphabet (Σ)
A finite, non-empty set of symbols. Also called a vocabulary. Each element
is called a symbol or letter.
Def 2String (Word)
A finite sequence of symbols from an alphabet Σ. The length of a string \(
w \), denoted \( |w| \), is the number of symbols in it.
Def 3Empty String (ε)
The unique string of length zero, containing no symbols. We have \(
|\varepsilon| = 0 \). Also denoted as \( \lambda \) in some texts.
Def 4Kleene Star (Σ*)
The set of all strings over alphabet Σ, including the empty string ε.
Formally: \( \Sigma^* = \bigcup_{i=0}^{\infty} \Sigma^i \) where \( \Sigma^0 = \{\varepsilon\}
\) and \( \Sigma^i \) is the set of all strings of length \( i \).
Def 5Kleene Plus (Σ⁺)
The set of all non-empty strings over Σ: \( \Sigma^+ = \Sigma^* \setminus
\{\varepsilon\} = \bigcup_{i=1}^{\infty} \Sigma^i \)
Def 6Concatenation
If \( u = a_1a_2...a_m \) and \( v = b_1b_2...b_n \), then \( uv =
a_1a_2...a_m b_1b_2...b_n \). Properties: \( |uv| = |u| + |v| \), \( \varepsilon w =
w\varepsilon = w \) for all \( w \).
Def 7Language
A set of strings over an alphabet Σ. Formally, a language \( L \) is any
subset of \( \Sigma^* \), i.e., \( L \subseteq \Sigma^* \). A language can be finite or
infinite.
String Indexing and Notation
Def 8String Indexing w[i]
For a string \( w = a_1 a_2 \cdots a_n \) of length \( n \), we define:
- \( w[i] = a_i \) — the \( i \)-th symbol of \( w \), where \( 1 \leq i \leq n \)
- Convention: indices are typically 1-indexed (first symbol is \( w[1] \))
- Some texts use 0-indexing where first symbol is \( w[0] \)
Def 9Substring Notation w[i..j]
For a string \( w = a_1 a_2 \cdots a_n \):
- \( w[i..j] = a_i a_{i+1} \cdots a_j \) for \( 1 \leq i \leq j \leq n \)
- \( w[1..i] \) = prefix of \( w \) of length \( i \)
- \( w[i..n] \) = suffix of \( w \) starting at position \( i \)
- If \( i > j \), then \( w[i..j] = \varepsilon \) (empty string)
Def 10Substring
A string \( v \) is a substring of \( w \) if \( w = xvy
\) for some strings \( x, y \in \Sigma^* \).
- A substring consists of contiguous symbols from \( w \)
- The empty string \( \varepsilon \) is a substring of every string
- Every string is a substring of itself
- Number of substrings of \( w \) with length \( |w| = n \): at most \( \frac{n(n+1)}{2} + 1 \)
Def 11Subsequence
A string \( v = b_1 b_2 \cdots b_k \) is a subsequence of
\( w = a_1 a_2 \cdots a_n \) if there exist indices \( 1 \leq i_1 < i_2 < \cdots < i_k \leq n \)
such that \( b_j=a_{i_j} \) for all \( j \).
- Subsequences need not be contiguous (can skip symbols)
- Every substring is a subsequence, but not vice versa
- Total number of subsequences of a string of length \( n \): \( 2^n \)
Def 12Prefix
A string \( u \) is a prefix of \( w \) if \( w = uv \)
for some string \( v \in \Sigma^* \).
- Equivalently, \( u = w[1..k] \) for some \( 0 \leq k \leq |w| \)
- \( \varepsilon \) is a prefix of every string (the empty prefix)
- \( w \) is a prefix of itself (the trivial prefix)
- A prefix \( u \) is proper if \( u \neq w \)
Def 13Suffix
A string \( v \) is a suffix of \( w \) if \( w = uv \)
for some string \( u \in \Sigma^* \).
- Equivalently, \( v = w[k..|w|] \) for some \( 1 \leq k \leq |w|+1 \)
- \( \varepsilon \) is a suffix of every string
- A suffix is proper if it is not equal to \( w \)
Def 14Reverse (wᴿ)
The reverse of a string \( w = a_1 a_2 \cdots a_n \) is:
\[ w^R = a_n a_{n-1} \cdots a_2 a_1 \]
- \( \varepsilon^R = \varepsilon \)
- \( (uv)^R = v^R u^R \)
- \( (w^R)^R = w \)
Recursive Definition of Strings
Def 15Recursive Definition of Σ*
The set \( \Sigma^* \) can be defined recursively:
- Base Case: \( \varepsilon \in \Sigma^* \)
- Recursive Case: If \( w \in \Sigma^* \) and \( a \in \Sigma \), then \( wa \in \Sigma^* \)
- Closure: Nothing else is in \( \Sigma^* \)
Def 16Recursive Definition of String
Length
The length function \( |w| \) is defined recursively:
- Base Case: \( |\varepsilon| = 0 \)
- Recursive Case: \( |wa| = |w| + 1 \) for \( w \in \Sigma^*, a \in \Sigma \)
Def 17Recursive Definition of
Concatenation
Concatenation of strings \( u \) and \( v \) is defined recursively:
- Base Case: \( u \cdot \varepsilon = u \)
- Recursive Case: \( u \cdot (va) = (u \cdot v)a \) for \( a \in \Sigma \)
Def 18Recursive Definition of
Reverse
The reverse \( w^R \) is defined recursively:
- Base Case: \( \varepsilon^R = \varepsilon \)
- Recursive Case: \( (wa)^R = a(w^R) \) for \( w \in \Sigma^*, a \in \Sigma \)
Examples: Indexing, Substrings, and Subsequences
Ex 1String Indexing
Let \( w = \text{abcde} \) (length 5). Then:
- \( w[1] = a \), \( w[2] = b \), \( w[3] = c \), \( w[4] = d \), \( w[5] = e \)
- \( w[1..3] = \text{abc} \) (first 3 characters)
- \( w[2..4] = \text{bcd} \) (characters 2 through 4)
- \( w[3..5] = \text{cde} \) (last 3 characters)
Ex 2Prefix Examples
For \( w = \text{compute} \):
- All prefixes: \( \varepsilon \), c, co, com, comp, compu, comput, compute
- Proper prefixes: \( \varepsilon \), c, co, com, comp, compu, comput
- \( w[1..4] = \text{comp} \) is a prefix
- A string of length \( n \) has exactly \( n+1 \) prefixes
Ex 3Suffix Examples
For \( w = \text{string} \):
- All suffixes: string, tring, ring, ing, ng, g, \( \varepsilon \)
- \( w[4..6] = \text{ing} \) is a suffix
- Note: "tri" is NOT a suffix (not at the end)
Ex 4Substring vs Subsequence
For \( w = \text{algorithm} \):
- Substrings: "algo", "rith", "gor", "hm" ✓ (contiguous)
- NOT substrings: "arm", "lot" ✗ (not contiguous in \( w \))
- Subsequences: "arm" ✓ (a...r......m), "aim" ✓ (a......i..m)
- "ago" is a subsequence: \( \underline{a}l\underline{g}\underline{o}rithm \)
Ex 5Counting Subsequences
For \( w = \text{abc} \) (length 3):
- All \( 2^3 = 8 \) subsequences: \( \varepsilon \), a, b, c, ab, ac, bc, abc
- Each subsequence corresponds to a subset of positions
- Note: "ba" is NOT a subsequence (order must be preserved)
Ex 6Reverse Examples
- \( (\text{hello})^R = \text{olleh} \)
- \( (\text{abba})^R = \text{abba} \) — a palindrome!
- \( (abc \cdot def)^R = (def)^R \cdot (abc)^R = fed \cdot cba = fedcba \)
Ex 7Recursive Length Computation
Compute \( |\text{cat}| \) recursively:
- \( |\text{cat}| = |\text{ca}| + 1 \) (since cat = ca·t)
- \( |\text{ca}| = |\text{c}| + 1 \)
- \( |\text{c}| = |\varepsilon| + 1 = 0 + 1 = 1 \)
- Therefore: \( |\text{c}| = 1 \), \( |\text{ca}| = 2 \), \( |\text{cat}| = 3 \) ✓
Ex 8Recursive Reverse Computation
Compute \( (\text{abc})^R \) recursively:
- \( (\text{abc})^R = c \cdot (\text{ab})^R \)
- \( (\text{ab})^R = b \cdot (\text{a})^R \)
- \( (\text{a})^R = a \cdot (\varepsilon)^R = a \cdot \varepsilon = a \)
- Therefore: \( (\text{ab})^R = ba \), \( (\text{abc})^R = cba \) ✓
Key Insight
Why Recursive Definitions Matter:
- They provide a foundation for inductive proofs about strings
- They show how complex strings are built from simpler ones
- They directly translate to recursive algorithms
More Examples of Strings
Ex 9Binary Alphabet
Let \( \Sigma = \{0, 1\} \). Then:
- \( \varepsilon \) is a string (the empty string)
- \( 0, 1, 00, 01, 10, 11, 000, ... \) are all strings in \( \Sigma^* \)
- \( |0110| = 4 \), \( |1| = 1 \)
Ex 10DNA Alphabet
\( \Sigma = \{A, C, G, T\} \) (Adenine, Cytosine, Guanine, Thymine)
- ACGT, GATTACA, AAAA are valid strings
- A DNA sequence like "ATCGATCG" has length 8
Ex 11Concatenation Examples
With \( u = \text{ab} \) and \( v = \text{cd} \):
- \( uv = \text{abcd} \), \( vu = \text{cdab} \)
- \( u\varepsilon = \varepsilon u = \text{ab} \)
- \( u^3 = uuu = \text{ababab} \)
Ex 12Language Examples
Over \( \Sigma = \{a, b\} \):
- \( L_1 = \{a, aa, aaa\} \) — finite language with 3 strings
- \( L_2 = \{a^n : n \geq 0\} = \{\varepsilon, a, aa, aaa, ...\} \) — infinite language
- \( L_3 = \emptyset \) — empty language (contains no strings)
- \( L_4 = \{\varepsilon\} \) — language containing only empty string
Important
Distinguish carefully:
- \( \emptyset \neq \{\varepsilon\} \): The empty set has no elements; \( \{\varepsilon\} \) has one element (the empty string)
- \( |\emptyset| = 0 \) but \( |\{\varepsilon\}| = 1 \)
Exercises
Exercise 1Properties of
Reversal
The reversal \( w^R \) of a string \( w \) is defined recursively as follows:
\[ w^R := \begin{cases} \varepsilon & \text{if } w = \varepsilon \\ x^R \cdot a & \text{if } w =
a \cdot x \text{ (where } a \in \Sigma, x \in \Sigma^*\text{)} \end{cases} \]
Prove the following properties for all strings \( w, x \in \Sigma^* \) and every symbol \( a \in
\Sigma \):
- Double Reversal: Prove that \( (w^R)^R = w \).
- Length Invariance: Prove that \( |w^R| = |w| \).
- Reversal of Concatenation: Prove that \( (w \cdot x)^R = x^R \cdot w^R \).
- Symbol Counting: Prove that \( \#(a, w^R) = \#(a, w) \), where \( \#(a, s) \) denotes the number of occurrences of symbol \( a \) in string \( s \).
Hint: Use mathematical induction on the length of the string \( w \).
Regular Languages
Defining Regular Languages
Def 19Regular Language
A language is regular if it can be recognized by a finite
automaton (DFA or NFA), or equivalently, if it can be described by a regular expression.
Def 20Regular Expression
A compact notation for describing regular languages. Defined recursively:
- \( \emptyset \) is a regex (denotes empty language)
- \( \varepsilon \) is a regex (denotes \( \{\varepsilon\} \))
- For any \( a \in \Sigma \), \( a \) is a regex (denotes \( \{a\} \))
- If \( R, S \) are regexes: \( (R+S) \), \( (RS) \), and \( (R^*) \) are regexes
Notation and Precedence
Regular languages are normally described using a compact notation called regular
expressions, which omit braces around one-string sets, use \( + \) to represent
union instead of \( \cup \), and juxtapose subexpressions to represent concatenation instead of
using an explicit operator \( \cdot \).
By convention, in the absence of parentheses, the operators have the following precedence:
By convention, in the absence of parentheses, the operators have the following precedence:
- The Kleene star (*) operator (highest precedence)
- The (implicit) concatenation operator
- The union (+) operator (lowest precedence)
Def 21Building Regular Languages
The set of regular languages over an alphabet \( \Sigma \) is defined
inductively:
- Base Cases:
- The empty language \( \emptyset \) is regular.
- The language containing only the empty string \( \{ \varepsilon \} \) is regular.
- For each \( a \in \Sigma \), the singleton language \( \{ a \} \) is regular.
- Inductive Step: If \( L_1 \) and \( L_2 \)
are
regular languages, then so are:
- \( L_1 \cup L_2 \) (Union)
- \( L_1 \cdot L_2 \) (Concatenation)
- \( L_1^* \) (Kleene Star)
Ex 13Building a Language Step-by-Step
To build the language \( L = \{0, 11\} \) over \( \Sigma = \{0, 1\} \):
- Base cases: \( \{0\} \) and \( \{1\} \) are regular languages.
- Concatenation: Since \( \{1\} \) is regular, \( \{1\} \cdot \{1\} = \{11\} \) is regular.
- Union: Since \( \{0\} \) and \( \{11\} \) are regular, \( \{0\} \cup \{11\} = \{0, 11\} \) is regular.
Def 22Operations on Regular
Languages
- Union: \( L_1 \cup L_2 = \{w : w \in L_1 \text{ or } w \in L_2\} \)
- Concatenation: \( L_1 L_2 = \{uv : u \in L_1, v \in L_2\} \)
- Kleene Star: \( L^* = \bigcup_{i=0}^{\infty} L^i \) where \( L^0 = \{\varepsilon\} \)
Def 23Closure Properties
Regular languages are closed under: union, intersection, complement, concatenation, Kleene
star,
reversal, and homomorphism.
Regular Expression Examples
Ex 14Basic Regular Expressions
Over \( \Sigma = \{0, 1\} \):
- \( 0^* \) = all strings of zeros: \( \{\varepsilon, 0, 00, 000, ...\} \)
- \( (0+1)^* \) = all binary strings = \( \Sigma^* \)
- \( 0^*1^* \) = zeros followed by ones: \( \{\varepsilon, 0, 1, 00, 01, 11, 001, ...\} \)
Ex 15Pattern Matching
Strings ending in 01: \( (0+1)^*01 \)
Examples: 01, 001, 101, 0101, 1101, ...
Examples: 01, 001, 101, 0101, 1101, ...
Ex 16Even Length Strings
All strings of even length: \( ((0+1)(0+1))^* \)
Examples: \( \varepsilon \), 00, 01, 10, 11, 0000, 0101, ...
Examples: \( \varepsilon \), 00, 01, 10, 11, 0000, 0101, ...
Ex 17Contains Substring
All strings containing "010" as substring: \( (0+1)^*010(0+1)^* \)
Examples: 010, 0010, 0101, 11010111, ...
Examples: 010, 0010, 0101, 11010111, ...
TheoremKleene's Theorem
A language is regular if and only if it can be described by a regular expression.
Equivalently,
a language is regular iff it is accepted by some DFA (or NFA).
Exercises
Exercise 2Divisibility by
5
Construct a regular expression for the language \( L = \{ w \in \{0, 1\}^* : \text{value}(w)
\text{ is divisible by 5} \} \).
Note: Strings are interpreted as binary numbers (e.g., "101" is 5).
Smallest known Regex: \[ (0 + 1(10)^*(0 + 11)(01^*01 + 01^*00(10)^*(0 + 11))^*1)^* \]
Note: Strings are interpreted as binary numbers (e.g., "101" is 5).
Hint: Use the state elimination method on a 5-state DFA where each state represents a remainder modulo 5.
Smallest known Regex: \[ (0 + 1(10)^*(0 + 11)(01^*01 + 01^*00(10)^*(0 + 11))^*1)^* \]
Deterministic Finite Automata (DFA)
Formal Definition
Def 24Deterministic Finite
Automaton
A DFA is a 5-tuple \( M = (Q, \Sigma, \delta, q_0, F) \) where:
- \( Q \) = finite set of states
- \( \Sigma \) = input alphabet
- \( \delta: Q \times \Sigma \to Q \) = transition function
- \( q_0 \in Q \) = start state
- \( F \subseteq Q \) = set of accept (final) states
Def 25Extended Transition
Function
\( \hat{\delta}: Q \times \Sigma^* \to Q \) defined recursively:
- \( \hat{\delta}(q, \varepsilon) = q \)
- \( \hat{\delta}(q, wa) = \delta(\hat{\delta}(q, w), a) \) for \( w \in \Sigma^*, a \in \Sigma \)
Def 26Language Accepted by
DFA
The language of a DFA \( M \) is: \( L(M) = \{w \in \Sigma^* : \hat{\delta}(q_0, w) \in F\}
\)
A string is accepted if processing it from \( q_0 \) ends in an accept state.
A string is accepted if processing it from \( q_0 \) ends in an accept state.
Def 27Dead State (Trap State)
A non-accepting state \( q_d \) from which no accepting state is reachable. All transitions
from
\( q_d \) lead back to \( q_d \).
DFA Examples
Ex 18Strings ending in 1
\( L = \{w \in \{0,1\}^* : w \text{ ends with } 1\} \)
DFA: \( Q = \{q_0, q_1\}, \Sigma = \{0,1\}, q_0 = q_0, F = \{q_1\} \)
Transitions: \( \delta(q_0, 0) = q_0, \delta(q_0, 1) = q_1, \delta(q_1, 0) = q_0, \delta(q_1, 1) = q_1 \)
DFA: \( Q = \{q_0, q_1\}, \Sigma = \{0,1\}, q_0 = q_0, F = \{q_1\} \)
Transitions: \( \delta(q_0, 0) = q_0, \delta(q_0, 1) = q_1, \delta(q_1, 0) = q_0, \delta(q_1, 1) = q_1 \)
Ex 19Even number of 0s
\( L = \{w \in \{0,1\}^* : w \text{ has even number of 0s}\} \)
\( Q = \{q_{even}, q_{odd}\}, F = \{q_{even}\} \)
\( \delta(q_{even}, 0) = q_{odd}, \delta(q_{even}, 1) = q_{even} \)
\( \delta(q_{odd}, 0) = q_{even}, \delta(q_{odd}, 1) = q_{odd} \)
\( Q = \{q_{even}, q_{odd}\}, F = \{q_{even}\} \)
\( \delta(q_{even}, 0) = q_{odd}, \delta(q_{even}, 1) = q_{even} \)
\( \delta(q_{odd}, 0) = q_{even}, \delta(q_{odd}, 1) = q_{odd} \)
Ex 20Divisibility by 3
Binary numbers divisible by 3 (interpreting string as binary number).
\( Q = \{q_0, q_1, q_2\} \) representing remainder mod 3.
Accept state: \( F = \{q_0\} \) (remainder 0)
Key insight: if current remainder is \( r \), reading bit \( b \) gives remainder \( (2r + b) \mod 3 \)
\( Q = \{q_0, q_1, q_2\} \) representing remainder mod 3.
Accept state: \( F = \{q_0\} \) (remainder 0)
Key insight: if current remainder is \( r \), reading bit \( b \) gives remainder \( (2r + b) \mod 3 \)
Ex 21Contains "01" as substring
\( Q = \{q_0, q_1, q_2\} \) where:
- \( q_0 \): haven't seen pattern yet
- \( q_1 \): just saw a 0
- \( q_2 \): have seen "01" (accept state, \( F = \{q_2\} \))
Key Property
Determinism: For every state and input symbol, there is exactly one next state.
No
ambiguity in computation.
Non-deterministic Finite Automata (NFA)
Formal Definition
Def 28Non-deterministic Finite
Automaton
An NFA is a 5-tuple \( N = (Q, \Sigma, \delta, q_0, F) \) where:
- \( Q, \Sigma, q_0, F \) are as in DFA
- \( \delta: Q \times (\Sigma \cup \{\varepsilon\}) \to \mathcal{P}(Q) \) — transition function returns a set of states
Def 29ε-Closure
For a state \( q \), the ε-closure is the set of all states reachable from \( q \) via zero
or
more ε-transitions:
\( \varepsilon\text{-closure}(q) = \{p : p \text{ is reachable from } q \text{ using only } \varepsilon\text{-transitions}\} \)
\( \varepsilon\text{-closure}(q) = \{p : p \text{ is reachable from } q \text{ using only } \varepsilon\text{-transitions}\} \)
Def 30NFA Acceptance
An NFA accepts a string \( w \) if there exists at least one computation
path
that ends in an accept state. The language \( L(N) \) is the set of all accepted strings.
Def 31ε-NFA
An NFA that can make transitions without consuming input symbols (ε-transitions). These
allow
"free" moves between states.
NFA Examples
Ex 22Strings ending in 01
NFA with 3 states is simpler than DFA:
- \( q_0 \): loops on 0 and 1, transitions to \( q_1 \) on 0
- \( q_1 \): transitions to \( q_2 \) on 1
- \( q_2 \): accept state
Ex 23Contains "web" or "ebay"
NFA can "guess" which pattern to look for:
- Start state branches (non-deterministically) to search for "web" or "ebay"
- Each branch matches its pattern sequentially
- Either branch reaching accept means the string is accepted
Ex 24ε-NFA Example
Language: strings that are either all 0s or all 1s.
Use ε-transitions from start to branch to "0-only" path or "1-only" path.
Regex: \( 0^* + 1^* \)
Use ε-transitions from start to branch to "0-only" path or "1-only" path.
Regex: \( 0^* + 1^* \)
Ex 25Third symbol from end is 1
\( L = \{w \in \{0,1\}^* : |w| \geq 3 \text{ and 3rd from end is 1}\} \)
NFA: 4 states, non-deterministically guess when to start counting last 3 symbols.
Much simpler than equivalent DFA (which needs 8 states)!
NFA: 4 states, non-deterministically guess when to start counting last 3 symbols.
Much simpler than equivalent DFA (which needs 8 states)!
TheoremSubset Construction
For every NFA \( N \), there exists an equivalent DFA \( D \) with \( L(D) = L(N) \). The
DFA
may have up to \( 2^{|Q|} \) states (exponential blowup in worst case).
Key Insight
NFAs and DFAs recognize exactly the same class of languages (regular languages), but NFAs can be
exponentially more succinct.
Context-Free Languages
Beyond Regular Languages
Limitation
Regular languages cannot count! Languages like \( \{a^n b^n : n \geq 0\} \)
(equal
a's and b's), balanced parentheses, and palindromes are NOT regular.
Def 32Context-Free Grammar
(CFG)
A CFG is a 4-tuple \( G = (V, \Sigma, R, S) \) where:
- \( V \) = finite set of variables (non-terminals)
- \( \Sigma \) = finite set of terminals (alphabet)
- \( R \) = finite set of production rules of form \( A \to \alpha \) where \( A \in V \) and \( \alpha \in (V \cup \Sigma)^* \)
- \( S \in V \) = start variable
Def 33Derivation
A derivation is a sequence of rule applications that transforms the start
symbol into a string of terminals.
We write \( \alpha \Rightarrow \beta \) if \( \beta \) is obtained from \( \alpha \) by one rule application.
\( \alpha \Rightarrow^* \beta \) means zero or more steps.
We write \( \alpha \Rightarrow \beta \) if \( \beta \) is obtained from \( \alpha \) by one rule application.
\( \alpha \Rightarrow^* \beta \) means zero or more steps.
Def 34Context-Free Language
A language \( L \) is context-free if \( L = L(G) \) for some CFG \( G \), where:
\( L(G) = \{w \in \Sigma^* : S \Rightarrow^* w\} \)
\( L(G) = \{w \in \Sigma^* : S \Rightarrow^* w\} \)
Def 35Pushdown Automaton
(PDA)
A PDA is like an NFA but with an additional stack memory. It can push and
pop
symbols from the stack. PDAs recognize exactly the context-free languages.
CFG Examples
Ex 26Equal a's and b's
\( L = \{a^n b^n : n \geq 0\} \)
Grammar: \( S \to aSb \mid \varepsilon \)
Derivation of \( aabb \): \( S \Rightarrow aSb \Rightarrow aaSbb \Rightarrow aabb \)
Grammar: \( S \to aSb \mid \varepsilon \)
Derivation of \( aabb \): \( S \Rightarrow aSb \Rightarrow aaSbb \Rightarrow aabb \)
Ex 27Balanced Parentheses
Grammar: \( S \to (S) \mid SS \mid \varepsilon \)
Examples: \( \varepsilon, (), (()), ()(), (())(), ... \)
Key: stack keeps track of unmatched opening parens.
Examples: \( \varepsilon, (), (()), ()(), (())(), ... \)
Key: stack keeps track of unmatched opening parens.
Ex 28Palindromes over {a,b}
\( L = \{w : w = w^R\} \) where \( w^R \) is reverse of \( w \)
Grammar: \( S \to aSa \mid bSb \mid a \mid b \mid \varepsilon \)
Examples: \( \varepsilon, a, b, aa, bb, aba, bab, abba, ... \)
Grammar: \( S \to aSa \mid bSb \mid a \mid b \mid \varepsilon \)
Examples: \( \varepsilon, a, b, aa, bb, aba, bab, abba, ... \)
Ex 29Arithmetic Expressions
Grammar for expressions with +, ×, and parentheses:
\( E \to E + T \mid T \)
\( T \to T \times F \mid F \)
\( F \to (E) \mid \text{id} \)
This grammar encodes operator precedence!
\( E \to E + T \mid T \)
\( T \to T \times F \mid F \)
\( F \to (E) \mid \text{id} \)
This grammar encodes operator precedence!
TheoremPumping Lemma for CFLs
If \( L \) is context-free, there exists \( p \) such that for any \( w \in L \) with \( |w|
\geq p \), we can write \( w = uvxyz \) with: (1) \( |vxy| \leq p \), (2) \( |vy| > 0 \),
(3) \(
uv^i xy^i z \in L \) for all \( i \geq 0 \).
Used to prove languages are NOT context-free.
Used to prove languages are NOT context-free.
Hierarchy
Regular ⊂ Context-Free: Every regular language is context-free (convert regex
to
CFG), but not vice versa.
Introduction to Turing Machines
The Chomsky Hierarchy
Recursively Enumerable (Turing Machines)
⊃
Context-Free Languages (PDAs, CFGs)
⊃
Regular Languages (DFA, NFA, Regex)
⊃
Finite Languages
Motivation for Turing Machines
Why TMs?Limitations of Finite
Automata &
PDAs
- DFA/NFA: Finite memory, cannot count arbitrarily
- PDA: Stack gives LIFO memory, but cannot match patterns like \( \{a^n b^n c^n\} \)
- Need: A model with unlimited memory and full read/write access
Def 36Turing Machine
(Informal)
A Turing Machine consists of:
- An infinite tape divided into cells (memory)
- A read/write head that moves left or right
- A finite state control
- Transition rules based on current state and symbol under head
Def 37Turing Machine (Formal)
A TM is a 7-tuple \( M = (Q, \Sigma, \Gamma, \delta, q_0, q_{accept}, q_{reject}) \) where:
- \( Q \) = finite set of states
- \( \Sigma \) = input alphabet (not containing blank ⊔)
- \( \Gamma \) = tape alphabet, \( \Sigma \cup \{⊔\} \subseteq \Gamma \)
- \( \delta: Q \times \Gamma \to Q \times \Gamma \times \{L, R\} \) = transition function
- \( q_0 \) = start state, \( q_{accept} \) = accept state, \( q_{reject} \) = reject state
Def 38Configuration
A snapshot of computation: current state, tape contents, and head position. Written as \(
uqv \)
meaning state \( q \), head on first symbol of \( v \), tape contains \( uv \).
Def 39Decidable vs.
Recognizable
- Decidable: TM always halts (accepts or rejects) on every input
- Recognizable: TM accepts strings in language, but may loop forever on others
Turing Machine Examples
Ex 30Language \( \{0^n 1^n : n \geq 0\}
\)
Algorithm: Cross off matching 0s and 1s repeatedly.
- Scan right, mark a 0 as X
- Continue right to find a 1, mark it as Y
- Return left to find next unmarked 0
- Repeat until all matched or mismatch found
Ex 31Language \( \{a^n b^n c^n : n \geq 0\}
\)
This is NOT context-free, but TM can recognize it:
- Mark an 'a', scan to mark a 'b', scan to mark a 'c'
- Return to beginning, repeat
- Accept if all symbols marked simultaneously
Ex 32Binary Addition
Input: two binary numbers separated by #.
TM can add them bit by bit, carrying as needed.
Uses tape for intermediate results.
TM can add them bit by bit, carrying as needed.
Uses tape for intermediate results.
Ex 33Palindrome Checker
For \( w = w^R \) over any alphabet:
- Read and erase first symbol, remember it
- Move to end, compare with last symbol
- Erase if match, reject if mismatch
- Repeat until tape is empty (accept)
ThesisChurch-Turing Thesis
Any function that can be computed by an algorithm can be computed by a Turing Machine. TMs
capture the intuitive notion of "effective computability."
Undecidability
The Halting Problem: There is no TM that can decide, for all TM-input pairs,
whether the TM will halt. This was Turing's profound negative answer to Hilbert's
Entscheidungsproblem!
🎯 Summary: The Computational Hierarchy
Finite Automata: Recognize regular languages (no memory beyond states)
Pushdown Automata: Recognize CFLs (stack memory)
Turing Machines: Recognize recursively enumerable languages (unlimited tape
memory)
Key Result: Some languages are not recognizable by ANY TM!