CS 637 - Lecture 24: Random Walks, Hitting Times, and Basic Properties

Lecture Roadmap

Topics for Today

The definition of a simple random walk on a graph and its transition matrix.
The expected time to reach a vertex or a set of vertices.
Stationary distribution, return times, and reversibility.
Some interesting qualitative properties: periodicity, convergence, and commute behavior.
Concrete calculations on complete graphs, cycles, and stars.

The Unifying Theme

A random walk is a process with very simple local rules:

\[ \text{from the current vertex, choose a neighbor uniformly at random.} \]

But the quantities it creates are global. Expected hitting times remember the geometry of the graph, stationary distributions remember the degree sequence, and long-run behavior distinguishes bipartite from non-bipartite structure.

Notation to Track

\(X_t\): position at time \(t\) \(P(u,v)\): one-step transition probability \(\tau_v\): hitting time of vertex \(v\) \(\tau_A\): hitting time of a set \(A\) \(H_{uv}=\mathbb{E}_u[\tau_v]\) \(\pi\): stationary distribution \(m=|E(G)|\) \(\deg(v)\): degree of \(v\)

Setting

Throughout, we usually work with a finite connected undirected graph \(G=(V,E)\). Connectivity ensures that every vertex is reachable from every other vertex, so the walk can explore the whole graph.

When we later mention convergence to stationarity, we will also point out the bipartite obstruction. That issue is not a technical nuisance; it is one of the most important structural features of random walks on graphs.

Simple Random Walk on a Graph

Definition

A simple random walk on \(G\) is a sequence of random vertices \(X_0, X_1, X_2, \dots\) such that if \(X_t=u\), then \(X_{t+1}\) is chosen uniformly from the neighbors of \(u\).

\[ P(u,v)= \begin{cases} \dfrac{1}{\deg(u)}, & \text{if } uv \in E(G), \\ 0, & \text{otherwise}. \end{cases} \]

Markov Property

The next position depends only on the current position, not on the earlier history:

\[ \Pr(X_{t+1}=v \mid X_t=u, X_{t-1}, \dots, X_0)=P(u,v). \]

So a random walk on a graph is a finite-state Markov chain.

Transition Matrix View

If the vertices are indexed as \(v_1,\dots,v_n\), the matrix \(P\) with entries \(P(v_i,v_j)\) is the transition matrix of the walk.

The \(t\)-step probabilities are given by matrix powers:

\[ \Pr_u(X_t=v) = (P^t)_{uv}. \]

This is why linear algebra becomes unavoidable once we start asking long-run questions.

Three Standard Examples

Graph	Walk Rule	Comment
\(K_n\)	Move to any other vertex with probability \(1/(n-1)\)	Everything is perfectly symmetric.
\(C_n\)	Move left or right with probability \(1/2\)	Behaves like a walk on a circle.
Star \(K_{1,d}\)	Leaves must jump to the center	Shows strong degree imbalance.

Connected Graphs Give Irreducible Walks

If \(G\) is connected and \(u,v \in V(G)\), then there is a path from \(u\) to \(v\). Following that path has positive probability, because every step along the path has positive probability. Hence for some \(t\),

\[ (P^t)_{uv} > 0. \]

This is exactly the irreducibility condition for the Markov chain.

Time to Reach Vertices: Hitting Times

Definitions

For a vertex \(v\), the hitting time of \(v\) is

\[ \tau_v = \min \{ t \ge 0 : X_t = v \}. \]

More generally, for a set \(A \subseteq V(G)\),

\[ \tau_A = \min \{ t \ge 0 : X_t \in A \}. \]

The expected hitting time from \(u\) to \(v\) is denoted by

\[ H_{uv} = \mathbb{E}_u[\tau_v]. \]

First-Step Equations

Fix a target vertex \(v\). Then

\[ H_{vv}=0, \qquad H_{uv} = 1 + \frac{1}{\deg(u)} \sum_{x \sim u} H_{xv} \quad \text{for } u \neq v. \]

So expected hitting times are obtained by solving a linear system.

Proof

If we start at \(v\), the hitting time is already \(0\), so \(H_{vv}=0\).

Now suppose \(u \neq v\). The walk takes one step immediately. From \(u\), each neighbor \(x\) is chosen with probability \(1/\deg(u)\). After that first step, the remaining expected time to hit \(v\) is \(H_{xv}\). Conditioning on the first step gives

\[ H_{uv} = 1 + \sum_{x \sim u} \frac{1}{\deg(u)} H_{xv} = 1 + \frac{1}{\deg(u)} \sum_{x \sim u} H_{xv}. \]

This is the required recurrence.

Example: Complete Graph \(K_n\)

Fix a target vertex \(v\). By symmetry, every other vertex has the same hitting time \(h\) to \(v\).

From a non-target vertex, with probability \(1/(n-1)\) we hit \(v\) in one step, and with probability \((n-2)/(n-1)\) we move to another non-target vertex and start over. Hence

\[ h = 1 + \frac{n-2}{n-1} h. \]

Solving gives

\[ H_{uv}=n-1 \qquad (u \neq v). \]

A Useful General Principle

Whenever the graph has strong symmetry, the system for hitting times collapses to only a few unknowns. That is exactly what happened in \(K_n\), and later it will happen again for cycles and stars.

On arbitrary graphs the system is larger, but the same first-step idea still does all the conceptual work.

Stationary Distribution and Return Time

Stationary Distribution of the Simple Random Walk

For a connected undirected graph with \(m\) edges, the distribution

\[ \pi(v)=\frac{\deg(v)}{2m} \]

is stationary for the simple random walk.

Proof by Detailed Balance

If \(uv \in E(G)\), then

\[ \pi(u)P(u,v) = \frac{\deg(u)}{2m} \cdot \frac{1}{\deg(u)} = \frac{1}{2m} = \frac{\deg(v)}{2m} \cdot \frac{1}{\deg(v)} = \pi(v)P(v,u). \]

If \(uv \notin E(G)\), both sides are \(0\). Therefore the detailed balance equations hold for every pair \(u,v\):

\[ \pi(u)P(u,v)=\pi(v)P(v,u). \]

Summing over \(u\) gives

\[ \sum_{u \in V(G)} \pi(u)P(u,v)=\pi(v), \]

so \(\pi\) is stationary.

Immediate Consequences

If \(G\) is \(d\)-regular, then \(\pi\) is uniform.
Higher-degree vertices are seen more often in the long run.
Under stationarity, each oriented edge is used with probability \(1/(2m)\) at each step.

Reversibility

The detailed balance identity means that the chain looks statistically the same when time is reversed under the stationary distribution. This is called reversibility.

For random walks on undirected graphs, reversibility is the bridge from probability to electrical network ideas.

Expected Return Time

Let

\[ \tau_v^+ = \min \{ t \ge 1 : X_t = v \}. \]

Then for the simple random walk on a connected graph,

\[ \mathbb{E}_v[\tau_v^+] = \frac{1}{\pi(v)} = \frac{2m}{\deg(v)}. \]

Proof Idea

In stationarity, the long-run fraction of time spent at \(v\) is \(\pi(v)\). So if we watch the walk for a very long time, the mean gap between successive visits to \(v\) must be the reciprocal frequency \(1/\pi(v)\).

For the simple random walk, \(\pi(v)=\deg(v)/(2m)\), so the average return time is \(2m/\deg(v)\). Thus highly connected vertices are revisited more quickly on average.

Some Interesting Properties of Random Walks

A Short List Worth Remembering

Property	Statement	Meaning
Irreducibility	Connected graph \(\Longrightarrow\) every vertex can reach every other with positive probability	The walk can explore the whole graph.
Reversibility	\(\pi(u)P(u,v)=\pi(v)P(v,u)\)	Time reversal is tractable.
Stationarity	\(\pi(v)=\deg(v)/(2m)\)	Degree controls long-run mass.
Return time	\(\mathbb{E}_v[\tau_v^+]=2m/\deg(v)\)	Large-degree vertices are revisited faster.
Commute time	\(H_{uv}+H_{vu}\) is symmetric in \(u,v\)	Round-trip travel has a hidden symmetry.

Periodicity and Bipartite Obstruction

If \(G\) is bipartite with parts \(A\) and \(B\), then every step flips the walk from one part to the other. So starting in \(A\), all even times lie in \(A\) and all odd times lie in \(B\).

Therefore the walk cannot converge pointwise to a stationary distribution unless the graph has some laziness built in. The parity oscillation never disappears.

Non-Bipartite Graphs Converge

If \(G\) is finite, connected, and non-bipartite, then the simple random walk is irreducible and aperiodic, and hence

\[ \Pr_u(X_t=v) \to \pi(v) \qquad \text{as } t \to \infty. \]

This is the basic convergence theorem for finite Markov chains.

Every Vertex Is Visited Infinitely Often

Because the chain is finite and connected, the simple random walk is positive recurrent. In particular, every vertex is visited infinitely many times with probability \(1\).

So on a finite connected graph, a vertex may take a while to appear, but it is never lost forever.

Commute Time

The commute time between \(u\) and \(v\) is

\[ C_{uv}=H_{uv}+H_{vu}. \]

It is automatically symmetric, and in fact a deep theorem says

\[ C_{uv}=2m \, R_{\mathrm{eff}}(u,v), \]

where \(R_{\mathrm{eff}}(u,v)\) is the effective electrical resistance between \(u\) and \(v\). This is one of the most beautiful connections between graph theory, probability, and physics.

Worked Examples

Cycle \(C_n\): Hitting a Fixed Vertex

Label the cycle \(0,1,\dots,n-1\), and let the target be \(0\). Write \(h_k=H_{k0}\). Then

\[ h_0=0, \qquad h_k = 1 + \frac{h_{k-1}+h_{k+1}}{2} \quad (1 \le k \le n-1), \]

where the indices are read modulo \(n\). Rearranging gives

\[ h_{k+1}-2h_k+h_{k-1}=-2. \]

The quadratic function \(h_k=k(n-k)\) satisfies this recurrence and the boundary conditions \(h_0=h_n=0\). Therefore

\[ H_{k0}=k(n-k). \]

The expected time is largest when the starting point is opposite the target, giving a worst case of about \(n^2/4\).

Star \(K_{1,d}\): Center to a Chosen Leaf

Let \(c\) be the center and \(t\) a fixed target leaf. Write \(h_c=H_{ct}\), and let \(h_o\) be the hitting time from any other leaf to \(t\).

From any non-target leaf, the next step must go to the center, so

\[ h_o=1+h_c. \]

From the center, with probability \(1/d\) we go directly to \(t\), and with probability \((d-1)/d\) we go to another leaf. Hence

\[ h_c = 1 + \frac{d-1}{d} h_o. \]

Solving gives

\[ H_{ct}=2d-1, \qquad H_{ot}=2d. \]

Comparison Table

Graph	Stationary Distribution	Typical Hitting Fact
\(K_n\)	Uniform	\(H_{uv}=n-1\) for \(u \neq v\)
\(C_n\)	Uniform	\(H_{k0}=k(n-k)\)
Star \(K_{1,d}\)	\(\pi(c)=1/2,\ \pi(\text{leaf})=1/(2d)\)	\(H_{ct}=2d-1\)

What the Examples Teach

Symmetry can collapse a large linear system to a one-variable equation.
Degree imbalance distorts stationary behavior sharply, as the star graph shows.
Even on a very regular graph like a cycle, hitting times grow quadratically with distance scale.

Takeaways

Main Results

A simple random walk chooses a uniformly random neighbor at each step.
Hitting times satisfy first-step linear equations.
For an undirected graph, the stationary distribution is \(\pi(v)=\deg(v)/(2m)\).
The expected return time to \(v\) is \(2m/\deg(v)\).
Bipartite graphs create parity oscillation, while connected non-bipartite graphs converge to stationarity.
Commute time reveals a deep connection with effective resistance.

Proof Patterns to Remember

Condition on the first step to derive equations for expectations.
Use symmetry to reduce many unknowns to a small number.
Verify stationarity through detailed balance rather than direct summation whenever possible.
Look at parity in bipartite graphs before making any convergence claim.