Why Single-Cycle Design Doesn't Scale — The Bridge to Pipelining

Lesson 2, Part 5 of 5 — the closing chapter. Parts 1 through 4 designed a tiny MIPS-style ISA, wired it into a complete single-cycle datapath, derived every control signal from the opcode, and traced a real seven-instruction program through that datapath cycle by cycle. It works. This post is about the uncomfortable fact hiding inside that success: the CPU we built is a genuinely bad idea to actually manufacture, and the reason why is something we already have the tools to prove.

The clocking rule, recalled from Part 3

Lesson 1, Part 3 derived a rule that applies to any synchronous digital circuit, not just this CPU:

T_clock ≥ T_register + T_combinational + T_setup

In words: the clock period has to be at least as long as the slowest path a signal can take between one clock edge and the next, plus the setup time the destination flip-flop needs before it can safely latch a new value. That "slowest path" has a name — the critical path — and Part 3 showed that a single 400 ps path sitting among a sea of 200 ps paths forces the entire circuit to run at the 400 ps path's pace. Every other path gets zero credit for finishing early.

A single-cycle CPU is one enormous synchronous circuit: registers (the PC, the register file, the pipeline-free datapath state) on one side, a wall of combinational logic in between, registers again on the other side, all sharing one clock. So the rule applies directly — we just have to work out what the critical path actually is.

Assigning stage delays (illustrative, not measured silicon)

To make this concrete we need numbers. These are illustrative teaching numbers, not measurements off a real chip — real fabrication-process delays depend on transistor sizing, wire capacitance, voltage, and a dozen other variables we haven't touched in this series. But the relative ordering — which stages are slow and which are fast — is realistic and matches the qualitative shape you'd see in any real single-cycle design.

Stage                          Illustrative delay
─────────────────────────────  ──────────────────
PC read (register access)         50 ps
Instruction memory access        150 ps
Register file read               100 ps
Sign-extend unit                   20 ps
ALU operation                    150 ps
Data memory access               200 ps
Register file write (setup)       50 ps
Mux overhead (per mux, ~3 in path) 15 ps each

These aren't arbitrary — they roughly track real relative costs: a memory access (instruction or data) is slower than a register-file access because it involves address decoding across a much larger, denser array; a MUX is cheap because it's just a handful of transistors; sign-extension is nearly free because it's pure wiring (the top 16 bits are just copies of bit 15, no logic gates required).

Walking every instruction's path

Recall the datapath from Part 2. Every instruction starts at PC read and ends with either a register-file write or (for SW/BEQ) no write at all — but every instruction's signal still has to physically reach wherever it's going before the next clock edge, whether or not that path does anything useful.

LW (load word) — the longest path in the whole design:

PC read → Instruction memory → Register read (base address) → mux (ALUSrc)
  → ALU (address calc) → Data memory read → mux (MemToReg) → Register write
   50  +      150       +        100       +      15
  + 150 + 200 + 15 + 50 (setup)
  = 730 ps

LW touches every single functional unit in the datapath: it reads a register, computes an address through the ALU, accesses data memory, and writes a register. There is no instruction in our seven-instruction ISA that does more work than this.

SW (store word) — almost as long, but skips the final register write:

PC read → IMem → RegRead(×2) → mux(ALUSrc) → ALU(address) → DMem write
   50   + 150   +    100      +     15      +     150      +    200
  = 665 ps

(No register-file write means no write-setup term — SW never touches the destination register at all.)

ADD / SUB / AND / OR (R-type) — much shorter, because it never touches data memory:

PC read → IMem → RegRead(×2) → ALU(compute) → mux(MemToReg) → RegWrite
   50   + 150   +    100      +     150      +      15       +   50
  = 515 ps

BEQ — shorter still, because it never writes any register:

PC read → IMem → RegRead(×2) → ALU(compare, Zero flag) → mux(PCSrc, branch-target adder in parallel)
   50   + 150   +    100      +          150             +            15
  = 465 ps

The number that decides everything

Instruction    Critical path (illustrative)
────────────   ────────────────────────────
LW                    730 ps
SW                    665 ps
R-type                515 ps
BEQ                   465 ps

Apply the clocking rule. T_clock must be at least as long as the longest of these — LW's 730 ps — because on any given cycle, the datapath has to be ready to correctly execute whatever instruction the program counter happens to be pointing at. The hardware doesn't get to say "give me extra time, this one's a load." It has exactly one clock period, set once at design time, and it must be long enough for the worst case, every single time.

T_clock = 730 ps (≈ 1.37 GHz), for every instruction, no matter which one actually executes.

Now look at what that means for BEQ. Its own critical path is 465 ps. But it still has to wait out the full 730 ps clock period before the next instruction can start — 265 ps of every BEQ cycle is pure waste, hardware sitting idle because the clock hasn't ticked yet even though the answer has been ready for a while. That's not a rounding error: 265 out of 730 ps is roughly 36% of every non-LW cycle thrown away, and for a program that's mostly R-type arithmetic and branches (which most real programs are — memory instructions are typically a minority of dynamic instruction count), that waste compounds across billions of cycles.

This is the entire, complete, first-principles motivation for every CPU design more sophisticated than a single-cycle machine. Not "single-cycle CPUs are old-fashioned" — they waste a measurable, derivable fraction of every clock cycle on every instruction that isn't the single slowest one in the ISA.

Two different fixes, two different families of CPU

Fix 1: let instructions take a variable number of shorter, uniform cycles — the multi-cycle CPU. Instead of one clock period sized for LW, break the datapath into stages roughly matching the delays above (say, ~150 ps each) and let each instruction take as many cycles as it actually needs: BEQ might finish in 3 cycles, LW in 5. The clock period shrinks to the size of the slowest single stage (≈200 ps for data memory access) rather than the slowest entire instruction. This requires new hardware — a small state machine controlling which stage is active, and extra registers holding intermediate results between cycles — but the core datapath components (ALU, register file, memories) can even be reused across cycles instead of duplicated, which single-cycle design couldn't do (a single-cycle CPU technically uses its ALU only once per instruction, for a tiny fraction of the clock period, then lets it sit idle).

Fix 2: keep every instruction at a fixed number of cycles, but let different instructions occupy different stages of the datapath simultaneously — the pipelined CPU. This is exactly the mechanism Lesson 1, Part 7 introduced at the hazard level (RAW/WAR/WAW dependencies, forwarding, the load-use stall, branch misprediction flushes) without yet showing the literal circuit change that makes it possible. The literal change is small and satisfying given everything built in this lesson: insert a pipeline register after each of the datapath's natural stage boundaries — after instruction fetch, after decode/register-read, after execute, after memory access — so that while instruction N is in its memory-access stage, instruction N+1 can simultaneously be in its execute stage, instruction N+2 in decode, and instruction N+3 being fetched. The clock period shrinks to the size of the slowest single stage between two pipeline registers — the same ≈200 ps as the multi-cycle design's win — but now instructions complete (on average) every cycle instead of every several cycles, because four of them are in flight at once.

Both fixes attack the exact same 730 ps number derived above; they just spend the savings differently. Multi-cycle design fixes it by shrinking cycle length while letting instruction latency stay high (an LW might now take 5 short cycles instead of 1 long one — total time is similar, but every other instruction gets to finish faster). Pipelining fixes it by shrinking cycle length and keeping instruction latency roughly fixed, while dramatically increasing throughput by overlapping instructions — precisely the latency-vs-throughput distinction Lesson 1, Part 3 introduced with the washing-machine-factory analogy, now anchored to a real number instead of an abstract example.

What Lesson 2 actually built

Zoom out for a moment on the whole five-part arc:

Part 1 designed a real, citable instruction set — the classic MIPS-style 32-bit encoding, R-type and I-type formats, seven instructions, hand-encoded down to actual hex.
Part 2 wired every one of those instructions into a single physical datapath: one register file, one ALU, one data memory, a handful of muxes making all the necessary choices.
Part 3 derived, from the instruction encoding alone, the complete control-signal truth table that drives every mux and every enable line in that datapath — control logic as a lookup table, not something hand-waved.
Part 4 proved the whole thing works by hand-tracing a real seven-instruction program through it, cycle by cycle, arriving at verified final register and memory state.
Part 5 (this post) showed why the thing we just built and verified is nonetheless the wrong architecture to actually ship, using nothing but the clocking math from Lesson 1.

That's a complete, internally consistent single-cycle CPU — ISA, datapath, control unit, and a working execution trace — built entirely from first principles across two lessons, with every design choice derived rather than asserted.

What's Next

The roadmap sketched at the end of Lesson 1, Part 10 remains the long-term arc: multi-cycle and pipelined datapaths taken to the RTL level with real hazard-detection and forwarding logic (formalizing what Lesson 1, Part 7 introduced conceptually), the memory hierarchy built down to SRAM cell layouts and real TLB structures, superscalar out-of-order execution with register renaming and reorder buffers implemented rather than just described, SIMD and multicore with cache-coherence protocols, GPU/NPU accelerator architecture, and advanced topics like NUMA and on-chip interconnects.

A future lesson picks up exactly where this post leaves off: taking the datapath and control unit built across these five parts and formally pipelining them — inserting the IF/ID, ID/EX, EX/MEM, and MEM/WB pipeline registers, then building the actual forwarding-unit and hazard-detection circuitry that Lesson 1, Part 7 described in principle. Everything needed to understand that lesson — the hazards, the datapath, the control signals — is already in place.