Instruction Memory, Decode, and the Control Unit

This is Lesson 1, Part 5 of Computer Architecture From First Principles. Part 4 built the register file and the program counter — named storage plus one special register that always holds the address of "what to do next." This part answers the obvious follow-up: what does the PC actually point at, and how does a stream of bits at that address turn into RegWrite, ALUSrc, MemRead, MemWrite, and Branch — the signals that steer everything built in Part 2 (gates, MUXes, the ALU) and Part 3 (the clock and flip-flops)?

Instructions Are Just Data Sitting in Memory

The first thing to unlearn is any notion that instructions are special. An instruction is a number, sitting in memory, exactly like any other number. The only thing that makes it an "instruction" is that the CPU has agreed, by convention, to interpret whatever is at address PC as an operation to perform rather than as a piece of data to add or compare.

Address       Contents (still just bits)
0x1000        00000000 00000001 00001000 11000000
0x1004        00000100 00000010 00010000 00000000
0x1008        ...

Fetching an instruction is the same read operation as fetching any word from memory:

Address = PC
Instruction = Memory[Address]

Instruction Memory Is Built From the Same Idea as the Register File

Part 4 built the register file as an array of storage cells with an address decoder in front of it: feed in a 5-bit register number, and a decoder activates exactly one cell's read or write port. Instruction memory is the identical idea at a larger scale — an array of storage cells (here, whole 32-bit words instead of single registers) addressed by a decoder, except the address is PC (typically 32 bits, giving up to 2^32 addressable byte locations) instead of a 5-bit register number. Nothing new had to be invented to get instruction memory; it's a bigger, differently-addressed version of a block already built.

For this lesson, treat instruction memory and data memory as two separate arrays — a Harvard-style split that keeps the diagrams clean. Real general-purpose CPUs mostly present a single unified address space to software (a von Neumann machine) and only split instructions from data inside the cache hierarchy. That split, and why it matters for performance, is a topic for the memory-hierarchy lessons later in this series — it doesn't change anything about decode or control, so it's safe to defer.

A Concrete Instruction Encoding (a Toy ISA)

To talk about "decoding" concretely, the instruction has to have an actual, fixed layout — specific bits at specific positions meaning specific things. So before building a decoder, this section designs one.

A quick note on scope: the encoding below is illustrative. It exists to make the mechanics of decode and control concrete with real bit positions and a real worked example. Lesson 2 formalizes an actual, complete instruction set — closely modeled on RISC-V — with its real opcode and funct fields, its real immediate-encoding tricks, and its real instruction classes. Everything built here is about the mechanism; the specific bit assignments are throwaway.

Why Fixed Width at All?

Instructions could, in principle, be encoded with a variable number of bits — shorter codes for common operations, longer codes for rare or complex ones. x86 does exactly this, and it's part of why x86 decode logic is famously gnarly: before you know where an instruction ends, you may need to partially decode it, because the length itself depends on earlier fields. A fixed-width instruction sidesteps that problem entirely — every instruction is, say, exactly 32 bits, so the boundaries between instructions are known in advance from the PC alone, and every field lives at the same bit position in every instruction of a given format. That uniformity is what makes single-cycle wire-only field extraction possible, which is the whole point of the next section. Most classic RISC ISAs (MIPS, RISC-V's base encoding, ARM's A32) make this same fixed-width trade — it costs some code density but massively simplifies the decoder.

The Layout

The toy ISA here uses 32-bit instructions and 32 general-purpose registers (R0–R31), which is exactly why register fields below are 5 bits wide: 2^5 = 32, matching the register file built in Part 4. Two instruction formats cover everything needed for this lesson's four instruction types (ADD, LOAD, STORE, BEQ):

R-type — register-register operations (ADD, SUB, ...):

I-type — one register, one immediate (LOAD, STORE, BEQ):

Neither layout is arbitrary — the widths are a budget allocation over exactly 32 bits. For I-type: 6 + 5 + 5 + 16 = 32. Six opcode bits buys up to 64 distinct operations, which is far more than this toy ISA needs, but it's a realistic order of magnitude (real ISAs typically spend somewhere between 6 and 10 primary-opcode bits). Whatever isn't spent on opcode and register fields goes to the immediate, since a useful LOAD/STORE offset or branch displacement needs real range.

Opcode Assignments

These five values are all that's needed to work through decode and control end to end. Adding a sixth instruction later is just claiming the next unused opcode and adding one row to every table that follows — which is exactly the process real ISA designers go through, just at much larger scale and with far more careful attention to encoding compatibility across processor generations.

The Decoder: Mostly Wires, a Little Logic

This is the part that tends to sound more mysterious than it is. "Decoding an instruction" conjures images of a complex piece of hardware doing real computation. In a fixed-width ISA, most of decode is not computation at all.

Field Extraction Costs Zero Gates

When the instruction word arrives from instruction memory, it's captured in an instruction register — 32 D flip-flops, exactly the storage primitive built in Part 3, latching the 32 bits on a clock edge. Once those 32 bits are sitting in the instruction register, "extracting the opcode field" doesn't require adding, comparing, or transforming anything. Bits 31 through 26 of the instruction register are the opcode. Bits 25 through 21 are rs. There is nothing to compute — those wires already carry exactly the values in question. "Decoding the fields" is a matter of running wires from each flip-flop output to whichever downstream block needs that particular bit, and giving each bundle of wires a name.

Contrast this with a variable-length encoding: before you can even say "these five bits are rs," you might first need to evaluate a length field, or check whether a prefix byte is present, or walk through the instruction byte by byte. That's real combinational — sometimes even iterative — logic sitting in the critical path before any field is available. Fixed width buys the field extraction step for free; that's the trade being made.

Turning Opcode Bits Into an Instruction Identity

Field extraction gets you six wires carrying the opcode bits. What you actually want, for control purposes, is a single, unambiguous answer to "is this instruction an ADD?" — a clean 1-or-0 signal. That's exactly the decoder primitive from digital logic: a circuit with n inputs and up to 2^n mutually-exclusive, one-hot outputs, where exactly one output line is high for any given input pattern (this is the same style of address decoder already used inside the register file and instruction memory earlier in this lesson — it's not a new circuit, just a familiar one wearing a different hat).

Concretely, the ADD output line is just an AND of inverted opcode bits, since ADD's opcode is 000000:

ADD = ¬opcode5 ∧ ¬opcode4 ∧ ¬opcode3 ∧ ¬opcode2 ∧ ¬opcode1 ∧ ¬opcode0

LOAD (000010) is the same shape with one bit uninverted:

LOAD = ¬opcode5 ∧ ¬opcode4 ∧ ¬opcode3 ∧ ¬opcode2 ∧ opcode1 ∧ ¬opcode0

Each of these is a 6-input AND gate with a handful of inverters on its inputs — six gates, none of them exotic, all of them already covered in Part 2. With 6 opcode bits there are 64 possible patterns and only 5 are assigned; the other 59 simply produce no active output line (or, in a real design, get treated as illegal-instruction traps — a detail worth flagging but not building out here).

These one-hot lines (ADD, SUB, LOAD, STORE, BEQ) feed two separate consumers downstream: the ALU control (which op-select code to hand the ALU's operation MUX, built in Part 2), and the control unit — the block that decides everything else about what the rest of the datapath should do this cycle. The control unit is the subject of the rest of this lesson.

Worked Example: Decoding One Instruction, Bit by Bit

Take the 32-bit word 0x00221800 sitting at some address in instruction memory. In binary:

0000 0000 0010 0010 0001 1000 0000 0000

Regrouped along the R-type field boundaries defined above (6 / 5 / 5 / 5 / 11 bits):

opcode   rs      rt      rd      unused
000000   00001   00010   00011   00000000000

Step through it exactly as the decoder would:

1. opcode = 000000. Looking it up in the opcode table: 000000 is ADD, and it's an R-type format, so the remaining fields split as rs / rt / rd, not rs / rt / immediate.
2. rs = 00001 = 1. Source register 1 is R1.
3. rt = 00010 = 2. Source register 2 is R2.
4. rd = 00011 = 3. Destination register is R3.
5. unused = all zero. No meaning yet — reserved for whatever this lesson's toy ISA never grows into.

Put together: 0x00221800 is ADD R3, R1, R2 — read R1 and R2, add them, write the result into R3. If R1 holds 10 and R2 holds 20 (the same register file contents used as an example in earlier parts of this series), the decoder alone is enough to know what should happen; it says nothing yet about when the register reads occur, when the ALU computes, or when the write-back lands — that's the fetch-decode-execute timeline, and it's the subject of Part 6. What decode produces here is a complete, static description of the operation: ADD, operands R1 and R2, destination R3.

The Control Unit as a Truth Table

The decoder answers "which instruction is this?" The control unit answers the next question: "given that this is an ADD (or a LOAD, or a STORE, or a BEQ), which parts of the datapath should be active this cycle?" That second question, it turns out, is entirely captured by a truth table — one row per instruction type, one column per control signal, values taken straight from the ISA's specification of what each instruction does.

Six signals are enough to cover ADD, LOAD, STORE, and BEQ:

• RegWrite — does this instruction write a result back into the register file?
• ALUSrc — does the ALU's second operand come from a register (0) or from the immediate field (1)?
• MemRead — does this instruction read data memory?
• MemWrite — does this instruction write data memory?
• MemtoReg — when writing a register, does the value come from the ALU result (0) or from memory (1)?
• Branch — does this instruction potentially redirect the PC instead of just incrementing it?

Filling in the table means, for each instruction, reading off what its semantics require:

Reading the LOAD row as an example of how the table gets built: LOAD computes an address by adding a base register to an immediate offset, so ALUSrc = 1 (second ALU operand is the immediate, not a register). It then reads data memory at that address, so MemRead = 1. The value it reads has to end up in a register, so RegWrite = 1 and MemtoReg = 1 (write-back source is memory, not the ALU result). It never writes memory and never branches, so MemWrite = 0 and Branch = 0.

STORE and BEQ both carry an X — a don't-care — in MemtoReg. Neither instruction writes a register (RegWrite = 0), so the write-back MUX that MemtoReg selects is never even sampled that cycle; its select line can be anything without affecting behavior. Don't-cares aren't a shortcut taken for convenience — they're real information handed to the logic-minimization step. A synthesis tool (or a designer doing it by hand) is free to pick whichever value for those cells produces the smallest or fastest circuit, and it often results in a different, cheaper expression than filling every cell in with a definite 0 or 1. This is the same cost-consciousness that mattered when building gates from transistors in Part 2 — fewer, simpler terms means less silicon and less delay.

From Table to Gates

Each column of the truth table is a Boolean function of the one-hot decoder lines from the previous section, and it synthesizes directly into a sum-of-products expression:

RegWrite = ADD ∨ LOAD
ALUSrc   = LOAD ∨ STORE
MemRead  = LOAD
MemWrite = STORE
MemtoReg = LOAD
Branch   = BEQ

Every one of these is either a bare wire (MemRead, MemWrite, MemtoReg, Branch — each driven directly by a single decoder output line) or a two-input OR gate (RegWrite, ALUSrc). That's the entire control unit for this instruction set: the opcode decoder built earlier, plus two OR gates. Nothing about it is clocked, nothing about it stores state — it's pure combinational logic, a direct descendant of the AND/OR/NOT gates from Part 2, just wired up according to a table derived from what the instructions are supposed to do rather than from arithmetic.

Control-Unit Design Is Literally "Derive a Truth Table From a Spec"

It's worth pausing on how mechanical the process above actually was. Nothing about deriving that table required cleverness — it required reading an instruction's definition ("LOAD reads a register, adds an immediate, reads memory at that address, writes the result to a register") and translating each clause directly into a 0 or 1 in the appropriate column. Adding a sixth instruction to this ISA — say ADDI, register plus immediate, no memory access — means adding exactly one row (RegWrite=1, ALUSrc=1, MemRead=0, MemWrite=0, MemtoReg=0, Branch=0) and folding one more term into the existing OR expressions (ALUSrc = LOAD ∨ STORE ∨ ADDI). There is no design insight buried in this step beyond correctly reading the spec — which is exactly why, in real hardware description languages, control logic this simple is often generated near-mechanically from a table, rather than hand-crafted gate by gate.

It can help to picture the table as software, purely as a mental model — never as what the hardware is actually doing:

// A software mirror of the truth table above -- useful for intuition,
// NOT a description of the real circuit. Real control logic evaluates
// every row in parallel, combinationally, with no branching at all.
struct ControlSignals {
    bool regWrite, aluSrc, memRead, memWrite, memToReg, branch;
};
 
ControlSignals decode(Opcode op) {
    switch (op) {
        case ADD:   return {true,  false, false, false, false, false};
        case LOAD:  return {true,  true,  true,  false, true,  false};
        case STORE: return {false, true,  false, true,  false, false};
        case BEQ:   return {false, false, false, false, false, true };
        default:    return {false, false, false, false, false, false};
    }
}

The switch is a fine way to think about it, but it's misleading if taken literally: there is no sequencing, no "checking cases one at a time" in the hardware. Every OR gate in the real circuit evaluates simultaneously, within a single gate-delay of the opcode bits becoming stable — the same combinational-versus-sequential distinction drawn all the way back in Part 3. No clock edge is needed to produce control signals; they're a pure function of the current opcode, available (after gate delay) the instant the opcode bits are.

Hardwired Versus Microcoded Control

Everything built above — decoder plus a handful of AND/OR gates wired directly from a truth table — is called a hardwired control unit. The name is literal: the mapping from opcode to control signals is baked directly into the arrangement of gates. It's fast (one gate-delay, no memory access on the control path) but rigid — changing an instruction's behavior, or adding a new one, means changing the physical gate layout, which for a manufactured chip means a new silicon revision.

There's a second, historically important approach: microprogrammed (microcoded) control. Instead of wiring the opcode straight to control-signal gates, the opcode is used as an address into a small on-chip ROM (or PLA) called the control store. Each entry in that ROM — a microinstruction — is itself a control-signal word, plus typically an address for the next microinstruction to fetch. A small internal sequencer, essentially a tiny CPU inside the CPU, steps through one or more microinstructions per macro-instruction, asserting a different set of control signals on each micro-step. A single complex instruction can expand into a short microprogram — a sequence of simple control-signal words applied over several cycles.

The trade-off mirrors the hardwired-versus-microcoded phrasing directly: hardwired control is faster because signals come straight from gates, but every instruction's behavior is frozen into the wiring; microcoded control is slower — an extra memory access sits on the critical path for each micro-step — but far more flexible, since extending or even patching an instruction's behavior can mean rewriting entries in the control store rather than redesigning gates (per GeeksforGeeks' summary of the two styles (Hardwired and Micro-programmed Control Unit)).

The real history lines up with the theory. The RISC processors introduced in the 1980s — SPARC and MIPS among them — deliberately kept instruction sets simple and uniform specifically so that hardwired control was practical: a small, uniform instruction set produces a small truth table, and a small truth table produces a small, fast gate network, exactly like the one built earlier in this lesson (Mark Smotherman's history of microprogramming traces this shift in detail: A Brief History of Microprogramming). x86, by contrast, carried decades of increasingly complex CISC instructions — the original 8086 alone shipped with a control store of over 500 microinstruction entries — and microcode was the practical way to implement operations too irregular to reduce to one clean truth-table row. Modern x86 chips are a hybrid of both ideas: common, simple instructions get decoded directly into one or a few fixed micro-ops without ever touching the control store — a fast, hardwired-flavored path — while the rare, genuinely complex legacy instructions still trigger a multi-cycle fetch from an on-chip microcode ROM.

For this series, everything from here forward stays hardwired — it's simpler to trace, and it matches the small, uniform toy ISA (and, later, the RISC-V base ISA) this series is building toward. Microcode is worth knowing exists and roughly why, but it isn't the model used in the datapath being assembled in this lesson.

Assembling the Front End

Putting instruction memory, the decoder, the control unit, and the register file and ALU from earlier parts together gives the front half of a CPU — everything needed before the machine can act on an instruction:

Two blocks are deliberately left out of this diagram: data memory (needed for LOAD/STORE) and the write-back MUX that MemtoReg selects. They're omitted here on purpose rather than by oversight — wiring them in only makes sense once each instruction's cycle-by-cycle behavior has been traced, which is exactly the job of the next part. What's here is everything that has to exist before any instruction can execute: a place instructions live, a way to split them into fields that costs no logic at all, and a control unit that is nothing more exotic than a truth table pulled straight from the ISA's own definition of what each instruction means.

Further Reading

• MIT OpenCourseWare, 6.004 Computation Structures (Spring 2017) — Chris Terman's course builds from gates through combinational and sequential logic up to a complete processor, including control-logic design.
• GeeksforGeeks, Introduction to Control Unit and its Design
• GeeksforGeeks, Hardwired and Micro-programmed Control Unit
• Mark Smotherman (Clemson University), A Brief History of Microprogramming — detailed history of microcode, including the 8086's control store and the RISC-era shift to hardwired control.
• David A. Patterson and John L. Hennessy, Computer Organization and Design, RISC-V Edition: The Hardware Software Interface — the canonical source for deriving a single-cycle control unit's truth table from RegWrite/ALUSrc/MemRead/MemWrite/MemtoReg/Branch-style signals. Publisher page.
• Sarah L. Harris and David Harris, Digital Design and Computer Architecture, RISC-V Edition — builds the same control-unit reasoning up from digital-logic fundamentals (decoders, truth tables, don't-cares) rather than starting from the ISA down. Publisher page.

With instruction memory, a decoder that splits fields for free, and a control unit that's just a synthesized truth table now in place, the next part traces what actually happens, cycle by cycle, when each of these instructions runs — Lesson 1, Part 6: "The Fetch-Decode-Execute Cycle: Tracing ADD, LOAD, STORE, and BEQ."