Memory Planning and Zero-Copy Execution

Lesson 4 spent its energy on kernels — tiling, fusion, quantization, the instruction-level fight for FLOP/s. But even the fastest kernel is wasted if the memory system around it is poorly planned: a beautifully tiled matmul still stalls, still fragments, still blows an embedded RAM budget if the runtime around it allocates and frees tensors with no more foresight than malloc/free scattered through a loop. Lesson 5 turns from computing on tensors to deciding, ahead of time, exactly where every one of them will live.

1. Why Inference Can Plan Memory Ahead of Time (and Training Can't)

Start with the claim this whole chapter rests on: an inference runtime can know, before it executes a single operator, the complete shape and lifetime of every tensor in the computation graph. That is a strong claim, and it is worth being precise about why it is true for inference and false — or at least much weaker — for training.

Inference is a fixed, forward-only DAG. Once a model is exported for deployment (traced, scripted, compiled to ONNX or a TVM Relay graph or a TFLite FlatBuffer), the sequence of operators, their input/output shapes, and their data dependencies are all baked in. For a given input shape there is exactly one execution path — no branch is taken conditionally based on tensor values, no loop runs a data-dependent number of iterations. That determinism is precisely what lets a compiler simulate the whole graph's execution on paper, at compile time, and record, for every tensor, the exact operator that produces it and the exact operator that consumes it last.

Training defeats this in three separate ways:

• Dynamic control flow. Many training frameworks run in an eager, define-by-run mode (this is PyTorch's default execution model): the graph is built as Python executes, one operator call at a time, so the memory allocator sees only the operator currently running — it has no visibility into what comes next. Architectures with genuinely data-dependent branching (recursive tree networks, certain mixture-of-experts routing, early-exit models) make the shape itself of future work depend on runtime values, which is fundamentally unknowable ahead of time no matter how the framework is structured.
• Variable batch sizes. Training pipelines commonly vary batch size across steps — the last batch of an epoch is often a remainder, some pipelines do dynamic batching by token count rather than by example count, and gradient accumulation changes effective shapes mid-run. Every shape change invalidates a static memory plan built for a different shape (or forces the planner to plan conservatively for the worst case, giving back much of the savings).
• Backpropagation needs the forward activations to survive. This is the deep one. Static memory planning works by discovering that most tensors have short lifetimes — produced by one operator, consumed by the very next, dead almost immediately — so their buffers can be aggressively reused. Backprop breaks that assumption at the root: every activation produced during the forward pass may be needed again during the backward pass, potentially dozens or hundreds of operators later in wall-clock terms, because the chain rule needs it to compute a local gradient. A tensor's live range stretches from its forward-pass producer all the way to its backward-pass consumer, which means most activations are alive simultaneously across the entire forward pass. That is close to the worst possible case for buffer reuse — it is precisely why training memory scales roughly linearly with model depth in the naive case, and precisely why techniques like gradient checkpointing (recomputing activations during the backward pass instead of storing them) exist at all: checkpointing is a direct, deliberate trade of compute for memory, applied because static reuse isn't available to training the way it is to inference.

Put the three together and the picture is: inference is a fixed forward-only DAG with known shapes and short, non-overlapping tensor lifetimes by construction. Training is a graph whose shape can vary, whose control flow can be dynamic, and whose activation lifetimes are deliberately stretched by the algorithm itself. Static memory planning is not a technique someone forgot to apply to training — it is a technique that is only sound to apply to inference in the first place.

2. Two Kinds of Objects: Logical Tensors and Physical Buffers

The single design idea that makes memory planning possible is separating two things that eager execution conflates:

• A logical tensor — an entry in the graph with an identity: a shape, a dtype, a producing operator, and a set of consuming operators. It exists at the level of "the model."
• A physical buffer — a fixed-size region of raw memory with a base address, that exists at the level of "the hardware."

In eager execution these two collapse into one: calling an operator allocates a new physical buffer right then, and the tensor's identity is that buffer. There is no room to plan, because there is no gap in time between "the tensor is created" and "the memory for it is claimed" — they are the same event.

A static memory planner reintroduces that gap deliberately. It walks the whole graph first, purely as data — no operator actually executes — and produces a mapping from every logical tensor to a (buffer_id, offset) pair, where several logical tensors may map to the same buffer as long as they are never alive at the same moment. Only after that plan exists does the runtime allocate the (much smaller) set of physical buffers once, up front, and then execute the graph by having each operator read from and write to the offsets the plan assigned it. No allocator call happens during the actual inference pass — allocation has been moved entirely from runtime to compile time.

3. Static Memory Planning as Interval Scheduling and Graph Coloring

Once tensor lifetimes are known ahead of time, the reuse question becomes a clean, well-studied combinatorial problem instead of a heuristic guess.

Define, for every tensor T, a live range [birth(T), death(T)] measured in graph steps: birth(T) is the index of the operator that produces T, and death(T) is the index of the operator that consumes T for the last time. Two tensors conflict — cannot share a buffer — exactly when their live ranges overlap, because at some shared step both would need to be resident in memory simultaneously. Build a graph where every tensor is a node and every conflicting pair is an edge; this is the tensor interference graph (the same construction compilers have used for decades under the name interference graph, just built over tensor lifetimes instead of variable lifetimes).

Assigning tensors to a minimal number of physical buffer "slots" such that no two tensors sharing an edge get the same slot is exactly graph coloring: each color is a buffer slot, and the minimum number of colors needed is the graph's chromatic number — the minimum number of buffers that must exist simultaneously at the graph's point of peak concurrency. This is not an analogy of convenience; it is the identical mathematical structure behind register allocation, where Chaitin's 1981 algorithm treats CPU registers as colors and variable live ranges as the graph to color, spilling a variable to memory whenever the graph can't be colored with the number of physical registers available. Memory planning for tensors is register allocation with the register file replaced by a heap.

A tiny three-tensor example makes the construction concrete before the full worked example below. Suppose X and Y overlap in time but Z starts only after X has died:

Live ranges:            Interference graph:
 
X:  ████████               X ─────── Y
Y:      ████████                 \
Z:              ████████          (no edge)   Z
 
X-Y overlap  → edge (cannot share a buffer)
X-Z disjoint → no edge (can share a buffer)
Y-Z disjoint → no edge (can share a buffer)
 
Chromatic number = 2:  color 1 = {X, Z},  color 2 = {Y}
→ two physical buffers suffice for three logical tensors

There is one useful simplification. Because every tensor's live range is a contiguous span of graph steps (nothing "returns to life" after dying), the interference graph is always an interval graph — a well-behaved, chordal special case. For interval graphs, a simple greedy algorithm — sort nodes by birth, then assign each one the lowest-numbered color not already used by a still-live conflicting node — is provably optimal in the number of colors used. This is the identical result behind the classic "minimum number of meeting rooms to schedule N overlapping meetings" problem; tensor lifetimes and meeting-room bookings are the same math.

The one place the analogy to registers breaks — and it matters — is size. A register is a fixed-width slot; any variable fits in any register (spill aside). A tensor buffer is not fixed-width: a slot shared by three tensors of different byte sizes must be sized to fit the largest of the three. Minimizing the number of colors (buffers) and minimizing total bytes consumed are not the same objective once sizes vary, and that distinction is exactly what the worked example below is built to expose.

4. Worked Example: Planning a Six-Tensor Graph With a Branch and Merge

4.1 The graph

Take a small graph with a chain, a branch, and a merge — a shape common to real CNN and Transformer sub-blocks (a residual branch, a two-headed attention split, a parallel expert path):

Six logical tensors flow through five operators. B is produced once by Conv1 and consumed twice — by both branch operators — which is exactly the pattern that makes branch-and-merge graphs harder to plan than a plain chain: a tensor with more than one consumer has to stay alive until the last of them runs, not the first.

4.2 Liveness intervals

Number the operators as execution steps 1 through 6 (treating the input load itself as step 1, since it "produces" the input tensor):

A tensor's live range spans from the step that writes it to the step that reads it last. With byte sizes attached (chosen so the arithmetic below comes out to round numbers):

As an interval chart, one row per tensor, one column per step:

step:      1     2     3     4     5     6
A:       ████████
B:               ████████████████
C:                     ████████████████
D:                           ████████
E:                                 ████████
F:                                       ████ →  (returned to caller)

4.3 The conflict graph, and why it forces exactly three buffers

Two tensors conflict when their live ranges overlap, using the convention that a tensor is still "alive" through the entire step that last reads it, and a newly produced tensor becomes alive during the step that writes it — so ranges that merely touch at a shared endpoint (one dies exactly when the other is born, in the same step) still conflict, because that step needs both tensors resident at once. Checking every pair:

• A-B overlap at step 2 → conflict (Conv1 needs to read A and write B in the same step)
• B-C overlap at step 3 → conflict
• B-D overlap at step 4 → conflict
• C-D overlap at step 4-5 → conflict (both feed the Add)
• C-E overlap at step 5 → conflict
• D-E overlap at step 5 → conflict
• every other pair (A-C, A-D, A-E, A-F, B-E, B-F, C-F, D-F, E-F) is disjoint → no conflict, safe to share

Notice B, C, and D are mutually conflicting — a triangle in the interference graph, because at step 4 all three are simultaneously required (B is still being read, C already exists and is waiting for the Add, D is being written). A triangle needs three distinct colors no matter how cleverly the rest of the graph is arranged, which means three buffers is a hard lower bound for this graph, not just a planning choice. This is the direct memory cost of the branch: forking a chain into two live paths that later merge forces at least that many tensors to coexist at the fork's far edge.

4.4 Coloring the graph: the buffer assignment

Greedy coloring in birth order — assign each tensor the lowest-numbered slot not already occupied by a conflicting, still-live tensor:

Three slots, exactly matching the lower bound from the B-C-D triangle. Slot 1 is handed A → C → F in sequence; slot 2 is handed B → E; slot 3 holds only D.

4.5 The arithmetic: naive vs. planned peak memory

Naive allocation — every tensor gets its own dedicated buffer, freed only when the whole graph finishes:

256 + 512 + 512 + 256 + 256 + 128 = 1,920 KB  (1.875 MB)

Planned allocation — each slot must be sized to fit the largest tensor ever assigned to it, since the slot is physically one buffer reused over time:

Slot 1 (A, C, F):  max(256, 512, 128) = 512 KB
Slot 2 (B, E):     max(512, 256)      = 512 KB
Slot 3 (D):        256 KB
 
Total = 512 + 512 + 256 = 1,280 KB  (1.25 MB)

Savings:

1,920 KB − 1,280 KB = 640 KB saved
640 / 1,920 ≈ 33.3% reduction in peak memory

A third of the naive footprint disappears purely from scheduling — no operator, no kernel, no numerical result changed at all.

4.6 The lower bound: why 1.25 MB is provably optimal here

It's worth checking that 1,280 KB isn't just what one particular greedy ordering happened to produce — that it's the best any valid plan could do for this graph. There is a clean, independent way to compute the true floor: at every step, sum the sizes of every tensor that must be resident at that instant, and take the maximum across all steps. No valid plan can ever use less memory than that instantaneous peak, because that many bytes are provably needed at that one moment regardless of how anything is scheduled.

The maximum is 1,280 KB at step 4 — exactly the branch point where the B-C-D triangle lives. That matches the coloring result exactly, which confirms the plan in section 4.4 is not merely a good plan, it is the optimal one for this graph: it drives peak memory all the way down to the theoretical floor with zero slack. That equality won't always hold in general — see the next section for when it can't — but for interval graphs like this one, a correct greedy coloring reliably gets there.

5. Why Exact Optimal Planning Is Hard in Practice (and What Real Planners Do Instead)

The worked example above was small enough to solve by hand, and its buffer count (three) was guaranteed optimal by the interval-graph coloring result from section 3. But real graphs complicate the picture in a way the classic register-allocation analogy hides: registers are all the same size, so minimizing the count of colors used is the whole problem. Tensor buffers are not uniform size, so minimizing total bytes is a different, harder objective — formally the dynamic storage allocation problem, and it is NP-hard in the general case. Two tensors that don't conflict can still be wasteful to co-locate if their sizes are mismatched (a 4 KB tensor sharing a slot sized for a 4 MB one wastes almost all of that slot for its entire lifetime), and with hundreds or thousands of tensors in a real model, the number of ways to pack them is combinatorially large.

Production memory planners respond the way most NP-hard-adjacent engineering problems get handled: fast, well-tuned heuristics instead of exact solvers, because near-optimal-in-milliseconds beats optimal-in-hours for a compile step that runs on every model build.

• Greedy by size. Sort tensors largest-first, then place each one into the first existing buffer region that both fits its size and has no lifetime conflict with anything already assigned there, opening a new buffer only when nothing fits. Placing the biggest, most memory-dominant tensors first tends to produce a tighter overall packing than birth-order greedy, because it front-loads the decisions that matter most. In outline:

sort tensors by size, descending
buffers = []  # each buffer tracks its size and the live ranges already placed in it
 
for tensor T in sorted order:
    for buffer B in buffers:
        if T.live_range does not overlap any range already placed in B
           and B.size >= T.size:
            place T in B
            next tensor
    # nothing fit — open a new buffer sized exactly to T
    buffers.append(new Buffer(size = T.size, ranges = [T.live_range]))
 
peak_memory = sum(B.size for B in buffers)

• Local-search refinement. Apache TVM's memory planner (discussed below) supports a hill_climb allocation algorithm alongside a simpler greedy one: start from a greedy solution, then repeatedly try small perturbations — swap two tensors' slot assignments, shift an offset — accepting any change that reduces total footprint, until no local move helps. This is a standard local-search pattern for exactly this class of packing problem.
• Offline vs. online planning. TensorFlow Lite Micro distinguishes computing the plan once, ahead of time, on a developer's workstation and baking the resulting offsets into the model file (offline planning) from computing the plan at model-load time on the target device itself using its GreedyMemoryPlanner (online planning, the default). Offline planning can afford a more expensive search because it only has to run once per model build; online planning has to be fast enough to run at every boot on a microcontroller with a few hundred MHz to spare.

The pattern across all of these: treat the interval-graph coloring result as a strong starting point and lower bound, not the final answer, because real tensor sizes are never uniform enough for pure graph-coloring theory to hand back the literal optimum on its own.

6. Zero-Copy Execution

Static planning solves where a tensor lives. A second, complementary question is whether a tensor needs to be copied at all as it moves between pipeline stages. Every unnecessary copy is bandwidth spent moving bytes that were already sitting correctly in memory — pure overhead with no arithmetic behind it, and on small, latency-sensitive inference workloads those copies can dominate total execution time even though they show up nowhere in the model's FLOP count. Zero-copy execution is the discipline of eliminating them wherever the data dependency structure allows it.

6.1 In-place elementwise operators

Consider y = relu(x). Every output element y[i] depends on exactly one input element x[i] and nothing else — no output element ever needs a different input element to have already been computed. If x's buffer has no other consumer after this operator (its live range ends exactly here), the operator can write its result directly back into x's own buffer, element by element, with no separate output allocation at all. The three conditions that make an operator safe to run in-place are worth stating precisely, because violating any one of them silently produces wrong answers rather than a crash:

1. The output size must not exceed the input buffer's size (elementwise ops with matching shape trivially satisfy this).
2. Each output element must depend only on input element(s) that have not yet been overwritten by the time they're read — true automatically for strictly elementwise maps like ReLU, sigmoid, or a scalar multiply, but not automatically true for something like a reduction or a convolution, where many output elements read overlapping windows of input.
3. The input buffer must have exactly this operator as its last consumer — if some other, later operator also needs to read the original, unmodified input, writing over it in-place would silently corrupt that later read.

Binary elementwise ops extend the same idea: an in-place add A += B can write its result into A's buffer instead of allocating a fresh output tensor, as long as nothing downstream still needs A's original value — this is exactly the "accumulate" pattern used throughout residual-connection and gradient-accumulation code, and it is a standard fusion target for graph compilers, which actively look for elementwise chains where every intermediate result is immediately consumable in-place.

6.2 Reshape and transpose as metadata, not data movement

A tensor in memory is really two things: a flat buffer of bytes, and a view over that buffer described by a shape and a set of strides (how many elements to skip to advance one step along each dimension). Many operations that look like they rearrange data are, underneath, only rearranging the view.

A reshape that preserves the tensor's row-major memory order — flattening [batch, height, width] into [batch, height*width], say — changes only the shape metadata; the underlying bytes are already laid out correctly for the new shape, so no element ever needs to move. The runtime updates the tensor's shape descriptor and hands back a pointer into the exact same buffer.

A transpose is more interesting because it looks like it must physically move data — swapping rows and columns feels like real work — but if the consuming kernel is written to accept arbitrary strides rather than assuming a fixed row-major layout, a transpose can also be represented purely as a permutation of the stride metadata, with zero bytes copied. This is precisely the distinction PyTorch surfaces between .reshape() (may copy, if the requested shape isn't compatible with the current strides) and .transpose() / .view() (never copy — they only ever manipulate the stride and shape fields, deferring any real data movement until, and unless, some later operation demands a contiguous layout it doesn't have). In an inference runtime, whether a transpose is "free" is entirely a property of whether the next kernel in the graph was written to be stride-aware; if it wasn't, the runtime is forced to materialize a physically transposed copy, and the "zero-copy" transpose reverts to an ordinary, bandwidth-costing one.

Same underlying buffer, two different "shapes" laid over it:
 
  buffer (12 elements, unchanged):  [ a b c d e f g h i j k l ]
 
  view as shape [3, 4], strides [4, 1]     view as shape [4, 3], strides [1, 4]
  (row-major "as stored")                  (a transpose of the above — same
                                             bytes, different stride order)
 
   a b c d                                  a e i
   e f g h        ── reshape/transpose ──▶  b f j
   i j k l           metadata only,         c g k
                      zero bytes moved       d h l
 
  a stride-aware kernel reads element [r,c] via buffer[r*stride0 + c*stride1] —
  it never cares whether the strides came from the "natural" layout or a
  transpose, so the transpose costs nothing until a stride-naive kernel forces
  a physical copy into contiguous order

6.3 Memory-mapping weights instead of copying them

Loading a model traditionally means reading its weight file from storage into a freshly allocated heap buffer — a full copy of however many megabytes the weights occupy, every single time the model is loaded, before a single inference has run. Memory-mapping the weight file instead (mmap on POSIX systems, or execute-in-place directly from external flash/QSPI on many microcontrollers) maps the file's bytes straight into the process's address space and lets the operating system — or, on bare-metal embedded targets, the memory controller — page or fetch the bytes in on demand as the model actually reads them, rather than up front and all at once.

This buys three concrete things that matter disproportionately on constrained and embedded targets:

• No duplicate buffer. The weights already exist as bytes on flash or disk; mmap reuses that storage as the live buffer instead of allocating a second, separate copy in RAM, which is often the difference between a model fitting in available memory and not fitting at all.
• Shared, read-only pages across processes. Weights don't change during inference, so multiple processes (or multiple model instances) running the same model can share the identical physical pages, with the OS enforcing copy-on-write only if something ever tries to modify them — which nothing legitimately does for frozen inference weights.
• Format designed for zero deserialization. TensorFlow Lite's model format is a FlatBuffer specifically because a FlatBuffer's on-disk byte layout is its in-memory layout — there is no parse-then-construct step between "bytes from flash" and "structure the runtime can read directly," which makes mapping the file and using it in place a genuinely free operation rather than a fast one.

The tradeoff is real and worth naming rather than glossing over: memory-mapped weight access trades a fast, predictable up-front copy for access latency spread across the model's execution, paid the first time each page is actually touched (a page fault on general-purpose OSes, or a flash-read stall on embedded XIP). For a model whose weight-access pattern is itself fairly predictable and sequential — most feedforward inference is — that tradeoff is close to strictly a win, since it converts one large blocking copy into many small overlapping reads.

6.4 I/O binding: zero-copy across the accelerator boundary

The last copy worth eliminating sits at the CPU-accelerator boundary rather than inside a single device's memory. A naive pipeline copies input data from a CPU buffer into a separate device (NPU/GPU) buffer before execution, then copies the result back out afterward — two full-tensor transfers bracketing every inference call, on top of whatever the model itself does. For small, latency-sensitive workloads, those two transfers can easily outweigh the actual compute time.

ONNX Runtime exposes this control explicitly as I/O binding: instead of letting the runtime silently allocate and copy input/output buffers on each call, the caller pre-allocates buffers directly on the target device (or in memory the device can access without a copy) and hands the runtime pointers to them, so the accelerator reads its input and writes its output in place, with no CPU-to-device or device-to-CPU transfer at all for that call. This is the same zero-copy principle as sections 6.1-6.3, just applied at the boundary between two different physical memories rather than within one.

Without I/O binding:                     With I/O binding:
 
  CPU buffer                               CPU-visible / device-shared buffer
     │  copy in                                      │
     ▼                                                ▼
  device buffer  ──▶ [ accelerator runs ]     [ accelerator runs ] directly
     │  copy out                              on the bound buffer — reads
     ▼                                        input and writes output with
  CPU buffer                                  no transfer either direction
 
  2 full-tensor transfers per call         0 full-tensor transfers per call
  (can dominate latency on small models)   (transfer cost paid once, at bind time)

7. How Real Runtimes Do This

7.1 TVM's Unified Static Memory Planning (USMP)

Apache TVM's memory planning history is itself a case study in why unified matters. Before USMP, TVM planned memory separately at two different graph levels — the inter-operator level (Relay) and the intra-operator level (TIR, inside each operator's own generated code) — and the USMP RFC states plainly that sharing memory inside an operator first, then trying to share the result with inter-operator tensors afterward, is sub-optimal, because decisions made early at one level foreclose better sharing opportunities visible only from the other. USMP replaces both with a single TIR-to-TIR compiler pass that analyzes liveness conflicts across every buffer in the whole compiled program at once — spanning operator and inter-operator tensors together — and rewrites what would otherwise be a runtime call to allocate a workspace (TVMBackendAllocWorkspace) into a compile-time-computed, fixed offset into a preallocated pool. The interface accepts pluggable allocation algorithms, including a greedy mode and a hill_climb local-search mode (selected via --pass-config tir.usmp.algorithm=hill_climb), which is precisely the greedy-plus-refinement pattern described in section 5. This is the piece of TVM specifically aimed at microcontroller and bare-metal deployment (microTVM), where every kilobyte of avoided allocation overhead is visible in the bill of materials.

Before USMP (runtime call, on every inference):
 
  buf = TVMBackendAllocWorkspace(size, device, ...)   # dynamic, per call
  ... use buf ...
  TVMBackendFreeWorkspace(buf)
 
After USMP (compile-time offset, resolved once, never called again):
 
  buf = pool_base + 512   # literal, compile-time-computed offset
  ... use buf ...         # no allocator call exists in the compiled binary

7.2 TensorFlow Lite Micro's tensor arena

TFLite Micro sidesteps dynamic allocation entirely by requiring the caller to provide one single, contiguous buffer up front — the tensor arena — sized by the application, out of which the entire interpreter's memory needs (persistent metadata, operator scratch space, and every intermediate tensor) are carved by a MicroAllocator. The arena grows from two ends toward the middle: a head, holding the non-persistent tensor buffers that get reused across operators as execution proceeds, and a tail, holding persistent allocations like operator metadata and quantization parameters that must survive for the interpreter's whole lifetime. The actual tensor-lifetime reuse inside the head region is the job of the GreedyMemoryPlanner, which is exactly the greedy-by-lifetime algorithm from section 5 — and, matching the offline/online distinction, TFLite Micro also supports precomputing tensor offsets on a host machine ahead of time and embedding them in the model's metadata, letting the on-device allocator simply honor fixed positions instead of re-deriving the plan at every boot.

7.3 ONNX Runtime's memory pattern and arena allocator

ONNX Runtime attacks the same problem from the allocator side. Its CPU execution provider defaults to an arena allocator (enable_cpu_mem_arena), which claims one large region from the OS up front and services individual tensor allocation requests out of that region instead of calling the system allocator per tensor — directly cutting allocation overhead even before any reuse logic runs. Layered on top of that is memory pattern optimization: when a session runs with a given set of input shapes, ONNX Runtime traces every internal allocation that execution makes, records the resulting reuse pattern once, and on subsequent calls with the same shapes replays that pattern as a single big allocation carved into the previously-determined offsets — turning "many small mallocs" into "one allocation, many memory-planned slices," which is the exact static-planning payoff from section 4, just triggered by a runtime trace on the first call rather than a full offline compiler pass.

The three systems arrive at the same underlying idea from three different entry points — a compiler pass, a fixed-arena embedded allocator, and a runtime trace — which is itself evidence that static memory planning isn't one runtime's clever trick, it's the structurally correct answer to a problem every inference stack independently runs into:

Further Reading

• Apache TVM RFC 0009, "Unified Static Memory Planning (USMP)" — the authoritative design doc for holistic, compile-time memory planning on microcontroller and embedded TVM targets.
• Apache TVM, USMP tracking issue #8404 — implementation history, including the greedy and hill-climbing allocation algorithms.
• TensorFlow Lite Micro, "Memory Management" — the tensor arena, head/tail layout, and GreedyMemoryPlanner documented directly by the TFLM team.
• ONNX Runtime, "Memory Consumption" — arena allocator behavior, memory pattern optimization, and shared-arena configuration across sessions.
• GeeksforGeeks, "Register Allocation Algorithms in Compiler Design" — the classic graph-coloring formulation this chapter's buffer-reuse problem is built on.
• Smith, M. D., Ramsey, N., & Holloway, G., "Register Allocation by Graph Coloring" — a deeper treatment of interference graphs, Chaitin-style coloring, and spilling, for readers who want the compiler-theory side in full.

Part 2 of Lesson 5 stays inside the runtime but moves from where tensors live to how execution actually proceeds: operator placement across heterogeneous cores, scheduling order, and overlapping DMA transfers with compute so the accelerator is never left waiting on a copy it could have started earlier.