CompleteOct – Dec 2025MIT 6.1910 Computation StructuresSolo project

A RISC-V processor, and the one instruction that made it 32× faster.

I designed a 4-stage pipelined RV32I CPU with caches and bypassing, then extended its instruction set with packmul, a packed 4×INT8 multiply-accumulate, and rewrote a neural network's inner loop around it.

MinispecRV32IPipeliningCachesISA extensionCSynthesis

00At a glance

0×

faster inference

24.62 ms → 0.772 ms · one MNIST image

0×

fewer clock cycles

42,071,387 → 873,434

+0%

logic area

29,156 → 31,394 µm², memories excluded

cycles per multiply-accumulate

down from 158 · 266,200 MACs per image

Mine

Pipeline control for all four stages: stalls, annulment, bypassing and branch redirect. Instruction decoder, ALU, shifter and adder. The direct-mapped cache. The packmul instruction end to end, and the rewritten inference loop in C.

Course-provided

Project skeleton and type definitions, memory models, the test programs, the trained network and its weights, a software reference for the packed multiply, and the synthesis flow.

01The problem

The assignment was open-ended: make a processor run a quantized neural network as fast as possible, where "fast" is clock period multiplied by cycle count, and area is watched. The workload is an INT8 multilayer perceptron for MNIST (784-300-100-10), which is 266,200 multiply-accumulates per image.

The base RV32I instruction set has no multiply. On the first logged run every one of those 8-bit products went through a software shift-and-add routine, and one image cost 42 million cycles: about 158 cycles for each multiply-accumulate. The pipeline was not the bottleneck. The instruction set was.

02The processor

The CPU is a four-stage pipeline: Fetch, Decode, Execute, Writeback. Instructions and data each go through a 4 KB direct-mapped cache (64 sets of 64-byte lines) in front of main memory; the data cache is write-back. Branches are predicted not-taken and resolved in Execute, so a misprediction costs one cycle when the instruction cache hits.

Fig. 1 · Datapath

Amber dashes: ALU results bypass from Execute into Decode. Teal dashes: load data bypasses from Writeback. The dashed grey path redirects Fetch when a branch was mispredicted.

The interesting part of a pipeline is what happens when instructions depend on each other. My Decode stage checks its source registers against what is in flight and picks one of three outcomes: take the value from Execute, take load data from Writeback, or stall. Step through it:

Fig. 2 · Step through the hazards

A simplified model of the hazard rules in my Decode stage, running the inner loop of the network with cache hits assumed. Red: stall. Teal edge: bypassed operand. Struck through: annulled.

03The instruction: packmul

A 32-bit register holds four 8-bit values. So one instruction can do four of the network's multiplies and add them up: packmul rd, rs1, rs2 treats rs1 as four unsigned bytes and rs2 as four signed bytes, multiplies lane by lane, and writes the 32-bit sum of the four products. It lives in the ALU as four 9×8-bit signed multipliers feeding a two-level adder, and it is encoded in the R-type space (opcode 0110011, funct7 0000001) so the standard assembler can emit it.

There is a catch. The network's activations are signed bytes with an input offset of +128, and the weights are signed too. Rather than build a signed×signed datapath plus an offset adder per lane, I used an identity: flipping the top bit of a signed byte is the same as adding 128 and reading it as unsigned. One XOR with 0x80808080 converts four activations at once, in the form the multiplier wants. Try it:

Fig. 3 · packmul, lane by lane

Drag any activation or weight across the full INT8 range. The packed result always equals Σ (x + 128) · w, which is exactly the term the quantized layer needs. The identity holds across the whole INT8 range, which is what makes the shortcut safe in hardware.

With the instruction in place, I rewrote the layer's inner loop in C: step four elements at a time, load activations and weights as whole words instead of bytes, XOR, packmul, accumulate. Four byte loads, four software multiplies and four adds became two word loads and two single-cycle instructions.

04Results

Fig. 4 · Before and after

first logged runfinal design

Clock cycles per image48.2× fewer

42,071,387

873,434

Clock period+51%

585.20 ps

883.53 ps

Runtime per image (period × cycles)31.9× faster

24.62 ms

0.772 ms

Logic area, memories excluded+7.7%

29,156 µm²

31,394 µm²

Both rows are the same test (one MNIST image, prediction correct) from the course's synthesis and simulation flow. Runtime is clock period × cycle count.

The result is a trade, and I think the honest way to show it is all four numbers together. The final design's clock is 51% slower than where I started. It does not matter: the cycle count fell 48-fold, so the product that counts fell 32-fold, for 7.7% more logic.

05Decisions and trade-offs

Fix the instruction set before the pipeline. At 158 cycles per multiply-accumulate, no amount of pipeline tuning would have mattered. Profiling by arithmetic first told me where the 42 million cycles were.

Pack four lanes, not one. A plain 32-bit multiply would still spend an instruction per byte. Packing matches the datapath to the data: INT8 values, four to a word.

Move the offset into an XOR. Handling the +128 input offset inside the multiplier would cost adders in every lane. The XOR identity costs one existing ALU operation per word.

Accept a slower clock, knowingly. The final critical path runs from the decoded immediate through the branch-target logic into the program counter, 74 cells long. Cutting it was the obvious next step; the cycle savings were worth far more.

Stall rather than bypass from everywhere. ALU results bypass from Execute only. A result already in Writeback costs one stall cycle instead of another wide mux on the critical Decode path.

06What I took from it

Architecture is accounting. Every choice shows up in cycles, picoseconds or square microns, and the only way to know whether an idea is good is to multiply the columns. The second lesson was about correctness: the signedness of each lane in packmul is the kind of detail that produces a network that is almost right, so I pinned down the arithmetic identity across the whole input range before debugging anything else.

If I continued: retime the branch-target path to recover clock period, measure stall and annulment rates with real counters, and compare a two-way cache against the direct-mapped one.