04. npu_sim — Cycle / Memory Simulation

If cpu_ref answers "is the result correct?", npu_sim answers "how long does it take and how much does it use?". This is where cycles / memory / bandwidth get predicted before silicon bring-up.

It is the key tool for keeping the Measured design policy — no PL offload decision without measurement.

What's in it

Cycle model (eulernpu/hls/cycle_model.py) — 17 per-kernel lambdas
Memory model — weight / activation / KV / scratch separation, BRAM/DSP/m_axi limits
Trace — per-op latency, bandwidth peak
Profile — JSON timeline for visualisation

The cycle model is locked against P06 silicon measurement: test_matmul_systolic_matches_p06_silicon fails CI if the formula drifts more than ±10 %.

Running

eulernpu compile --spec ... --backend npu_sim --out /tmp/sim
eulernpu run     --artifact /tmp/sim --input ... --out /tmp/sim.out.npz \
                 --profile /tmp/sim.profile.json

--profile emits the timeline JSON.

Reading the Profile

eulernpu profile show /tmp/sim.profile.json

Total cycles      : 2,164,512  (≈ 3.25 ms @ 666 MHz)
Bottleneck op     : conv2d_3 (980,224 cyc, 45.3%)
Bandwidth peak    : 412 MB/s @ depthwise_conv2d_5
Memory breakdown  :
  weights         : 132 KB / 140 KB BRAM (94%)
  activations     :  18 KB
  KV cache        :   0 KB
  scratch         :   4 KB

Board Profile Comparison

eulernpu profile --all-boards /tmp/sim.profile.json

See in one table how the same graph behaves on XC7Z020 / XC7Z045 / XCZU7EV / XCZU9EG — which board it first fits, where BRAM/DSP breaks.

Sweep — Board × systolic_size Pareto

eulernpu sweep \
    --spec    examples/industrial_kws/spec.yaml \
    --boards  xc7z020,xc7z045,xczu7ev,xczu9eg \
    --systolic 8,16,32,64 \
    --out     /tmp/kws_sweep.json

Each (board, systolic) combination produces latency / area / power, and the Pareto frontier is highlighted. Which trade-offs matter becomes visible in one view.

Trace — Per-op Latency

eulernpu trace /tmp/sim.profile.json --top 10

op                  cycles    %     bandwidth     mem
─────────────────────────────────────────────────────
conv2d_3           980,224   45.3   312 MB/s     BRAM
depthwise_conv2d_5 412,800   19.1   189 MB/s     BRAM
linear_7           198,432    9.2    24 MB/s     BRAM
gru_cell_0         156,224    7.2    12 MB/s     BRAM
...

Where the bottleneck lives, which ops are bandwidth-bound (memory hierarchy) vs compute-bound (DSP).

Roofline — Bandwidth-bound vs Compute-bound

eulernpu profile --roofline /tmp/sim.profile.json --board xc7z020

Each op's placement on the roofline (peak compute vs measured throughput). Left of the ridge point = bandwidth-bound, right = compute-bound. The basis for choosing which optimization to invest in.

Comparison with Measured Silicon

The cycle model is calibrated against P06:

kernel	model predict	silicon measured	error
`matmul_systolic` (M=32 K=48 N=8)	456 cyc	460 cyc	+0.9%
`kws_npu_top` (full graph)	230 K cyc	2.11 M cyc (DMA-included wall)	bandwidth-bound regime

tests/hls/test_cycle_model_hls.py::test_matmul_systolic_matches_p06_silicon is the lock-test. CI fails if the formula drifts beyond ±10 %.

To shrink weight/activation cost with INT8 quantization → 05. quantize.