100x slowed down
Detailed20x
BATCH · 1 request
1512
GPU mostly waiting on memory · checking guesses is almost free
TARGET PASSES0
TOKENS0
TOKENS / PASS–
LAST STEP–
AT REALTIME–
VS NORMAL–
NormalSpeculative
TARGET0.0 GB
DRAFT HEAD · 0.8 GB a step—
16 GB ÷ 1.6 TB/s = 10 ms a passREAD PER WORD –
Now: speculative decoding ↗