Field Notes

Paper Notes: DeepSeek-v3 Technical Report

last updated 2026-07-13

• llm
• speculative-decoding

DeepSeek trained MTP-1, that is a single predicted token. Interestingly, the focus is on getting better representations during training, rather than specifically as a speculative decoding trick.

They make passing note in section 2.2 about it being usable for speculative decoding, but without a serious discussion or analysis (unlike their later work on dSpark).

Our MTP strategy mainly aims to improve the performance of the main model, so during inference, we can directly discard the MTP modules and the main model can function independently and normally. Additionally, we can also repurpose these MTP modules for speculative decoding to further improve the generation latency.

However, in section 5 they do note that it gives a ~1.8x speedup with an 85-90% acceptance rate.

Based on our evaluation, the acceptance rate of the second token prediction ranges between 85% and 90% across various generation topics, demonstrating consistent reliability. This high acceptance rate enables DeepSeek-V3 to achieve a significantly improved decoding speed, delivering 1.8 times TPS (Tokens Per Second).

From Yifei Hu at reducto, they were seeing MTP-1 acceptance rates even higher:

fun fact: after some post-training, MTP-1 on our finetuned Qwen3.5 models has 98%+ acceptance rate. This allows us to run inference at high concurrency with MTP enabled.

— Yifei Hu (@hu_yifei) June 11, 2026