FineWeb WCS Model Suite

Decoder

Metric

Line Style

Base / pre-trained Instruct / tuned / distilled

Model Controls

Parameter reference markers: all plotted FineWeb curves remain evaluated at T=1.0.

Qwen2.5-14B-Instruct: top-p 0.8, top-k 20

Llama-3.1-8B-Instruct: top-p 0.9

DeepSeek-R1-Distill-Qwen-14B: top-p 0.95

Mistral-7B-Instruct-v0.3: top-p 0.9, standard vLLM/Transformers-style default when not overridden

top-p Target-word context coverage

Could not load fineweb_wcs_word_summary.csv automatically. If you opened this page directly from disk, choose the CSV file here.