Qwen 2.5

Qwen 2.5 model family

Qwen 2.5 (Alibaba Cloud, September 2024) is a family of open-weight language models ranging from 0.5B to 72B parameters. The series uses GQA, RoPE positional encoding, and SwiGLU activation. Context window: 128k tokens for most sizes. The family includes three specialised variants: Qwen2.5-Coder (7B, 14B, 32B, 72B) for programming, Qwen2.5-Math (1.5B, 7B, 72B) for mathematics, and the general-purpose Qwen2.5 series for instruction following.

Key capabilities

Qwen 2.5 is particularly notable for: (1) Multilingual coverage — 29 languages including Chinese, Japanese, Korean, Arabic, French, German, Spanish, and more. This is broader than Llama 3 which focuses primarily on English and code. (2) Coding — Qwen2.5-Coder-32B outperforms GPT-4o on HumanEval and several competitive programming benchmarks. (3) Math reasoning — Qwen2.5-Math-72B achieves top performance on MATH, GSM8K, and competition mathematics. (4) Instruction following — strong performance on IFEval and complex instruction chains.

Running Qwen 2.5 in Python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "Qwen/Qwen2.5-7B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain the difference between GQA and MHA."},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
    )

# Decode only newly generated tokens
new_tokens = output_ids[0][inputs.input_ids.shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

Qwen2.5-Coder

Qwen2.5-Coder is pre-trained on 5.5T tokens of code and code-related text, then instruction-tuned for programming tasks. The 32B variant is particularly strong — it outperforms GPT-4o on HumanEval (92.7% vs 90.2% pass@1) and achieves state-of-the-art on EvalPlus, LiveCodeBench, and SWE-Bench verified. It supports 92 programming languages with dedicated tokenization. For code generation, code explanation, debugging, and code completion tasks, Qwen2.5-Coder-32B is one of the best open models available.

model_name = "Qwen/Qwen2.5-Coder-7B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype=torch.bfloat16, device_map="auto"
)
messages = [
    {"role": "user", "content": "Write a Python function to find all prime numbers up to n using the Sieve of Eratosthenes."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=512)
new_tokens = output[0][inputs.input_ids.shape[1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

Quantised versions

GPTQ and AWQ quantised versions are available on HuggingFace for reduced VRAM requirements:

• Qwen2.5-7B-Instruct-GPTQ-Int4: ~4.5 GB VRAM (vs ~15 GB bfloat16)
• Qwen2.5-14B-Instruct-AWQ: ~8.5 GB VRAM (vs ~29 GB bfloat16)
• Qwen2.5-72B-Instruct-GPTQ-Int4: ~40 GB VRAM — fits on 2×A100 40GB

GGUF versions are also available for use with llama.cpp and Ollama: ollama run qwen2.5:7b.

Multilingual usage

# Qwen 2.5 handles mixed-language contexts well
messages = [
    {"role": "system", "content": "你是一个有帮助的助手。"},  # Chinese system prompt
    {"role": "user", "content": "Summarize this in Japanese: The capital of France is Paris."},
]
# The model will respond in Japanese as instructed
# Also works for: Arabic, Korean, Vietnamese, Thai, and 25 other languages

Gotchas

• Chat template is mandatory: Qwen 2.5 instruct models require the chat template via tokenizer.apply_chat_template(). Feeding raw text without the template produces poor results.
• Trust remote code: Some Qwen model configurations require trust_remote_code=True when loading with HuggingFace. This executes code from the repo — only do this from the official Qwen HuggingFace repo.
• Context window in practice: 128k context is the official limit, but performance degrades on tasks requiring retrieval from >32k tokens in many models. Test with your specific use case before relying on very long contexts.