Quality
Task success, factuality, instruction following, tool-call validity.
Qwen3.8 27B benchmark
Qwen3.8 27B benchmark scores need their test conditions.
These are selected results reported by Qwen for the original checkpoint. Use them to form a test plan—not as a guarantee for your quant, runtime, prompt, or hardware.
Selected official results
Scores are reproduced from the model card available on August 22, 2026.
| Benchmark | Reported score | What it probes |
|---|---|---|
| TerminalBench 2.1 | 73.0 | Agentic terminal coding |
| SWE-bench Pro | 61.7 | Repository-level software engineering |
| LiveCodeBench v6 | 90.3 | Competitive code generation |
| OSWorld | 84.3 | Visual computer-use tasks |
| WebArena | 64.8 | Web agent tasks |
Source: Official Qwen3.8 27B model card and evaluation settings
Read scores safely
Four questions before you compare.
- 01
Was thinking enabled?
Reasoning effort and token budget can materially change both score and cost.
- 02
Was the original checkpoint used?
FP8, GGUF, AWQ, and other conversions may behave differently from the published evaluation.
- 03
Were tools and scaffolds identical?
Agent benchmarks depend on tool permissions, environment images, timeouts, and retry rules.
- 04
Does the benchmark match your task?
A coding score does not validate your support workflow, extraction schema, or domain language.
Your own evaluation
A small, honest test beats a large borrowed table.
Build 30–100 representative cases, freeze the prompt and runtime, record latency and failure categories, and review the same outputs across every model candidate. Keep the test set private if it mirrors production data.
Efficiency
Time to first token, completion latency, input and output tokens.
Reliability
Timeouts, malformed output, repetition, unsafe or ungrounded answers.