Hardware leaderboard
The hardware leaderboard ranks systems that serve open-weights models. A system is one server with a standard hardware name, for example "Dell PowerEdge XE9680 - 8x AMD Instinct MI300X". The board shows how much work each system does per GPU and how fast each request feels.
Two boards use the same ranking rules:
| Board | Where | Who sees it | What it lists |
|---|---|---|---|
| Public board | The landing page and /leaderboard/<entry> pages | Everyone, without an account | Entries that Metrum AI verified and published |
| Workspace board | Leaderboard in the left sidebar | Signed-in members of a workspace on the Pro plan or higher | Completed runs in your own workspace |
The public board is also available as a public API.
What an entry is
An entry is one published series of measurements. All points in an entry share these five values:
- The system (the hardware).
- The model.
- The precision, for example FP8 or BF16.
- The workload preset.
- The serving shape: the framework, the framework version, and the tensor parallel size.
An entry holds many concurrency points. Each point is one measured number of concurrent requests.
What the board compares
The board only compares entries that share the same model, precision, and workload preset. Together these three values are the comparability key. You pick them at the top of the board, and the board lists only the entries that match.
The workload preset sets the Input Sequence Length and the Output Sequence Length. For example, one preset sends 1,024 input tokens and asks for 1,024 output tokens. The board shows this as "1,024 in, 1,024 out".
The framework is not part of the key. Entries that use vLLM, SGLang, or TensorRT-LLM rank together, and each row shows its framework, version, and tensor parallel size.
Categories
The board has three categories. Each category ranks the entries of one comparability key.
| Category | What it ranks | Better is |
|---|---|---|
| Throughput per GPU | Total tokens per second divided by the GPU count, at the best measured point whose interactivity meets the target | Higher |
| Peak interactivity | The highest output tokens per second per request across all measured points | Higher |
| Time to first token | The median time to first token at the throughput point | Lower |
Throughput per GPU
The board divides the total tokens per second by the GPU count, so a system with 8 GPUs and a system with 4 GPUs compare fairly.
The board picks the point with the highest throughput per GPU among the points whose interactivity is at least the target. If no point meets the target, the entry shows "Target not reached" and has no rank.
Peak interactivity
Interactivity is the output tokens per second that one request receives. A higher value means a reader sees text appear faster. The board uses the best point of the entry. The value does not depend on the target.
Time to first token
Time to first token is the time from sending a request to receiving the first output token. The board shows the median (p50) at the throughput point. If the run did not measure it, the entry has no rank in this category.
Interactivity target
The throughput category needs a target. The target is the minimum interactivity a point must reach to count.
| Target | Meaning |
|---|---|
| 20 | At least 20 output tokens per second per request |
| 50 | At least 50 output tokens per second per request. This is the default. |
| 100 | At least 100 output tokens per second per request |
The board accepts only these three values. A higher target forces a lower concurrency, so throughput per GPU drops.
Rank and ties
Entries that reached the target receive a rank, highest value first. Entries with equal values share a rank and show a tie mark. Entries that did not reach the target appear after the ranked entries, sorted by system name.
Each row
Each row shows these values:
| Column | Description |
|---|---|
| Rank | Position in the current category |
| System | Public system name |
| GPUs | Number of GPUs in the system |
| Framework | Framework, version, and tensor parallel size |
| Throughput per GPU | Total tokens per second per GPU at the throughput point |
| Interactivity | Output tokens per second per request at the throughput point |
| Peak interactivity | Best output tokens per second per request |
| Time to first token | Median at the throughput point, when measured |
| Energy | Joules per output token at the throughput point, when measured |
| Release | The benchmark tool release that produced the data, for example metrumbench-llm 0.1.80 |
Select a row to open its entry page. The entry page shows the model, precision, Input Sequence Length, Output Sequence Length, GPU driver, CUDA or ROCm version, dataset, run completion time, publish time, and every concurrency point with its metrics.
Hardware names
An entry uses the accelerator name only, for example "AMD Instinct MI300X", unless the hardware partner has cleared the use of its name. Metrum AI adds the partner name to the system name only after that clearance.
How entries are verified and published
Only Metrum AI publishes entries. You cannot submit a run to the public board.
- A run that Metrum AI verified produces the measurements.
- A Metrum AI administrator opens the admin console and reviews the candidates. The console lists a run as eligible only when it meets the methodology rules below. For an ineligible run, the console states the reason in one sentence.
- If the run does not record its precision, the administrator chooses it.
- The administrator sets the public system name and records whether the hardware partner cleared its name.
- The administrator publishes the entry.
The board refuses a second publish of the same system, model, precision, workload preset, and framework configuration. The administrator must replace the earlier entry on purpose.
Frozen values
A published entry does not change. The board copies the measurements at publish time. A later edit to the source project, or a deletion of it, does not change the entry.
Rename and withdrawal
An administrator can rename an entry. An administrator can also withdraw an entry. A withdrawn entry leaves the board at once. Its entry page and its entry API route return "not found". A publish, rename, or withdrawal refreshes the cached copy of the public board.
Methodology
The board accepts a run only when it meets these rules:
| Rule | Value |
|---|---|
| Request count at each point | Locked to the larger of 100 and 10 times the concurrency |
| Streaming | Required, because time to first token needs the first streamed token |
| Tool release | Recorded with every entry as the release tag |
| Verification | A run that Metrum AI verified |
For the full load-generation method, see Performance methodology.
Workspace board
The workspace board ranks the completed runs in your own workspace. It uses the same three categories, the same interactivity targets, and the same ranking rules as the public board.
Open Leaderboard in the left sidebar. The workspace board needs the Pro plan or higher.
| Control | What it does |
|---|---|
| Model, Precision, Workload | Sets the comparability key |
| Category | Throughput per GPU, Peak interactivity, or Time to first token |
| Interactivity target | 20, 50, or 100 output tokens per second per request |
| Framework | Optional. Limits the list to one framework |
| Include private projects | Adds runs from private projects. Only workspace owners and admins can turn it on. |
Each workspace row links to the project, the run, and the run report. A row also shows whether Metrum AI has published the series on the public board.
If you turn on Include private projects without an owner or admin role, the request fails with an insufficient privilege error. The board does not quietly hide the private runs.
Your workspace runs never appear on the public board unless Metrum AI verifies and publishes them.
KYAI leaderboard
KYAI results rank by mean quality score, not by throughput. The v_kyai_leaderboard view returns the ranking through the API.