Hardware Leaderboard
The Hardware Leaderboard ranks every completed benchmark job in your team's workspace across ten distinct performance dimensions. It answers one question: given the models and hardware your team has already benchmarked, which configuration delivers the most useful output per unit of resource?
Navigate to Leaderboard in the left sidebar (/leaderboard).
What the Leaderboard shows
Each row represents a single benchmark job: one GPU configuration, one model, one framework version, one scenario (concurrency level, input length, output length). The leaderboard aggregates results from all completed benchmark jobs that have been shared within your workspace, including jobs from all projects where visibility is set to Organization or Public.
You can optionally include results from your own Private projects by toggling Include private Team projects in the scope controls at the top of the page (only available to users with admin access to the workspace).
Reading the top of the page
When you open the Leaderboard you will see, in order from top to bottom:
- Scope controls - team label, private project toggle, and a staleness timestamp showing when the most recent entry in the current view was last updated.
- Metric mode selector - switches what the page ranks on (see below).
- About these metrics - a collapsible panel explaining the scoring philosophy and each metric mode in plain language. Click the row to expand it.
- Podium - the top three configurations for the current metric mode, shown as highlighted cards.
- Pareto frontier chart - a scatter plot comparing two dimensions at once, with the Pareto-dominant configurations highlighted.
- Ranked table - the full ranked list, 50 rows per page.
Metric modes
The Leaderboard has ten metric modes. Switch between them using the row of mode buttons at the top right of the page. The table re-ranks and the podium updates immediately.
| Mode | What it measures | Direction |
|---|---|---|
| Recommended | Weighted score: interactivity 40%, per-GPU efficiency 20%, p95 latency 20%, TTFT 10%, cost 10% (when available). Error-penalized. This is the default. | Higher is better |
| Interactivity | Output tokens per second per concurrent user. This is what each individual user feels while reading a streaming reply. | Higher is better |
| Raw throughput | Total output tokens per second, before any quality or reliability discount. | Higher is better |
| Latency | P95 end-to-end response latency in milliseconds, with TTFT as a tie-breaker. | Lower is better |
| Cost | USD per million output tokens, derived from your server's hourly price and measured throughput. Only rows with cost data qualify. | Lower is better |
| Per GPU | Output tokens per second divided by GPU count. Measures hardware utilization efficiency. | Higher is better |
| Requests | Completed requests per second. Useful for capacity planning. | Higher is better |
| Energy | Joules of measured system power per output token. Only rows where power telemetry was recorded qualify. | Lower is better |
| Useful tok/s per GPU | Tokens per second that passed the quality threshold, divided by GPU count. Requires a KYAI evaluation on the same model, framework, and GPU. | Higher is better |
| Useful throughput | Total tokens per second that passed the quality threshold. Requires a KYAI evaluation. | Higher is better |
Modes marked Needs quality data only rank rows that have a matching KYAI evaluation for the same model, framework, and GPU combination. Rows without a quality evaluation stay in the table for all other modes but are excluded from the quality-gated ranking.
Tip. Start with Recommended to get a balanced view of your results. Switch to Cost when comparing cloud GPU options. Switch to Useful throughput after running a KYAI evaluation to see which configuration delivers the most correct answers per second.
The podium
The three cards at the top of the page show the first-, second-, and third-ranked configurations for the current metric mode. Each card shows:
- GPU model, vendor, and total VRAM
- Model name
- Primary metric value (the number the mode ranks on)
- Quality badge: KYAI pass rate percentage if a quality evaluation was run, or "No quality eval" if not. A Truncated answers warning appears if the model hit its context limit during the KYAI evaluation, which makes the score unreliable.
- Framework and scenario (concurrency, ISL, OSL)
- Supporting metrics: tok/s/GPU and cost per million tokens where available
- Rank reason - one sentence explaining exactly why this configuration holds this rank
- A link to the full benchmark report for the run that produced this result
Pareto frontier chart
Below the podium, the Pareto frontier chart shows a scatter plot of all entries on the current page across two axes at once. You can switch between three axis pairs:
- Useful tok/s/GPU vs latency - throughput efficiency on the x-axis, p95 latency on the y-axis.
- Cost vs throughput - raw throughput on the x-axis, cost per million tokens on the y-axis.
- Energy vs throughput - raw throughput on the x-axis, joules per token on the y-axis.
Points on the Pareto frontier - configurations where no other entry is strictly better on both axes at the same time - are highlighted. Everything below-and-right of the frontier is dominated by at least one other entry.
If the page is showing a filtered subset, the chart adds a note: the Pareto frontier is bounded to the entries currently visible, so a configuration on a later page could still dominate a point shown here.
Click any point on the chart to scroll the table to that row.
Ranked table
The table lists all entries in rank order for the current metric mode. Each row shows:
| Column | Description |
|---|---|
| Rank | Numeric rank for the current mode. Gold/silver/bronze styling on the top three. |
| GPU | GPU model name and count. |
| Hardware | System label (the hostname or hardware label from your server configuration). |
| Model | Model name being benchmarked. |
| Framework | Inference framework (vLLM, SGLang, TensorRT-LLM, etc.). |
| Scenario | Concurrency, input sequence length, and output sequence length. |
| Primary metric | The value the current mode ranks on, formatted with its unit. |
| p95 latency | P95 end-to-end response latency. |
| Cost / 1M tok | USD per million output tokens (shown as a dash if not available). |
| Quality | KYAI pass rate if available, otherwise empty. |
Click any row to expand the Row details panel, which shows the full breakdown grouped into four sections:
Identity - project name, run name, job name, workload, scenario, tool, completion time, and a direct link to the run's benchmark report.
Hardware - GPU model, vendor, GPU count, VRAM total, system label, CUDA version, ROCm version, driver version, and hostname.
Model and stack - model name, model family, framework name, framework version, and build profile (CUDA, cu129, ROCm, etc.).
All metrics - every metric the leaderboard computes: throughput, interactivity, per-GPU throughput, requests/sec, latency (TTFT, p50, p95, p99), cost, energy, and the quality-gated useful throughput variants.
Filters
The Filters panel runs down the left sidebar. All filters are multi-select chip buttons. Click a chip to include only rows that match; click it again to deselect. Active filters are shown as filled chips.
| Filter | Description |
|---|---|
| GPU | Restrict to specific GPU models (e.g. H100, H200, A100). |
| Model | Restrict to specific models (e.g. Llama 3.1 70B, Qwen 2.5 72B). |
| Hardware | Restrict to specific hardware labels (the system label on your server). |
| Framework | Restrict to specific inference frameworks. |
| ISL | Restrict to specific input sequence lengths. |
| OSL | Restrict to specific output sequence lengths. |
| Concurrency | Restrict to specific concurrency values. |
| GPU count | Restrict to 1-GPU, 2-GPU, 4-GPU, 8-GPU, and multi-node configurations. |
| Precision | Restrict to specific weight dtype/precision (e.g. fp16, bf16, fp8). |
| Quantization | Restrict to specific quantization methods. |
Use the search box at the top of the filter panel to search across model name, hardware label, framework, and project name simultaneously.
Click Clear filters (the X button next to the Filters heading) to reset everything at once.
How entries get into the Leaderboard
The Leaderboard automatically picks up every completed benchmark job from all Organization and Public projects in your workspace. No manual submission is required. Results appear within a few minutes of the job completing.
If a job is in a Private project, it only appears if you have toggled Include private Team projects on, and you must be an admin of the workspace to see that toggle.
The Leaderboard does not de-duplicate or average results. If you ran the same model on the same hardware ten times, all ten jobs appear as separate rows. This is intentional: it lets you see run-to-run variance and track improvements over time.
Quality data and KYAI
The two quality-gated modes (Useful tok/s per GPU and Useful throughput) require a matching KYAI evaluation. A "matching" evaluation is one that ran on the same model, the same framework version, and the same GPU type as the benchmark job.
If you have not run a KYAI evaluation yet, those two modes will show an empty state. All other modes work without quality data.
A Truncated answers badge on a podium card or a warning icon in the quality column of the table means that one or more of the model's responses during the KYAI evaluation were cut off at the model's context limit. A truncated answer cannot be scored properly, so the quality pass rate for that entry is not directly comparable to an entry where all answers were complete.
Pagination
Results are shown 50 per page. Use the Previous and Next buttons at the bottom of the table, or click a specific page number, to move between pages. The URL updates with the current page number so you can bookmark or share a specific page.
Interpreting the results
A few patterns to look for:
- Throughput saturation. If raw throughput at concurrency 32 is not much higher than at concurrency 8, the GPU memory bandwidth or compute capacity is fully saturated. The concurrency at the elbow is your operating sweet spot for this hardware-model combination.
- Per-GPU efficiency. A 4-GPU configuration that scores the same total throughput as a 1-GPU configuration means three GPUs are essentially idle at this model size. The per-GPU mode will show this immediately.
- Cost vs speed tradeoff. In Cost mode, a smaller GPU that runs a quantized model may rank above a larger GPU running fp16. That is expected: fewer dollars per token, even at lower absolute throughput.
- Quality discount. Switching from Raw throughput to Useful throughput sometimes reorders the ranking significantly. A configuration that is fast but produces short, failing answers may rank lower in quality-weighted modes.