Quickstart

Run your first LLM inference benchmark on Metrum Insights in under 15 minutes. By the end of this guide you'll have a project, a workload, a completed job, and a results view on the Benchmarks page that you can share with your team.

What you'll do

1. Sign in and land on the dashboard
2. Pick a GPU server to run on
3. Create a project with a single workload
4. Save and execute the run
5. Read the results on the Benchmarks page

If you only have a few minutes, jump to Save and execute; the rest is context.

Before you start

You'll need:

• An invitation to a Metrum Insights workspace, or the ability to create your own account at https://<your-control-plane>. The UI calls a tenancy a Workspace in the top-right switcher; the underlying data model still uses org_id.
• At least one registered GPU server visible to your workspace. If you don't see any servers under Hardware in the sidebar, ask your platform admin to onboard one (see the Admin Guide) before continuing.
• A rough idea of what you want to measure. For this Quickstart we'll benchmark Llama 3.1 70B Instruct on vLLM at three concurrency levels ~ a good "is this hardware healthy?" smoke test.

---

1. Sign in

1. Open https://<your-control-plane>. The app does not show a marketing landing page here ~ unauthenticated visitors are redirected straight through auth.metrum.ai (Auth0) to your identity provider.
2. Complete the SSO flow. Google is enabled by default; SAML/OIDC providers (Microsoft Entra, Okta, custom OIDC) are configurable per workspace.
3. After sign-in you'll land on /dashboard.

You'll land on the Dashboard:

Screenshot placeholder: Dashboard after first login. The dashboard greets you with four KPI tiles (PROJECTS, ACTIVE JOBS, SERVERS ONLINE, ACTIVE RUNS) and five widgets: Throughput Leaderboard, Recent Activity, Server Fleet, Recent Runs, and Cloud. None of these will be populated yet on first login ~ that's expected.

The left-hand sidebar is your primary navigation. The pages you'll touch in this Quickstart are Hardware, Projects, and Benchmarks.

---

2. Confirm a server is available

Before creating the project, verify there's a GPU server you can target.

1. Click Hardware in the sidebar (URL: /hardware).
2. You should see at least one server row in the Hardware table with status Online and a recent heartbeat.
3. Note the Hostname and Config Code of the server you want to use ~ you'll select it in step 4.

Screenshot placeholder: Hardware page with one online server.

If no servers are listed, click Add Server to register one yourself (you'll need shell access to the GPU box), or ask an admin to do it. The bootstrap flow is covered in the Admin Guide.

If a server is listed but offline, the agent isn't reporting heartbeats. Check that the metrum-agent service is running on the host and that the machine can reach the control plane.

---

3. Create a project

A project in Metrum Insights is a named container for one or more workloads. Each workload describes a model + framework + scenario matrix to benchmark.

1. Click Projects in the sidebar.
2. Click New Project in the top-right corner of the projects page.

You'll land on /projects/new.

Screenshot placeholder: New Project page, empty state.

Fill in the project section

Field	Value for this Quickstart
Project Name	Llama 3.1 70B Quickstart
Visibility	Private (the default in the live UI is Organization; change it to Private for this Quickstart)

Visibility controls who in your workspace can see the project:

• Private ~ only you.
• Organization ~ anyone in your workspace with the appropriate role. This is the form default.
• Public ~ visible across the platform (rare; usually reserved for reference benchmarks).

Set it to Private for now ~ you can change it later.

Fill in the workload card

A new project starts with one empty workload card titled Workload 1. Configure it as follows:

Field	Value
Workload Name	llama-70b-vllm-baseline
Model	pick a Llama 70B-class entry from the Model dropdown (the dropdown lists the registered model catalog)
Framework	vLLM
Version	0.6.3 (stable) (selectable once Framework is set)
Concurrency chips	toggle on 1, 8, 32
Input Sequence Length chips	toggle on 512
Output Sequence Length chips	toggle on 256
Requests per scenario	10 (the default; total requests per scenario)
Streaming	leave checked (default on)

A few notes:

• The Scenario Matrix section uses chip-button multi-select rather than free text. Available concurrency chips are 1, 4, 8, 16, 32, 64, 128. ISL chips are 128, 256, 512, 1024, 2048. OSL chips are 64, 128, 256, 512. Selecting 1, 8, 32 for concurrency with one ISL and one OSL produces a 3×1×1 = 3-scenario matrix.
• Click Show details under the workload card to preview the rolled-out scenarios; they're named c<concurrency>-isl<isl>-osl<osl> (for example c1-isl512-osl256).
• Requests per scenario controls the total number of requests fired per scenario. Streaming controls whether the benchmark consumes the response as a server-sent stream.
• Each workload card also offers Add Benchmark and Add KYAI Run buttons for adding additional benchmark or quality-evaluation steps. Leave these alone for the Quickstart.

Don't add a second workload yet ~ Save and Execute only works when there's exactly one workload (see the note below).

Pick the server

Scroll to the Server section. You should see the server you confirmed in step 2 as a card with a radio button. Each card shows the hostname on the first line and the server config code on the second line (for example localhost / local-dummy).

1. Click the card to select it.
2. Both action buttons at the bottom-right (Save Project and Save and Execute) should now be enabled. A Reset button sits next to them.

Screenshot placeholder: Workload and server filled in, Save and Execute enabled.

Why only one workload?

Save and Execute runs the project immediately on the selected server. To keep that simple and reliable, the action is restricted to projects with a single workload and an explicit server. Multi-workload projects are saved as drafts via Save Project and executed later via the run flow ~ covered in the User Guide.

---

4. Save and execute

Click Save and Execute.

You'll be redirected to the Project Detail page at /projects/<id>, then to the Run Detail page once the run is created. The header shows:

• Run name ~ auto-generated from the workload (e.g. llama-70b-vllm-baseline-r1)
• Status badge ~ starts as pending, transitions to queued, then running
• Started at / Last updated ~ refreshed on each status change

The Jobs table lists one row per scenario. Job names follow the pattern c<concurrency>-isl<isl>-osl<osl>. For our 3-scenario matrix you'll see three rows:

Job name	Tool	Concurrency	ISL	OSL
c1-isl512-osl256	metrumbench-llm	1	512	256
c8-isl512-osl256	metrumbench-llm	8	512	256
c32-isl512-osl256	metrumbench-llm	32	512	256

Screenshot placeholder: Run detail with three jobs running.

Watching progress

You can stay on the Run Detail page (it polls for status updates) or switch to the Monitoring page for a live view of every running job across the platform.

A healthy run progresses through these statuses on each job:

pending → queued → running → completed

If a job lands in failed, the Tool column links to the job's logs. The most common first-run failure is a model download timeout (Llama 3.1 70B is ~140 GB in fp16); give the server a few minutes if it's the first time pulling this model. If the failure looks more serious, see Troubleshooting (post-launch).

For Llama 3.1 70B on vLLM, expect each scenario to take 2 to 6 minutes on a single H100/H200 node depending on concurrency, plus a one-time model load on the first job.

---

5. Read the results in Benchmarks

Once at least one job is completed, head to Benchmarks in the sidebar (/benchmarks). The page subtitle reads "Full results across all benchmark runs. Filter, compare, and analyse performance."

Screenshot placeholder: Benchmarks page with results from the Quickstart run.

Filter to your run

The Benchmarks page exposes a Filters dropdown and a My runs toggle pill at the top of the results table.

1. Click Filters to open the panel and narrow by run/project/model/framework as needed.
2. Toggle My runs on to restrict to runs you launched.

You should see your three jobs in the results table. Click a row to expand its detailed metrics.

Read the key metrics

Four numbers will tell you most of what you need to know:

• Throughput (tokens/sec) ~ total output tokens generated per second across all concurrent requests. Should scale up as concurrency rises, until you saturate the GPU.
• TTFT P50 / P99 ~ Time To First Token. P50 is typical user experience; P99 is your tail latency. Both grow with concurrency.
• TPOT ~ Time Per Output Token after the first one. This is the steady-state generation speed.
• GPU power / utilization ~ from telemetry. A healthy benchmark pushes utilization above 80% at the higher concurrency levels.

Visualize across scenarios

Scroll below the table to the chart panel. The chart sub-tabs are Model, Framework, Quantization, Concurrency, Cost, and Hardware. Open the Concurrency tab to see throughput and latency plotted as a function of concurrency. The classic shape is:

• Throughput rises sharply, then flattens ~ that's your saturation point.
• P99 latency stays flat, then climbs ~ that's queueing.

The concurrency at the elbow of the throughput curve is the rough "sweet spot" for this hardware at this sequence-length shape.

---

What's next

You've now run a single-workload benchmark end-to-end. From here:

• Run a richer matrix. Add more concurrency values, more ISL/OSL combinations, or a second framework (try SGLang or TensorRT-LLM on the same model) to compare side-by-side.
• Add a second workload. Same model, different quantization (e.g. fp16 vs fp8). The Quantization chart tab on Benchmarks will show you the throughput-vs-quality tradeoff.
• Try Sizer. If you're picking hardware for a new use case, the Sizer Usage Guide walks through getting a recommendation before you commit to a deployment.
• Evaluate quality with KYAI. Throughput numbers don't tell you whether the model's outputs are still good. KYAI runs a two-phase generation + LLM-judge pipeline. The KYAI execution path is wired into projects via the Add KYAI Run action on each workload card; a dedicated KYAI navigation surface is planned but not yet exposed in the UI.
• Bring your own endpoint or workload. BYOE lets you point Metrum at an existing inference endpoint (your own deployment, a third-party API). BYOW lets you supply your own prompt dataset. Both are covered in the User Guide.

Common pitfalls

A few things first-time users trip over:

• Concurrency too high for the model. Setting concurrency to 128 on a 70B model with 80 GB of GPU memory will OOM. Start at 8, find the saturation point, then push higher.
• Mismatched ISL/OSL. If your real workload sends 4k-token prompts and you benchmark at 512, your numbers will be optimistic. Match the shape of production traffic.
• One server, many concurrent runs. Each run holds the GPU exclusively for the duration of its jobs. Queue up runs on the project; don't fire them off in parallel on the same server.
• Stale browser tab. The Run Detail page polls for updates, but if your tab has been backgrounded for a long time, refresh it to force a re-fetch.

Getting help

• Email notifications ~ planned per-user setting (the in-product Notifications panel is not yet exposed; see the User Guide for the current workaround).
• Docs ~ the User Guide covers every workflow this Quickstart skipped; the Feature Reference is the per-feature deep-dive.