Skip to main content

Pulse

Pulse is a collaborative benchmark intelligence workspace. It lets you select any set of completed benchmark jobs from across your team's projects, group them into a named View, and then explore the combined performance data through charts and tables - all in one place, without navigating project-by-project.

Navigate to Pulse in the left sidebar (/pulse).

Core concepts​

Job - any single completed benchmark job (one concurrency level, one model, one hardware target, one tool). Pulse can include jobs from any project and workload.

View - a named, curated collection of benchmark jobs. A View is the unit you explore, share, and save. Think of it as a custom benchmark report you assemble from your team's job history.

Curator - the panel where you browse the full pool of available jobs and select which ones belong in a View.

Lens - the panel where you read the charts, tables, and quality reports for the active View.

The two tabs​

The workspace is divided into two top-level tabs:

Curator tab​

The Curator tab is visible to workspace members with write access to Pulse. It shows you the full pool of completed benchmark jobs and lets you curate them into Views.

  • The left panel lists all your saved Views (the View List).
  • The main area shows the Curator job browser: a searchable, filterable list of every completed benchmark job in scope.
  • Each job appears as a card showing the project name, workload name, model, framework, hardware label, tool name, and scenario details.
  • Tick the checkbox on any job card to add it to your current selection.
  • The job browser has a tool filter tab strip at the top: All, MetrumBench LLM, MetrumBench VLM, MetrumBench ASR, MetrumBench ImageGen, GenAI-Perf, InferenceX. Switch tabs to narrow the list to jobs from a specific tool.
  • Use the search box to search by project name, model, hardware, framework, or scenario text.

You can also toggle private projects in the Curator. When enabled, jobs from your own private projects appear in the pool alongside the shared workspace jobs.

Lens tab​

The Lens tab shows the full benchmark data for whichever View is currently selected in the View List on the left. If no View is selected, the Lens is empty.

The View List​

The left sidebar in both tabs shows your list of saved Views. Pinned Views appear first, sorted in your chosen order. Unpinned Views follow alphabetically.

  • Click any View name to open it in the Lens.
  • Pinned Views can be reordered by drag-and-drop or by using the up/down arrow buttons on each row (keyboard-accessible).
  • A pin icon appears next to pinned View names.

Creating a View​

  1. Go to the Curator tab.
  2. Browse the job pool and tick the checkboxes on the jobs you want to include.
  3. Click Create view in the View List sidebar.
  4. Enter a name for your View.
  5. The View is saved and immediately becomes the active View in the Lens.

You must select at least one job before you can create a View. The selection count is shown next to the Create view button.

Editing a View's job list​

  1. Open the View in the Lens (click its name in the View List).
  2. Click the Edit button in the View toolbar.
  3. The Curator opens with the current View's jobs pre-selected.
  4. Tick or untick jobs to add or remove them.
  5. Save when done.

Renaming a View​

  1. Open the View in the Lens.
  2. Click the Rename button in the View toolbar.
  3. Type the new name in the inline dialog and confirm.

What the Lens shows​

When a View is open in the Lens you will see, from top to bottom:

View toolbar​

The header bar shows the View name and a row of action buttons. The buttons available to you depend on your permissions for this View:

ActionWhat it does
EditOpens the Curator to change which jobs are in this View.
RenameRenames the View inline without opening the Curator.
ShareOpens the Share dialog to invite specific email addresses.
PinPins or unpins the View to the top of your View List.
PublishMakes the View publicly accessible via a shareable link.
CopyDuplicates the View under a new name.
Download CSVExports all benchmark results in this View as a CSV file.
DeletePermanently removes the View.

A Published badge appears on the toolbar when the View is public. A Pinned badge appears when it is pinned.

Filter bar​

Below the toolbar is a filter bar that narrows the charts and table to a subset of the jobs in the View. Filters available in Pulse:

  • Model
  • Framework
  • Hardware label
  • Tool
  • Concurrency
  • Input sequence length (ISL)
  • Output sequence length (OSL)
  • GPU count
  • Precision

Filters work across all sections in the Lens simultaneously. The charts, data table, and KYAI section all reflect the same active filters.

Charts​

Charts are organized into categories matching the Reporting page:

CategoryChart types included
ThroughputTokens per second vs concurrency, requests per second vs concurrency, total throughput
LatencyTTFT, TPOT, p50/p95/p99 latency vs concurrency
EfficiencyTokens per second per GPU, interactivity (tokens per user)
QualityKYAI pass rate, quality score (when KYAI results are present in the View)
CostUSD per million tokens, tokens per dollar
Hardware utilizationGPU utilization, GPU power, GPU memory, CPU utilization, system power

Each category header is collapsible. You can collapse sections you are not interested in to focus on what matters for your current comparison.

Charts in Pulse use the same rendering as the Reporting page, so results look familiar if you have used Reporting before. When a View contains jobs across multiple workloads, the charts show each workload as a separate section, labeled with the workload name and tool type. This makes it easy to compare, for example, a vLLM workload and a SGLang workload for the same model on the same hardware side by side.

Data table​

Below the charts is a Data Table showing one row per benchmark job in the View. The table includes the same columns as the Reporting page: model, framework, hardware, scenario, throughput, latency, cost, and quality metrics. You can sort by any column.

KYAI section​

If any of the jobs in the View are linked to a KYAI evaluation run, the KYAI section appears below the benchmark data. It shows the quality evaluation results for those jobs: judge scores, pass rates, sample-level details, and truncation warnings.

Sharing a View​

Share by email​

  1. Open the View in the Lens.
  2. Click the Share button in the toolbar.
  3. Enter one or more email addresses in the input field (comma or space-separated).
  4. Click Send invitations.
  5. The recipients receive an email with a direct link to the View.

Shared users can view the View and its data. They cannot edit or delete it unless you explicitly give them write access.

To revoke a share, open the Share dialog again. Existing shares are listed below the email input. Click the X next to any share to remove it.

Publish a View​

Publishing makes the View accessible to anyone with the link, inside or outside your workspace, without requiring them to be logged in.

  1. Open the View in the Lens.
  2. Click the Publish button in the toolbar.
  3. Confirm the action. A Published badge appears and a shareable URL is shown.

To unpublish, click the Publish button again (it toggles). The View becomes private immediately and the shareable link stops working.

Note. Only users with write access to Pulse can publish or unpublish Views.

Pinning a View​

Pinning a View moves it to the top of your View List. Pinned Views appear before unpinned Views and are reorderable by drag-and-drop or keyboard arrows.

Click the Pin button in the toolbar to toggle pinning on or off.

Copying a View​

Click Copy in the toolbar to duplicate the View. The copy gets the name Copy of <original name>. It starts unpinned and unpublished and contains the same jobs and filters as the original. You own the copy and can edit, share, or delete it independently.

Downloading as CSV​

Click Download CSV to export all benchmark result rows in the active View (after filters are applied) as a comma-separated file. The exported file contains the same columns shown in the data table, plus additional raw metric columns not shown in the table by default.

The filename is based on the View name and the current date.

Permission model​

Who can do whatWrite access to PulseRead-only access
Open Pulse and view ViewsYesYes
See the Curator tabYesNo
Create, edit, rename, delete ViewsYesNo
Share or publish ViewsYes (on Views they have access to)No
Pin ViewsYesYes (pin for themselves)
Download CSVYesYes

Write access to Pulse is granted at the workspace level by a workspace admin.

Tips for effective Pulse use​

  • Narrow your selection in the Curator. A View with 200 jobs is hard to read. Aim for 5 to 20 jobs per View, grouped around a specific question: "vLLM vs SGLang on this model" or "How does this hardware scale with concurrency?"
  • One View per question. Pulse works best when each View has a clear purpose. Name it to reflect that purpose: Llama 4 Scout - throughput comparison is more useful six months from now than Test view 3.
  • Combine performance and quality. If you have run a KYAI evaluation alongside your benchmark jobs, include the KYAI jobs in the same View. The KYAI section in the Lens will appear automatically alongside the performance charts.
  • Share before the meeting. Use Publish or Share-by-email to send a View to teammates before a design review or planning session. They can explore it in the browser without needing a Metrum Insights account (if published publicly) or with just a team invitation.
  • Use filters in the Lens. If a View spans multiple concurrency levels or ISL values, apply a filter in the Lens to focus a single dimension at a time. The charts update immediately.