Hardware Sizer API

This page documents the Sizer API surface exposed through Metrum Insights. It is intended for product integrations, workspace automation, and users who need to understand the exact request and response shape behind the Sizer UI.

This reference focuses on the product endpoints available through Metrum Insights.

Endpoints

Method	Path	Purpose
`GET`	`/api/sizer/status`	Check workspace configuration: whether Sizer is enabled for the workspace.
`GET`	`/api/sizer/health`	Check service readiness: whether the Sizer service is reachable right now.
`GET`	`/api/sizer/gpu-skus`	List GPU models accepted by Sizer inventory and hardware filters.
`POST`	`/api/sizer/solve`	Run a structured sizing request.
`POST`	`/api/sizer/agent`	Run a natural-language Sizer chat request.
`POST`	`/api/sizer/export`	Enqueue benchmark export to sizer.
`GET`	`/api/sizer/export-jobs`	List sizer export/withdraw jobs for an org.
`POST`	`/api/sizer/train`	Trigger sizer training for an org.

Use the same authentication pattern as the rest of your Metrum Insights deployment.

0. API Environment

Set the control-plane URLs and identity values. Sizer endpoints are exposed through the Insights frontend at /api/sizer/*.

export METRUM_INSIGHTS_URL="https://<control-plane-host-or-dns>"
export METRUM_API_URL="https://api.<control-plane-domain>"
export METRUM_ORG_ID="<org-uuid>"

Get your API token from the Insights frontend:

Open the Insights frontend in your browser.
Go to Profile → Tokens and generate a new token.
Copy the token value and export it:

export METRUM_JWT_TOKEN="<paste-your-token-here>"

This token is a signed JWT that can be used directly as a Bearer token for all Sizer and PostgREST API calls. Tokens created from the Profile page expire based on the TTL you select during creation.

Verify the token before creating sizer requests:

curl -fsS -X POST "$METRUM_API_URL/rpc/get_current_user_context" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}' | jq

Do not paste bearer tokens, provider keys, onboarding tokens, or bootstrap script URLs into tickets or chat.

1. Check Sizer Availability

curl -fsS "$METRUM_INSIGHTS_URL/api/sizer/status" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" | jq

curl -fsS "$METRUM_INSIGHTS_URL/api/sizer/health" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" | jq

If enabled is false or health returns 503 SIZER_NOT_CONFIGURED, stop here and ask an admin to set HARDWARE_SIZER_URL and SIZER_JWT_SECRET.

2. Discover Available Catalog Data

GPU SKUs

List GPU models the sizer recognizes. Use these slugs in allowed_gpu_models and agent prompts.

curl -fsS "$METRUM_INSIGHTS_URL/api/sizer/gpu-skus" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" | jq '.gpu_skus[] | {gpu_model, gpu_vram_gb, vendor}'

Reference the same catalog via PostgREST:

curl -fsS "$METRUM_API_URL/v_sizer_reference_gpu_models?select=gpu_model,gpu_display_name,gpu_vendor_code,gpu_vram_gb&order=gpu_model.asc" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" | jq

Common GPU codes in the seed catalog: h100, h200, b200, b300, mi300x, mi325x, a100, a100-80g, l40s, rtx-pro-6000, rtx-4090.

AI Models

Reference models available for sizing:

curl -fsS "$METRUM_API_URL/v_sizer_reference_ai_models?select=model_code,model_name,model_params_b,family_name&order=model_code.asc" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" | jq

Common models: llama-3.1-70b-instruct, llama-3.1-405b-instruct, deepseek-v3, deepseek-r1, deepseek-v4-pro, deepseek-v4-flash, qwen-3-8b, qwen-3.5-9b, microsoft-phi-4, mistral-small-3.1-24b-instruct, mistral-large-instruct-2411, kimi-k2-instruct, llama-4-scout-17b-16e-instruct.

3. Structured Solve API

The main structured route is:

POST /api/sizer/solve

Modality (fit.modality_code) is required. Each request also needs at least one sizing signal beyond modality. The sizing_goal field tells the sizer what type of question you are asking.

3.1 Find Viable Hardware

Ask the sizer to find the smallest or most efficient hardware footprint for a specific model stack.

Hardware-focused: smallest footprint for a 70B model

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "find_viable_hardware",
      "display_model": "Llama 3.1 70B Instruct",
      "framework": "vllm",
      "precision": "fp8",
      "allowed_gpu_models": ["h200", "b200", "h100"]
    }
  }' | jq

Model-focused: can DeepSeek-V3 fit on 2x H100?

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "find_viable_hardware",
      "display_model": "DeepSeek V3",
      "framework": "vllm",
      "precision": "fp8",
      "gpu_count": 2,
      "allowed_gpu_models": ["h100"]
    }
  }' | jq

Workload-focused: minimum GPU config for Qwen2.5 72B with vLLM at 32 concurrency

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "find_viable_hardware",
      "display_model": "Qwen3.6 Plus",
      "framework": "vllm",
      "precision": "fp8",
      "conc": 32,
      "isl": 1024,
      "osl": 512
    }
  }' | jq

3.2 Find Viable Workload Plan

Ask the sizer to size a workload when you know the target shape but not the exact hardware.

Workload-focused: size a chat-assistant deployment for 500 concurrent users

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "find_viable_workload_plan",
      "conc": 500,
      "isl": 1024,
      "osl": 300,
      "workload_mix": {
        "coding_assistant": 10,
        "customer_support": 80,
        "agentic_workflows": 5,
        "rag_search": 5
      }
    }
  }' | jq

3.3 Meet Throughput

Ask the sizer to hit a specific tokens-per-second target.

Hardware-focused: what GPUs do I need for 10k tok/s on a 70B model?

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "meet_throughput",
      "display_model": "Llama 3.1 70B Instruct",
      "framework": "vllm",
      "precision": "fp8",
      "target_tokens_per_sec": 10000,
      "conc": 32,
      "isl": 1024,
      "osl": 512
    }
  }' | jq

Workload-focused: throughput target for 1k/500 ISL/OSL at 128 concurrency

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "meet_throughput",
      "display_model": "Qwen3.6 Plus",
      "target_tokens_per_sec": 25000,
      "conc": 128,
      "isl": 1024,
      "osl": 512
    }
  }' | jq

3.4 Meet Latency SLO

Ask the sizer to stay under a latency ceiling.

Hardware-focused: sub-500ms p99 E2E latency for 200 users on H200

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "meet_latency_slo",
      "display_model": "Llama 3.1 70B Instruct",
      "framework": "vllm",
      "precision": "fp8",
      "max_latency_p99_ms": 500,
      "conc": 200,
      "isl": 1024,
      "osl": 300,
      "allowed_gpu_models": ["h200"]
    }
  }' | jq

Model-focused: which model meets 300ms TTFT under 50ms TPOT for 100 users?

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "meet_latency_slo",
      "max_latency_p99_ms": 300,
      "conc": 100,
      "isl": 1024,
      "osl": 300
    }
  }' | jq

Workload-focused: latency-only sizing for a RAG workload with 8k input, 500 output

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "meet_latency_slo",
      "max_latency_p99_ms": 2000,
      "conc": 32,
      "isl": 8192,
      "osl": 512,
      "workload_mix": {
        "coding_assistant": 0,
        "customer_support": 0,
        "agentic_workflows": 0,
        "rag_search": 100
      }
    }
  }' | jq

3.5 Stay Within Budget

Ask the sizer to respect a dollar cap.

Hardware-focused: best LLM hardware under $50k capex

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "stay_within_budget",
      "budget_usd": 50000,
      "budget_basis": "capex",
      "conc": 32,
      "isl": 1024,
      "osl": 512
    }
  }' | jq

Model-focused: size Mistral Small under $10k capex

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "stay_within_budget",
      "display_model": "Mistral Small 3.1 24B Instruct",
      "framework": "vllm",
      "precision": "fp8",
      "budget_usd": 10000,
      "budget_basis": "capex",
      "conc": 16,
      "isl": 1024,
      "osl": 300
    }
  }' | jq

Workload-focused: budget-constrained sizing for 1k concurrent customer-support users

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "stay_within_budget",
      "budget_usd": 200000,
      "budget_basis": "capex",
      "conc": 1000,
      "isl": 1024,
      "osl": 300,
      "workload_mix": {
        "coding_assistant": 0,
        "customer_support": 100,
        "agentic_workflows": 0,
        "rag_search": 0
      }
    }
  }' | jq

3.6 Limit Nodes

Ask the sizer to respect a node-count ceiling.

Hardware-focused: best single-node config for a 405B model

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "limit_nodes",
      "display_model": "Llama 3.1 405B Instruct",
      "framework": "vllm",
      "precision": "fp8",
      "max_nodes": 1,
      "conc": 32,
      "isl": 1024,
      "osl": 512
    }
  }' | jq

Model-focused: which models fit in 2 nodes at 100 concurrency?

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "limit_nodes",
      "max_nodes": 2,
      "conc": 100,
      "isl": 1024,
      "osl": 300
    }
  }' | jq

Workload-focused: 2-node max for a coding-assistant workload with 64 concurrency

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": { "modality_code": "llm" },
    "solve_for": "sizing_plan",
    "features": {
      "sizing_goal": "limit_nodes",
      "max_nodes": 2,
      "conc": 64,
      "isl": 512,
      "osl": 256,
      "workload_mix": {
        "coding_assistant": 100,
        "customer_support": 0,
        "agentic_workflows": 0,
        "rag_search": 0
      }
    }
  }' | jq

4. Natural-Language Agent API

The agent endpoint turns plain-language questions into sizer requests. It maintains conversation history for multi-turn refinement.

Request shape

Field	Type	Required	Description
`message`	`string`	Yes	The natural-language prompt.
`history`	`SizerAgentMessage[]`	Yes	Previous turns (`{role, content}`). Empty array for first message.
`form`	`SizerFormState`	Yes	Current UI form context (minimal defaults work for API-only callers).
`previous_request`	`SizerSolveRequest`	No	The solve request from the previous turn, if any.
`previous_response`	`SizerSolveResponse`	No	The solve response from the previous turn, if any.
`org_id`	`string \| null`	No	Target org; defaults to caller's primary org.

Response shape

Field	Type	Description
`status`	`"needs_clarification" \| "solved" \| "failed"`	Outcome of the agent turn.
`assistant_message`	`string`	Human-readable answer or clarification question.
`steps`	`SizerAgentStep[]`	Trace of what the agent did.
`request`	`SizerSolveRequest`	Structured solve request built by the agent (when `status=solved`).
`response`	`SizerSolveResponse`	Raw sizer result (when `status=solved`).
`form_patch`	`Partial<SizerFormState>`	Suggested UI form updates.
`assumptions`	`string[]`	Notable assumptions the agent made.
`error_code`	`string`	Error code if `status=failed`.

Base form stub

The examples below reuse this minimal form. In practice you only need to set modality, sizingGoal, selectedRequestInputs, and the fields relevant to your question.

{
  "modality": "llm",
  "sizingGoal": "findViableHardware",
  "hardware": "",
  "allowedGpuModels": [],
  "allowedGpuVendors": [],
  "framework": "any",
  "displayModel": "",
  "precision": "any",
  "specMethod": "none",
  "concurrency": 1,
  "inputSequenceLength": 128,
  "outputSequenceLength": 128,
  "tensorParallelism": 1,
  "servingProcessorCount": 1,
  "targetTokensPerSecond": 0,
  "targetThroughputScope": "totalFleet",
  "recommendationSort": "recommended",
  "maxP99E2EMs": 0,
  "maxNodes": 0,
  "activeUsersPeak": 0,
  "budgetBasis": "capex",
  "budgetUsd": 0,
  "horizonYears": 1,
  "latencySlo": "production",
  "growthPct": 0,
  "tokenProfileMode": "workloadMix",
  "workloadMix": {
    "codingAssistant": 25,
    "customerSupport": 25,
    "agenticWorkflows": 25,
    "ragSearch": 25
  },
  "targets": [],
  "selectedRequestInputs": [],
  "availableInventoryRows": [],
  "availableInventoryJson": ""
}

4.1 Find Viable Hardware

Hardware-focused

Prompt: "What is the smallest hardware footprint to run Llama 3.1 70B?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What is the smallest hardware footprint to run Llama 3.1 70B?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "findViableHardware",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "Llama 3.1 70B Instruct",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 1,
      "inputSequenceLength": 128,
      "outputSequenceLength": 128,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["modelServing"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

Workload-focused

Prompt: "What's the minimum GPU config for Qwen3.6 Plus with vLLM at 32 concurrency?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What is the minimum GPU config for Qwen3.6 Plus with vLLM at 32 concurrency?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "findViableHardware",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "vllm",
      "displayModel": "Qwen3.6 Plus",
      "precision": "fp8",
      "specMethod": "none",
      "concurrency": 32,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 512,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "explicitTokens",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["modelServing", "frameworkFilter", "precisionFilter", "workloadShape"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

4.2 Find Viable Workload Plan

Workload-focused

Prompt: "Size a deployment for a chat assistant workload with 100 concurrency."

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Size a deployment for a chat assistant workload with 100 concurrency.",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "findViableWorkloadPlan",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 100,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 300,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 0,
        "customerSupport": 100,
        "agenticWorkflows": 0,
        "ragSearch": 0
      },
      "targets": [],
      "selectedRequestInputs": ["workloadMix", "workloadShape"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

4.3 Meet Throughput

Hardware-focused

Prompt: "I need 10,000 tokens/sec aggregate throughput for Llama 3.1 70B. What hardware?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "I need 10,000 tokens/sec aggregate throughput for Llama 3.1 70B. What hardware?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "meetThroughput",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "Llama 3.1 70B Instruct",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 32,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 512,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 10000,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "explicitTokens",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["modelServing", "targetThroughput", "workloadShape"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

Model-focused

Prompt: "What GPUs do I need to hit 50k tok/s on a 70B model?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What GPUs do I need to hit 50k tok/s on a 70B model?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "meetThroughput",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "Llama 3.1 70B Instruct",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 64,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 512,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 50000,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["modelServing", "targetThroughput"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

Workload-focused

Prompt: "I need to serve 500 concurrent users with 1k/500 token ISL/OSL. What do I need?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "I need to serve 500 concurrent users with 1k/500 token ISL/OSL. What do I need?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "meetThroughput",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 500,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 500,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 0,
      "activeUsersPeak": 500,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "explicitTokens",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["workloadShape", "activeUsersPeak"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

4.4 Meet Latency SLO

Hardware-focused

Prompt: "What setup gives me sub-500ms P99 E2E latency for 200 users?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What setup gives me sub-500ms P99 E2E latency for 200 users?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "meetLatencySlo",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 200,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 300,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 500,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["latencyLimit", "workloadShape"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

Model-focused

Prompt: "Which model meets 300ms TTFT and 50ms TPOT for 100 concurrent users?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Which model meets 300ms TTFT and 50ms TPOT for 100 concurrent users?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "meetLatencySlo",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 100,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 300,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 300,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "strict",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["latencyLimit", "workloadShape"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

Workload-focused

Prompt: "Find a low-latency LLM serving option under 500 ms p99 for 64 concurrency."

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Find a low-latency LLM serving option under 500 ms p99 for 64 concurrency.",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "meetLatencySlo",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 64,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 300,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 500,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["latencyLimit", "workloadShape"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

4.5 Stay Within Budget

Hardware-focused

Prompt: "I have a $50k budget. What's the best LLM hardware I can buy?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "I have a $50k budget. What is the best LLM hardware I can buy?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "stayWithinBudget",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 32,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 300,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 50000,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["budget"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

Model-focused

Prompt: "Size a deployment under $10k capex for Mistral Small."

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Size a deployment under $10k capex for Mistral Small.",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "stayWithinBudget",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "Mistral Small 3.1 24B Instruct",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 16,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 300,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 10000,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["modelServing", "budget"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

Workload-focused

Prompt: "Recommend a model and hardware option under a $500k capex budget for 200 concurrent users."

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Recommend a model and hardware option under a $500k capex budget for 200 concurrent users.",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "stayWithinBudget",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 200,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 300,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 0,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 500000,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["budget", "workloadShape"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

4.6 Limit Nodes

Hardware-focused

Prompt: "I can only use 1 node. What's the best model I can serve?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "I can only use 1 node. What is the best model I can serve?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "limitNodes",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 32,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 300,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 1,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["nodeLimit"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

Model-focused

Prompt: "Limit to 2 nodes max. What works for 405B at 100 concurrency?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Limit to 2 nodes max. What works for 405B at 100 concurrency?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "limitNodes",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "Llama 3.1 405B Instruct",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 100,
      "inputSequenceLength": 1024,
      "outputSequenceLength": 300,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 2,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 25,
        "customerSupport": 25,
        "agenticWorkflows": 25,
        "ragSearch": 25
      },
      "targets": [],
      "selectedRequestInputs": ["modelServing", "nodeLimit", "workloadShape"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

Workload-focused

Prompt: "Can I run a coding-assistant workload with 64 concurrency on a single node?"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Can I run a coding-assistant workload with 64 concurrency on a single node?",
    "history": [],
    "form": {
      "modality": "llm",
      "sizingGoal": "limitNodes",
      "hardware": "",
      "allowedGpuModels": [],
      "allowedGpuVendors": [],
      "framework": "any",
      "displayModel": "",
      "precision": "any",
      "specMethod": "none",
      "concurrency": 64,
      "inputSequenceLength": 512,
      "outputSequenceLength": 256,
      "tensorParallelism": 1,
      "servingProcessorCount": 1,
      "targetTokensPerSecond": 0,
      "targetThroughputScope": "totalFleet",
      "recommendationSort": "recommended",
      "maxP99E2EMs": 0,
      "maxNodes": 1,
      "activeUsersPeak": 0,
      "budgetBasis": "capex",
      "budgetUsd": 0,
      "horizonYears": 1,
      "latencySlo": "production",
      "growthPct": 0,
      "tokenProfileMode": "workloadMix",
      "workloadMix": {
        "codingAssistant": 100,
        "customerSupport": 0,
        "agenticWorkflows": 0,
        "ragSearch": 0
      },
      "targets": [],
      "selectedRequestInputs": ["nodeLimit", "workloadMix", "workloadShape"],
      "availableInventoryRows": [],
      "availableInventoryJson": ""
    },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

5. Multi-Turn Agent Chaining

Use previous_request and previous_response to refine an answer without repeating context.

Example: refine a recommendation with latency constraints

Turn 1 — broad recommendation:

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Start with an LLM recommendation for Qwen3.6 Plus at 10k tokens per second.",
    "history": [],
    "form": { ... },
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | tee /tmp/sizer-turn-1.json | jq

Turn 2 — refine with a latency constraint:

export PREV_REQ="$(jq '.request' /tmp/sizer-turn-1.json)"
export PREV_RESP="$(jq '.response' /tmp/sizer-turn-1.json)"

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/agent" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Now add a strict latency constraint: under 300ms P99 E2E. Show which candidates were rejected.",
    "history": [
      { "role": "user", "content": "Start with an LLM recommendation for Qwen3.6 Plus at 10k tokens per second." },
      { "role": "assistant", "content": "..." }
    ],
    "form": {
      ...
      "selectedRequestInputs": ["modelServing", "targetThroughput", "latencyLimit"],
      "latencySlo": "strict",
      "maxP99E2EMs": 300
    },
    "previous_request": '"$PREV_REQ"',
    "previous_response": '"$PREV_RESP"',
    "org_id": "'"$METRUM_ORG_ID"'"
  }' | jq

6. Response Parsing and Chaining

Extract the primary recommendation for downstream use:

# After a solve or agent call, extract primary recommendation
curl ... > /tmp/sizer-response.json

jq -r '
  .primary_recommendation // .recommendations[0] // empty |
  {
    model: .model_name,
    gpu: .gpu_model,
    gpu_count: .gpus_needed,
    nodes: .nodes_needed,
    framework: .framework,
    precision: .precision,
    projected_tps: .expected_tps_total,
    p99_ms: .expected_p99_e2e_ms,
    confidence: .confidence,
    capex: .capex_usd
  }
' /tmp/sizer-response.json

List rejected candidates and their reasons:

jq '.rejected[] | {gpu: .gpu_model, count: .gpus_needed, reasons: .constraint_failures}' /tmp/sizer-response.json

Check if the response needs clarification:

jq -e '.status == "needs_clarification"' /tmp/sizer-response.json && echo "NEEDS_CLARIFICATION"

7. Error Recovery

Message or code	Meaning	User action
`SIZER_NOT_CONFIGURED`	Sizer is not enabled for this workspace.	Ask a workspace admin to enable or check Sizer availability.
`SIZER_UPSTREAM_UNAVAILABLE`	Sizer could not answer right now.	Retry later or ask an admin to check Sizer availability.
`MISSING_SIZING_SIGNAL`	The request is missing a modality or at least one user-provided sizing signal.	Select a modality and add a sizing signal such as a model, hardware filter, workload input, objective, or constraint.
`TRAINING_REQUIRED`	Metrum's sizing dataset does not yet cover the selected target or exact filter combination.	Use the closest supported options, relax exact-match filters, or contact Metrum about adding coverage.
`INSUFFICIENT_SCOPE`	The caller is not authorized to use the requested Sizer capability.	Sign in with the right account or ask an admin to update access.
`NO_VIABLE_RECOMMENDATION`	Constraints are too tight for any candidate.	Remove or relax budget, latency, node, or hardware filters and retry.

If TRAINING_REQUIRED is returned:

Remove framework or precision filters.
Expand allowed_gpu_models to include more SKUs.
Switch to a smaller or more common display_model.
Retry with solve_for: "model_fit" instead of "sizing_plan".

Appendix A: Optional Feature Keys

The Insights frontend sends only selected inputs. None of these feature keys is required by itself, but at least one sizing signal must be supplied beyond modality:

Feature	Meaning
`display_model`	Model name or display label.
`framework`	Serving framework, for example `vllm`.
`precision`	Precision or quantization label, for example `fp8`.
`spec_method`	Speculative decoding method.
`workload_mix`	Weighted workload categories.
`conc`, `isl`, `osl`	Concurrency, input sequence length, output sequence length.
`decode_tp`, `prefill_tp`	Tensor parallelism for decode and prefill.
`num_decode_gpu`, `num_prefill_gpu`, `gpu_count`	Explicit GPU-count hints.
`target_tokens_per_sec`	Fleet-level throughput target after scope scaling.
`target_tokens_per_sec_per_user`	Original per-user throughput input when relevant.
`target_tokens_per_sec_per_request`	Original per-request throughput input when relevant.
`target_tokens_per_sec_per_user_request`	Original per-active-user-request input when relevant.
`active_users_peak`	Peak active users used for scaling and context.
`budget_usd`, `budget_basis`	Budget setting. `budget_usd` is the dollar cap, and `budget_basis` selects which cost metric to compare against. If `budget_basis` is omitted, Sizer uses capex.
`horizon_years`, `growth_pct`	Business horizon and growth assumptions.
`latency_slo`, `max_latency_p99_ms`	Latency profile and hard p99 limit.
`max_nodes`	Node-count limit.
`allowed_gpu_models`, `allowed_vendors`	Hardware filters.
`recommendation_sort`	Ranking strategy when not using the default recommended order.

Appendix B: Solve Response Fields

For solve_for: "sizing_plan", the response can include:

Field	Meaning
`status`	Whether the request completed or needs clarification.
`plan_type`	The type of sizing result returned, such as model-fit-only or capacity-sized.
`primary_recommendation`	The selected recommendation.
`recommendations`	Ranked viable candidates.
`rejected`	Candidates rejected by memory, latency, budget, hardware, or node constraints.
`fit_check`	Model-fit summary and minimum compatible hardware when a complete model stack is supplied.
`evaluated_profile`	The concurrency and token profile used for the recommendation.
`scalar_results`	Advanced scalar estimates included with the response.
`warnings`	Non-fatal caveats, missing pricing notices, or confidence notes.

Recommendation rows can include GPU model, vendor, GPUs needed, nodes needed, expected tokens/sec, expected p99 latency, VRAM headroom, confidence, cost metadata, deployment notes, bill of materials, and sensitivity points.

Appendix C: Scalar Estimates

For deployment recommendations, request sizing_plan. Advanced integrations can request scalar estimates with solve_for_many:

curl -fsS -X POST "$METRUM_INSIGHTS_URL/api/sizer/solve" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "fit": {
      "modality_code": "llm"
    },
    "solve_for_many": [
      "m_tput_per_gpu",
      "vram_per_gpu_gb",
      "ttft_p50_ms",
      "kv_cache_gb"
    ],
    "features": {
      "conc": 32,
      "isl": 1024,
      "osl": 4096,
      "gpu_count": 8,
      "display_model": "Qwen3.6 Plus"
    }
  }' | jq

Scalar estimates are useful for advanced dashboards and diagnostics. For procurement or deployment decisions, prefer the full recommendation response.

See Hardware Sizer for UI workflows and Sizer Dataset And Administration for dataset coverage guidance.

Endpoints​

0. API Environment​

1. Check Sizer Availability​

2. Discover Available Catalog Data​

GPU SKUs​

AI Models​

3. Structured Solve API​

3.1 Find Viable Hardware​

Hardware-focused: smallest footprint for a 70B model​

Model-focused: can DeepSeek-V3 fit on 2x H100?​

Workload-focused: minimum GPU config for Qwen2.5 72B with vLLM at 32 concurrency​

3.2 Find Viable Workload Plan​

Workload-focused: size a chat-assistant deployment for 500 concurrent users​

3.3 Meet Throughput​

Hardware-focused: what GPUs do I need for 10k tok/s on a 70B model?​

Workload-focused: throughput target for 1k/500 ISL/OSL at 128 concurrency​

3.4 Meet Latency SLO​

Hardware-focused: sub-500ms p99 E2E latency for 200 users on H200​

Model-focused: which model meets 300ms TTFT under 50ms TPOT for 100 users?​

Workload-focused: latency-only sizing for a RAG workload with 8k input, 500 output​

3.5 Stay Within Budget​

Hardware-focused: best LLM hardware under $50k capex​

Model-focused: size Mistral Small under $10k capex​

Workload-focused: budget-constrained sizing for 1k concurrent customer-support users​

3.6 Limit Nodes​

Hardware-focused: best single-node config for a 405B model​

Model-focused: which models fit in 2 nodes at 100 concurrency?​

Workload-focused: 2-node max for a coding-assistant workload with 64 concurrency​

4. Natural-Language Agent API​

Request shape​

Response shape​

Base form stub​

4.1 Find Viable Hardware​

Hardware-focused​

Workload-focused​

4.2 Find Viable Workload Plan​

Workload-focused​

4.3 Meet Throughput​

Hardware-focused​

Model-focused​

Workload-focused​

4.4 Meet Latency SLO​

Hardware-focused​

Model-focused​

Workload-focused​

4.5 Stay Within Budget​

Hardware-focused​

Model-focused​

Workload-focused​

4.6 Limit Nodes​

Hardware-focused​

Model-focused​

Workload-focused​

5. Multi-Turn Agent Chaining​

Example: refine a recommendation with latency constraints​

6. Response Parsing and Chaining​

7. Error Recovery​

Appendix A: Optional Feature Keys​

Appendix B: Solve Response Fields​

Appendix C: Scalar Estimates​

Endpoints

0. API Environment

1. Check Sizer Availability

2. Discover Available Catalog Data

GPU SKUs

AI Models

3. Structured Solve API

3.1 Find Viable Hardware

Hardware-focused: smallest footprint for a 70B model

Model-focused: can DeepSeek-V3 fit on 2x H100?

Workload-focused: minimum GPU config for Qwen2.5 72B with vLLM at 32 concurrency

3.2 Find Viable Workload Plan

Workload-focused: size a chat-assistant deployment for 500 concurrent users

3.3 Meet Throughput

Hardware-focused: what GPUs do I need for 10k tok/s on a 70B model?

Workload-focused: throughput target for 1k/500 ISL/OSL at 128 concurrency

3.4 Meet Latency SLO

Hardware-focused: sub-500ms p99 E2E latency for 200 users on H200

Model-focused: which model meets 300ms TTFT under 50ms TPOT for 100 users?

Workload-focused: latency-only sizing for a RAG workload with 8k input, 500 output

3.5 Stay Within Budget

Hardware-focused: best LLM hardware under $50k capex

Model-focused: size Mistral Small under $10k capex

Workload-focused: budget-constrained sizing for 1k concurrent customer-support users

3.6 Limit Nodes

Hardware-focused: best single-node config for a 405B model

Model-focused: which models fit in 2 nodes at 100 concurrency?

Workload-focused: 2-node max for a coding-assistant workload with 64 concurrency

4. Natural-Language Agent API

Request shape

Response shape

Base form stub

4.1 Find Viable Hardware

Hardware-focused

Workload-focused

4.2 Find Viable Workload Plan

Workload-focused

4.3 Meet Throughput

Hardware-focused

Model-focused

Workload-focused

4.4 Meet Latency SLO

Hardware-focused

Model-focused

Workload-focused

4.5 Stay Within Budget

Hardware-focused

Model-focused

Workload-focused

4.6 Limit Nodes

Hardware-focused

Model-focused

Workload-focused

5. Multi-Turn Agent Chaining

Example: refine a recommendation with latency constraints

6. Response Parsing and Chaining

7. Error Recovery

Appendix A: Optional Feature Keys

Appendix B: Solve Response Fields

Appendix C: Scalar Estimates