Skip to main content

Runbook Control Plane APIs

Private Stack Exposure​

Remote benchmark workers must reach the Metrum Insights control plane over a public URL. When you run a private or local stack, expose PostgREST before launch and use the public HTTPS URL for METRUM_PUBLIC_API_URL, INSIGHTS_API_URL, and INSIGHTS_CONTROL_PLANE_URL:

cloudflared tunnel --url http://localhost:${POSTGREST_PORT:-3000}
export METRUM_PUBLIC_API_URL="https://<cloudflared-host>.trycloudflare.com"
export INSIGHTS_API_URL="$METRUM_PUBLIC_API_URL"
export INSIGHTS_CONTROL_PLANE_URL="$METRUM_PUBLIC_API_URL"
curl -fsS "$METRUM_PUBLIC_API_URL/" >/dev/null

Do not give a remote worker localhost, 127.0.0.1, host.docker.internal, or a private-only DNS name. Keep the tunnel alive through onboarding, polling, result upload, and teardown; expose any private package/WebUI endpoint through a public URL too when the worker downloads artifacts from that stack.

The pages in this section document the HTTP API surface used by supported runbooks. They intentionally show PostgREST and WebUI calls directly, not container-local helper commands.

Start with the model and hardware target, then choose a provider subsection inside that target runbook. Provider-only pages are appendices for inspection and teardown, not primary benchmark launch instructions.

See Model And Hardware Runbooks for the maintained target matrix and operator rules.

Project Lifecycle​

  • POST /rpc/get_current_user_context
  • POST /rpc/list_projects
  • POST /rpc/create_project
  • GET /v_project_runs_with_status

See Project Lifecycle for parameters and examples. See End To End Benchmark API Example for the complete ordered sequence.

Catalog And Scenario Setup​

  • GET /ai_models
  • GET /frameworks
  • GET /framework_versions
  • POST /rpc/create_workload
  • POST /rpc/create_scenario
  • POST /rpc/list_workloads
  • POST /rpc/list_scenarios

See metrumbench-llm Workload And Scenario for parameters and examples.

Hardware Onboarding​

  • POST /rpc/create_server_config
  • POST /rpc/create_server_config_gpu
  • POST /rpc/create_server_config_instance
  • POST /rpc/list_server_config_instances
  • POST /api/onboarding/bootstrap
  • POST /rpc/validate_server_readiness
  • GET /v_server_current_status

See Hardware Onboarding for parameters and examples.

Cloud Benchmark Execution​

  • POST /api/cloud/onboard
  • POST /rpc/execute_scenario
  • POST /rpc/get_job_status
  • POST /rpc/list_jobs

See Cloud Benchmark Execution for parameters and examples.

Results And Reports​

  • GET /v_benchmark_results
  • POST /rpc/get_telemetry_summary_for_job
  • POST /rpc/get_telemetry_summary_for_run
  • GET /v_metrumbench_llm_outputs
  • GET /v_run_cost_summary
  • POST /api/ai-eval
  • GET /api/benchmark-report/:runId
  • GET /api/run-report/:runId
  • GET /api/project-report/:projectId
  • GET /api/project-run-report/:projectId

See Results And Reports for parameters and examples.

KYAI​

  • POST /rpc/create_kyai_generation_run
  • POST /rpc/get_kyai_generation_run
  • GET /v_kyai_generation_outputs
  • POST /rpc/create_kyai_judge_run
  • POST /rpc/get_kyai_judge_run
  • GET /kyai_results

See KYAI Evaluation for parameters and examples.

Document KYAI endpoints only for the generation/evaluation runbook flow. Avoid publishing raw KYAI table CRUD pages unless a runbook calls those tables directly.