Bring Your Own Endpoint

Private Stack Exposure

Remote benchmark workers must reach the Metrum Insights control plane over a public URL. When you run a private or local stack, expose PostgREST before launch and use the public HTTPS URL for METRUM_PUBLIC_API_URL, INSIGHTS_API_URL, and INSIGHTS_CONTROL_PLANE_URL:

cloudflared tunnel --url http://localhost:${POSTGREST_PORT:-3000}
export METRUM_PUBLIC_API_URL="https://<cloudflared-host>.trycloudflare.com"
export INSIGHTS_API_URL="$METRUM_PUBLIC_API_URL"
export INSIGHTS_CONTROL_PLANE_URL="$METRUM_PUBLIC_API_URL"
curl -fsS "$METRUM_PUBLIC_API_URL/" >/dev/null

Do not give a remote worker localhost, 127.0.0.1, host.docker.internal, or a private-only DNS name. Keep the tunnel alive through onboarding, polling, result upload, and teardown; expose any private package/WebUI endpoint through a public URL too when the worker downloads artifacts from that stack.

BYOE benchmarks an endpoint you operate. Metrum Insights does not start or stop the model server for that workload.

If you have SSH access to raw hardware and want Metrum Insights to set up the worker and model server before benchmarking, use Bring Your Own Hardware instead.

Use BYOE when:

• the model server is custom;
• the accelerator is non-NVIDIA or vendor-specific;
• the endpoint already runs in Kubernetes, on-prem, or another cloud.

Web UI

1. Create or open the project.
2. Add a BYOE workload.
3. Enter endpoint URL, transport, model name, and auth reference.
4. Create scenarios.
5. Run against the endpoint.
6. Attach external telemetry or SUT evidence if the worker cannot collect it.

Placeholder: if BYOE workload creation is not exposed in your Web UI build, use the BYOE API and view the resulting jobs in the project.

API

Use an HTTPS endpoint that the benchmark worker can reach. For a keyless OpenAI-compatible endpoint:

curl -fsS -X POST "$METRUM_API_URL/rpc/create_model_endpoint" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"p_endpoint_code\":\"custom-qwen15b-vllm\",
    \"p_endpoint_name\":\"Custom Qwen 1.5B vLLM\",
    \"p_provider_code\":\"custom_openai_compatible\",
    \"p_transport_code\":\"openai_chat_completions\",
    \"p_api_base_url\":\"https://<endpoint-host>/v1\",
    \"p_model_identifier\":\"deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B\",
    \"p_owner_account_id\":\"$METRUM_ACCOUNT_ID\",
    \"p_healthcheck_path\":\"/v1/models\",
    \"p_client_reported_framework\":\"vllm\",
    \"p_client_reported_framework_version\":\"0.20.0\",
    \"p_max_concurrent_requests\":32,
    \"p_advisory_cost_class\":\"self_hosted\",
    \"p_org_id\":\"$METRUM_ORG_ID\"
  }" | jq

List endpoints without exposing stored keys:

curl -fsS -X POST "$METRUM_API_URL/rpc/list_model_endpoints" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"p_owner_account_id\":\"$METRUM_ACCOUNT_ID\",
    \"p_org_id\":\"$METRUM_ORG_ID\",
    \"p_include_inactive\":false
  }" | jq

Create a BYOE workload bound to the endpoint:

curl -fsS -X POST "$METRUM_API_URL/rpc/create_shared_workload_byoe" \
  -H "Authorization: Bearer $METRUM_JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"p_workload_name\":\"Custom Qwen 1.5B metrumbench-llm BYOE\",
    \"p_workload_code\":\"custom-qwen15b-byoe-metrumbench-llm\",
    \"p_tool_code\":\"metrumbench-llm\",
    \"p_endpoint_code\":\"custom-qwen15b-vllm\",
    \"p_owner_account_id\":\"$METRUM_ACCOUNT_ID\",
    \"p_org_id\":\"$METRUM_ORG_ID\"
  }" | jq

Attach the workload to the project through the Web UI when project attachment is available. If your build exposes only the shared-workload RPC, keep this page as the endpoint/workload registry step and run the scenario from the project after the workload appears there.

Use the normal scenario and execution APIs after the project workload exists. BYOE jobs set p_manage_model_server to false; Metrum does not launch or teardown your endpoint.

Report Caveats

State which responsibilities are outside Metrum Insights:

• model-server startup and teardown;
• autoscaling and load balancer behavior;
• endpoint auth;
• external telemetry;
• cloud costs;
• custom accelerator health.