metrumbench-imagegen Workload And Scenario
Use this flow to create an image generation workload and its scenario matrix. Image generation is served by vLLM-Omni or SGLang diffusion.
Set these environment variables for the examples:
export METRUM_API_URL="https://api.<control-plane-domain>"
export METRUM_JWT_TOKEN="<12-hour-db-minted-postgrest-jwt>"
export METRUM_ACCOUNT_ID="<account-uuid>"
Create Workload
Creates a benchmark workload for a project, tool, model, framework, version, and engine args set.
curl -fsS -X POST "$METRUM_API_URL/rpc/create_workload" \
-H "Authorization: Bearer $METRUM_JWT_TOKEN" \
-H "Content-Type: application/json" \
-d "{
\"p_owner_account_id\":\"$METRUM_ACCOUNT_ID\",
\"p_project_name\":\"zimage-rtxpro6000-smoke\",
\"p_workload_code\":\"zimage-vllm-omni\",
\"p_workload_name\":\"zimage-vllm-omni\",
\"p_tool_code\":\"metrumbench-imagegen\",
\"p_model_code\":\"Tongyi-MAI/Z-Image-Turbo\",
\"p_framework_code\":\"vllm-omni\",
\"p_version\":\"0.24.0\",
\"p_engine_args_set_code\":\"vllm-omni-metrumbench-imagegen\"
}" | jq
Parameters:
| Name | Required | Description |
|---|---|---|
p_owner_account_id | Yes | Account UUID that owns the project. |
p_project_name | Yes | Existing project name. |
p_workload_code | Yes | Stable workload code. Use lowercase words separated by hyphens. |
p_workload_name | Yes | Display name. Often the same as p_workload_code. |
p_tool_code | Yes | Benchmark tool code, usually metrumbench-imagegen. |
p_model_code | Yes | Image generation model code, for example Tongyi-MAI/Z-Image-Turbo. |
p_framework_code | Yes | vllm-omni or sglang. |
p_version | Yes | Framework version. |
p_quantization_code | No | Quantization code, when applicable. |
p_engine_args_set_code | No | Engine argument preset, for example vllm-omni-metrumbench-imagegen. |
Create Scenario
Creates one benchmark scenario for an image generation workload. Request count
is derived from concurrency (not user-configurable). Keep n = 1 on SGLang
unless the model supports multi-image requests.
curl -fsS -X POST "$METRUM_API_URL/rpc/create_scenario" \
-H "Authorization: Bearer $METRUM_JWT_TOKEN" \
-H "Content-Type: application/json" \
-d "{
\"p_owner_account_id\":\"$METRUM_ACCOUNT_ID\",
\"p_project_name\":\"zimage-rtxpro6000-smoke\",
\"p_workload_code\":\"zimage-vllm-omni\",
\"p_scenario_code\":\"c4-1024x1024\",
\"p_scenario_name\":\"c4-1024x1024\",
\"p_concurrency\":4,
\"p_num_images_batch\":1
}" | jq
Parameters:
| Name | Required | Description |
|---|---|---|
p_owner_account_id | Yes | Account UUID that owns the project. |
p_project_name | Yes | Existing project name. |
p_workload_code | Yes | Workload code created earlier. |
p_scenario_code | Yes | Stable scenario code. Include concurrency. |
p_scenario_name | Yes | Display name. Often the same as p_scenario_code. |
p_concurrency | No | Concurrent requests. Default 1. |
p_num_images_batch | No | Images per request (n). Default 1. |
p_size | No | Image size WxH. Default 1024x1024. |
p_response_format | No | b64_json or url. Default b64_json. |