Skip to main content
Simplismart’s benchmarking suite lets you evaluate any deployment for Performance (speed & throughput), Quality (accuracy & relevance), or Advanced evaluation (predefined evaluator-based assessment). Follow these steps to create, configure, and run benchmarks end-to-end.

Performance

Measures Throughput, TTFT (time‑to‑first‑token), and TPOT (time‑per‑output‑token).

​

Quality

Evaluates model responses for accuracy, relevance, and output quality on selected datasets.

Advanced

Measures quality using a suite of advanced predefined evaluators for deeper assessment.

​
Model Types

ModelTypeStatus
LLMText generation✅ Available
WhisperSpeech-to-Text (STT)⏳ Coming soon
FluxImage generation⏳ Coming soon

​
Prerequisites

You have at least one Deployment in an active Cluster.