Simplismart’s benchmarking suite lets you evaluate any deployment for Performance (speed & throughput), Quality (accuracy & relevance), or Advanced evaluation (predefined evaluator-based assessment). Follow these steps to create, configure, and run benchmarks end-to-end.Documentation Index
Fetch the complete documentation index at: https://docs.simplismart.ai/llms.txt
Use this file to discover all available pages before exploring further.
Performance
Measures Throughput, TTFT (time‑to‑first‑token), and TPOT (time‑per‑output‑token).
Quality
Evaluates model responses for accuracy, relevance, and output quality on selected datasets.
Advanced
Measures quality using a suite of advanced predefined evaluators for deeper assessment.
Model Types
| Model | Type | Status |
|---|---|---|
| LLM | Text generation | ✅ Available |
| Whisper | Speech-to-Text (STT) | ⏳ Coming soon |
| Flux | Image generation | ⏳ Coming soon |