Skip to main content
Every deployment runs on a number of pods. Each pod is a running copy of your model, ready to serve requests. Too few pods, and requests queue up while response times climb. Too many, and you’re paying for GPU capacity that sits idle. Autoscaling keeps this balanced automatically — Simplismart monitors your traffic and adds or removes pods accordingly.
If you’re just getting started: you don’t need to read this whole page to configure autoscaling. Choose a metric and a threshold on the deployment form and leave the rest at their defaults — see Picking a starting point for sensible defaults by workload type. The rest of this page explains the reasoning behind that behavior, for when you need finer control.
This page walks through how autoscaling decides to add or remove pods. For the step-by-step form fields, see Creating a Deployment.

Scaling Parameters

Pick the metric, threshold, and min/max range that drive scaling.

Schedule-Based Autoscaling

Guarantee capacity for traffic patterns you already know about.

Autoscaling Policy

Control how fast pods are added or removed once a threshold is crossed.

Scale to Zero

Drop idle deployments to 0 pods and stop paying for them.

​
How it decides

At regular intervals, Simplismart checks one number — your chosen metric — against a target you’ve set, called the threshold. If the number is above the threshold, it adds pods. If it’s below, it removes pods. At the threshold, it leaves things as they are. Two settings bound this decision on either side:
  • Min pods — the floor. Your deployment never drops below this, even at zero traffic, unless Scale to Zero is enabled (covered below).
  • Max pods — the ceiling, capped by your account quota. Your deployment never grows past this, regardless of demand.
The metric determines the direction; min and max determine how far it’s allowed to go.

​
Choosing a metric

The metric is simply the number autoscaling watches. It falls into one of two categories:
  • Resource metrics describe the pod itself — how hard the CPU or GPU is working, how much memory it’s using.
  • Request-shape metrics describe your traffic — how many requests are arriving, how quickly they’re answered.
If you’re unsure where to start, use Concurrency as your primary metric with GPU Utilization as a backup. This combination works well for most model deployments.
Resource metrics Request-shape metrics
Some model runtimes expose a few extra specialized metrics too (like queue depth for async workers). Available metrics can vary by model and container type — see Scaling Parameters for the exact list on your deployment.
Whichever metric you choose, the threshold is the value that determines the direction: above it, pods are added; below it, pods are removed.

​
Try it: the autoscale console

Scaling up and scaling down are deliberately asymmetric. Simplismart adds pods quickly when demand rises, so requests don’t queue. It removes pods more cautiously, so a brief lull doesn’t remove a pod you need again moments later. The console below runs a live simulation rather than a static illustration. Choose a traffic pattern, adjust the sliders, and watch the pod count respond in real time — no deployment required. Two things worth exploring once it’s loaded:
  • The Scaling ledger logs every scale event as it happens. Select the Flapping pattern with a short scale-down period, and the ledger will flag (in rose) a scale-down that occurs sooner than expected. This is a real effect, explained in the next section.
  • Under Advanced settings, enable Schedule window. It guarantees a minimum pod count during a time window you define, regardless of what the traffic metric shows — you’ll see it hold that floor (shaded on the chart) and return control to standard scaling once the window ends.
Notice the shape of the two lines: traffic snaps up and pods follow within a step or two, but on the way down the pod line descends in a staircase, spread out well after traffic has already dropped. That staircase is controlled by the Autoscaling Policy, covered next.

​
Controlling the speed: Autoscaling Policy

By default, Simplismart scales between your min and max pods with no additional rate-limiting — most deployments don’t need to change this. Autoscaling Policy is an optional, advanced layer for controlling how fast pods are added or removed:
  • Stabilization window — how long the autoscaler observes the metric before acting, so a brief spike or dip doesn’t trigger a decision it would otherwise reverse moments later. Scale-up defaults to a short 60s window (react quickly); scale-down defaults to a longer 180s window (avoid overreacting to a blip).
  • Policies — the size of each step: a fixed number of Pods, or a Percent of the current count, taken over a Period of time (1–1800s).
For most deployments, this is all you need to know — see Autoscaling Policy for the full field reference. The remainder of this section covers one setting that’s easy to misread: the scale-down Period.

​
A subtlety in the scale-down period

A policy such as “remove 1 pod every 1200 seconds” reads like a strict rate limit — one pod removed per 20-minute window, no exceptions. In practice, a second pod can be removed well before that window is up, if your traffic has been fluctuating.
Why: the autoscaler doesn’t simply track time since the last scale-down. It looks at every scaling event — both additions and removals — within the trailing period, and a recent addition effectively offsets an earlier removal in that calculation. So if your metric has been crossing the threshold repeatedly, a scale-down can become eligible again well before the period would suggest.
You can observe this directly in the console above: select Flapping, set a short scale-down period, and watch the ledger flag it in real time.
Here’s what actually happens with a 1 pod / 1200s (20 min) scale-down policy:At 17:05 — currently at 4 pods. The trailing 1200s window holds 1 down and 2 ups. periodStartReplicas = 4 − 2 + 1 = 3, so the floor is 3 − 1 = 2. Scaling to 3 keeps us at 3 ≥ 2 → allowed. Pods go 4 → 3.At 17:07 — currently at 3 pods. The window now holds 2 downs and 2 ups. periodStartReplicas = 3 − 2 + 2 = 3, so the floor is still 3 − 1 = 2. Scaling to 2 keeps us at 2 ≥ 2 → allowed. Pods go 3 → 2 — just 2 minutes after the previous scale-down, inside a policy meant to allow one pod removed every 20 minutes.
How to mitigate it:
  • Give your threshold some headroom rather than tuning it to the exact edge, so ordinary noise doesn’t flip the decision repeatedly.
  • If your traffic is naturally variable, a smoother metric (such as Throughput) will trigger this less often than a spikier one (such as instantaneous Latency).
  • Treat the period as a lookback window the autoscaler uses for its calculation, not as a timer that resets cleanly after each scale-down.

​
Scale to Zero

Well suited to deployments that sit idle for extended periods — development and test environments, low-traffic endpoints. When there’s no incoming traffic, Scale to Zero reduces your pod count to 0, so you stop paying for GPU capacity entirely. When a request arrives, a pod starts automatically, with a brief cold-start delay while it comes up. A cooldown period controls how long the deployment waits after traffic stops before scaling to zero, so a short pause doesn’t trigger an unnecessary cold start.
Scale to Zero and Schedule-Based Autoscaling’s “scale to 0 outside windows” option are mutually exclusive — use whichever fits how predictable your traffic is.

​
Rapid Autoscaling

For sudden traffic bursts, Rapid Autoscaling reacts faster than the standard evaluation windows allow, reducing the latency spike that would otherwise occur while new pods are still starting up.
Rapid Autoscaling is available on Simplismart Cloud (Dedicated) deployments only. Contact support@simplismart.tech to enable it.

​
Schedule-Based Autoscaling

If you already know your traffic pattern in advance — busy during office hours, quiet overnight, a recurring batch job — you don’t need to rely on metrics for those periods. Define a time window with a guaranteed minimum pod count, and the deployment falls back to standard scaling (or zero) outside it. If a window and the regular metric-based scaling disagree, whichever calls for more pods wins — a window sets a floor, never a ceiling. To see this in the console above, open Advanced settings and enable Schedule window. The window appears shaded on the chart, and the pod count moves to the guaranteed minimum the instant the window opens, with none of the usual wait. See Schedule-Based Autoscaling for how to configure this, with both a guided picker and cron expressions.

​
Picking a starting point

These are starting points, not rules — tune each setting against your own traffic once the deployment is live.

​
Configuring autoscaling

  • Creating a Deployment walks through every field on the deployment form, including Scaling Parameters, Schedule-Based Autoscaling, and Autoscaling Policy.
  • Editing a Deployment covers adjusting any of these settings after the deployment is already live, applied as a rolling update.