Rate limits

Limits exist to keep the Grove fast and fair for everyone. They are generous for interactive and production use, and every limited response tells you exactly how long to wait.

The limits

Limit Value What happens when you hit it
Requests per minute 600 per account 429 with a Retry-After header
Concurrent requests 10 in-flight per account 429 with Retry-After: 2
Output tokens 32,768 per request Your max_tokens is capped, the request still succeeds

Notes on how these behave:

  • The request limit is a continuously refilling budget, not a fixed window. You can burst up to 600 requests at once; capacity then refills at about ten requests per second. Steady traffic at or below 10 requests/second never hits it.
  • Streaming responses count toward concurrency for their full duration. Ten long-running streams will occupy all ten slots until they finish.
  • Limits apply per account, across all of your API keys. Multiple keys are encouraged for organization and safe rotation, but they share one budget: creating more keys does not add throughput.
  • Limits may be adjusted over time (see the Terms of Use, Section 1). This page always shows the current values.

Two kinds of 429

Not every 429 means you sent too much. Check the message:

  1. "Rate limit reached for your account": you hit your account's request or concurrency limit. These return instantly. Back off for Retry-After seconds and continue.
  2. A busy model: when a model is congested, we first ride it out for you with an internal retry ladder (up to ~20 seconds). Only if the model stays busy do you see a 429 (with a longer Retry-After) or a 503. These are not about your usage; honor the header and retry.

Full status-code reference and backoff recipes: Errors & retries.

Best practices

  • Honor Retry-After. It is present on every 429 and is the fastest safe retry time. The OpenAI SDKs do this automatically (max_retries).
  • Add jitter to retries in high-concurrency workers so they don't retry in lockstep.
  • Keep a client-side concurrency cap (≤10) in batch jobs: a semaphore in your worker pool is simpler than handling 429s. The concurrency limit, not the per-minute limit, is what a large fan-out hits first: a small stagger (even ~100ms between launches) keeps you comfortably under it.
  • For sustained batch workloads, pace to ~10 requests/second and the request limit disappears entirely.

Requesting higher limits

To request higher rate limits, please email support@pgsgroveinternal.com with your use case and expected volume. Higher rate limits are not guaranteed to be granted.