API Rate Limit Response
This page documents the rate limiting behavior of our API and explains how to handle different types of rate limit errors.
Overview
Our API implements rate limiting to ensure fair usage and maintain service stability. There are three types of rate limits:
- Daily rate limit
- Burst rate limit
- Chat-based rate limit
When a rate limit is exceeded, the API returns a 429 status code along with specific headers that provide information about the limit, remaining requests/tokens, and reset time.
Daily Rate Limit
This is a daily limit applied to publish-related API endpoints (new message requests like publish, enqueue, or batch). Other API requests, such as fetching events or messages, do not count toward this limit.
Headers:
RateLimit-Limit
: Maximum number of requests allowed per dayRateLimit-Remaining
: Remaining number of requests for the dayRateLimit-Reset
: Time (in unix timestamp) when the daily limit will reset
Burst Rate Limit
This is a short-term limit per second to prevent rapid bursts of requests. This limit applies to all API endpoints.
Headers:
Burst-RateLimit-Limit
: Maximum number of requests allowed in the burst window (1 second)Burst-RateLimit-Remaining
: Remaining number of requests in the burst window (1 second)Burst-RateLimit-Reset
: Time (in unix timestamp) when the burst limit will reset
Chat-based Rate Limit
This limit is applied to chat-related API endpoints.
Headers:
x-ratelimit-limit-requests
: Maximum number of requests allowed per dayx-ratelimit-limit-tokens
: Maximum number of tokens allowed per dayx-ratelimit-remaining-requests
: Remaining number of requests for the dayx-ratelimit-remaining-tokens
: Remaining number of tokens for the dayx-ratelimit-reset-requests
: Time (in unix timestamp) until the request limit resetsx-ratelimit-reset-tokens
: Time (in unix timestamp) when the token limit will reset
Example Rate Limit Error Handling
import { QstashDailyRatelimitError } from "@upstash/qstash";
try {
// Example of a publish request that could hit the daily rate limit
const result = await client.publishJSON({
url: "https://my-api...",
// or urlGroup: "the name or id of a url group"
body: {
hello: "world",
},
});
} catch (error) {
if (error instanceof QstashDailyRatelimitError) {
console.log("Daily rate limit exceeded. Retry after:", error.reset);
// Implement retry logic or notify the user
} else {
console.error("An unexpected error occurred:", error);
}
}
import { QstashRatelimitError } from "@upstash/qstash";
try {
// Example of a request that could hit the burst rate limit
const result = await client.publishJSON({
url: "https://my-api...",
// or urlGroup: "the name or id of a url group"
body: {
hello: "world",
},
});
} catch (error) {
if (error instanceof QstashRatelimitError) {
console.log("Burst rate limit exceeded. Retry after:", error.reset);
// Implement exponential backoff or delay before retrying
} else {
console.error("An unexpected error occurred:", error);
}
}
import { QstashChatRatelimitError, Client, openai } from "@upstash/qstash";
try {
// Example of a chat-related request that could hit the chat rate limit
const client = new Client({
token: "<QSTASH_TOKEN>",
});
const result = await client.publishJSON({
api: {
name: "llm",
provider: openai({ token: process.env.OPENAI_API_KEY! }),
},
body: {
model: "gpt-3.5-turbo",
messages: [
{
role: "user",
content: "Where is the capital of Turkey?",
},
],
},
callback: "https://oz.requestcatcher.com/",
});
} catch (error) {
if (error instanceof QstashChatRatelimitError) {
console.log("Chat rate limit exceeded. Retry after:", error.resetRequests);
// Handle chat-specific rate limiting, perhaps by queueing requests
} else {
console.error("An unexpected error occurred:", error);
}
}
Was this page helpful?