# null



# Mintlify Starter Kit

Click on `Use this template` to copy the Mintlify starter kit. The starter kit
contains examples including

*   Guide pages
*   Navigation
*   Customizations
*   API Reference pages
*   Use of popular components

### 👩‍💻 Development

Install the [Mintlify CLI](https://www.npmjs.com/package/mintlify) to preview
the documentation changes locally. To install, use the following command

```
npm i -g mintlify
```

Run the following command at the root of your documentation (where mint.json is)

```
mintlify dev
```

### 😎 Publishing Changes

Changes will be deployed to production automatically after pushing to the
default branch.

You can also preview changes using PRs, which generates a preview link of the
docs.

#### Troubleshooting

*   Mintlify dev isn't running - Run `mintlify install` it'll re-install
    dependencies.
*   Page loads as a 404 - Make sure you are running in a folder with `mint.json`


# Add a Payment Method



Upstash does not require a credit card for Free databases. However, for paid databases, you need to add at least one payment method. To add a payment method, follow these steps:

1.  Click on your profile at the top right.
2.  Select  `Account` from the dropdown menu.
3.  Navigate to the `Billing` tab.
4.  On the screen, click the `Add Your Card` button.
5.  Enter your name and credit card information in the following form:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/addpaymentmethod/ccform.png" />
</Frame>

You can enter multiple credit cards and set one of them as the default one. The
payments will be charged from the default credit card.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/addpaymentmethod/cardlist.png" />
</Frame>

## Payment Security

Upstash does not store users' credit card information in its servers. We use
Stripe Inc payment processing company to handle payments. You can read more
about payment security in Stripe
[here](https://stripe.com/docs/security/stripe).


# Audit Logs



Audit logs give you a chronological set of activity records that have affected
your databases and Upstash account. You can see the list of all activities on a
single page. You can access your audit logs under `Account > Audit Logs` in your
console:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/auditlogs/audit.png" />
</Frame>

Here the `Source` column shows if the action has been called by the console or via
an API key. The `Entity` column gives you the name of the resource that has been
affected by the action. For example, when you delete a database, the name of the
database will be shown here. Also, you can see the IP address which performed the
action.

## Security

You can track your audit logs to detect any unusual activity on your account and
databases. When you suspect any security breach, you should delete the API key
related to suspicious activity and inform us by emailing
[support@upstash.com](mailto:support@upstash.com)

## Retention period

After the retention period, the audit logs are deleted. The retention period for free databases is 7 days, for pay-as-you-go databases, it is 30 days, and for the Pro tier, it is one year.


# AWS Marketplace



<Check>
  **Prerequisite**

  You need an Upstash account before subscribing on AWS, create one
  [here](https://console.upstash.com).
</Check>

Upstash is available on the AWS Marketplace, which is particularly beneficial for users who already get other services from AWS Marketplace and can consolidate Upstash under a single bill.

You can search "Upstash" on AWS Marketplace or just click [here](https://aws.amazon.com/marketplace/pp/prodview-fssqvkdcpycco).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/awsmarketplace/AWS-marketplace-1.png" />
</Frame>

Once you click subscribe, you will be prompted to select which personal or team account you wish to link with your AWS Subscription.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/awsmarketplace/AWS-marketplace-2.png" />
</Frame>

Once your account is linked, regardless of which Upstash product you use, all of your usage will be billed to your AWS Account. You can also upgrade or downgrade your subscription through Upstash console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/awsmarketplace/AWS-marketplace-3.png" />
</Frame>


# Cost Explorer



The Cost Explorer pages allow you to view your current and previous months’ costs. To access the Cost Explorer, navigate to the left menu and select Account > Cost Explorer. Below is an example report:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/costexplorer/costexplorer.png" />
</Frame>

You can select a specific month to view the cost breakdown for that period. Here's the explanation of the fields in the report:

**Request:** This represents the total number of requests sent to the database.

**Storage:** This indicates the average size of the total storage consumed. Upstash database includes a persistence layer for data durability. For example, if you have 1 GB of data in your database throughout the entire month, this value will be 1 GB. Even if your database is empty for the first 29 days of the month and then expands to 30 GB on the last day, this value will still be 1 GB.

**Cost:** This field represents the total cost of your database in US Dollars.

> The values for the current month is updated hourly, so values can be stale up
> to 1 hour.


# Create an Account



You can sign up for <a href="https://console.upstash.com" target="_blank">Upstash</a> using your Amazon, Github or Google accounts. Alternatively, if you prefer not to use these authentication providers or want to sign up with a corporate email address, you can also sign up using email and password.

<Note>
  We do not access your information other than:

  *   Your email
  *   Your name
  *   Your profile picture and we never share your information with third parties.
</Note>


# Developer API



Using Upstash API, you can develop applications that can create and manage
Upstash databases and Upstash kafka clusters. You can automate everything that
you can do in the console. To use developer API, you need to create an API key
in the console.

See [DevOps](/devops) for details.


# Account and Billing FAQ



## How can I delete my account?

You can delete your account from `Account` > `Settings` > `Delete Account`. You should first delete all your databases and clusters. After you delete your account, all your data and payment information will be deleted and you will not be able to recover it.

## How can I delete my credit card?

You can delete your credit card from `Account` > `Billing` page. However, you should first add a new credit card to be able to delete the existing one. If you want to delete all of your payment information, you should delete your account.

## How can I change my email address?

You can change your account e-mail address in `Account` > `Settings` page. In order to change your billing e-mail adress, please see `Account` > `Billing` page. If you encounter any issues, please contact us at [support@upstash.com](mailto:support@upstash.com) to change your email address.

## Can I set an upper spending limit, so I don't get surprises after an unexpected amount of high traffic?

On Pay as You Go model, you can set a budget for your Redis instances. When your monthly cost reaches the max budget, we send an email to inform you and throttle your instance. You will not be charged beyond your set budget.

To set the budget, you can go to the "Usage" tab of your Redis instance and click "Change Budget" under the cost metric.

## What happens if my payment fails?

If a payment failure occurs, we will retry the payment three more times before suspending the account. During this time, you will receive email notifications about the payment failure. If the account is suspended, all resources in the account will be inaccessible. If you add a valid payment method after the account suspension, your account will be automatically unsuspended during the next payment attempt.

## What happens if I unsubscribe from AWS Marketplace but I don't have any other payment methods?

We send a warning email three times before suspending an account. If no valid payment method is added, we suspend the account. Once the account is suspended, all resources within the account will be inaccessible. If you add a valid payment method after the account suspension, your account will be automatically unsuspended during the next system check.

## I have a question about my bill, who should I contact?

Please contact us at [support@upstash.com](mailto:support@upstash.com).


# Payment History



The Payment History page gives you information about your payments. You can open your
payment history in the left menu under Account > Payment History. Here an example
report:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/paymenthistory/paymenthistory.png" />
</Frame>

You can download receipt. If one of your payments failed, you can retry your
payment on this page.


# Teams and Users



Team management enables collaboration with other users. You can create a team and invite people to join by using their email addresses. Team members will have access to databases created under the team based on their assigned roles.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-member.png" />
</Frame>

## Create Team

You can create a team using the menu `Account > Teams`

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-page.png" />
</Frame>

<br />

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-create.png" />
</Frame>

> A user can create up to 5 teams. You can be part of even more teams but only
> be the owner of 5 teams. If you need to own more teams please email us at
> [support@upstash.com](mailto:support@upstash.com).

You can still continue using your personal account or switch to a team.

> The databases in your personal account are not shared with anyone. If you want
> your database to be accessible by other users, you need to create it under a
> team.

## Switch Team

You need to switch to the team to create databases shared with other team
members. You can switch to the team via the switch button in the team table. Or
you can click your profile pic in the top right and switch to any team listed
there.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/change-team.png" />
</Frame>

## Add/Remove Team Member

After switching to a team, if you are the Owner or an Admin of the team, you can add team members by navigating to `Account > Teams`. Simply enter their email addresses.It's not an issue if the email addresses are not yet registered with Upstash. Once the user registers with that email, they will gain access to the team. We do not send invitations; when you add a member, they become a member directly. You can also remove members from the same page.

> Only Admins or the Owner can add/remove users.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-member-create.png" />
</Frame>

## Roles

While adding a team member, you will need to select a role. Here are the access rights associated with each role:

*   Admin: This role has full access, including the ability to add and remove members, manage databases, and payment methods.

*   Dev: This role can create, manage, and delete databases but cannot manage users or payment methods.

*   Finance: This role is limited to managing payment methods and cannot manage databases or users.

*   Owner: The Owner role has all the access rights of an Admin and, in addition to having the ability to delete the team. This role is automatically assigned to the user who created the team, and you cannot assign it to other members.

> If you want to change a user's role, you will need to delete and re-add them with the desired access rights.

## Delete Team

Only the original creator (owner) can delete a team. Also the team should not
have any active databases, namely all databases under the team should be deleted
first. To delete your team, first you need to switch your personal account then
you can delete your team in the team list under `Account > Teams`.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-member.png" />
</Frame>


# Access Anywhere



Upstash has integrated REST APIs into all its products to facilitate access from various runtime environments. This integration is particularly beneficial for edge runtimes like Cloudflare Workers and Vercel Edge, which do not permit TCP connections, and for serverless functions such as AWS Lambda, which are stateless and do not retain connection information between invocations.

### Rationale

The absence of TCP connection support in edge runtimes and the stateless nature of serverless functions necessitate a different approach for persistent connections typically used in traditional server setups. The stateless REST API provided by Upstash addresses this gap, enabling consistent and reliable communication with data stores from these platforms.

### REST API Design

The REST APIs for Upstash services are thoughtfully designed to align closely with the conventions of each product. This ensures that users who are already familiar with these services will find the interactions intuitive and familiar. Our API endpoints are self-explanatory, following standard REST practices to guarantee ease of use and seamless integration.

<img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/access-anywhere/restclient.png" />

### SDKs for Popular Languages

To enhance the developer experience, Upstash is developing SDKs in various popular programming languages. These SDKs simplify the process of integrating Upstash services with your applications by providing straightforward methods and functions that abstract the underlying REST API calls.

### Resources

[Redis REST API Docs](https://upstash.com/docs/redis/features/restapi)

[Kafka REST API Docs](https://upstash.com/docs/kafka/rest/restintro)

[QStash REST API Docs](https://upstash.com/docs/qstash/api/authentication)

[Redis SDK - Typescript](https://github.com/upstash/upstash-redis)

[Redis SDK - Python](https://github.com/upstash/redis-python)

[Kafka SDK - Typescript](https://github.com/upstash/upstash-kafka)

[QStash SDK - Typescript](https://github.com/upstash/sdk-qstash-ts)


# Global Replication

Fast anywhere.

Upstash Redis replicates your data to user-selected regions. You can add or remove regions from a running cluster without experiencing downtime. Think of each region as a replica that maintains a copy of your data, ensuring low latency and high availability.

## Designed for the Edge Functions

Edge runtimes, such as Cloudflare Workers and Vercel Edge, enhance speed by executing your code at locations closest to your users. However, if your use case involves data storage, it's crucial to position your data store near your users to achieve optimal performance. Upstash Global tackles this need by replicating your data across multiple regions. While edge runtimes come with certain limitations, Upstash provides a HTTP-based Redis® client designed and tested to integrate seamlessly with popular edge runtimes, including Vercel, Cloudflare, Fastly, and Deno.

## Read Regions Everywhere, Low Latency Anywhere

The global database is designed to minimize latency for read operations. It consists of a single primary replica and multiple read replicas. Write commands are sent to and processed by the primary replica, then replicated across all read replicas. When a client issues a read command, it retrieves the response from the nearest read replica based on the region. Conversely, all write requests are directed to the primary replica to maintain consistency.

Our tests indicate sub-millisecond latency for clients located in the same AWS region as the Redis® instance.

**Read operations are processed at the the nearest replica.**

<img src="https://upstash.com/blog/global2/reads.png" />

**Writes are processed at the primary replica.**

<img src="https://upstash.com/blog/global2/writes.png" />

## Choose your regions

The global database requires you to choose primary and read regions:

→ Choose a primary region where your write operations will take place to ensure faster write speeds.

→ Select read regions that are closest to the majority of your user base for optimized read speeds.

Below are the supported regions:

*   AWS US-East-1 (North Virginia)
*   AWS US-West-1 (North California)
*   AWS US-West-2 (Oregon)
*   AWS EU-West-1 (Ireland)
*   AWS EU-Central-1 (Frankfurt)
*   AWS AP-Southeast-1 (Singapore)
*   AWS AP-Southeast-2 (Sydney)
*   AWS SA-East-1 (São Paulo)

    <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/globaldb/map2.png" width="1000" />

Check out [our blog post](https://upstash.com/blog/global-database) to learn more about. Also see our [live benchmark](https://latency.upstash.com/) to check the latency number from different locations.


# Scale to Zero

Only pay for what you really use.

Traditionally, cloud services required users to predict their resource needs and provision servers or instances based on those predictions. This often led to over-provisioning to handle potential peak loads, resulting in paying for unused resources during periods of low demand.

By *scaling to zero*, our pricing model aligns more closely with actual usage.

## Pay for usage

You're only charged for the resources you actively use. When your application experiences low activity or no incoming requests, the system automatically scales down resources to a minimal level. This means you're no longer paying for idle capacity, resulting in cost savings.

<img src="https://upstash.com/blog/awsmsk/charts.png" />

## Flexibility

"Scaling to zero" offers flexibility in scaling both up and down. As your application experiences traffic spikes, the system scales up resources to meet demand. Conversely, during quiet periods, resources scale down.

## Focus on Innovation

Developers can concentrate on building and improving the application without constantly worrying about resource optimization. Upstash handles the scaling, allowing developers to focus on creating features that enhance user experiences.

In essence, this aligns pricing with actual utilization, increases cost efficiency, and promotes a more sustainable approach to resource consumption. This model empowers businesses to leverage cloud resources without incurring unnecessary expenses, making cloud computing more accessible and attractive to a broader range of organizations.


# Serverless

What do we mean by serverless?

Upstash is a modern serverless data platform. But what do we mean by serverless?

## No Server Management

In a serverless setup, developers don't need to worry about configuring or managing servers. We take care of server provisioning, scaling, and maintenance.

## Automatic Scaling

As traffic or demand increases, Upstash automatically scales the required resources to handle the load. This means applications can handle sudden spikes in traffic without manual intervention.

## Granular Billing

We charge based on the actual usage of resources rather than pre-allocated capacity. This can lead to more cost-effective solutions, as users only pay for what they consume. [Read more](/common/concepts/scale-to-zero)

## Stateless Functions

In serverless architectures, functions are typically stateless. However, the traditional approach involves establishing long-lived connections to databases, which can lead to issues in serverless environments if connections aren't properly managed after use. Additionally, there are scenarios where TCP connections may not be feasible. Upstash addresses this issue by offering access via HTTP, a universally available protocol across all platforms.

## Rapid Deployment

Fast iteration is the key to success in today's competitive environment. You can create a new Upstash database in seconds, with minimal required configuration.


# Account & Teams



## Create an Account

You can sign up to <a href="https://console.upstash.com" target="_blank">Upstash</a> using your Amazon, Github or Google accounts. Alternatively you can sign up using
email/password registration if you don't want to use these auth providers, or you
want to sign up using a corporate email address.

<Note>
  We do not access your information other than:

  *   Your email
  *   Your name
  *   Your profile picture and we never share your information with third parties.
</Note>

Team management allows you collaborate with other users. You can create a team
and invite people to the team by email addresses. The team members will have
access to the databases created under the team depending on their roles.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-member.png" />
</Frame>

## Teams

### Create Team

You can create a team using the menu `Account > Teams`

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-page.png" />
</Frame>

<br />

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-create.png" />
</Frame>

> A user can create up to 5 teams. You can be part of even more teams but only
> be the owner of 5 teams. If you need to own more teams please email us at
> [support@upstash.com](mailto:support@upstash.com).

You can still continue using your personal account or switch to a team.

> The databases in your personal account are not shared with anyone. If you want
> your database to be accessible by other users, you need to create it under a
> team.

### Switch Team

You need to switch to the team to create databases shared with other team
members. You can switch to the team via the switch button in the team table. Or
you can click your profile pic in the top right and switch to any team listed
there.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/change-team.png" />
</Frame>

### Add/Remove Team Member

Once you switched to a team, you can add team members in `Account > Teams` if
you are Owner or Admin for of the team. Entering email will be enough. The email
may not registered to Upstash yet, it is not a problem. Once the user registers
with that email, he/she will be able to switch to the team. We do not send
invitation, so when you add a member, he/she becomes a member directly. You can
remove the members from the same page.

> Only Admins or the Owner can add/remove users.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-member-create.png" />
</Frame>

### Roles

While adding a team member you need to select a role. Here the privileges of
each role:

*   Admin: This role has full access including adding removing members, databases,
    payment methods.

*   Dev: This role can create, manage and delete databases. It can not manage
    users and payment methods.

*   Finance: This role can only manage payment methods. It can not manage the
    databases and users.

*   Owner: Owner has all the privileges that admin has. In addition he is the only
    person who can delete the team. This role is assigned to the user who created
    the team. So you can not create a member with Owner role.

> If you want change role of a user, you need to delete and add again.

### Delete Team

Only the original creator (owner) can delete a team. Also the team should not
have any active databases, namely all databases under the team should be deleted
first. To delete your team, first you need to switch your personal account then
you can delete your team in the team list under `Account > Teams`.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/teams/team-member.png" />
</Frame>


# Announcements

Upstash Announcements!

Removal of GraphQL API and edge caching (Redis) (October 1, 2022) These two
features have been already deprecated. We are planning to deactivate them
completely on November 1st. We recommend use of REST API to replace GraphQL API
and Global databases instead of Edge caching.

Removal of strong consistency (Redis) (October 1, 2022) Upstash supported Strong
Consistency mode for the single region databases. We decided to deprecate this
feature because its effect on latency started to conflict with the performance
expectations of Redis use cases. Moreover, we improved the consistency of
replication to guarantee Read-Your-Writes consistency. Strong consistency will
be disabled on existing databases on November 1st.

#### Redis pay-as-you-go usage cap (October 1, 2022)

We are increasing the max usage cap to \$160 from \$120 as of October 1st. This
update is needed because of the increasing infrastructure cost due to
replicating all databases to multiple instances. After your database exceeds the
max usage cost, your database might be rate limited.

#### Replication is enabled (Sep 29, 2022)

All new and existing paid databases will be replicated to multiple replicas.
Replication enables high availability in case of system and infrastructure
failures. Starting from October 1st, we will gradually upgrade all databases
without downtime. Free databases will stay single replica.

<hr />

#### QStash Price Decrease (Sep 15, 2022)

The price is \$1 per 100K requests.

<hr />

#### [Pulumi Provider is available](https://upstash.com/blog/upstash-pulumi-provider)  (August 4, 2022)

<hr />

#### [QStash is released and announced](https://upstash.com/blog/qstash-announcement)  (July 18, 2022)

<hr />

#### [Announcing Upstash CLI](https://upstash.com/blog/upstash-cli)  (May 16, 2022)

<hr />

#### [Introducing Redis 6 Compatibility](https://upstash.com/blog/redis-6)  (April 10, 2022)

<hr />

#### Strong Consistency Deprecated (March 29, 2022)

We have deprecated Strong Consistency mode for Redis databases due to its
performance impact. This will not be available for new databases. We are
planning to disable it on existing databases before the end of 2023. The
database owners will be notified via email.

<hr />

#### [Announcing Upstash Redis SDK v1.0.0](https://upstash.com/blog/upstash-redis-sdk-v1) (March 14, 2022)

<hr />

#### Support for Kafka (Nov 29, 2021)

Kafka Support is released. Check the
[the blog post](https://blog.upstash.com/serverless-kafka-launch).

<hr />

#### Support for Google Cloud (June 8, 2021)

Google Cloud is available for Upstash Redis databases. We initially support
US-Central-1 (Iowa) region. Check the
[get started guide](https://docs.upstash.com/redis/howto/getstartedgooglecloudfunctions).

<hr />

#### Support for AWS Japan (March 1, 2021)

こんにちは日本

Support for AWS Tokyo Region was the most requested feature by our users. Now
our users can create their database in AWS Asia Pacific (Tokyo) region
(ap-northeast-1). In addition to Japan, Upstash is available in the regions
us-west-1, us-east-1, eu-west-1.

Click [here](https://console.upstash.com) to start your database for free.

Click [here](https://roadmap.upstash.com) to request new regions to be
supported.

<hr />

#### Vercel Integration (February 22, 2021)

Upstash\&Vercel integration has been released. Now you are able to integrate
Upstash to your project easily. We believe Upstash is the perfect database for
your applications thanks to its:

*   Low latency data
*   Per request pricing
*   Durable storage
*   Ease of use

Below are the resources about the integration:

See [how to guide](https://docs.upstash.com/redis/howto/vercelintegration).

See [integration page](https://vercel.com/integrations/upstash).

See
[Roadmap Voting app](https://github.com/upstash/roadmap)
as a showcase for the integration.


# Compliance



## Upstash Legal & Security Documents

*   [Upstash Terms of Service](https://upstash.com/static/trust/terms.pdf)
*   [Upstash Privacy Policy](https://upstash.com/static/trust/privacy.pdf)
*   [Upstash Data Processing Agreement](https://upstash.com/static/trust/dpa.pdf)
*   [Upstash Technical and Organizational Security Measures](https://upstash.com/static/trust/security-measures.pdf)
*   [Upstash Subcontractors](https://upstash.com/static/trust/subprocessors.pdf)

## Is Upstash SOC2 Compliant?

Upstash Redis databases under Pro and Enterprise support plans are SOC2 compliant. Check our [trust page](https://trust.upstash.com/) for details.

## Is Upstash ISO-27001 Compliant?

We are in process of getting this certification. Contact us
([support@upstash.com](mailto:support@upstash.com)) to learn about the expected
date.

## Is Upstash GDPR Compliant?

Yes. For more information, see our
[Privacy Policy](https://upstash.com/static/trust/privacy.pdf). We acquire DPAs
from each [subcontractor](https://upstash.com/static/trust/subprocessors.pdf)
that we work with.

## Is Upstash HIPAA Compliant?

We are in process of getting this certification. Contact us
([support@upstash.com](mailto:support@upstash.com)) to learn about the expected
date.

## Is Upstash PCI Compliant?

Upstash does not store personal credit card information. We use Stripe for
payment processing. Stripe is a certified PCI Service Provider Level 1, which is
the highest level of certification in the payments industry.

## Does Upstash conduct vulnerability scanning and penetration tests?

Yes, we use third party tools and work with pen testers. We share the results
with Enterprise customers. Contact us
([support@upstash.com](mailto:support@upstash.com)) for more information.

## Does Upstash take backups?

Yes, we take regular snapshots of the data cluster to the AWS S3 platform.

## Does Upstash encrypt data?

Customers can enable TLS when creating a database or cluster, and we recommend this for production environments. Additionally, we encrypt data at rest upon customer request.


# Integration with Third Parties & Partnerships



<Frame>
  <img height="100" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/integration/upstash-integration-diagram.png" />
</Frame>

## Introduction

In this guideline we will outline the steps to integrate Upstash into your platform (GUI or Web App) and allow your users to create and manage Upstash databases without leaving your interfaces. We will explain how to use OAuth2.0 as the underlying foundation to enable this access seamlessly.

If your product or service offering utilizes Redis, Kafka or QStash or if there is a common use case that your end users enable by leveraging these database resources, we invite you to be a partner with us. By integrating Upstash into your platform, you can offer a more complete package for your customers and become a one stop shop. This will also position yourself at the forefront of innovative cloud computing trends such as serverless and expand your customer base.

This is the most commonly used partnership integration model that can be easily implemented by following this guideline. Recently [Cloudflare workers integration](https://blog.cloudflare.com/cloudflare-workers-database-integration-with-upstash/) is implemented through this methodology. For any further questions or partnership discussions please send us an email at [partnerships@upstash.com](mailto:partnerships@upstash.com)

<Info>
  Before starting development to integrate Upstash into your product, please
  send an email to [partnerships@upstash.com](mailto:partnerships@upstash.com) for further assistance and guidance.
</Info>

**General Flow (High level user flow)**

1.  User clicks **`Connect Upstash`** button on your platform’s surface (GUI, Web App)
2.  This initiates the OAuth 2.0 flow, which opens a new browser page displaying the **`Upstash Login Page`**.
3.  If this is an existing user, user logins with their Upstash credentials otherwise they can directly sign up for a new Upstash account.
4.  Browser window redirects to **`Your account has been connected`** page and authentication window automatically closes.
5.  After the user returns to your interface, they see their Upstash Account is now connected.

## Technical Design (SPA - Regular Web Application)

1.  Users click `Connect Upstash` button from Web App.
2.  Web App initiate Upstash OAuth 2.0 flow. Web App can use
    [Auth0 native libraries](https://auth0.com/docs/libraries).

<Note>
  Please reach [partnerships@upstash.com](mailto:partnerships@upstash.com) to receive client id and callback url.
</Note>

3.  After user returns from OAuth 2.0 flow then web app will have JWT token. Web
    App can generate Developer Api key:

```bash
curl -XPOST https://api.upstash.com/apikey \
    -H "Authorization: Bearer JWT_KEY" \
    -H "Content-Type: application/json" \
    -d '{ "name": "APPNAME_API_KEY_TIMESTAMP" }'
```

4.  Web App need to save Developer Api Key to the backend.

## Technical Design ( GUI Apps )

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/integration/oauth2-integration.png" />
</Frame>

1.  User clicks **`Connect Upstash`** button from web app.
2.  Web app initiates Upstash OAuth 2.0 flow and it can use **[Auth0 native libraries](https://auth0.com/docs/libraries)**.
3.  App will open new browser:

```
https://auth.upstash.com/authorize?response_type=code&audience=upstash-api&scope=offline_access&client_id=XXXXXXXXXX&redirect_uri=http%3A%2F%2Flocalhost:3000
```

<Note>Please reach [partnerships@upstash.com](mailto:partnerships@upstash.com) to receive client id.</Note>

4.  After user authenticated Auth0 will redirect user to
    `localhost:3000/?code=XXXXXX`

5.  APP can return some nice html response when Auth0 returns to `localhost:3000`

6.  After getting `code` parameter from the URL query, GUI App will make http
    call to the Auth0 code exchange api. Example CURL request

```bash
curl -XPOST 'https://auth.upstash.com/oauth/token' \
  --header 'content-type: application/x-www-form-urlencoded' \
  --data 'grant_type=authorization_code --data audience=upstash-api' \
  --data 'client_id=XXXXXXXXXXX' \
  --data 'code=XXXXXXXXXXXX' \
  --data 'redirect_uri=localhost:3000'
```

Response:

```json
{
  "access_token": "XXXXXXXXXX",
  "refresh_token": "XXXXXXXXXXX",
  "scope": "offline_access",
  "expires_in": 172800,
  "token_type": "Bearer"
}
```

7.  After 6th Step the response will include `access_token`, it has 3 days TTL.
    GUI App will call Upstash API to get a developer api key:

```bash
curl https://api.upstash.com/apikey -H "Authorization: Bearer JWT_KEY" -d '{ "name" : "APPNAME_API_KEY_TIMESTAMP" }'
```

8.  GUI App will save Developer Api key locally. Then GUI App can call any
    Upstash Developer API [developer.upstash.com/](https://developer.upstash.com/)

## Managing Resources

After obtaining Upstash Developer Api key, your platform surface (web or GUI) can call Upstash API. For example **[Create Database](https://developer.upstash.com/#create-database-global)**, **[List Database](https://developer.upstash.com/#list-databases)**

In this flow, you can ask users for region information and name of the database then can call Create Database API to complete the task

Example CURL request:

```bash
curl -X POST \
  https://api.upstash.com/v2/redis/database \
  -u 'EMAIL:API_KEY' \
  -d '{"name":"myredis", "region":"global", "primary_region":"us-east-1", "read_regions":["us-west-1","us-west-2"], "tls": true}'
```


# Legal



## Upstash Legal Documents

*   [Upstash Terms of Service](https://upstash.com/trust/terms.pdf)
*   [Upstash Privacy Policy](https://upstash.com/trust/privacy.pdf)
*   [Upstash Subcontractors](https://upstash.com/trust/subprocessors.pdf)


# Professional Support



For all Upstash products, we manage everything for you and let you focus on more important things. If you ever need further help, our dedicated Professional Support team are here to ensure you get the most out of our platform, whether you’re just starting or scaling to new heights.

Professional Support is strongly recommended especially for customers who use Upstash as part of their production systems.

# Expert Guidance

Get direct access to our team of specialists who can provide insights, troubleshooting, and best practices tailored to your unique use case. In any urgent incident you might have, our Support team will be standing by and ready to join you for troubleshooting.

Professional Support package includes:

*   **Guaranteed Response Time:** Rapid Response Time SLA to urgent support requests, ensuring your concerns are addressed promptly with a **24/7 coverage**.
*   **Customer Onboarding:** A personalized session to guide you through utilizing our support services and reviewing your specific use case for a seamless start.
*   **Quarterly Use Case Review & Health Check:** On-request sessions every quarter to review your use case and ensure optimal performance.
*   **Dedicated Slack Channel:** Direct access to our team via a private Slack channel, so you can reach out whenever you need assistance.
*   **Incident Support:** Video call support during critical incidents to provide immediate help and resolution.
*   **Root Cause Analysis:** Comprehensive investigation and post-mortem analysis of critical incidents to identify and address the root cause.

# Response Time SLA

We understand that timely assistance is critical for production workloads, so your access to our Support team comes with 24/7 coverage and below SLA:

| Severity                        | Response Time |
| ------------------------------- | ------------- |
| P1 - Production system down     | 30 minutes    |
| P2 - Production system impaired | 2 hours       |
| P3 - Minor issue                | 12 hours      |
| P4 - General guidance           | 24 hours      |

## How to Reach Out?

As a Professional Support Customer, below are the **two methods** to reach out to Upstash Support Team, in case you need to utilize our services:

#### Starting a Chat

You will see a chatbox on the bottom right when viewing Upstash console, docs and website. Once you initiate a chat, Professional Support customers will be prompted to select a severity level:

<img noZoom width="300" height="100" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pro-support/image.png" />

<Note>
  To be able to see these options in chat, remember to sign into your Upstash Account first.
</Note>

If you select "P1 - Production down, no workaround", or "P2 - Production impaired with workaround" options, you will be triggering an alert for our team to urgently step in.

#### Sending an Email

Sending an email with details to [support@upstash.com](mailto:support@upstash.com) is another way to submit a support request. In case of an urgency, sending an email with details by using "urgent" keyword in email subject is another alternative to alert our team about a possible incident.

# Pricing

For pricing and further details about Professional Support, please contact us at [support@upstash.com](mailto:support@upstash.com)


# Uptime SLA



This Service Level Agreement ("SLA") applies to the use of the Upstash services,
offered under the terms of our Terms of Service or other agreement with us
governing your use of Upstash. This SLA does not apply to Upstash services in
the Upstash Free and Pay-as-you-go Tier. It is clarified that this SLA is subject to the terms of
the Agreement, and does not derogate therefrom (capitalized terms, unless
otherwise indicated herein, have the meaning specified in the Agreement).
Upstash reserves the right to change the terms of this SLA by publishing updated
terms on its website, such change to be effective as of the date of publication.

### Upstash Database SLA

Upstash will use commercially reasonable efforts to make
databases available with a Monthly Uptime Percentage of at least 99.99%.

In the event any of the services do not meet the SLA, you will be eligible to
receive a Service Credit as described below.

| Monthly Uptime Percentage                           | Service Credit Percentage |
| --------------------------------------------------- | ------------------------- |
| Less than 99.99% but equal to or greater than 99.0% | 10%                       |
| Less than 99.0% but equal to or greater than 95.0%  | 30%                       |
| Less than 95.0%                                     | 60%                       |

### SLA Credits

Service Credits are calculated as a percentage of the monthly bill (excluding
one-time payments such as upfront payments) for the service in the affected
region that did not meet the SLA.

Uptime percentages are recorded and published in the
[Upstash Status Page](https://status.upstash.com).

To receive a Service Credit, you should submit a claim by sending an email to
[support@upstash.com](mailto:support@upstash.com). Your credit request should be
received by us before the end of the second billing cycle after the incident
occurred.

We will apply any service credits against future payments for the applicable
services. At our discretion, we may issue the Service Credit to the credit card
you used. Service Credits will not entitle you to any refund or other payment. A
Service Credit will be applicable and issued only if the credit amount for the
applicable monthly billing cycle is greater than one dollar (\$1 USD). Service
Credits may not be transferred or applied to any other account.


# Support & Contact Us



## Community

[Upstash Discord Channel](https://upstash.com/discord) is the best way to
interact with the community.

## Team

Regardless of your subscription plan, you can contact the team
via [support@upstash.com](mailto:support@upstash.com) for technical support as
well as questions and feedback.

## Follow Us

Follow us on [X](https://x.com/upstash).

## Bugs & Issues

You can help us improve Upstash by reporting issues, suggesting new features and
giving general feedback in
our [Community Github Repo](https://github.com/upstash/issues/issues/new).

## Enterprise Support

Get [Enterprise Support](/common/help/prosupport) for your organization from the Upstash team.


# Uptime Monitor



## Status Page

You can track the uptime status of Upstash databases in
[Upstash Status Page](https://status.upstash.com)

## Latency Monitor

You can see the average latencies for different regions in
[Upstash Latency Monitoring](https://latency.upstash.com) page


# Trials



<Info>
  If you want to try Upstash paid and pro plans, we can offer **Free
  Trials**. Email us at [support@upstash.com](mailto:support@upstash.com)
</Info>


# Overview



Manage Upstash resources in your terminal or CI.

You can find the Github Repository [here](https://github.com/upstash/cli).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/oss/cli/banner.svg" />
</Frame>

<br />

# Installation

## npm

You can install upstash's cli directly from npm

```bash
npm i -g @upstash/cli
```

It will be added as `upstash` to your system's path.

## Compiled binaries:

`upstash` is also available from the
[releases page](https://github.com/upstash/cli/releases/latest) compiled
for windows, linux and mac (both intel and m1).

# Usage

```bash
> upstash

  Usage:   upstash
  Version: development

  Description:

    Official cli for Upstash products

  Options:

    -h, --help               - Show this help.
    -V, --version            - Show the version number for this program.
    -c, --config   <string>  - Path to .upstash.json file

  Commands:

    auth   - Login and logout
    redis  - Manage redis database instances
    kafka  - Manage kafka clusters and topics
    team   - Manage your teams and their members

  Environment variables:

    UPSTASH_EMAIL    <string>  - The email you use on upstash
    UPSTASH_API_KEY  <string>  - The api key from upstash
```

## Authentication

When running `upstash` for the first time, you should log in using
`upstash auth login`. Provide your email and an api key.
[See here for how to get a key.](https://docs.upstash.com/redis/howto/developerapi#api-development)

As an alternative to logging in, you can provide `UPSTASH_EMAIL` and
`UPSTASH_API_KEY` as environment variables.

## Usage

Let's create a new redis database:

```
> upstash redis create --name=my-db --region=eu-west-1
  Database has been created

  database_id          a3e25299-132a-45b9-b026-c73f5a807859
  database_name        my-db
  database_type        Pay as You Go
  region               eu-west-1
  type                 paid
  port                 37090
  creation_time        1652687630
  state                active
  password             88ae6392a1084d1186a3da37fb5f5a30
  user_email           andreas@upstash.com
  endpoint             eu1-magnetic-lacewing-37090.upstash.io
  edge                 false
  multizone            false
  rest_token           AZDiASQgYTNlMjUyOTktMTMyYS00NWI5LWIwMjYtYzczZjVhODA3ODU5ODhhZTYzOTJhMTA4NGQxMTg2YTNkYTM3ZmI1ZjVhMzA=
  read_only_rest_token ApDiASQgYTNlMjUyOTktMTMyYS00NWI5LWIwMjYtYzczZjVhODA3ODU5O_InFjRVX1XHsaSjq1wSerFCugZ8t8O1aTfbF6Jhq1I=


  You can visit your database details page: https://console.upstash.com/redis/a3e25299-132a-45b9-b026-c73f5a807859

  Connect to your database with redis-cli: redis-cli -u redis://88ae6392a1084d1186a3da37fb5f5a30@eu1-magnetic-lacewing-37090.upstash.io:37090
```

## Output

Most commands support the `--json` flag to return the raw api response as json,
which you can parse and automate your system.

```bash
> upstash  redis create --name=test2113 --region=us-central1 --json | jq '.endpoint'

 "gusc1-clean-gelding-30208.upstash.io"
```


# Authentication

Authentication for the Upstash Developer API

The Upstash API requires API keys to authenticate requests. You can view and
manage API keys at the Upstash Console.

Upstash API uses HTTP Basic authentication. You should pass `EMAIL` and
`API_KEY` as basic authentication username and password respectively.

With a client such as `curl`, you can pass your credentials with the `-u`
option, as the following example shows:

```curl
curl https://api.upstash.com/v2/redis/database -u EMAIL:API_KEY
```

Replace `EMAIL` and `API_KEY` with your email and API key.


# HTTP Status Codes

The Upstash API uses the following HTTP Status codes:

| Code | Description               |                                                                                 |
| ---- | ------------------------- | ------------------------------------------------------------------------------- |
| 200  | **OK**                    | Indicates that a request completed successfully and the response contains data. |
| 400  | **Bad Request**           | Your request is invalid.                                                        |
| 401  | **Unauthorized**          | Your API key is wrong.                                                          |
| 403  | **Forbidden**             | The kitten requested is hidden for administrators only.                         |
| 404  | **Not Found**             | The specified kitten could not be found.                                        |
| 405  | **Method Not Allowed**    | You tried to access a kitten with an invalid method.                            |
| 406  | **Not Acceptable**        | You requested a format that isn't JSON.                                         |
| 429  | **Too Many Requests**     | You're requesting too many kittens! Slow down!                                  |
| 500  | **Internal Server Error** | We had a problem with our server. Try again later.                              |
| 503  | **Service Unavailable**   | We're temporarily offline for maintenance. Please try again later.              |


# Getting Started



Using Upstash API, you can develop applications that can create and manage
Upstash databases and Upstash kafka clusters. You can automate everything that
you can do in the console. To use developer API, you need to create an API key
in the console.

### Create an API key

1.  Log in to the console then in the left menu click the
    `Account > Management API` link.

2.  Click the `Create API Key` button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/developerapi/api-key-list.png" />
</Frame>

3.  Enter a name for your key. You can not use the same name for multiple keys.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/developerapi/api-key-create.png" />
</Frame>

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/developerapi/api-key-secret.png" />
</Frame>

You need to download or copy/save your API key. Upstash does not remember or
keep your API for security reasons. So if you forget your API key, it becomes
useless; you need to create a new one.

<br />

You can create multiple keys. It is recommended to use different keys in
different applications. By default one user can create up to 37 API keys. If you
need more than that, please send us an email at
[support@upstash.com](mailto:support@upstash.com)

### Deleting an API key

When an API key is exposed (e.g. accidentally shared in a public repository) or
not being used anymore; you should delete it. You can delete the API keys in
`Account > API Keys` screen.

### Roadmap

**Role based access:** You will be able to create API keys with specific
privileges. For example you will be able to create a key with read-only access.

**Stats:** We will provide reports based on usage of your API keys.


# Create Kafka Cluster

POST https://api.upstash.com/v2/kafka/cluster
This endpoint creates a new kafka cluster.

## Request Parameters

<ParamField body="name" type="string" required>
  Name of the new Kafka cluster
</ParamField>

<ParamField body="region" type="string" required>
  The region the cluster will be deployed in
  <br /> **Options:** `eu-west-1` or `us-east-1`
</ParamField>

<ParamField body="multizone" type="boolean" required>
  Set true to enable multi-zone replication
</ParamField>

## Response Parameters

<ResponseField name="cluster_id" type="string">
  ID of the created kafka cluster
</ResponseField>

<ResponseField name="name" type="string">
  Name of the kafka cluster
</ResponseField>

<ResponseField name="region" type="string">
  The region the kafka cluster is deployed in
</ResponseField>

<ResponseField name="type" type="string">
  Shows whether the cluster is free or paid
</ResponseField>

<ResponseField name="multizone" type="boolean">
  Whether the multizone replication is enabled for the cluster or not
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint to connect to the kafka cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint to connect to the kafka cluster
</ResponseField>

<ResponseField name="state" type="string">
  Current state of the cluster(active, deleted)
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="int">
  Max retention size will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="int">
  Max retention time will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_messages_per_second" type="int">
  Max messages allowed to be produced per second
</ResponseField>

<ResponseField name="creation_time" type="int">
  Cluster creation timestamp
</ResponseField>

<ResponseField name="max_message_size" type="int">
  Max message size will be allowed in topics in the cluster
</ResponseField>

<ResponseField name="max_partitions" type="int">
  Max total number of partitions allowed in the cluster
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/cluster \
    -u 'EMAIL:API_KEY' \
    -d '{"name":"mykafkacluster","region":"eu-west-1","multizone":true}'
  ```

  ```python Python
  import requests

  data = '{"name":"mykafkacluster","region":"eu-west-1","multizone":true}'

  response = requests.post('https://api.upstash.com/v2/kafka/cluster', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "name": "test_kafka_cluster_4",
      "region": "eu-west-1",
      "multizone": true
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/cluster", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9",
      "name": "mykafkacluster",
      "region": "eu-west-1",
      "type": "paid",
      "multizone": true,
      "tcp_endpoint": "sharing-mastodon-12819-eu1-kafka.upstashdev.com",
      "rest_endpoint": "sharing-mastodon-12819-eu1-rest-kafka.upstashdev.com",
      "state": "active",
      "username": "c2hhcmluZy1tYXN0b2Rvbi0xMjgxOSRV1ipriSBOwd0PHzw2KAs_cDrTXzvUKIs",
      "password": "zlQgc0nbgcqF6MxOqnh7tKjJsGnSgLFS89uS-FXzMVqhL2dgFbmHwB-IXAAsOYXzUYj40g==",
      "max_retention_size": 1073741824000,
      "max_retention_time": 2592000000,
      "max_messages_per_second": 1000,
      "creation_time": 1643978975,
      "max_message_size": 1048576,
      "max_partitions": 100
  }
  ```
</ResponseExample>


# Delete Kafka Cluster

DELETE https://api.upstash.com/v2/kafka/cluster/{id}
This endpoint deletes a kafka cluster.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka cluster to be deleted
</ParamField>

<RequestExample>
  ```shell curl
  curl -X DELETE \
    https://api.upstash.com/v2/kafka/cluster/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.delete('https://api.upstash.com/v2/kafka/cluster/:id' auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("DELETE", "https://api.upstash.com/v2/kafka/cluster/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK" 
  ```
</ResponseExample>


# Get Kafka Cluster

GET https://api.upstash.com/v2/Kafka/cluster/{id}
This endpoint gets details of a Kafka cluster. 

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka cluster
</ParamField>

## Response Parameters

<ResponseField name="cluster_id" type="string">
  ID of the created Kafka cluster
</ResponseField>

<ResponseField name="name" type="string">
  Name of the Kafka cluster
</ResponseField>

<ResponseField name="region" type="string">
  The region the Kafka cluster is deployed in
</ResponseField>

<ResponseField name="type" type="string">
  Shows whether the cluster is free or paid
</ResponseField>

<ResponseField name="multizone" type="boolean">
  Whether the multizone replication is enabled for the cluster or not
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="state" type="string">
  Current state of the cluster(active, deleted)
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="int">
  Max retention size will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="int">
  Max retention time will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_messages_per_second" type="int">
  Max messages allowed to be produced per second
</ResponseField>

<ResponseField name="creation_time" type="int">
  Cluster creation timestamp
</ResponseField>

<ResponseField name="max_message_size" type="int">
  Max message size will be allowed in topics in the cluster
</ResponseField>

<ResponseField name="max_partitions" type="int">
  Max total number of partitions allowed in the cluster
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/kafka/cluster/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/kafka/cluster/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/kafka/cluster/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9",
      "name": "test_kafka_cluster",
      "region": "eu-west-1",
      "type": "paid",
      "multizone": true,
      "tcp_endpoint": "sharing-mastodon-12819-eu1-kafka.upstashdev.com",
      "rest_endpoint": "sharing-mastodon-12819-eu1-rest-kafka.upstashdev.com",
      "state": "active",
      "username": "c2hhcmluZy1tYXN0b2Rvbi0xMjgxOSRV1ipriSBOwd0PHzw2KAs_cDrTXzvUKIs",
      "password": "zlQgc0nbgcqF6MxOqnh7tKjJsGnSgLFS89uS-FXzMVqhL2dgFbmHwB-IXAAsOYXzUYj40g==",
      "max_retention_size": 1073741824000,
      "max_retention_time": 2592000000,
      "max_messages_per_second": 1000,
      "creation_time": 1643978975,
      "max_message_size": 1048576,
      "max_partitions": 100
  }
  ```
</ResponseExample>


# List Kafka Clusters

GET https://api.upstash.com/v2/kafka/clusters
This endpoint lists all kafka clusters of user.

## Response Parameters

<ResponseField name="cluster_id" type="string">
  ID of the created kafka cluster
</ResponseField>

<ResponseField name="name" type="string">
  Name of the kafka cluster
</ResponseField>

<ResponseField name="region" type="string">
  The region the kafka cluster is deployed in
</ResponseField>

<ResponseField name="type" type="string">
  Shows whether the cluster is free or paid
</ResponseField>

<ResponseField name="multizone" type="boolean">
  Whether the multizone replication is enabled for the cluster or not
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint to connect to the kafka cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint to connect to the kafka cluster
</ResponseField>

<ResponseField name="state" type="string">
  Current state of the cluster(active, deleted)
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="int">
  Max retention size will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="int">
  Max retention time will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_messages_per_second" type="int">
  Max messages allowed to be produced per second
</ResponseField>

<ResponseField name="creation_time" type="int">
  Cluster creation timestamp
</ResponseField>

<ResponseField name="max_message_size" type="int">
  Max message size will be allowed in topics in the cluster
</ResponseField>

<ResponseField name="max_partitions" type="int">
  Max total number of partitions allowed in the cluster
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/kafka/clusters \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/kafka/clusters', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/kafka/clusters", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
      {
          "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9",
          "name": "test_kafka_cluster",
          "region": "eu-west-1",
          "type": "paid",
          "multizone": true,
          "tcp_endpoint": "sharing-mastodon-12819-eu1-kafka.upstashdev.com",
          "rest_endpoint": "sharing-mastodon-12819-eu1-rest-kafka.upstashdev.com",
          "state": "active",
          "username": "c2hhcmluZy1tYXN0b2Rvbi0xMjgxOSRV1ipriSBOwd0PHzw2KAs_cDrTXzvUKIs",
          "password": "zlQgc0nbgcqF6MxOqnh7tKjJsGnSgLFS89uS-FXzMVqhL2dgFbmHwB-IXAAsOYXzUYj40g==",
          "max_retention_size": 1073741824000,
          "max_retention_time": 2592000000,
          "max_messages_per_second": 1000,
          "creation_time": 1643978975,
          "max_message_size": 1048576,
          "max_partitions": 100
      }
  ]
  ```
</ResponseExample>


# Rename Kafka Cluster

POST https://api.upstash.com/v2/kafka/rename-cluster/{id}
This endpoint gets details of a Kafka cluster. 

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka cluster
</ParamField>

## Request Parameters

<ParamField body="name" type="string" required>
  The new name of the kafka cluster
</ParamField>

## Response Parameters

<ResponseField name="cluster_id" type="string">
  ID of the created Kafka cluster
</ResponseField>

<ResponseField name="name" type="string">
  Name of the Kafka cluster
</ResponseField>

<ResponseField name="region" type="string">
  The region the Kafka cluster is deployed in
</ResponseField>

<ResponseField name="type" type="string">
  Shows whether the cluster is free or paid
</ResponseField>

<ResponseField name="multizone" type="boolean">
  Whether the multizone replication is enabled for the cluster or not
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="state" type="string">
  Current state of the cluster(active, deleted)
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="int">
  Max retention size will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="int">
  Max retention time will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_messages_per_second" type="int">
  Max messages allowed to be produced per second
</ResponseField>

<ResponseField name="creation_time" type="int">
  Cluster creation timestamp
</ResponseField>

<ResponseField name="max_message_size" type="int">
  Max message size will be allowed in topics in the cluster
</ResponseField>

<ResponseField name="max_partitions" type="int">
  Max total number of partitions allowed in the cluster
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/rename-cluster/:id \
    -u 'EMAIL:API_KEY' \
    -d '{"name":"mykafkacluster-2"}'
  ```

  ```python Python
  import requests

  data = '{"name":"mykafkacluster-2"}'

  response = requests.post('https://api.upstash.com/v2/kafka/rename-cluster/:id', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "name":"mykafkacluster-2"
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/rename-cluster/:id", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9",
      "name": "mykafkacluster-2",
      "region": "eu-west-1",
      "type": "paid",
      "multizone": true,
      "tcp_endpoint": "sharing-mastodon-12819-eu1-kafka.upstashdev.com",
      "rest_endpoint": "sharing-mastodon-12819-eu1-rest-kafka.upstashdev.com",
      "state": "active",
      "username": "c2hhcmluZy1tYXN0b2Rvbi0xMjgxOSRV1ipriSBOwd0PHzw2KAs_cDrTXzvUKIs",
      "password": "zlQgc0nbgcqF6MxOqnh7tKjJsGnSgLFS89uS-FXzMVqhL2dgFbmHwB-IXAAsOYXzUYj40g==",
      "max_retention_size": 1073741824000,
      "max_retention_time": 2592000000,
      "max_messages_per_second": 1000,
      "creation_time": 1643978975,
      "max_message_size": 1048576,
      "max_partitions": 100
  }
  ```
</ResponseExample>


# Reset Kafka Cluster Password

POST https://api.upstash.com/v2/kafka/reset-password/{id}
This endpoint updates the password of a kafka cluster 

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka cluster to reset password
</ParamField>

## Response Parameters

<ResponseField name="cluster_id" type="string">
  ID of the created Kafka cluster
</ResponseField>

<ResponseField name="name" type="string">
  Name of the Kafka cluster
</ResponseField>

<ResponseField name="region" type="string">
  The region the Kafka cluster is deployed in
</ResponseField>

<ResponseField name="type" type="string">
  Shows whether the cluster is free or paid
</ResponseField>

<ResponseField name="multizone" type="boolean">
  Whether the multizone replication is enabled for the cluster or not
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="state" type="string">
  Current state of the cluster(active, deleted)
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="int">
  Max retention size will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="int">
  Max retention time will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_messages_per_second" type="int">
  Max messages allowed to be produced per second
</ResponseField>

<ResponseField name="creation_time" type="int">
  Cluster creation timestamp
</ResponseField>

<ResponseField name="max_message_size" type="int">
  Max message size will be allowed in topics in the cluster
</ResponseField>

<ResponseField name="max_partitions" type="int">
  Max total number of partitions allowed in the cluster
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/reset-password/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/kafka/reset-password/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/reset-password/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9",
      "name": "mykafkacluster-2",
      "region": "eu-west-1",
      "type": "paid",
      "multizone": true,
      "tcp_endpoint": "sharing-mastodon-12819-eu1-kafka.upstashdev.com",
      "rest_endpoint": "sharing-mastodon-12819-eu1-rest-kafka.upstashdev.com",
      "state": "active",
      "username": "c2hhcmluZy1tYXN0b2Rvbi0xMjgxOSRV1ipriSBOwd0PHzw2KAs_cDrTXzvUKIs",
      "password": "7ea02715ceeb4fd3ba1542a5f3bf758e",
      "max_retention_size": 1073741824000,
      "max_retention_time": 2592000000,
      "max_messages_per_second": 1000,
      "creation_time": 1643978975,
      "max_message_size": 1048576,
      "max_partitions": 100
  }
  ```
</ResponseExample>


# Get Kafka Cluster Stats

GET  https://api.upstash.com/v2/kafka/stats/topic/{id}
This endpoint gets detailed stats of a database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka cluster
</ParamField>

## Response Parameters

<ResponseField name="throughput" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Number of monthly messages in kafka cluster
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="produce_throughput" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Number of monthly messages produced in kafka cluster
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="consume_throughput" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Number of monthly messages consumed in kafka cluster
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="diskusage" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total disk usage of the kafka cluster
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="days" type="Array<string>">
  String representation of last 5 days of the week starting from the current day
</ResponseField>

<ResponseField name="dailyproduce" type="int">
  Last 5 days daily produced message count in kafka cluster
</ResponseField>

<ResponseField name="dailyconsume" type="int">
  Last 5 days daily consumed message count in kafka cluster
</ResponseField>

<ResponseField name="total_monthly_storage" type="int">
  Average storage size of the kafka cluster in the current month
</ResponseField>

<ResponseField name="total_monthly_billing" type="int">
  Total cost of the kafka cluster in current month
</ResponseField>

<ResponseField name="total_monthly_produce" type="int">
  Total number of produced message in current month
</ResponseField>

<ResponseField name="total_monthly_consume" type="int">
  Total number of consumed message in current month
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/kafka/stats/cluster/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/kafka/stats/cluster/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/kafka/stats/cluster/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "throughput": [
      {
        "x": "2022-02-07 11:30:28",
        "y": 0
      }
      ...
    ],
    "produce_throughput": [
      {
        "x": "2022-02-07 11:30:28",
        "y": 0
      }
      ...
    ],
    "consume_throughput": [
      {
        "x": "2022-02-07 11:30:28",
        "y": 0
      }
      ...
    ],
    "diskusage": [
      {
        "x": "2022-02-07 11:45:28",
        "y": 0
      }
      ...
    ],
    "days": [
      "Thursday",
      "Friday",
      "Saturday",
      "Sunday",
      "Monday"
    ],
    "dailyproduce": [
      {
        "x": "2022-02-07 11:30:28.937259962 +0000 UTC",
        "y": 0
      }
      ...
    ],
    "dailyconsume": [
      {
        "x": "2022-02-07 11:30:28.937256776 +0000 UTC",
        "y": 0
      }
      ...
    ],
    "total_monthly_storage": 0,
    "total_monthly_billing": 0,
    "total_monthly_produce": 0,
    "total_monthly_consume": 0
  }
  ```
</ResponseExample>


# Create Kafka Connector

POST https://api.upstash.com/v2/kafka/connector
This endpoint creates a new kafka connector in a cluster.

## Request Parameters

<ParamField body="name" type="string" required>
  Name of the new kafka topic
</ParamField>

<ParamField body="cluster_id" type="string" required>
  ID of the cluster the topic will be deployed in
</ParamField>

<ParamField body="properties" type="Object" required>
  Properties of the connector. Custom config for different types of connectors.
</ParamField>

## Response Parameters

<ResponseField name="connector_id" type="string">
  ID of the new kafka connector
</ResponseField>

<ResponseField name="name" type="string">
  Name of the new kafka connector
</ResponseField>

<ResponseField name="customer_id" type="string">
  Owner of the connector
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the kafka cluster of the connector
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the topic
</ResponseField>

<ResponseField name="deletion_time" type="int">
  Creation time of the topic
</ResponseField>

<ResponseField name="state" type="string">
  State of the connector
</ResponseField>

<ResponseField name="state_error_message" type="string">
  Error message, if the connector failed
</ResponseField>

<ResponseField name="connector_state" type="string">
  State of the connector
</ResponseField>

<ResponseField name="tasks" type="object">
  Tasks for the connector
</ResponseField>

<ResponseField name="topics" type="string">
  Topics that are given with properties config
</ResponseField>

<ResponseField name="connector_class" type="string">
  Class of the created connector
</ResponseField>

<ResponseField name="encoded_username" type="string">
  Encoded username for the connector
</ResponseField>

<ResponseField name="TTL" type="int">
  Time to live for connector
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/connector \
    -u 'EMAIL:API_KEY' \
    -d '{"name":"connectorName","cluster_id":"7568431c-88d5-4409-a808-2167f22a7133", "properties":{"connector.class": "com.mongodb.kafka.connect.MongoSourceConnector","connection.uri": "connection-uri"}}'
  ```

  ```python Python
  import requests

  data = '{"name":"connectorName","cluster_id":"7568431c-88d5-4409-a808-2167f22a7133", "properties":{"connector.class": "com.mongodb.kafka.connect.MongoSourceConnector","connection.uri": "connection-uri"}}'

  response = requests.post('https://api.upstash.com/v2/kafka/connector', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "name": "connectorName",
      "cluster_id": "7568431c-88d5-4409-a808-2167f22a7133",
      "properties":{"connector.class": "com.mongodb.kafka.connect.MongoSourceConnector","connection.uri": "connection-uri"}
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/connector", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "connector_id":"431ec970-b59d-4b00-95fe-5f3abcc52c2f",
    "name":"connectorName",
    "customer_id":"EMAIL",
    "cluster_id":"7568431c-88d5-4409-a808-2167f22a7133",
    "creation_time":1684369147,
    "deletion_time":0,
    "state":"pending",
    "state_error_message":"",
    "connector_state":"",
    "tasks":[],
    "topics":[],
    "connector_class":"com.mongodb.kafka.connect.MongoSourceConnector",
    "encoded_username":"YXBwYXJlbnQta2l0ZS0xMTMwMiTIqFhTItzgDdE56au6LgnnbtlN7ITzh4QATDw",
    "TTL":1684370947
  }
  ```
</ResponseExample>


# Delete Kafka Connector

DELETE https://api.upstash.com/v2/kafka/connector/{id}
This endpoint deletes a Kafka Connector.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka Connector to be deleted
</ParamField>

<RequestExample>
  ```shell curl
  curl -X DELETE \
    https://api.upstash.com/v2/kafka/connector/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.delete('https://api.upstash.com/v2/kafka/connector/:id' auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("DELETE", "https://api.upstash.com/v2/kafka/connector/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Get Kafka Connector

GET https://api.upstash.com/v2/kafka/connector/{id}
This endpoint gets details of a kafka connector.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka Connector
</ParamField>

## Response Parameters

<ResponseField name="connector_id" type="string">
  ID of the Kafka connector
</ResponseField>

<ResponseField name="name" type="string">
  Name of the Kafka connector
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the kafka cluster of the connector
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the topic
</ResponseField>

<ResponseField name="customer_id" type="string">
  Owner of the connector
</ResponseField>

<ResponseField name="state" type="string">
  State of the connector
</ResponseField>

<ResponseField name="state_error_message" type="string">
  Error message, if the connector failed
</ResponseField>

<ResponseField name="connector_state" type="string">
  State of the connector
</ResponseField>

<ResponseField name="tasks" type="Array<object>">
  Tasks for the connector
</ResponseField>

<ResponseField name="topics" type="Array<string>">
  Topics that are given with properties config
</ResponseField>

<ResponseField name="connector_class" type="string">
  Class of the created connector
</ResponseField>

<ResponseField name="properties" type="object">
  Properties that the connector was configured with
</ResponseField>

<ResponseField name="encoded_username" type="string">
  Encoded username for the connector
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/kafka/connector/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/kafka/connector/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/kafka/connector/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "connector_id": "431ec970-b59d-4b00-95fe-5f3abcc52c2f",
    "name": "connectorName",
    "customer_id": "EMAIL",
    "cluster_id": "7568431c-88d5-4409-a808-2167f22a7133",
    "creation_time": 1684369147,
    "deletion_time": 0,
    "state": "failed",
    "state_error_message": "Connector configuration is invalid and contains the following 1 error(s):\nInvalid value connection-uri-update for configuration connection.uri: The connection string is invalid. Connection strings must start with either 'mongodb://' or 'mongodb+srv://\n",
    "connector_state": "",
    "tasks": [],
    "topics": [],
    "connector_class": "com.mongodb.kafka.connect.MongoSourceConnector",
    "properties": {
      "connection.uri": "connection-uri-update",
      "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector"
    },
    "encoded_username": "YXBwYXJlbnQta2l0ZS0xMTMwMiTIqFhTItzgDdE56au6LgnnbtlN7ITzh4QATDw"
  }
  ```
</ResponseExample>


# List Kafka Connectors in Cluster

GET https://api.upstash.com/v2/kafka/connectors/{id}
This endpoint lists kafka connectors in a cluster.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka Cluster
</ParamField>

## Response Parameters

<ResponseField name="connector_id" type="string">
  ID of the Kafka connector
</ResponseField>

<ResponseField name="name" type="string">
  Name of the Kafka connector
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the kafka cluster of the connector
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the topic
</ResponseField>

<ResponseField name="customer_id" type="string">
  Owner of the connector
</ResponseField>

<ResponseField name="state" type="string">
  State of the connector
</ResponseField>

<ResponseField name="state_error_message" type="string">
  Error message, if the connector failed
</ResponseField>

<ResponseField name="connector_state" type="string">
  State of the connector
</ResponseField>

<ResponseField name="tasks" type="Array<object>">
  Tasks for the connector
</ResponseField>

<ResponseField name="topics" type="Array<string>">
  Topics that are given with properties config
</ResponseField>

<ResponseField name="connector_class" type="string">
  Class of the created connector
</ResponseField>

<ResponseField name="properties" type="object">
  Properties that the connector was configured with
</ResponseField>

<ResponseField name="encoded_username" type="string">
  Encoded username for the connector
</ResponseField>

<ResponseField name="TTL" type="int">
  Time to live for connector
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/kafka/connectors/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/kafka/connectors/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/kafka/connectors/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
    {
      "connector_id": "431ec970-b59d-4b00-95fe-5f3abcc52c2f",
      "name": "connectorName",
      "customer_id": "EMAIL",
      "cluster_id": "7568431c-88d5-4409-a808-2167f22a7133",
      "creation_time": 1684369147,
      "deletion_time": 0,
      "state": "failed",
      "state_error_message": "Connector configuration is invalid and contains the following 1 error(s):\nInvalid value connection-uri-update for configuration connection.uri: The connection string is invalid. Connection strings must start with either 'mongodb://' or 'mongodb+srv://\n",
      "connector_state": "",
      "tasks": [],
      "topics": [],
      "connector_class": "com.mongodb.kafka.connect.MongoSourceConnector",
      "properties": {
        "connection.uri": "connection-uri-update",
        "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector"
      },
      "encoded_username": "YXBwYXJlbnQta2l0ZS0xMTMwMiTIqFhTItzgDdE56au6LgnnbtlN7ITzh4QATDw",
      "TTL": 1684370947
    }
  ]
  ```
</ResponseExample>


# Pause Kafka Connector

POST https://api.upstash.com/v2/kafka/connector/{id}/pause
This endpoint pauses an existing connector.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka Connector to be paused
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/connector/:id/pause \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/kafka/connector/:id/start', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/connector/:id/start", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Reconfigure Kafka Connector

POST https://api.upstash.com/v2/kafka/update-connector/{id}
This endpoint reconfigures an existing kafka connector.

## Request Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka Connector
</ParamField>

## Request Parameters

<ParamField body="properties" type="string" required>
  Custom property values, depending on the connector type. Given values will be
  changed on the connector. You can check the documentation of the related
  connector.
</ParamField>

## Response Parameters

<ResponseField name="connector_id" type="string">
  ID of the Kafka connector
</ResponseField>

<ResponseField name="name" type="string">
  Name of the Kafka connector
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the kafka cluster of the connector
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the topic
</ResponseField>

<ResponseField name="customer_id" type="string">
  Owner of the connector
</ResponseField>

<ResponseField name="state" type="string">
  State of the connector
</ResponseField>

<ResponseField name="state_error_message" type="string">
  Error message, if the connector failed
</ResponseField>

<ResponseField name="connector_state" type="string">
  State of the connector
</ResponseField>

<ResponseField name="tasks" type="Array<object>">
  Tasks for the connector
</ResponseField>

<ResponseField name="topics" type="Array<string>">
  Topics that are given with properties config
</ResponseField>

<ResponseField name="connector_class" type="string">
  Class of the created connector
</ResponseField>

<ResponseField name="encoded_username" type="string">
  Encoded username for the connector
</ResponseField>

<ResponseField name="TTL" type="int">
  Time to live for connector
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/update-connector/:id \
    -u 'EMAIL:API_KEY' \
    -d '{"connector.class": "com.mongodb.kafka.connect.MongoSourceConnector","connection.uri": "connection-uri-update"}'
  ```

  ```python Python
  import requests

  data = '{"connector.class": "com.mongodb.kafka.connect.MongoSourceConnector","connection.uri": "connection-uri-update"}'

  response = requests.post('https://api.upstash.com/v2/kafka/update-connector/:id', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector",
      "connection.uri": "connection-uri-update"
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/update-connector/:id", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "connector_id": "431ec970-b59d-4b00-95fe-5f3abcc52c2f",
    "name": "connectorName",
    "customer_id": "EMAIL",
    "cluster_id": "7568431c-88d5-4409-a808-2167f22a7133",
    "creation_time": 1684369147,
    "deletion_time": 0,
    "state": "failed",
    "state_error_message": "Connector configuration is invalid and contains the following 1 error(s):\nInvalid value connection-uri-update for configuration connection.uri: The connection string is invalid. Connection strings must start with either 'mongodb://' or 'mongodb+srv://\n",
    "connector_state": "",
    "tasks": [],
    "topics": [],
    "connector_class": "com.mongodb.kafka.connect.MongoSourceConnector",
    "encoded_username": "YXBwYXJlbnQta2l0ZS0xMTMwMiTIqFhTItzgDdE56au6LgnnbtlN7ITzh4QATDw",
    "TTL": 1684370947
  }
  ```
</ResponseExample>


# Restart Kafka Connector

POST https://api.upstash.com/v2/kafka/connector/{id}/restart
This endpoint restarts an existing connector.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka Connector to be restarted
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/connector/:id/restart \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/kafka/connector/:id/restart', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/connector/:id/restart", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Start Kafka Connector

POST https://api.upstash.com/v2/kafka/connector/{id}/start
This endpoint starts an existing connector.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka Connector to be started
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/connector/:id/start \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/kafka/connector/:id/start', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/connector/:id/start", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Create Kafka Credential

POST https://api.upstash.com/v2/kafka/credential
This endpoint creates a kafka credential.

## Request Parameters

<ParamField body="credential_name" type="string" required>
  The ID of the kafka topic
</ParamField>

<ParamField body="cluster_id" type="string" required>
  ID of the kafka cluster
</ParamField>

<ParamField body="topic" type="string" required>
  Name of the kafka topic the credential will be used for
</ParamField>

<ParamField body="permissions" type="string" required>
  Permission scope of the credential <br />
  **Options:** `ALL`, `PRODUCE` or `CONSUME`
</ParamField>

## Response Parameters

<ResponseField name="credential_id" type="string">
  ID of the created Kafka credential
</ResponseField>

<ResponseField name="credential_name" type="string">
  Name of the created Kafka credential
</ResponseField>

<ResponseField name="topic" type="string">
  Name of the topic of the created Kafka credential
</ResponseField>

<ResponseField name="permissions" type="string">
  Permission scope given to the kafka credential
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the kafka cluster
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used for the kafka credential
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the credential
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="state" type="string">
  State of the credential\ `active` or `deleted`
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/credential \
    -u 'EMAIL:API_KEY' \
    -d '{"credential_name": "mycreds", "cluster_id":"1793bfa1-d96e-46de-99ed-8f91f083209d", "topic": "testtopic", "permissions": "ALL"}'
  ```

  ```python Python
  import requests

  data = '{"credential_name": "mycreds", "cluster_id":"1793bfa1-d96e-46de-99ed-8f91f083209d", "topic": "testtopic", "permissions": "ALL"}'

  response = requests.post('https://api.upstash.com/v2/credential', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "credential_name": "mycreds",
      "cluster_id":"1793bfa1-d96e-46de-99ed-8f91f083209d",
      "topic": "testopic",
      "permissions": "ALL"
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/credential", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "credential_id": "27172269-da05-471b-9e8e-8fe4195871bc",
    "credential_name": "mycreds",
    "topic": "testtopic",
    "permissions": "ALL",
    "cluster_id": "1793bfa1-d96e-46de-99ed-8f91f083209d",
    "cluster_slug":"easy-haddock-7753",
    "username":"ZWFzeS1oYWRkb2NrLTc3NTMkeeOs0FG4DZ3GxK99cArT0slAC37KLJgbe0fs7dA",
    "creation_time": 1655886853,
    "password": "xE1ypRHMq50jAhpbzu8qBb8jHNAxzezn6bkuRUvc2RZr7X1sznbhampm9p-feT61jnz6ewHJjUd5N6cQHhs84zCjQiP5somCY17FTQ7t6n0uPhWeyf-Fcw==",
    "state": "active"
  }
  ```
</ResponseExample>


# Delete Kafka Credential

DELETE https://api.upstash.com/v2/kafka/credential/{id}
This endpoint deletes a kafka credential.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the kafka credential to delete
</ParamField>

<RequestExample>
  ```shell curl
  curl -X DELETE \
    https://api.upstash.com/v2/kafka/credential/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.delete('https://api.upstash.com/v2/kafka/credential/:id' auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("DELETE", "https://api.upstash.com/v2/kafka/credential/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# List Kafka Credentials

GET https://api.upstash.com/v2/kafka/credentials
This endpoint lists created kafka credentials other than the default one.

## Request Parameters

## Response Parameters

<ResponseField name="credential_id" type="string">
  ID of the created Kafka credential
</ResponseField>

<ResponseField name="credential_name" type="string">
  Name of the created Kafka credential
</ResponseField>

<ResponseField name="topic" type="string">
  Name of the topic of the created Kafka credential
</ResponseField>

<ResponseField name="permissions" type="string">
  Permission scope given to the kafka credential
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the kafka cluster
</ResponseField>

<ResponseField name="cluster_slug" type="string">
  ID of the kafka cluster
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used for the kafka credential
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the credential
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="state" type="string">
  State of the credential\ `active` or `deleted`
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/kafka/credentials \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/kafka/credentials', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/kafka/credentials", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
    {
      "credential_id": "27172269-da05-471b-9e8e-8fe4195871bc",
      "credential_name": "mycreds",
      "topic": "testopic",
      "permissions": "ALL",
      "cluster_id": "1793bfa1-d96e-46de-99ed-8f91f083209d",
      "cluster_slug":"noted-hamster-9151",
      "username":"bm90ZWQtaGFtc3Rlci05MTUxJPGKdKDkmwdObf8yMzmJ8jUqhmN1UQ7VmDe1xkk",
      "creation_time": 1655886853,
      "password": "xE1ypRHMq50jAhpbzu8qBb8jHNAxzezn6bkuRUvc2RZr7X1sznbhampm9p-feT61jnz6ewHJjUd5N6cQHhs84zCjQiP5somCY17FTQ7t6n0uPhWeyf-Fcw==",
      "state": "active"
    }
  ]
  ```
</ResponseExample>


# Create Kafka Topic

POST https://api.upstash.com/v2/kafka/topic
This endpoint creates a new kafka topic in a cluster.

## Request Parameters

<ParamField body="name" type="string" required>
  Name of the new kafka topic
</ParamField>

<ParamField body="partitions" type="int" required>
  The number of partitions the topic will have
</ParamField>

<ParamField body="retention_time" type="int" required>
  Retention time of messsages in the topic `-1` for highest possible value
</ParamField>

<ParamField body="retention_size" type="int" required>
  Retention size of the messages in the topic `-1` for highest possible value
</ParamField>

<ParamField body="max_message_size" type="int" required>
  Max message size in the topic `-1` for highest possible value
</ParamField>

<ParamField body="cleanup_policy" type="string" required>
  Cleanup policy will be used in the topic `compact` or `delete`
</ParamField>

<ParamField body="cluster_id" type="string" required>
  ID of the cluster the topic will be deployed in
</ParamField>

## Response Parameters

<ResponseField name="topic_id" type="string">
  ID of the new kafka topic
</ResponseField>

<ResponseField name="topic_name" type="string">
  Name of the new kafka topic
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the created Kafka cluster
</ResponseField>

<ResponseField name="name" type="string">
  Name of the Kafka cluster
</ResponseField>

<ResponseField name="region" type="string">
  The region the Kafka cluster is deployed in
</ResponseField>

<ResponseField name="type" type="string">
  Shows whether the cluster is free or paid
</ResponseField>

<ResponseField name="multizone" type="boolean">
  Whether the multizone replication is enabled for the cluster or not
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="state" type="string">
  Current state of the cluster(active, deleted)
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="int">
  Max retention size will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="int">
  Max retention time will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_messages_per_second" type="int">
  Max messages allowed to be produced per second
</ResponseField>

<ResponseField name="creation_time" type="int">
  Cluster creation timestamp
</ResponseField>

<ResponseField name="max_message_size" type="int">
  Max message size will be allowed in topics in the cluster
</ResponseField>

<ResponseField name="max_partitions" type="int">
  Max total number of partitions allowed in the cluster
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/topic \
    -u 'EMAIL:API_KEY' \
    -d '{"name":"test-kafka-topic","partitions":1,"retention_time":1234,"retention_size":4567,"max_message_size":8912,"cleanup_policy":"delete","cluster_id":"9bc0e897-cbd3-4997-895a-fd77ad00aec9"}'
  ```

  ```python Python
  import requests

  data = '{"name":"test-kafka-topic","partitions":1,"retention_time":1234,"retention_size":4567,"max_message_size":8912,"cleanup_policy":"delete","cluster_id":"9bc0e897-cbd3-4997-895a-fd77ad00aec9"}'

  response = requests.post('https://api.upstash.com/v2/kafka/topic', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "name": "test-kafka-topic",
      "partitions": 1,
      "retention_time": 1234,
      "retention_size": 4567,
      "max_message_size": 8912,
      "cleanup_policy": "delete",
      "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9"
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/topic", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "topic_id": "0f458c88-2dc6-4f69-97bb-05060e0be934",
      "topic_name": "test-kafka-topic",
      "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9",
      "region": "eu-west-1",
      "creation_time": 1643981720,
      "state": "active",
      "partitions": 1,
      "multizone": true,
      "tcp_endpoint": "sharing-mastodon-12819-eu1-kafka.upstashdev.com",
      "rest_endpoint": "sharing-mastodon-12819-eu1-rest-kafka.upstashdev.com",
      "username": "c2hhcmluZy1tYXN0b2Rvbi0xMjgxOSRV1ipriSBOwd0PHzw2KAs_cDrTXzvUKIs",
      "password": "eu8K3rYRS-ma0AsINDo7MMemmHjjRSldHJcG3c1LUMZkFfdSf9u_Kd4xCWO9_oQc",
      "cleanup_policy": "delete",
      "retention_size": 4567,
      "retention_time": 1234,
      "max_message_size": 8912
  }
  ```
</ResponseExample>


# Delete Kafka Topic

DELETE https://api.upstash.com/v2/kafka/topic/{id}
This endpoint deletes a kafka topic in a cluster.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka Topic to be deleted
</ParamField>

<RequestExample>
  ```shell curl
  curl -X DELETE \
    https://api.upstash.com/v2/kafka/topic/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.delete('https://api.upstash.com/v2/kafka/topic/:id' auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("DELETE", "https://api.upstash.com/v2/kafka/topic/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Get Kafka Topic

GET https://api.upstash.com/v2/kafka/topic/{id}
This endpoint gets details of a kafka topic.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the kafka topic
</ParamField>

## Response Parameters

<ResponseField name="topic_id" type="string">
  ID of the new kafka topic
</ResponseField>

<ResponseField name="topic_name" type="string">
  Name of the new kafka topic
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the created Kafka cluster
</ResponseField>

<ResponseField name="region" type="string">
  The region the Kafka cluster is deployed in
</ResponseField>

<ResponseField name="creation_time" type="int">
  Cluster creation timestamp
</ResponseField>

<ResponseField name="state" type="string">
  State of the topic\ `active` or `deleted`
</ResponseField>

<ResponseField name="partitions" type="int">
  Number of partitions the topic has
</ResponseField>

<ResponseField name="multizone" type="boolean">
  Whether the multizone replication is enabled for the cluster or not
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="username" type="string">
  Cleanup policy to be used in the topic\ `compact` or `delete`
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="cleanup_policy" type="string">
  Max total number of partitions allowed in the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="int">
  Max retention size will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="int">
  Max retention time will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_message_size" type="int">
  Max message size will be allowed in topics in the cluster
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/topic \
    -u 'EMAIL:API_KEY' \
    -d '{"name":"test-kafka-topic","partitions":1,"retention_time":1234,"retention_size":4567,"max_message_size":8912,"cleanup_policy":"delete","cluster_id":"9bc0e897-cbd3-4997-895a-fd77ad00aec9"}'
  ```

  ```python Python
  import requests

  data = '{"name":"test-kafka-topic","partitions":1,"retention_time":1234,"retention_size":4567,"max_message_size":8912,"cleanup_policy":"delete","cluster_id":"9bc0e897-cbd3-4997-895a-fd77ad00aec9"}'

  response = requests.post('https://api.upstash.com/v2/kafka/topic', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "name": "test-kafka-topic",
      "partitions": 1,
      "retention_time": 1234,
      "retention_size": 4567,
      "max_message_size": 8912,
      "cleanup_policy": "delete",
      "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9"
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/topic", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "topic_id": "0f458c88-2dc6-4f69-97bb-05060e0be934",
      "topic_name": "test-kafka-topic",
      "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9",
      "region": "eu-west-1",
      "creation_time": 1643981720,
      "state": "active",
      "partitions": 1,
      "multizone": true,
      "tcp_endpoint": "sharing-mastodon-12819-eu1-kafka.upstashdev.com",
      "rest_endpoint": "sharing-mastodon-12819-eu1-rest-kafka.upstashdev.com",
      "username": "c2hhcmluZy1tYXN0b2Rvbi0xMjgxOSRV1ipriSBOwd0PHzw2KAs_cDrTXzvUKIs",
      "password": "eu8K3rYRS-ma0AsINDo7MMemmHjjRSldHJcG3c1LUMZkFfdSf9u_Kd4xCWO9_oQc",
      "cleanup_policy": "delete",
      "retention_size": 4567,
      "retention_time": 1234,
      "max_message_size": 8912
  }
  ```
</ResponseExample>


# List Kafka Topics in Cluster

GET https://api.upstash.com/v2/kafka/topics/{id}
This endpoint list kafka topics in a cluster.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka cluster
</ParamField>

## Response Parameters

<ResponseField name="topic_id" type="string">
  ID of the new kafka topic
</ResponseField>

<ResponseField name="topic_name" type="string">
  Name of the new kafka topic
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the created Kafka cluster
</ResponseField>

<ResponseField name="region" type="string">
  The region the Kafka cluster is deployed in
</ResponseField>

<ResponseField name="creation_time" type="int">
  Cluster creation timestamp
</ResponseField>

<ResponseField name="state" type="string">
  State of the topic\ **Options**: `active` or `deleted`
</ResponseField>

<ResponseField name="partitions" type="int">
  Number of partitions the topic has
</ResponseField>

<ResponseField name="multizone" type="boolean">
  Whether the multizone replication is enabled for the cluster or not
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="username" type="string">
  Cleanup policy to be used in the topic\ **Options**: `compact` or `delete`
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="cleanup_policy" type="int">
  Max total number of partitions allowed in the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="int">
  Max retention size will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="int">
  Max retention time will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_message_size" type="int">
  Max message size will be allowed in topics in the cluster
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/kafka/topics/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/kafka/topics/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/kafka/topics/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
      {
          "topic_id": "0f458c88-2dc6-4f69-97bb-05060e0be934",
          "topic_name": "test-kafka-topic",
          "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9",
          "region": "eu-west-1",
          "creation_time": 1643981720,
          "state": "active",
          "partitions": 1,
          "multizone": true,
          "tcp_endpoint": "sharing-mastodon-12819-eu1-kafka.upstashdev.com",
          "rest_endpoint": "sharing-mastodon-12819-eu1-rest-kafka.upstashdev.com",
          "username": "c2hhcmluZy1tYXN0b2Rvbi0xMjgxOSRV1ipriSBOwd0PHzw2KAs_cDrTXzvUKIs",
          "password": "eu8K3rYRS-ma0AsINDo7MMemmHjjRSldHJcG3c1LUMZkFfdSf9u_Kd4xCWO9_oQc",
          "cleanup_policy": "delete",
          "retention_size": 4568,
          "retention_time": 1235,
          "max_message_size": 8913
      }
  ]
  ```
</ResponseExample>


# Reconfigure Kafka Topic

POST https://api.upstash.com/v2/kafka/update-topic/{id}
This endpoint reconfigures an existing kafka topic.

## URL Parameters

<ParamField path="id" type="string" required>
  The unique ID of the topic
</ParamField>

## Request Parameters

<ParamField body="retention_time" type="int" optional>
  Retention time of messsages in the topic\ `-1` for highest possible value
</ParamField>

<ParamField body="retention_size" type="int" optional>
  Retention size of the messages in the topic\ `-1` for highest possible value
</ParamField>

<ParamField body="max_message_size" type="int" optional>
  Max message size in the topic\\
</ParamField>

## Response Parameters

<ResponseField name="topic_id" type="string">
  ID of the new kafka topic
</ResponseField>

<ResponseField name="topic_name" type="string">
  Name of the new kafka topic
</ResponseField>

<ResponseField name="cluster_id" type="string">
  ID of the created Kafka cluster
</ResponseField>

<ResponseField name="region" type="string">
  The region the Kafka cluster is deployed in
</ResponseField>

<ResponseField name="creation_time" type="int">
  Cluster creation timestamp
</ResponseField>

<ResponseField name="state" type="string">
  State of the topic\ `active` or `deleted`
</ResponseField>

<ResponseField name="partitions" type="int">
  Number of partitions the topic has
</ResponseField>

<ResponseField name="multizone" type="boolean">
  Whether the multizone replication is enabled for the cluster or not
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint to connect to the Kafka cluster
</ResponseField>

<ResponseField name="username" type="string">
  Cleanup policy to be used in the topic\ `compact` or `delete`
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="cleanup_policy" type="string">
  Max total number of partitions allowed in the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="int">
  Max retention size will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="int">
  Max retention time will be allowed to topics in the cluster
</ResponseField>

<ResponseField name="max_message_size" type="int">
  Max message size will be allowed in topics in the cluster
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/kafka/update-topic/:id \
    -u 'EMAIL:API_KEY' \
    -d '{"retention_time":1235,"retention_size":4568,"max_message_size":8913}'
  ```

  ```python Python
  import requests

  data = '{"retention_time":1235,"retention_size":4568,"max_message_size":8913}'

  response = requests.post('https://api.upstash.com/v2/kafka/update-topic/:id', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "retention_time": 1235,
      "retention_size": 4568,
      "max_message_size": 8913
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/kafka/update-topic/:id", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "topic_id": "0f458c88-2dc6-4f69-97bb-05060e0be934",
      "topic_name": "test-kafka-topic",
      "cluster_id": "9bc0e897-cbd3-4997-895a-fd77ad00aec9",
      "region": "eu-west-1",
      "creation_time": 1643981720,
      "state": "active",
      "partitions": 1,
      "multizone": true,
      "tcp_endpoint": "sharing-mastodon-12819-eu1-kafka.upstashdev.com",
      "rest_endpoint": "sharing-mastodon-12819-eu1-rest-kafka.upstashdev.com",
      "username": "c2hhcmluZy1tYXN0b2Rvbi0xMjgxOSRV1ipriSBOwd0PHzw2KAs_cDrTXzvUKIs",
      "password": "eu8K3rYRS-ma0AsINDo7MMemmHjjRSldHJcG3c1LUMZkFfdSf9u_Kd4xCWO9_oQc",
      "cleanup_policy": "delete",
      "retention_size": 4568,
      "retention_time": 1235,
      "max_message_size": 8913
  }
  ```
</ResponseExample>


# Get Kafka Topic Stats

GET https://api.upstash.com/v2/Kafka/stats/topic/{id}
This endpoint gets detailed stats of a Kafka cluster.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Kafka topic
</ParamField>

## Response Parameters

<ResponseField name="throughput" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Number of monthly messages in Kafka topic
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="produce_throughput" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Number of monthly messages produced in Kafka topic
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="consume_throughput" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Number of monthly messages consumed in Kafka topic
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="diskusage" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total disk usage of the Kafka topic
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="total_monthly_storage" type="int">
  Average storage size of the Kafka topic in the current month
</ResponseField>

<ResponseField name="total_monthly_produce" type="int">
  Total number of monthly produced messages to the Kafka topic
</ResponseField>

<ResponseField name="total_monthly_consume" type="int">
  Total number of monthly consumed messages from the Kafka topic
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/kafka/stats/topic/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/kafka/stats/topic/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/kafka/stats/topic/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "throughput": [
      {
        "x": "2022-02-07 12:05:11",
        "y": 0
      }
      ...
    ],
    "produce_throughput": [
      {
        "x": "2022-02-07 12:05:11",
        "y": 0
      }
      ...
    ],
    "consume_throughput": [
      {
        "x": "2022-02-07 12:05:11",
        "y": 0
      }
      ...
    ],
    "diskusage": [
      {
        "x": "2022-02-07 12:20:11",
        "y": 0
      }
      ...
    ],
    "total_monthly_storage": 0,
    "total_monthly_produce": 0,
    "total_monthly_consume": 0
  }
  ```
</ResponseExample>


# null





# Create Backup

POST https://api.upstash.com/v2/redis/create-backup/{id}
This endpoint creates a backup for a Redis database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Redis database
</ParamField>

## Request Parameters

<ParamField body="name" type="string" required>
  Name of the backup
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/create-backup/{id} \
    -u 'EMAIL:API_KEY' \
    -d '{"name" : "backup_name"}'
  ```

  ```python Python
  import requests

  data = '{"name" : "backup_name"}'

  response = requests.post('https://api.upstash.com/v2/redis/create-backup/{id}', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "name":"backup_name"
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/create-backup/{id}", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s
  ", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Delete Backup

DELETE https://api.upstash.com/v2/redis/delete-backup/{id}/{backup_id}
This endpoint deletes a backup of a Redis database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Redis database
</ParamField>

<ParamField path="backup_id" type="string" required>
  The ID of the backup to delete
</ParamField>

<RequestExample>
  ```shell curl
  curl -X DELETE \
    https://api.upstash.com/v2/redis/delete-backup/:id/:backup_id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.delete('https://api.upstash.com/v2/redis/delete-backup/:id/:backup_id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("DELETE", "https://api.upstash.com/v2/redis/delete-backup/:id/:backup_id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s
  ", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Disable Daily Backup

PATCH https://api.upstash.com/v2/redis/disable-dailybackup/{id}
This endpoint disables daily backup for a Redis database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Redis database
</ParamField>

<RequestExample>
  ```shell curl
  curl -X PATCH \
    https://api.upstash.com/v2/redis/disable-dailybackup/{id} \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.patch('https://api.upstash.com/v2/redis/disable-dailybackup/{id}', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("PATCH", "https://api.upstash.com/v2/redis/disable-dailybackup/{id}", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s
  ", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Enable Daily Backup

PATCH https://api.upstash.com/v2/redis/enable-dailybackup/{id}
This endpoint enables daily backup for a Redis database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Redis database
</ParamField>

<RequestExample>
  ```shell curl
  curl -X PATCH \
    https://api.upstash.com/v2/redis/enable-dailybackup/{id} \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.patch('https://api.upstash.com/v2/redis/enable-dailybackup/{id}', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("PATCH", "https://api.upstash.com/v2/redis/enable-dailybackup/{id}", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s
  ", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# List Backup

GET https://api.upstash.com/v2/redis/list-backup/{id}
This endpoint lists all backups for a Redis database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Redis database
</ParamField>

## Response Parameters

<ResponseField name="database_id" type="string">
  ID of the database
</ResponseField>

<ResponseField name="customer_id" type="string">
  Customer ID
</ResponseField>

<ResponseField name="name" type="string">
  Name of the backup
</ResponseField>

<ResponseField name="backup_id" type="string">
  ID of the backup
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the backup as Unix time
</ResponseField>

<ResponseField name="state" type="string">
  State of the backup (e.g., completed)
</ResponseField>

<ResponseField name="backup_size" type="int">
  Size of the backup
</ResponseField>

<ResponseField name="daily_backup" type="boolean">
  Daily backup status
</ResponseField>

<ResponseField name="hourly_backup" type="boolean">
  Hourly backup status
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/redis/list-backup/{id} \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/redis/list-backup/{id}', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/redis/list-backup/{id}", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s
  ", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
    {
      "database_id":"6gceaafd-9627-4fa5-8g71-b3359g19a5g4",
      "customer_id":"customer_id",
      "name":"test2",
      "backup_id":"1768e55b-c137-4339-b46e-449dcd33a62e",
      "creation_time":1720186545,
      "state":"completed",
      "backup_size":0,
      "daily_backup":"false",
      "hourly_backup":"false"
    },
    {
      "database_id":"6gceaafd-9627-4fa5-8g71-b3359g19a5g4",
      "customer_id":"customer_id",
      "name":"test1",
      "backup_id":"39310b84-21b3-45c3-5318-403553a2466d",
      "creation_time":1720096600,
      "state":"completed",
      "backup_size":0,
      "daily_backup":"false",
      "hourly_backup":"false"
    }
  ]
  ```
</ResponseExample>


# Restore Backup

POST https://api.upstash.com/v2/redis/restore-backup/{id}
This endpoint restores data from an existing backup.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the Redis database
</ParamField>

## Request Parameters

<ParamField body="backup_id" type="string" required>
  ID of the backup to restore
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/restore-backup/{id} \
    -u 'EMAIL:API_KEY' 
    -d '{"backup_id" : "backup_id"}'
  ```

  ```python Python
  import requests

  data = '{"backup_id" : "backup_id"}'

  response = requests.post('https://api.upstash.com/v2/redis/restore-backup/{id}', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "backup_id":"backup_id"
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/restore-backup/{id}", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s
  ", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Create a Redis Database (Regional - DEPRECATED)

POST https://api.upstash.com/v2/redis/database
This endpoint creates a new regional Redis database. This behaviour is deprecated in favor of Global databases and support for it will be removed in the upcoming releases.

## Request Parameters

<ParamField body="name" type="string" required>
  Name of the database
</ParamField>

<ParamField body="region" type="string" required>
  Region of the database.\ **Options:** `eu-west-1`, `us-east-1`, `us-west-1`,
  `ap-northeast-1` or `us-central1`
</ParamField>

<ParamField body="tls" type="boolean" required>
  Set true to enable tls.
</ParamField>

## Response Parameters

<ResponseField name="database_id" type="string">
  ID of the created database
</ResponseField>

<ResponseField name="database_name" type="string">
  Name of the database
</ResponseField>

<ResponseField name="database_type" type="string">
  Type of the database in terms of pricing model(Free, Pay as You Go or
  Enterprise)
</ResponseField>

<ResponseField name="region" type="string">
  The region where database is hosted
</ResponseField>

<ResponseField name="port" type="int">
  Database port for clients to connect
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the database as Unix time
</ResponseField>

<ResponseField name="state" type="string">
  State of database (active or deleted)
</ResponseField>

<ResponseField name="password" type="string">
  Password of the database
</ResponseField>

<ResponseField name="user_email" type="string">
  Email or team id of the owner of the database
</ResponseField>

<ResponseField name="endpoint" type="string">
  Endpoint URL of the database
</ResponseField>

<ResponseField name="tls" type="boolean">
  TLS/SSL is enabled or not
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/database \
    -u 'EMAIL:API_KEY' \
    -d '{"name":"myredis","region":"eu-west-1","tls": true}'
  ```

  ```python Python
  import requests

  data = '{"name":"myredis","region":"eu-west-1","tls":true}'

  response = requests.post('https://api.upstash.com/v2/redis/database', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "name":"myredis",
      "region":"eu-west-1",
      "tls": true
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/database", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "database_id": "96ad0856-03b1-4ee7-9666-e81abd0349e1",
    "database_name": "MyRedis",
    "database_type": "Pay as You Go",
    "region": "eu-central-1",
    "port": 30143,
    "creation_time": 1658909671,
    "state": "active",
    "password": "038a8e27c45e43068d5f186085399884",
    "user_email": "example@upstash.com",
    "endpoint": "eu2-sought-mollusk-30143.upstash.io",
    "tls": true,
    "rest_token": "AXW_ASQgOTZhZDA4NTYtMDNiMS00ZWU3LTk2NjYtZTgxYWJkMDM0OWUxMDM4YThlMjdjNDVlNDMwNjhkNWYxODYwODUzOTk4ODQ=",
    "read_only_rest_token": "AnW_ASQgOTZhZDA4NTYtMDNiMS00ZWU3LTk2NjYtZTgxYWJkMDM0OWUx8sbmiEcMm9u7Ks5Qx-kHNiWr_f-iUXSIH8MlziKMnpY="
  }
  ```
</ResponseExample>


# Create a Redis Database (Global)

POST https://api.upstash.com/v2/redis/database
This endpoint creates a new Redis database.

## Request Parameters

<ParamField body="name" type="string" required>
  Name of the database
</ParamField>

<ParamField body="region" type="string" default="global" required>
  Region of the database. Only valid option is `global`.
</ParamField>

<ParamField body="primary_region" type="string" required>
  Primary Region of the Global Database.

  Available regions: `us-east-1`, `us-west-1`, `us-west-2`, `eu-west-1`,
  `eu-central-1`, `ap-southeast-1`, `ap-southeast-2`, `sa-east-1`
</ParamField>

<ParamField body="read_regions" type="Array<string>">
  Array of Read Regions of
  the Database.

  Available regions: `us-east-1`, `us-west-1`, `us-west-2`, `eu-west-1`,
  `eu-central-1`, `ap-southeast-1`, `ap-southeast-2`, `ap-northeast-1`, `sa-east-1`
</ParamField>

## Response Parameters

<ResponseField name="database_id" type="string">
  ID of the created database
</ResponseField>

<ResponseField name="database_name" type="string">
  Name of the database
</ResponseField>

<ResponseField name="database_type" type="string">
  Type of the database in terms of pricing model(Free, Pay as You Go or
  Enterprise)
</ResponseField>

<ResponseField name="region" type="string">
  The region where database is hosted
</ResponseField>

<ResponseField name="port" type="int">
  Database port for clients to connect
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the database as Unix time
</ResponseField>

<ResponseField name="state" type="string">
  State of database (active or deleted)
</ResponseField>

<ResponseField name="password" type="string">
  Password of the database
</ResponseField>

<ResponseField name="user_email" type="string">
  Email or team id of the owner of the database
</ResponseField>

<ResponseField name="endpoint" type="string">
  Endpoint URL of the database
</ResponseField>

<ResponseField name="tls" type="boolean">
  TLS is always enabled for new databases
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/database \
    -u 'EMAIL:API_KEY' \
    -d '{"name":"myredis", "region":"global", "primary_region":"us-east-1", "read_regions":["us-west-1","us-west-2"], "tls": true}'
  ```

  ```python Python
  import requests

  data = '{"name":"myredis", "region":"global", "primary_region":"us-east-1", "read_regions":["us-west-1","us-west-2"], "tls":true}'

  response = requests.post('https://api.upstash.com/v2/redis/database', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "name":"myredis",
      "region":"global",
      "primary_region"":"us-east-1",
      "read_regions":["us-west-1","us-west-2"],
      "tls": true
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/database", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "database_id": "93e3a3e-342c-4683-ba75-344c08ae143b",
    "database_name": "global-test",
    "database_type": "Pay as You Go",
    "region": "global",
    "type": "paid",
    "port": 32559,
    "creation_time": 1674596896,
    "state": "active",
    "password": "dd1803832a2746309e118373549e574d",
    "user_email": "support@upstash.com",
    "endpoint": "steady-stud-32559.upstash.io",
    "tls": false,
    "rest_token": "AX8vACQgOTMyY2UyYy00NjgzLWJhNzUtMzQ0YzA4YWUxNDNiZMyYTI3NDYzMDllMTE4MzczNTQ5ZTU3NGQ=",
    "read_only_rest_token": "An8vACQg2UtMzQyYy00NjgzLWJhNzUtMzQ0YzA4YBVsUsyn19xDnTAvjbsiq79GRDrURNLzIYIOk="
  }
  ```
</ResponseExample>


# Delete Database

DELETE https://api.upstash.com/v2/redis/database/{id}
This endpoint deletes a database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the database to be deleted
</ParamField>

<RequestExample>
  ```shell curl
  curl -X DELETE \
    https://api.upstash.com/v2/redis/database/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.delete('https://api.upstash.com/v2/redis/database/:id' auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("DELETE", "https://api.upstash.com/v2/redis/database/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Disable Auto Upgrade

POST https://api.upstash.com/v2/redis/disable-autoupgrade/{id}
This endpoint disables Auto Upgrade for given database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the database to disable auto upgrade
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/disable-autoupgrade/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/redis/disable-autoupgrade/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/disable-autoupgrade/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>`json 200 OK "OK" `</ResponseExample>


# Disable Eviction

POST https://api.upstash.com/v2/redis/disable-eviction/{id}
This endpoint disables eviction for given database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the database to disable eviction
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/disable-eviction/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/redis/disable-eviction/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/disable-eviction/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Enable Auto Upgrade

POST https://api.upstash.com/v2/redis/enable-autoupgrade/{id}
This endpoint enables Auto Upgrade for given database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the database to enable auto upgrade
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/enable-autoupgrade/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/redis/enable-autoupgrade/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/enable-autoupgrade/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Enable Eviction

POST https://api.upstash.com/v2/redis/enable-eviction/{id}
This endpoint enables eviction for given database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the database to enable eviction
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/enable-eviction/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/redis/enable-eviction/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/enable-eviction/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Enable TLS

POST https://api.upstash.com/v2/redis/enable-tls/{id}
This endpoint enables tls on a database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the database to rename
</ParamField>

## Response Parameters

<ResponseField name="database_id" type="string">
  ID of the created database
</ResponseField>

<ResponseField name="database_name" type="string">
  Name of the database
</ResponseField>

<ResponseField name="database_type" type="string">
  Type of the database in terms of pricing model\ `Free`, `Pay as You Go` or
  `Enterprise`
</ResponseField>

<ResponseField name="region" type="string">
  The region where database is hosted
</ResponseField>

<ResponseField name="port" type="int">
  Database port for clients to connect
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the database as Unix time
</ResponseField>

<ResponseField name="state" type="string">
  State of database\ `active` or `deleted`
</ResponseField>

<ResponseField name="password" type="string">
  Password of the database
</ResponseField>

<ResponseField name="user_email" type="string">
  Email or team id of the owner of the database
</ResponseField>

<ResponseField name="endpoint" type="string">
  Endpoint URL of the database
</ResponseField>

<ResponseField name="tls" type="boolean">
  TLS/SSL is enabled or not
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/enable-tls/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/redis/enable-tls/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/enable-tls/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "database_id": "96ad0856-03b1-4ee7-9666-e81abd0349e1",
    "cluster_id": "dea1f974",
    "database_name": "MyRedis",
    "database_type": "Pay as You Go",
    "region": "eu-central-1",
    "port": 30143,
    "creation_time": 1658909671,
    "state": "active",
    "password": "49665a1710f3434d8be008aab50f38d2",
    "user_email": "example@upstash.com",
    "endpoint": "eu2-sought-mollusk-30143.upstash.io",
    "tls": true,
  }
  ```
</ResponseExample>


# Get Database

GET https://api.upstash.com/v2/redis/database/{id}
This endpoint gets details of a database.

## Request

<ParamField path="id" type="string" required>
  The ID of the database to reset password
</ParamField>

<ParamField query="credentials" type="string">
  Set to `hide` to remove credentials from the response.
</ParamField>

## Response

<ResponseField name="database_id" type="string">
  ID of the created database
</ResponseField>

<ResponseField name="database_name" type="string">
  Name of the database
</ResponseField>

<ResponseField name="database_type" type="string">
  Type of the database in terms of pricing model(Free, Pay as You Go or
  Enterprise)
</ResponseField>

<ResponseField name="region" type="string">
  The region where database is hosted
</ResponseField>

<ResponseField name="port" type="int">
  Database port for clients to connect
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the database as Unix time
</ResponseField>

<ResponseField name="state" type="string">
  State of database (active or deleted)
</ResponseField>

<ResponseField name="password" type="string">
  Password of the database
</ResponseField>

<ResponseField name="user_email" type="string">
  Email or team id of the owner of the database
</ResponseField>

<ResponseField name="endpoint" type="string">
  Endpoint URL of the database
</ResponseField>

<ResponseField name="tls" type="boolean">
  TLS/SSL is enabled or not
</ResponseField>

<ResponseField name="rest_token" type="string">
  Token for rest based communication with the database
</ResponseField>

<ResponseField name="read_only_rest_token" type="string">
  Read only token for rest based communication with the database
</ResponseField>

<ResponseField name="db_max_clients" type="int">
  Max number of concurrent clients can be opened on this database currently
</ResponseField>

<ResponseField name="db_max_request_size" type="int">
  Max size of a request that will be accepted by the database currently(in
  bytes)
</ResponseField>

<ResponseField name="db_disk_threshold" type="int">
  Total disk size limit that can be used for the database currently(in bytes)
</ResponseField>

<ResponseField name="db_max_entry_size" type="int">
  Max size of an entry that will be accepted by the database currently(in bytes)
</ResponseField>

<ResponseField name="db_memory_threshold" type="int">
  Max size of a memory the database can use(in bytes)
</ResponseField>

<ResponseField name="db_daily_bandwidth_limit" type="int">
  Max daily bandwidth can be used by the database(in bytes)
</ResponseField>

<ResponseField name="db_max_commands_per_second" type="int">
  Max number of commands can be sent to the database per second
</ResponseField>

<ResponseField name="db_request_limit" type="int">
  Total number of commands can be sent to the database
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/redis/database/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/redis/database/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/redis/database/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "database_id": "96ad0856-03b1-4ee7-9666-e81abd0349e1",
    "database_name": "MyRedis",
    "database_type": "Pay as You Go",
    "region": "eu-central-1",
    "port": 30143,
    "creation_time": 1658909671,
    "state": "active",
    "password": "038a8e27c45e43068d5f186085399884",
    "user_email": "example@upstash.com",
    "endpoint": "eu2-sought-mollusk-30143.upstash.io",
    "tls": true,
    "rest_token": "AXW_ASQgOTZhZDA4NTYtMDNiMS00ZWU3LTk2NjYtZTgxYWJkMDM0OWUxMDM4YThlMjdjNDVlNDMwNjhkNWYxODYwODUzOTk4ODQ=",
    "read_only_rest_token": "AnW_ASQgOTZhZDA4NTYtMDNiMS00ZWU3LTk2NjYtZTgxYWJkMDM0OWUx8sbmiEcMm9u7Ks5Qx-kHNiWr_f-iUXSIH8MlziKMnpY=",
    "db_max_clients": 1000,
    "db_max_request_size": 1048576,
    "db_disk_threshold": 107374182400,
    "db_max_entry_size": 104857600,
    "db_memory_threshold": 1073741824,
    "db_daily_bandwidth_limit": 53687091200,
    "db_max_commands_per_second": 1000,
    "db_request_limit": 9223372036854775808
  }
  ```
</ResponseExample>


# Get Database Stats

GET https://api.upstash.com/v2/redis/stats/{id}
This endpoint gets detailed stats of a database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the database
</ParamField>

## Response Parameters

<ResponseField name="connection_count" type="Object">
  <Expandable title="properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total number of connections momentarily
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="keyspace" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total number keys exists in the database
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="throughput" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Throughput seen on the database connections
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="produce_throughput" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Throughput seen on the database connections for write requests
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="consume_throughput" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Throughput seen on the database connections for read requests
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="diskusage" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total amount of this usage of the database
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="latencymax" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Maximum server latency observed in the last hour
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="latencymin" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Minimum server latency observed in the last hour
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="read_latency_mean" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      The average read latency value measured in the last hour
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="read_latency_99" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      The 99th percentile server read latency observed in the last hour
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="write_latency_mean" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      The average write latency value measured in the last hour
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="write_latency_99" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      The 99th percentile server write latency observed in the last hour
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="hits" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total number requests made to the database that are hit
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="misses" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total number requests made to the database that are miss
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="read" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total number read requests made to the database
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="write" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total number write requests made to the database
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="dailyrequests" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      Total number requests made to the database on the corresponding day
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="dailybandwidth" type="int">
  The total daily bandwidth usage (in bytes).
</ResponseField>

<ResponseField name="bandwidths" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      The total bandwidth size for that specific timestamp
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="days" type="Array<string>">
  A list of the days of the week for the measurement
</ResponseField>

<ResponseField name="dailybilling" type="Object">
  <Expandable title="Properties">
    <ResponseField name="x" type="string">
      Timestamp indicating when the measurement was taken.
    </ResponseField>

    <ResponseField name="y" type="int">
      The billing amount for that specific date.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="dailyproduce" type="int">
  Total number of daily produced commands
</ResponseField>

<ResponseField name="dailyconsume" type="int">
  Total number of daily consumed commands
</ResponseField>

<ResponseField name="total_monthly_requests" type="int">
  The total number of requests made in the current month.
</ResponseField>

<ResponseField name="total_monthly_read_requests" type="int">
  The total number of read requests made in the current month.
</ResponseField>

<ResponseField name="total_monthly_write_requests" type="int">
  The total number of write requests made in the current month.
</ResponseField>

<ResponseField name="total_monthly_storage" type="int">
  The total amount of storage used (in bytes) in the current month.
</ResponseField>

<ResponseField name="total_monthly_billing" type="int">
  Total cost of the database in the current month
</ResponseField>

<ResponseField name="total_monthly_produce" type="int">
  Total number of produce commands in the current month
</ResponseField>

<ResponseField name="total_monthly_consume" type="int">
  Total number of consume commands in the current month
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/redis/stats/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/redis/stats/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/redis/stats/:id", nil)
  if err != nil {
  log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
  log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
  log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "connection_count": [
      {
        "x": "2023-05-22 10:59:23.426 +0000 UTC",
        "y": 320
      },
      ...
    ],
    "keyspace": [
      {
        "x": "2023-05-22 10:59:23.426 +0000 UTC",
        "y": 344725564
      },
      ...
    ],
    "throughput": [
      {
        "x": "2023-05-22 11:00:23.426 +0000 UTC",
        "y": 181.88333333333333
      },
      ...
    ],
    "produce_throughput": null,
    "consume_throughput": null,
    "diskusage": [
      {
        "x": "2023-05-22 10:59:23.426 +0000 UTC",
        "y": 532362818323
      },
      ...
    ],
    "latencymean": [
      {
        "x": "2023-05-22 10:59:23.426 +0000 UTC",
        "y": 0.176289
      },
      ...
    ],
    "read_latency_mean": [
      {
        "x": "2023-05-22 11:00:23.426 +0000 UTC",
        "y": 0
      },
      ...
    ],
    "read_latency_99": [
      {
        "x": "2023-05-22 11:00:23.426 +0000 UTC",
        "y": 0
      },
      ...
    ],
    "write_latency_mean": [
      {
        "x": "2023-05-22 11:00:23.426 +0000 UTC",
        "y": 0
      },
      ...
    ],
    "write_latency_99": [
      {
        "x": "2023-05-22 11:00:23.426 +0000 UTC",
        "y": 0
      },
      ...
    ],
    "hits": [
      {
        "x": "2023-05-22 11:00:23.426 +0000 UTC",
        "y": 0
      },
      ...
    ],
    "misses": [
      {
        "x": "2023-05-22 11:00:23.426 +0000 UTC",
        "y": 0
      },
      ...
    ],
    "read": [
      {
        "x": "2023-05-22 11:00:23.426 +0000 UTC",
        "y": 82.53333333333333
      },
      ...
    ],
    "write": [
      {
        "x": "2023-05-22 11:00:23.426 +0000 UTC",
        "y": 99.35
      },
      ...
    ],
    "dailyrequests": [
      {
        "x": "2023-05-18 11:58:23.534505371 +0000 UTC",
        "y": 68844080
      },
      ...
    ],
    "days": [
      "Thursday",
      "Friday",
      "Saturday",
      "Sunday",
      "Monday"
    ],
    "dailybilling": [
      {
        "x": "2023-05-18 11:58:23.534505371 +0000 UTC",
        "y": 145.72694911244588
      },
      ...
    ],
    "dailybandwidth": 50444740913,
    "bandwidths": [
      {
        "x": "2023-05-18 11:58:23.534505371 +0000 UTC",
        "y": 125391861729
      },
      ...
    ],
    "dailyproduce": null,
    "dailyconsume": null,
    "total_monthly_requests": 1283856937,
    "total_monthly_read_requests": 1034567002,
    "total_monthly_write_requests": 249289935,
    "total_monthly_storage": 445942383672,
    "total_monthly_billing": 222.33902763855485,
    "total_monthly_produce": 0,
    "total_monthly_consume": 0
  }
  ```
</ResponseExample>


# List Databases

GET https://api.upstash.com/v2/redis/databases
This endpoint list all databases of user.

## Response Parameters

<ResponseField name="database_id" type="string">
  ID of the database
</ResponseField>

<ResponseField name="database_name" type="string">
  Name of the database
</ResponseField>

<ResponseField name="database_type" type="string">
  Type of the database in terms of pricing model\ `Free`, `Pay as You Go` or
  `Enterprise`
</ResponseField>

<ResponseField name="region" type="string">
  The region where database is hosted
</ResponseField>

<ResponseField name="port" type="int">
  Database port for clients to connect
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the database as Unix time
</ResponseField>

<ResponseField name="state" type="string">
  State of database\ `active` or `deleted`
</ResponseField>

<ResponseField name="password" type="string">
  Password of the database
</ResponseField>

<ResponseField name="user_email" type="string">
  Email or team id of the owner of the database
</ResponseField>

<ResponseField name="endpoint" type="string">
  Endpoint URL of the database
</ResponseField>

<ResponseField name="tls" type="boolean">
  TLS/SSL is enabled or not
</ResponseField>

<ResponseField name="rest_token" type="string">
  Token for rest based communication with the database
</ResponseField>

<ResponseField name="read_only_rest_token" type="string">
  Read only token for rest based communication with the database
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/redis/databases \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/redis/databases', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/redis/databases", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
      {
          "database_id": "96ad0856-03b1-4ee7-9666-e81abd0349e1",
          "database_name": "MyRedis",
          "database_type": "Pay as You Go",
          "region": "eu-central-1",
          "port": 30143,
          "creation_time": 1658909671,
          "state": "active",
          "password": "038a8e27c45e43068d5f186085399884",
          "user_email": "example@upstash.com",
          "endpoint": "eu2-sought-mollusk-30143.upstash.io",
          "tls": true,
          "rest_token": "AXW_ASQgOTZhZDA4NTYtMDNiMS00ZWU3LTk2NjYtZTgxYWJkMDM0OWUxMDM4YThlMjdjNDVlNDMwNjhkNWYxODYwODUzOTk4ODQ=",
          "read_only_rest_token": "AnW_ASQgOTZhZDA4NTYtMDNiMS00ZWU3LTk2NjYtZTgxYWJkMDM0OWUx8sbmiEcMm9u7Ks5Qx-kHNiWr_f-iUXSIH8MlziKMnpY="
      }
  ]
  ```
</ResponseExample>


# Move To Team

POST https://api.upstash.com/v2/redis/move-to-team
This endpoint moves database under a target team

## URL Parameters

<ParamField body="team_id" type="string" required>
  The ID of the target team
</ParamField>

<ParamField body="database_id" type="string" required>
  The ID of the database to be moved
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/move-to-team \
    -u 'EMAIL:API_KEY' \
    -d '{"team_id": "6cc32556-0718-4de5-b69c-b927693f9282","database_id": "67b6af16-acb2-4f00-9e38-f6cb9bee800d"}'
  ```

  ```python Python
  import requests

  data = '{"team_id": "6cc32556-0718-4de5-b69c-b927693f9282","database_id": "67b6af16-acb2-4f00-9e38-f6cb9bee800d"}'

  response = requests.post('https://api.upstash.com/v2/redis/move-to-team', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "team_id": "6cc32556-0718-4de5-b69c-b927693f9282",
      "database_id": "67b6af16-acb2-4f00-9e38-f6cb9bee800d"
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/move-to-team", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>`json 200 OK "OK" `</ResponseExample>


# Rename Database

POST https://api.upstash.com/v2/redis/rename/{id}
This endpoint renames a database.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the database to be renamed
</ParamField>

## Request Parameters

<ParamField body="name" type="string" required>
  The new name of the database
</ParamField>

## Response Parameters

<ResponseField name="database_id" type="string">
  ID of the created database
</ResponseField>

<ResponseField name="database_name" type="string">
  New name of the database
</ResponseField>

<ResponseField name="database_type" type="string">
  Type of the database in terms of pricing model\ `Free`, `Pay as You Go` or
  `Enterprise`
</ResponseField>

<ResponseField name="region" type="string">
  The region where database is hosted
</ResponseField>

<ResponseField name="port" type="int">
  Database port for clients to connect
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the database as Unix time
</ResponseField>

<ResponseField name="state" type="string">
  State of database\ `active` or `deleted`
</ResponseField>

<ResponseField name="password" type="string">
  Password of the database
</ResponseField>

<ResponseField name="user_email" type="string">
  Email or team id of the owner of the database
</ResponseField>

<ResponseField name="endpoint" type="string">
  Endpoint URL of the database
</ResponseField>

<ResponseField name="tls" type="boolean">
  TLS/SSL is enabled or not
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/reset-password/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/redis/reset-password/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/reset-password/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "database_id": "96ad0856-03b1-4ee7-9666-e81abd0349e1",
    "cluster_id": "dea1f974",
    "database_name": "MyRedis",
    "database_type": "Pay as You Go",
    "region": "eu-central-1",
    "port": 30143,
    "creation_time": 1658909671,
    "state": "active",
    "password": "49665a1710f3434d8be008aab50f38d2",
    "user_email": "example@upstash.com",
    "endpoint": "eu2-sought-mollusk-30143.upstash.io",
    "tls": true,

  }

  ```
</ResponseExample>

```
```


# Reset Password

POST https://api.upstash.com/v2/redis/reset-password/{id}
This endpoint updates the password of a database.

## Request

<ParamField path="id" type="string" required>
  The ID of the database to reset password
</ParamField>

## Response

<ResponseField name="database_id" type="string">
  ID of the created database
</ResponseField>

<ResponseField name="database_name" type="string">
  Name of the database
</ResponseField>

<ResponseField name="database_type" type="string">
  Type of the database in terms of pricing model\ `Free`, `Pay as You Go` or
  `Enterprise`
</ResponseField>

<ResponseField name="region" type="string">
  The region where database is hosted
</ResponseField>

<ResponseField name="port" type="int">
  Database port for clients to connect
</ResponseField>

<ResponseField name="creation_time" type="int">
  Creation time of the database as Unix time
</ResponseField>

<ResponseField name="state" type="string">
  State of database\ `active` or `deleted`
</ResponseField>

<ResponseField name="password" type="string">
  Password of the database
</ResponseField>

<ResponseField name="user_email" type="string">
  Email or team id of the owner of the database
</ResponseField>

<ResponseField name="endpoint" type="string">
  Endpoint URL of the database
</ResponseField>

<ResponseField name="tls" type="boolean">
  TLS/SSL is enabled or not
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/reset-password/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/redis/reset-password/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/reset-password/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "database_id": "96ad0856-03b1-4ee7-9666-e81abd0349e1",
    "cluster_id": "dea1f974",
    "database_name": "MyRedis",
    "database_type": "Pay as You Go",
    "region": "eu-central-1",
    "port": 30143,
    "creation_time": 1658909671,
    "state": "active",
    "password": "49665a1710f3434d8be008aab50f38d2",
    "user_email": "example@upstash.com",
    "endpoint": "eu2-sought-mollusk-30143.upstash.io",
    "tls": true,
    "consistent": false,
    "pool_id": "f886c7f3",
    "rest_token": "AXW_ASQgOTZhZDA4NTYtMDNiMS00ZWU3LTk2NjYtZTgxYWJkMDM0OWUxNDk2NjVhMTcxMGYzNDM0ZDhiZTAwOGFhYjUwZjM4ZDI=",
    "read_only_rest_token": "AnW_ASQgOTZhZDA4NTYtMDNiMS00ZWU3LTk2NjYtZTgxYWJkMDM0OWUxB5sRhCROkPsxozFcDzDgVGRAxUI7UUr0Y6uFB7jMIOI="
  }
  ```
</ResponseExample>


# Update Regions (Global)

POST https://api.upstash.com/v2/redis/update-regions/{id}
Update the regions of global database

## Request

<ParamField path="id" type="string" required>
  The ID of your database
</ParamField>

<ParamField body="read_regions" type="Array<string>" required>
  Array of read regions of the database

  **Options:** `us-east-1`, `us-west-1`, `us-west-2`, `eu-west-1`, `eu-central-1`,
  `ap-southeast-1`, `ap-southeast-2`, `sa-east-1`
</ParamField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/redis/update-regions/:id \
    -u 'EMAIL:API_KEY' \
    -d '{ "read_regions":["us-west-1"] }'
  ```

  ```python Python
  import requests

  data = '{"read_regions":["eu-west-1"]}'

  response = requests.post('https://api.upstash.com/v2/redis/database', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{,
      "read_regions":["us-west-1"]
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/redis/read-regions/:id", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Add Team Member

POST https://api.upstash.com/v2/teams/member
This endpoint adds a new team member to the specified team.

## Request Parameters

<ParamField body="team_id" type="string" required>
  Id of the team to add the member to
</ParamField>

<ParamField body="member_email" type="string" required>
  Email of the new team member
</ParamField>

<ParamField body="member_role" type="string" required>
  Role of the new team member
  <br /> **Options:** `admin`, `dev` or `finance`
</ParamField>

## Response Parameters

<ResponseField name="team_id" type="string">
  ID of the created team
</ResponseField>

<ResponseField name="team_name" type="string">
  Name of the created team
</ResponseField>

<ResponseField name="member_email" type="string">
  Email of the new team member
</ResponseField>

<ResponseField name="member_role" type="string">
  Role of the new team member
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/teams/member \
    -u 'EMAIL:API_KEY' \
    -d '{"team_id":"95849b27-40d0-4532-8695-d2028847f823","member_email":"example@upstash.com","member_role":"dev"}'
  ```

  ```python Python
  import requests

  data = '{"team_id":"95849b27-40d0-4532-8695-d2028847f823","member_email":"example@upstash.com","member_role":"dev"}'

  response = requests.post('https://api.upstash.com/v2/teams/member', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "team_id":"95849b27-40d0-4532-8695-d2028847f823",
      "member_email":"example@upstash.com",
      "member_role":"dev"
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/teams/member", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "team_id": "95849b27-40d0-4532-8695-d2028847f823",
      "team_name": "test_team_name",
      "member_email": "example@upstash.com",
      "member_role": "dev"
  }
  ```
</ResponseExample>


# Create Team

POST https://api.upstash.com/v2/team
This endpoint creates a new team.

## Request Parameters

<ParamField body="team_name" type="string" required>
  Name of the new team
</ParamField>

<ParamField body="copy_cc" type="boolean" required>
  Whether to copy existing credit card information to team or not\ Options:
  `true` or `false`
</ParamField>

## Response Parameters

<ResponseField name="team_id" type="string">
  ID of the created team
</ResponseField>

<ResponseField name="team_name" type="string">
  Name of the created team
</ResponseField>

<ResponseField name="copy_cc" type="boolean">
  Whether creditcard information added to team during creation or not
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/team \
    -u 'EMAIL:API_KEY' \
    -d '{"team_name":"myteam","copy_cc":true}'
  ```

  ```python Python
  import requests

  data = '{"team_name":"myteam","copy_cc":true}'

  response = requests.post('https://api.upstash.com/v2/team', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "team_name":"myteam",
      "copy_cc":true
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/team", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "team_id": "75b471f2-15a1-47b0-8ce5-12a57682bfc9",
      "team_name": "test_team_name_2",
      "copy_cc": true
  }
  ```
</ResponseExample>


# Delete Team

DELETE https://api.upstash.com/v2/team/{id}
This endpoint deletes a team.

## URL Parameters

<ParamField path="id" type="string" required>
  The ID of the team to delete
</ParamField>

## Response Parameters

"OK"

<RequestExample>
  ```shell curl
  curl -X DELETE \
    https://api.upstash.com/v2/team/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.delete('https://api.upstash.com/v2/team/:id' auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("DELETE", "https://api.upstash.com/v2/team/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Delete Team Member

DELETE https://api.upstash.com/v2/teams/member
This endpoint deletes a team member from the specified team.

## Request Parameters

<ParamField body="team_id" type="string" required>
  Id of the team to add the member to
</ParamField>

<ParamField body="member_email" type="string" required>
  Email of the new team member
</ParamField>

## Response Parameters

"OK"

<RequestExample>
  ```shell curl
  curl -X DELETE \
    https://api.upstash.com/v2/teams/member \
    -u 'EMAIL:API_KEY' \
    -d '{"team_id":"95849b27-40d0-4532-8695-d2028847f823","member_email":"example@upstash.com"}'
  ```

  ```python Python
  import requests

  data = '{"team_id":"95849b27-40d0-4532-8695-d2028847f823","member_email":"example@upstash.com"}'

  response = requests.delete('https://api.upstash.com/v2/teams/member', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "team_id":"95849b27-40d0-4532-8695-d2028847f823",
      "member_email":"example@upstash.com"
  }`)
  req, err := http.NewRequest("DELETE", "https://api.upstash.com/v2/teams/member", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Get Team Members

GET https://api.upstash.com/v2/teams/{team_id}
This endpoint list all members of a team.

## Request Parameters

<ParamField path="team_id" type="string" required>
  ID of the team
</ParamField>

## Response Parameters

<ResponseField name="team_id" type="string">
  ID of the team
</ResponseField>

<ResponseField name="team_name" type="string">
  Name of the team
</ResponseField>

<ResponseField name="member_email" type="string">
  Email of the team member
</ResponseField>

<ResponseField name="member_role" type="string">
  Role of the team member
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/teams/:id \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/teams/:id', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/teams/:id", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
      {
          "team_id": "3423cb72-e50d-43ec-a9c0-f0f359941223",
          "team_name": "test_team_name_2",
          "member_email": "example@upstash.com",
          "member_role": "dev"
      },
      {
          "team_id": "3423cb72-e50d-43ec-a9c0-f0f359941223",
          "team_name": "test_team_name_2",
          "member_email": "example_2@upstash.com",
          "member_role": "owner"
      }
  ]
  ```
</ResponseExample>


# List Teams

GET https://api.upstash.com/v2/teams
This endpoint lists all teams of user.

## Response Parameters

<ResponseField name="team_id" type="string">
  ID of the created team
</ResponseField>

<ResponseField name="member_role" type="string">
  Role of the user in this team
</ResponseField>

<ResponseField name="team_name" type="string">
  Name of the created team
</ResponseField>

<ResponseField name="copy_cc" type="boolean">
  Whether creditcard information added to team during creation or not
</ResponseField>

<RequestExample>
  ```shell curl
  url -X GET \
    https://api.upstash.com/v2/teams \
    -u 'EMAIL:API_KEY'
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/teams', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/teams", nil)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
      {
          "team_id": "95849b27-40d0-4532-8695-d2028847f823",
          "team_name": "test_team_name",
          "member_role": "owner",
          "copy_cc": true
      }
  ]
  ```
</ResponseExample>


# Create Index

POST https://api.upstash.com/v2/vector/index
This endpoint creates an index.

## Request Parameters

<ParamField body="name" type="string" required>
  Name of the index.
</ParamField>

<ParamField body="region" type="string" required>
  Region of the database.\
  **Options:** `eu-west-1`, `us-east-1`,
</ParamField>

<ParamField body="similarity_function" type="string" required>
  Similarity function that's used to calculate the distance between two
  vectors.\
  **Options:** `COSINE`, `EUCLIDIAN`, `DOT_PRODUCT`
</ParamField>

<ParamField body="dimension_count" type="number" required>
  The amount of values in a single vector.
</ParamField>

<ParamField body="type" type="string">
  The payment plan of your index.\
  **Options:** `payg`, `fixed`
</ParamField>

<ParamField body="embedding_model" type="string">
  The payment plan of your index.\
  **Options:** `BGE_SMALL_EN_V1_5`, `BGE_BASE_EN_V1_5`, `BGE_LARGE_EN_V1_5`, `BGE_M3`, `BERT_BASE_UNCASED`, `UAE_Large_V1`, `ALL_MINILM_L6_V2`, `MXBAI_EMBED_LARGE_V1`
</ParamField>

## Response Parameters

<ResponseField name="customer_id" type="string" required>
  The associated ID of the owner of the index
</ResponseField>

<ResponseField name="id" type="string" required>
  Unique ID of the index
</ResponseField>

<ResponseField name="name" type="string" required>
  The name of the index.
</ResponseField>

<ResponseField name="similarity_function" type="string" required>
  Similarity function that's used to calculate the distance between two
  vectors
</ResponseField>

<ResponseField name="dimension_count" type="number" required>
  The amount of values in a single vector
</ResponseField>

<ResponseField name="endpoint" type="string" required>
  The REST endpoint of the index
</ResponseField>

<ResponseField name="token" type="string" required>
  The REST authentication token for the index
</ResponseField>

<ResponseField name="read_only_token" type="string" required>
  The REST authentication read only token for the index
</ResponseField>

<ResponseField name="type" type="string" required>
  The payment plan of the index
</ResponseField>

<ResponseField name="region" type="string" required>
  The region where the index is currently deployed.
</ResponseField>

<ResponseField name="max_vector_count" type="number" required>
  The number of maximum that your index can contain.
</ResponseField>

<ResponseField name="max_daily_updates" type="number" required>
  The number of maximum update operations you can perform in a day. Only upsert operations are included in update count.
</ResponseField>

<ResponseField name="max_daily_queries" type="number" required>
  The number of maximum query operations you can perform in a day. Only query operations are included in query count.
</ResponseField>

<ResponseField name="max_monthly_bandwidth" type="number" required>
  The maximum amount of monthly bandwidth for the index. Unit is bytes. `-1` if the limit is unlimited.
</ResponseField>

<ResponseField name="max_writes_per_second" type="number" required>
  The number of maximum write operations you can perform per second. Only upsert operations are included in write count.
</ResponseField>

<ResponseField name="max_query_per_second" type="number" required>
  The number of maximum query operations you can perform per second. Only query operations are included in query count.
</ResponseField>

<ResponseField name="max_reads_per_request" type="number" required>
  The number of maximum vectors in a read operation. Query and fetch operations are included in read operations.
</ResponseField>

<ResponseField name="max_writes_per_request" type="number" required>
  The number of maximum vectors in a write operation. Only upsert operations are included in write operations.
</ResponseField>

<ResponseField name="max_total_metadata_size" type="number" required>
  The amount of maximum size for the total metadata sizes in your index.
</ResponseField>

<ResponseField name="reserved_price" type="number">
  Monthly pricing of your index. Only available for fixed and pro plans.
</ResponseField>

<ResponseField name="creation_time" type="number" required>
  The creation time of the vector index in UTC as unix timestamp.
</ResponseField>

<ResponseField name="embedding_model" type="string" required>
  The predefined embedding model to vectorize your plain text.
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X POST https://api.upstash.com/v2/vector/index \
    -u 'EMAIL:API_KEY' \
    -d '{
          "name": "myindex",
          "region": "eu-west-1",
          "similarity_function": "COSINE",
          "dimension_count": 1536
        }' \
  ```

  ```javascript JavaScript
  const axios = require('axios');

  const postData = {
    name: "myindex",
    region: "eu-west-1",
    similarity_function: "COSINE",
    dimension_count: 1536,
  };

  const config = {
    auth: {
      username: 'EMAIL',
      password: 'API_KEY',
    },
    headers: {
      'Content-Type': 'application/json',
    },
  };

  axios.post('https://api.upstash.com/v2/vector/index', postData, config)
    .then((response) => {
      console.log('Response:', response.data);
    })
    .catch((error) => {
      console.error('Error:', error);
    });
  ```

  ```python Python
  import requests

  data = '{"name":"myindex","region":"eu-west-1","similarity_function":"COSINE","dimension_count":1536}'

  response = requests.post('https://api.upstash.com/v2/vector/index', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader('{
    "name":"myindex",
    "region":"eu-west-1",
    "similarity_function":"COSINE",
    "dimension_count":1536}'
   )
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/vector/index", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "customer_id": "test@upstash.com",
      "id": "0639864f-ece6-429c-8118-86a287b0e808",
      "name": "myindex",
      "similarity_function": "COSINE",
      "dimension_count": 5,
      "embedding_model": "BGE_SMALL_EN_V1_5"
      "endpoint": "test-index-3814-eu1-vector.upstash.io",
      "token": "QkZGMk5heGltdW0tdXBkYXRlZC0zNzM1LWV1MkFkbWlOeGZGZ1J5Wm1GdE5tTXhNQzB1TmpsbExUb3hOekF0TVRJbFpqMTJORFUxTm1GZw==",
      "read_only_token": "QkZGRk1heGltdW0tdXBkYXRlZC0zNzM1LWV1MnJlYWRvbmx5TmtaZ05qS3JNWVV0Wm1aZ01pMDBOV1poTHRob05qY3RNR0U0TkRjejNqazJU"
      "type": "paid",
      "region": "eu-west-1",
      "max_vector_count": 400000000,
      "max_daily_updates": -1,
      "max_daily_queries": -1,
      "max_monthly_bandwidth": -1,
      "max_writes_per_second": 1000,
      "max_query_per_second": 1000,
      "max_reads_per_request": 1000,
      "max_writes_per_request": 1000,
      "max_total_metadata_size": 53687091200,
      "creation_time": 1707313165
  }
  ```
</ResponseExample>


# Delete Index

DELETE https://api.upstash.com/v2/vector/index/{id}
This endpoint deletes an index.

## Request Parameters

<ParamField path="id" type="string" required>
  The unique ID of the index to be deleted.
</ParamField>

## Response Parameters

`"OK"` on successfull deletion operation.

<RequestExample>
  ```shell curl
  curl -X DELETE https://api.upstash.com/v2/vector/index/0639864f-ece6-429c-8118-86a287b0e808 \
    -u 'EMAIL:API_KEY'
  ```

  ```javascript JavaScript
  const axios = require('axios');

  const config = {
    auth: {
      username: 'EMAIL',
      password: 'API_KEY',
    },
  };

  const url = 'https://api.upstash.com/v2/vector/index/0639864f-ece6-429c-8118-86a287b0e808';

  axios.delete(url, config)
    .then((response) => {
      console.log('Deleted successfully', response.data);
    })
    .catch((error) => {
      console.error('Error:', error);
    });
  ```

  ```python Python
  import requests

  id="0639864f-ece6-429c-8118-86a287b0e808"

  response = requests.delete(f"https://api.upstash.com/v2/vector/index/{id}", auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}

  req, err := http.NewRequest("DELETE", "https://api.upstash.com/v2/vector/index/0639864f-ece6-429c-8118-86a287b0e808", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Get Index

GET https://api.upstash.com/v2/vector/index/{id}
This endpoint returns the data associated to a index.

## Request Parameters

<ParamField path="id" type="string" required>
  The unique ID of the index to fetch.
</ParamField>

## Response Parameters

<ResponseField name="customer_id" type="string" required>
  The associated ID of the owner of the index
</ResponseField>

<ResponseField name="id" type="string" required>
  Unique ID of the index
</ResponseField>

<ResponseField name="name" type="string" required>
  The name of the index.
</ResponseField>

<ResponseField name="similarity_function" type="string" required>
  Similarity function that's used to calculate the distance between two
  vectors
</ResponseField>

<ResponseField name="dimension_count" type="number" required>
  The amount of values in a single vector
</ResponseField>

<ResponseField name="endpoint" type="string" required>
  The REST endpoint of the index
</ResponseField>

<ResponseField name="token" type="string" required>
  The REST authentication token for the index
</ResponseField>

<ResponseField name="read_only_token" type="string" required>
  The REST authentication read only token for the index
</ResponseField>

<ResponseField name="type" type="string" required>
  The payment plan of the index
</ResponseField>

<ResponseField name="region" type="string" required>
  The region where the index is currently deployed.
</ResponseField>

<ResponseField name="max_vector_count" type="number" required>
  The number of maximum that your index can contain.
</ResponseField>

<ResponseField name="max_daily_updates" type="number" required>
  The number of maximum update operations you can perform in a day. Only upsert operations are included in update count.
</ResponseField>

<ResponseField name="max_daily_queries" type="number" required>
  The number of maximum query operations you can perform in a day. Only query operations are included in query count.
</ResponseField>

<ResponseField name="max_monthly_bandwidth" type="number" required>
  The maximum amount of monthly bandwidth for the index. Unit is bytes. `-1` if the limit is unlimited.
</ResponseField>

<ResponseField name="max_writes_per_second" type="number" required>
  The number of maximum write operations you can perform per second. Only upsert operations are included in write count.
</ResponseField>

<ResponseField name="max_query_per_second" type="number" required>
  The number of maximum query operations you can perform per second. Only query operations are included in query count.
</ResponseField>

<ResponseField name="max_reads_per_request" type="number" required>
  The number of maximum vectors in a read operation. Query and fetch operations are included in read operations.
</ResponseField>

<ResponseField name="max_writes_per_request" type="number" required>
  The number of maximum vectors in a write operation. Only upsert operations are included in write operations.
</ResponseField>

<ResponseField name="max_total_metadata_size" type="number" required>
  The amount of maximum size for the total metadata sizes in your index.
</ResponseField>

<ResponseField name="reserved_price" type="number">
  Monthly pricing of your index. Only available for fixed and pro plans.
</ResponseField>

<ResponseField name="creation_time" type="number" required>
  The creation time of the vector index in UTC as unix timestamp.
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET https://api.upstash.com/v2/vector/index/0639864f-ece6-429c-8118-86a287b0e808 \
    -u 'EMAIL:API_KEY' \
  ```

  ```javascript JavaScript
  const axios = require('axios');

  const config = {
    auth: {
      username: 'EMAIL',
      password: 'API_KEY',
    },
  };

  const url = 'https://api.upstash.com/v2/vector/index/0639864f-ece6-429c-8118-86a287b0e808';

  axios.get(url, config)
    .then((response) => {
      console.log(response.data);
    })
    .catch((error) => {
      console.error('Error:', error);
    });
  ```

  ```python Python
  import requests

  id = "0639864f-ece6-429c-8118-86a287b0e808"

  response = requests.post(f"https://api.upstash.com/v2/vector/index/{id}", auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}

  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/vector/index/0639864f-ece6-429c-8118-86a287b0e808", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "customer_id": "test@upstash.com",
      "id": "0639864f-ece6-429c-8118-86a287b0e808",
      "name": "myindex",
      "similarity_function": "COSINE",
      "dimension_count": 5,
      "endpoint": "test-index-3814-eu1-vector.upstash.io",
      "token": "QkZGMk5heGltdW0tdXBkYXRlZC0zNzM1LWV1MkFkbWlOeGZGZ1J5Wm1GdE5tTXhNQzB1TmpsbExUb3hOekF0TVRJbFpqMTJORFUxTm1GZw==",
      "read_only_token": "QkZGRk1heGltdW0tdXBkYXRlZC0zNzM1LWV1MnJlYWRvbmx5TmtaZ05qS3JNWVV0Wm1aZ01pMDBOV1poTHRob05qY3RNR0U0TkRjejNqazJU"
      "type": "paid",
      "region": "eu-west-1",
      "max_vector_count": 400000000,
      "max_daily_updates": -1,
      "max_daily_queries": -1,
      "max_monthly_bandwidth": -1,
      "max_writes_per_second": 1000,
      "max_query_per_second": 1000,
      "max_reads_per_request": 1000,
      "max_writes_per_request": 1000,
      "max_total_metadata_size": 53687091200,
      "creation_time": 1707313165
  }
  ```
</ResponseExample>


# List Indices

GET https://api.upstash.com/v2/vector/index/
This endpoint returns the data related to all indices of an account as a list.

## Request Parameters

This endpoint doesn't require any additional data.

## Response Parameters

<ResponseField name="Array<Index>" type="Array" required>
  <Expandable defaultOpen="true">
    <ResponseField name="customer_id" type="string" required>
      The associated ID of the owner of the index
    </ResponseField>

    <ResponseField name="id" type="string" required>
      Unique ID of the index
    </ResponseField>

    <ResponseField name="name" type="string" required>
      The name of the index.
    </ResponseField>

    <ResponseField name="similarity_function" type="string" required>
      Similarity function that's used to calculate the distance between two
      vectors
    </ResponseField>

    <ResponseField name="dimension_count" type="number" required>
      The amount of values in a single vector
    </ResponseField>

    <ResponseField name="endpoint" type="string" required>
      The REST endpoint of the index
    </ResponseField>

    <ResponseField name="type" type="string" required>
      The payment plan of the index
    </ResponseField>

    <ResponseField name="region" type="string" required>
      The region where the index is currently deployed.
    </ResponseField>

    <ResponseField name="max_vector_count" type="number" required>
      The number of maximum that your index can contain.
    </ResponseField>

    <ResponseField name="max_daily_updates" type="number" required>
      The number of maximum update operations you can perform in a day. Only upsert operations are included in update count.
    </ResponseField>

    <ResponseField name="max_daily_queries" type="number" required>
      The number of maximum query operations you can perform in a day. Only query operations are included in query count.
    </ResponseField>

    <ResponseField name="max_monthly_bandwidth" type="number" required>
      The maximum amount of monthly bandwidth for the index. Unit is bytes. `-1` if the limit is unlimited.
    </ResponseField>

    <ResponseField name="max_writes_per_second" type="number" required>
      The number of maximum write operations you can perform per second. Only upsert operations are included in write count.
    </ResponseField>

    <ResponseField name="max_query_per_second" type="number" required>
      The number of maximum query operations you can perform per second. Only query operations are included in query count.
    </ResponseField>

    <ResponseField name="max_reads_per_request" type="number" required>
      The number of maximum vectors in a read operation. Query and fetch operations are included in read operations.
    </ResponseField>

    <ResponseField name="max_writes_per_request" type="number" required>
      The number of maximum vectors in a write operation. Only upsert operations are included in write operations.
    </ResponseField>

    <ResponseField name="max_total_metadata_size" type="number" required>
      The amount of maximum size for the total metadata sizes in your index.
    </ResponseField>

    <ResponseField name="reserved_price" type="number">
      Monthly pricing of your index. Only available for fixed and pro plans.
    </ResponseField>

    <ResponseField name="creation_time" type="number" required>
      The creation time of the vector index in UTC as unix timestamp.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```shell curl
  curl -X GET \
    https://api.upstash.com/v2/vector/index \
    -u 'EMAIL:API_KEY' \
  ```

  ```javascript JavaScript
  const axios = require('axios');

  const config = {
    auth: {
      username: 'EMAIL',
      password: 'API_KEY',
    },
  };

  const url = 'https://api.upstash.com/v2/vector/index';

  axios.get(url, config)
    .then((response) => {
      console.log(response.data);
    })
    .catch((error) => {
      console.error('Error:', error);
    });
  ```

  ```python Python
  import requests

  response = requests.get('https://api.upstash.com/v2/vector/index', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}

  req, err := http.NewRequest("GET", "https://api.upstash.com/v2/vector/index")
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
  	{
  	    "customer_id": "test@upstash.com",
  	    "id": "0639864f-ece6-429c-8118-86a287b0e808",
  	    "name": "myindex",
  	    "similarity_function": "COSINE",
  	    "dimension_count": 5,
  	    "endpoint": "test-index-3814-eu1-vector.upstash.io",
  	    "token": "QkZGMk5heGltdW0tdXBkYXRlZC0zNzM1LWV1MkFkbWlOeGZGZ1J5Wm1GdE5tTXhNQzB1TmpsbExUb3hOekF0TVRJbFpqMTJORFUxTm1GZw==",
  	    "read_only_token": "QkZGRk1heGltdW0tdXBkYXRlZC0zNzM1LWV1MnJlYWRvbmx5TmtaZ05qS3JNWVV0Wm1aZ01pMDBOV1poTHRob05qY3RNR0U0TkRjejNqazJU"
  	    "type": "paid",
  	    "region": "eu-west-1",
  	    "max_vector_count": 400000000,
  	    "max_daily_updates": -1,
  	    "max_daily_queries": -1,
  	    "max_monthly_bandwidth": -1,
  	    "max_writes_per_second": 1000,
  	    "max_query_per_second": 1000,
  	    "max_reads_per_request": 1000,
  	    "max_writes_per_request": 1000,
  	    "max_total_metadata_size": 53687091200,
  	    "creation_time": 1707313165
  	}
  ]
  ```
</ResponseExample>


# Rename Index

POST https://api.upstash.com/v2/vector/index/{id}/rename
This endpoint is used to change the name of an index.

## Request Parameters

<ParamField path="id" type="string" required>
  The unique ID of the index to be deleted.
</ParamField>

<ParamField body="name" type="string" required>
  The new name of the index.
</ParamField>

## Response Parameters

`"OK"` on successfull deletion operation.

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/rename \
    -u 'EMAIL:API_KEY' \
    -d '{"name":"myindex"}'
  ```

  ```javascript JavaScript
  const axios = require('axios');

  const postData = {
    name: "myindex",
  };

  const config = {
    auth: {
      username: 'EMAIL',
      password: 'API_KEY',
    },
    headers: {
      'Content-Type': 'application/json',
    },
  };

  const url = 'https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/rename';

  axios.post(url, postData, config)
    .then((response) => {
      console.log('Rename successful:', response.data);
    })
    .catch((error) => {
      console.error('Error:', error);
    });
  ```

  ```python Python
  import requests

  data = '{"name":"myindex"}'

  response = requests.post('https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/rename', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "name":"myindex",
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/rename", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Reset Index Passwords

POST https://api.upstash.com/v2/vector/index/{id}/reset-password
This endpoint is used to reset regular and readonly tokens of an index.

## Request Parameters

<ParamField path="id" type="string" required>
  The unique ID of the index to reset the password for..
</ParamField>

## Response Parameters

`"OK"` on successfull deletion operation.

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/reset-password \
    -u 'EMAIL:API_KEY' \
  ```

  ```javascript JavaScript
  const axios = require('axios');

  const config = {
    auth: {
      username: 'EMAIL',
      password: 'API_KEY',
    },
    headers: {
      'Content-Type': 'application/json',
    },
  };

  const url = 'https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/reset-password';

  axios.post(url, {}, config) // Sending an empty object as data since no payload is required.
    .then((response) => {
      console.log('Operation successful:', response.data);
    })
    .catch((error) => {
      console.error('Error:', error);
    });

  ```

  ```python Python
  import requests

  response = requests.post('https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/reset-password', auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}

  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/reset-password")

  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Set Index Plan

POST https://api.upstash.com/v2/vector/index/{id}/setplan
This endpoint is used to change the plan of an index.

## Request Parameters

<ParamField path="id" type="string" required>
  The unique ID of the index to be deleted.
</ParamField>

<ParamField body="target_plan" type="string" required>
  The new plan for the index.
</ParamField>

## Response Parameters

`"OK"` on successfull deletion operation.

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/setplan \
    -u 'EMAIL:API_KEY' \
    -d '{"target_plan":"fixed"}'
  ```

  ```javascript JavaScript
  const axios = require('axios');

  const postData = {
    target_plan: "fixed",
  };

  const config = {
    auth: {
      username: 'EMAIL',
      password: 'API_KEY',
    },
    headers: {
      'Content-Type': 'application/json',
    },
  };

  const url = 'https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/setplan';

  axios.post(url, postData, config)
    .then((response) => {
      console.log('Plan set successfully:', response.data);
    })
    .catch((error) => {
      console.error('Error:', error);
    });

  ```

  ```python Python
  import requests

  data = '{"target_plan":"fixed"}'

  response = requests.post('https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/setplan', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "target_plan":"fixed",
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/setplan", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Transfer Index

POST https://api.upstash.com/v2/vector/index/{id}/transfer
This endpoint is used to transfer an index to another team.

## Request Parameters

<ParamField path="id" type="string" required>
  The unique ID of the index to be deleted.
</ParamField>

<ParamField body="target_account" type="string" required>
  The ID of the target account. If the target is a team, then use the format `team:<TEAM_ID>`, else if the target is your personal account use the format `<YOUR_EMAIL>`.
</ParamField>

## Response Parameters

`"OK"` on successfull deletion operation.

<RequestExample>
  ```shell curl
  curl -X POST \
    https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/transfer \
    -u 'EMAIL:API_KEY' \
    -d '{"target_account":"team:team-id-1"}'
  ```

  ```javascript JavaScript
  const axios = require('axios');

  const postData = {
    target_account: "team:team-id-1",
  };

  const config = {
    auth: {
      username: 'EMAIL',
      password: 'API_KEY',
    },
    headers: {
      'Content-Type': 'application/json',
    },
  };

  const url = 'https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/transfer';

  axios.post(url, postData, config)
    .then((response) => {
      console.log('Transfer successful:', response.data);
    })
    .catch((error) => {
      console.error('Error:', error);
    });
  ```

  ```python Python
  import requests

  data = '{"target_account":"team:team-id-1"}'

  response = requests.post('https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/transfer', data=data, auth=('EMAIL', 'API_KEY'))
  response.content
  ```

  ```go Go
  client := &http.Client{}
  var data = strings.NewReader(`{
      "target_account":"team:team-id-1",
  }`)
  req, err := http.NewRequest("POST", "https://api.upstash.com/v2/vector/index/14841111-b834-4788-925c-04ab156d1123/transfer", data)
  if err != nil {
      log.Fatal(err)
  }
  req.SetBasicAuth("email", "api_key")
  resp, err := client.Do(req)
  if err != nil {
      log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText);
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  "OK"
  ```
</ResponseExample>


# Overview



The Upstash Pulumi Provider lets you manage [Upstash](https://upstash.com) Redis and Kafka resources programmatically.

You can find the Github Repository [here](https://github.com/upstash/pulumi-upstash).

## Installing

This package is available for several languages/platforms:

### Node.js (JavaScript/TypeScript)

To use from JavaScript or TypeScript in Node.js, install using either `npm`:

```bash
npm install @upstash/pulumi
```

or `yarn`:

```bash
yarn add @upstash/pulumi
```

### Python

To use from Python, install using `pip`:

```bash
pip install upstash_pulumi
```

### Go

To use from Go, use `go get` to grab the latest version of the library:

```bash
go get github.com/upstash/pulumi-upstash/sdk/go/...
```

## Configuration

The following configuration points are available for the `upstash` provider:

*   `upstash:apiKey` (environment: `UPSTASH_API_KEY`) - the API key for `upstash`. Can be obtained from the [console](https://console.upstash.com).
*   `upstash:email` (environment: `UPSTASH_EMAIL`) - owner email of the resources

## Some Examples

### TypeScript:

```typescript
import * as pulumi from "@pulumi/pulumi";
import * as upstash from "@upstash/pulumi";

// multiple redis databases in a single for loop

for (let i = 0; i < 5; i++) {
  new upstash.RedisDatabase("mydb" + i, {
    databaseName: "pulumi-ts-db" + i,
    region: "eu-west-1",
    tls: true,
  });
}
```

### Go:

```go
package main

import (
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
	"github.com/upstash/pulumi-upstash/sdk/go/upstash"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {

		createdTeam, err := upstash.NewTeam(ctx, "exampleTeam", &upstash.TeamArgs{
			TeamName: pulumi.String("pulumi go team"),
			CopyCc:   pulumi.Bool(false),
			TeamMembers: pulumi.StringMap{
				"<owner-email>": pulumi.String("owner"),
				"<some-other-user-email>":   pulumi.String("dev"),
			},

		})
		if err != nil {
			return err
		}
		return nil
	})
}

```

### Python:

```python
import  pulumi
import upstash_pulumi as upstash

created_cluster = upstash.KafkaCluster(
    resource_name="myCluster",
    cluster_name="pulumi-python-cluster",
    multizone=False,
    region="eu-west-1"
)
```


# null





# upstash_kafka_cluster_data



<RequestExample>
  ```hcl example.tf
  data "upstash_kafka_cluster_data" "clusterData" {
    cluster_id = resource.upstash_kafka_cluster.exampleCluster.cluster_id
  }

  ```
</RequestExample>

## Schema

### Required

<ParamField query="cluster_id" type="string" required>
  Unique Cluster ID for requested cluster
</ParamField>

### Read-Only

<ResponseField name="cluster_name" type="string">
  Name of the cluster
</ResponseField>

<ResponseField name="creation_time" type="number">
  Creation time of the cluster
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="max_message_size" type="number">
  Max Message Size for the cluster
</ResponseField>

<ResponseField name="max_messsages_per_second" type="number">
  Max Messages Per Second for the cluster
</ResponseField>

<ResponseField name="max_partitions" type="number">
  Max Partitions for the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="number">
  Max Retention Size of the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="number">
  Max Retention Time of the cluster
</ResponseField>

<ResponseField name="multizone" type="bool">
  Whether multizone replication is enabled
</ResponseField>

<ResponseField name="password" type="string">
  Password for the cluster
</ResponseField>

<ResponseField name="region" type="string">
  Region of the cluster. Possible values (may change) are: `eu-west-1`,
  `us-east-1`
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  Name of the cluster
</ResponseField>

<ResponseField name="state" type="string">
  Current state of the cluster

  <br />

  Possible values: `active` or `deleted`
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP Endpoint of the cluster
</ResponseField>

<ResponseField name="type" type="string">
  Type of the cluster
</ResponseField>

<ResponseField name="username" type="string">
  Base64 encoded username for the cluster
</ResponseField>


# upstash_kafka_connector_data



<RequestExample>
  ```hcl example.tf
  data "upstash_kafka_connector_data" "kafkaConnectorData" {
    topic_id = resource.upstash_kafka_connector.exampleKafkaConnector.connector_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="connector_id" type="string" required>
  Unique Connector ID for created connector
</ParamField>

### Read-Only

<ResponseField name="cluster_id" type="string">
  Unique Cluster ID for cluster that the connector is tied to
</ResponseField>

<ResponseField name="connector_class" type="string">
  Connector class of the connector
</ResponseField>

<ResponseField name="connector_state" type="string">
  State error message of the connector
</ResponseField>

<ResponseField name="creation_time" type="number">
  Creation time of the connector
</ResponseField>

<ResponseField name="encoded_username" type="string">
  Encoded username for the connector
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="name" type="string">
  Name of the connector
</ResponseField>

<ResponseField name="properties" type="string">
  Properties of the connector. Custom config for different types of connectors.
</ResponseField>

<ResponseField name="properties_encrypted" type="string">
  Encrypted properties for the connector
</ResponseField>

<ResponseField name="state" type="string">
  State of the connector
</ResponseField>

<ResponseField name="state_error_message" type="string">
  State error message of the connector
</ResponseField>

<ResponseField name="task" type="list(map(string))">
  Tasks of the connector
</ResponseField>

<ResponseField name="topics" type="list(string)">
  Topics for the connector
</ResponseField>

<ResponseField name="ttl" type="number">
  TTL for the connector
</ResponseField>

<ResponseField name="user_password" type="string">
  User password for the connector
</ResponseField>


# upstash_kafka_credential_data



<RequestExample>
  ```hcl example.tf
  data "upstash_kafka_credential_data" "kafkaCredentialData" {
    credential_id = upstash_kafka_credential.exampleKafkaCredential.credential_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="cluster_id" type="string" required>
  Unique ID of the kafka credential
</ParamField>

### Read-Only

<ResponseField name="cluster_id" type="string">
  ID of the kafka cluster
</ResponseField>

<ResponseField name="creation_time" type="number">
  Creation time of the credential
</ResponseField>

<ResponseField name="credential_name" type="string">
  Name of the kafka credential
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="permissions" type="string">
  Permission scope given to the kafka credential
</ResponseField>

<ResponseField name="state" type="string">
  State of the credential. `active` or `deleted`
</ResponseField>

<ResponseField name="topic" type="string">
  Name of the kafka topic
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used for the kafka credential
</ResponseField>


# upstash_kafka_topic_data



<RequestExample>
  ```hcl example.tf
  data "upstash_kafka_topic_data" "kafkaTopicData" {
    topic_id = resource.upstash_kafka_topic.exampleKafkaTopic.topic_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="topic_id" type="string" required>
  Unique Topic ID for requested kafka topic
</ParamField>

### Read-Only

<ParamField query="cleanup_policy" type="string">
  Cleanup policy will be used in the topic (`compact` or `delete`)
</ParamField>

<ParamField query="cluster_id" type="string">
  ID of the cluster the topic will be deployed in
</ParamField>

<ParamField query="creation_time" type="number">
  Creation time of the topic
</ParamField>

<ParamField query="id" type="string">
  The ID of this resource.
</ParamField>

<ParamField query="max_message_size" type="map(string)">
  Max message size in the topic
</ParamField>

<ParamField query="multizone" type="bool">
  Whether multizone replication is enabled
</ParamField>

<ParamField query="partitions" type="map(string)">
  The number of partitions the topic will have
</ParamField>

<ParamField query="password" type="string">
  Password to be used in authenticating to the cluster
</ParamField>

<ParamField query="region" type="string">
  Region of the kafka topic. Possible values (may change) are: `eu-west-1`,
  `us-east-1`
</ParamField>

<ParamField query="rest_endpoint" type="string">
  REST Endpoint of the kafka topic
</ParamField>

<ParamField query="retention_size" type="number">
  The number of partitions the topic will have
</ParamField>

<ParamField query="retention_time" type="string">
  Retention time of messages in the topic
</ParamField>

<ParamField query="state" type="string">
  State of the credential. `active` or `deleted`
</ParamField>

<ParamField query="tcp_endpoint" type="string">
  TCP Endpoint of the kafka topic
</ParamField>

<ParamField query="topic_id" type="string">
  Unique Cluster ID for created topic
</ParamField>

<ParamField query="username" type="string">
  Base64 encoded username to be used in authenticating to the cluster
</ParamField>


# upstash_qstash_endpoint_data



<RequestExample>
  ```hcl example.tf
  data "upstash_qstash_endpoint_data" "exampleQStashEndpointData" {
    endpoint_id = resource.upstash_qstash_endpoint.exampleQStashEndpoint.endpoint_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="topic_id" type="string" required>
  Topic Id that the endpoint is added to
</ParamField>

### Read-Only

<ResponseField name="endpoint_id" type="string">
  Unique QStash Endpoint ID
</ResponseField>

<ResponseField name="id" type="number">
  The ID of this resource.
</ResponseField>

<ResponseField name="url" type="string">
  Unique QStash Topic Name for Endpoint
</ResponseField>


# upstash_qstash_schedule_data



<RequestExample>
  ```hcl example.tf
  data "upstash_qstash_schedule_data" "exampleQStashScheduleData" {
    schedule_id = resource.upstash_qstash_schedule.exampleQStashSchedule.schedule_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="schedule_id" type="string" required>
  Unique QStash Schedule ID for requested schedule
</ParamField>

### Read-Only

<ResponseField name="body" type="string">
  Body to send for the POST request in string format. Needs escaping () double
  quotes.
</ResponseField>

<ResponseField name="created_at" type="number">
  Creation time for QStash Schedule
</ResponseField>

<ResponseField name="cron" type="string">
  Cron string for QStash Schedule
</ResponseField>

<ResponseField name="destination" type="string">
  Destination for QStash Schedule. Either Topic ID or valid URL
</ResponseField>

<ResponseField name="forward_headers" type="map(string)">
  Forward headers to your API
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="not_before" type="number">
  Start time for QStash Scheduling.
</ResponseField>

<ResponseField name="retries" type="number">
  Retries for QStash Schedule requests.
</ResponseField>


# upstash_qstash_topic_data



<RequestExample>
  ```hcl example.tf
  data "upstash_qstash_topic_data" "exampleQstashTopicData" {
    topic_id = resource.upstash_qstash_topic.exampleQstashTopic.topic_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="topic_id" type="string" required>
  Unique QStash Topic ID for requested topic
</ParamField>

### Read-Only

<ResponseField name="endpoints" type="list(map(string))">
  Endpoints for the QStash Topic
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="name" type="string">
  Name of the QStash Topic
</ResponseField>


# upstash_redis_database_data



<RequestExample>
  ```hcl example.tf
  data "upstash_redis_database_data" "exampleDBData" {
    database_id = resource.upstash_redis_database.exampleDB.database_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="database_id" type="string" required>
  Unique Database ID for created database
</ParamField>

### Read-Only

<ResponseField name="auto_scale" type="bool">
  Upgrade to higher plans automatically when it hits quotas
</ResponseField>

<ResponseField name="creation_time" type="number">
  Creation time of the database
</ResponseField>

<ResponseField name="database_name" type="string">
  Name of the database
</ResponseField>

<ResponseField name="database_type" type="string">
  Type of the database
</ResponseField>

<ResponseField name="db_daily_bandwidth_limit" type="number">
  Daily bandwidth limit for the database
</ResponseField>

<ResponseField name="db_disk_threshold" type="number">
  Disk threshold for the database
</ResponseField>

<ResponseField name="db_max_clients" type="number">
  Max clients for the database
</ResponseField>

<ResponseField name="db_max_commands_per_second" type="number">
  Max commands per second for the database
</ResponseField>

<ResponseField name="db_max_entry_size" type="number">
  Max entry size for the database
</ResponseField>

<ResponseField name="db_max_request_size" type="number">
  Max request size for the database
</ResponseField>

<ResponseField name="db_memory_threshold" type="number">
  Memory threshold for the database
</ResponseField>

<ResponseField name="endpoint" type="string">
  Database URL for connection
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="password" type="string">
  Password of the database
</ResponseField>

<ResponseField name="port" type="number">
  Port of the endpoint
</ResponseField>

<ResponseField name="primary_region" type="string">
  Primary region for the database (Only works if region='global'. Can be one of
  \[us-east-1, us-west-1, us-west-2, eu-central-1, eu-west-1, sa-east-1,
  ap-southeast-1, ap-southeast-2])
</ResponseField>

<ResponseField name="read_only_rest_token" type="string">
  Rest Token for the database.
</ResponseField>

<ResponseField name="read_regions" type="set(string)">
  Read regions for the database (Only works if region='global' and
  primary\_region is set. Can be any combination of \[us-east-1, us-west-1,
  us-west-2, eu-central-1, eu-west-1, sa-east-1, ap-southeast-1,
  ap-southeast-2], excluding the one given as primary.)
</ResponseField>

<ResponseField name="region" type="string">
  Region of the database. Possible values are: `global`, `eu-west-1`,
  `us-east-1`, `us-west-1`, `ap-northeast-1` , `eu-central1`
</ResponseField>

<ResponseField name="rest_token" type="string">
  Rest Token for the database.
</ResponseField>

<ResponseField name="state" type="string">
  State of the database
</ResponseField>

<ResponseField name="tls" type="bool">
  When enabled, data is encrypted in transit. (If changed to false from true,
  results in deletion and recreation of the resource)
</ResponseField>

<ResponseField name="user_email" type="string">
  User email for the database
</ResponseField>


# upstash_team_data



<RequestExample>
  ```hcl example.tf
  data "upstash_team_data" "teamData" {
    team_id = resource.upstash_team.exampleTeam.team_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="team_id" type="string" required>
  Unique Cluster ID for created cluster
</ParamField>

### Read-Only

<ResponseField name="copy_cc" type="bool">
  Whether Credit Card is copied
</ResponseField>

<ResponseField name="id" type="number">
  The ID of this resource.
</ResponseField>

<ResponseField name="team_members" type="map(string)">
  Members of the team. (Owner must be specified, which is the owner of the api
  key.)
</ResponseField>

<ResponseField name="team_name" type="string">
  Name of the team
</ResponseField>


# Overview



The Upstash Terraform Provider lets you manage Upstash Redis and Kafka resources programmatically.

You can find the Github Repository for the Terraform Provider [here](https://github.com/upstash/terraform-provider-upstash).

## Installation

```hcl
terraform {
  required_providers {
    upstash = {
      source = "upstash/upstash"
      version = "x.x.x"
    }
  }
}

provider "upstash" {
  email    = var.email
  api_key  = var.api_key
}
```

`email` is your registered email in Upstash.

`api_key` can be generated from Upstash Console. For more information please check our [docs](https://docs.upstash.com/howto/developerapi).

## Create Database Using Terraform

Here example code snippet that creates database:

```hcl
resource "upstash_redis_database" "redis" {
  database_name = "db-name"
  region        = "eu-west-1"
  tls           = "true"
  multi_zone    = "false"
}
```

## Import Resources From Outside of Terraform

To import resources created outside of the terraform provider, simply create the resource in .tf file as follows:

```hcl
resource "upstash_redis_database" "redis" {}
```

after this, you can run the command:

```
terraform import upstash_redis_database.redis <db-id>
```

Above example is given for an Upstash Redis database. You can import all of the resources by changing the resource type and providing the resource id.

You can check full spec and [doc from here](https://registry.terraform.io/providers/upstash/upstash/latest/docs).

## Support, Bugs Reports, Feature Requests

If you need support then you can ask your questions Upstash Team in [upstash.com](https://upstash.com) chat widget.

There is also discord channel available for community. [Please check here](https://docs.upstash.com/help/support) for more information.


# upstash_kafka_cluster

Create and manage Kafka clusters on Upstash.

<RequestExample>
  ```hcl example.tf
  resource "upstash_kafka_cluster" "exampleCluster" {
    cluster_name = "TerraformCluster"
    region       = "eu-west-1"
    multizone    = false
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="cluster_name" type="string" required>
  Name of the cluster
</ParamField>

<ParamField query="region" type="string" required>
  Region of the cluster. Possible values (may change) are: `eu-west-1`,
  `us-east-1`
</ParamField>

### Optional

<ParamField query="multizone" type="bool">
  Whether cluster has multizone attribute
</ParamField>

### Read-Only

<ResponseField name="cluster_id" type="string">
  Unique cluster ID for created cluster
</ResponseField>

<ResponseField name="creation_time" type="number">
  Creation time of the cluster
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="max_message_size" type="number">
  Max message size for the cluster
</ResponseField>

<ResponseField name="max_messsages_per_second" type="number">
  Max messages per second for the cluster
</ResponseField>

<ResponseField name="max_partitions" type="number">
  Max partitions for the cluster
</ResponseField>

<ResponseField name="max_retention_size" type="number">
  Max retention size of the cluster
</ResponseField>

<ResponseField name="max_retention_time" type="number">
  Max retention time of the cluster
</ResponseField>

<ResponseField name="password" type="string">
  Password for the cluster
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint of the cluster
</ResponseField>

<ResponseField name="state" type="string">
  State, where the cluster is originated
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint of the cluster
</ResponseField>

<ResponseField name="type" type="string">
  Type of the cluster
</ResponseField>

<ResponseField name="username" type="string">
  Base64 encoded username for the cluster
</ResponseField>


# upstash_kafka_connector

Create and manage Kafka Connectors.

<RequestExample>
  ```hcl example.tf
  # Not necessary if the topic belongs to an already created cluster.
  resource "upstash_kafka_cluster" "exampleKafkaCluster" {
    cluster_name = "Terraform_Upstash_Cluster"
    region       = "eu-west-1"
    multizone    = false
  }

  resource "upstash_kafka_topic" "exampleKafkaTopic" {
    topic_name       = "TerraformTopic"
    partitions       = 1
    retention_time   = 625135
    retention_size   = 725124
    max_message_size = 829213
    cleanup_policy   = "delete"

    # Here, you can use the newly created kafka_cluster resource (above) named exampleKafkaCluster.
    # And use its ID so that the topic binds to it.

    # Alternatively, provide the ID of an already created cluster.
    cluster_id = resource.upstash_kafka_cluster.exampleKafkaCluster.cluster_id

  }

  resource "upstash_kafka_connector" "exampleKafkaConnector" {
    name = var.connector_name
    cluster_id = upstash_kafka_cluster.exampleKafkaCluster.cluster_id
    properties = {
      "collection": "user123",
      "connection.uri": "mongodb+srv://test:test@cluster0.fohyg7p.mongodb.net/?retryWrites=true&w=majority",
      "connector.class": "com.mongodb.kafka.connect.MongoSourceConnector",
      "database": "myshinynewdb2",
      "topics": "${upstash_kafka_topic.exampleKafkaTopic.topic_name}"
    }
    # OPTIONAL: change between restart-running-paused
    # running_state = "running"

  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="cluster_id" type="string" required>
  Unique cluster ID related to the connector
</ParamField>

<ParamField query="name" type="string" required>
  Name of the connector
</ParamField>

<ParamField query="properties" type="map(string)" required>
  Properties that the connector will have. Please check the documentation of the
  related connector.
</ParamField>

### Optional

<ParamField query="running_state" type="string">
  Running state of the connector
</ParamField>

### Read-Only

<ResponseField name="cluster_id" type="string">
  Unique connector ID for created connector
</ResponseField>

<ResponseField name="creation_time" type="number">
  Creation of the connector
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>


# upstash_kafka_credential

Create and manage credentials for a kafka cluster.

<RequestExample>
  ```hcl example.tf
  resource "upstash_kafka_cluster" "exampleKafkaCluster" {
    cluster_name = var.cluster_name
    region       = var.region
    multizone    = var.multizone
  }

  resource "upstash_kafka_topic" "exampleKafkaTopic" {
    topic_name       = var.topic_name
    partitions       = var.partitions
    retention_time   = var.retention_time
    retention_size   = var.retention_size
    max_message_size = var.max_message_size
    cleanup_policy   = var.cleanup_policy
    cluster_id       = resource.upstash_kafka_cluster.exampleKafkaCluster.cluster_id
  }

  resource "upstash_kafka_credential" "exampleKafkaCredential" {
    cluster_id      = upstash_kafka_cluster.exampleKafkaCluster.cluster_id
    credential_name = "credentialFromTerraform"
    topic           = upstash_kafka_topic.exampleKafkaTopic.topic_name
    permissions     = "ALL"
  }

  resource "upstash_kafka_credential" "exampleKafkaCredentialAllTopics" {
    cluster_id      = upstash_kafka_cluster.exampleKafkaCluster.cluster_id
    credential_name = "credentialFromTerraform"
    topic           = "*"
    permissions     = "ALL"
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="cluster_name" type="string" required>
  Name of the cluster
</ParamField>

<ParamField query="credential_name" type="string" required>
  Name of the kafka credential
</ParamField>

<ParamField query="permissions" type="map(string)" required>
  Properties that the connector will have. Please check the documentation of the
  related connector.
</ParamField>

<ParamField query="topic" type="map(string)" required>
  Name of the kafka topic
</ParamField>

### Read-Only

<ResponseField name="creation_time" type="number">
  Creation time of the credential
</ResponseField>

<ResponseField name="credential_id" type="string">
  Unique ID of the kafka credential
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="state" type="string">
  State of the credential. `active` or `deleted`
</ResponseField>

<ResponseField name="username" type="string">
  Username to be used for the kafka credential
</ResponseField>


# upstash_kafka_topic

Create and manage Kafka topics in Upstash.

<RequestExample>
  ```hcl example.tf
  # Not necessary if the topic belongs to an already created cluster.
  resource "upstash_kafka_cluster" "exampleKafkaCluster" {
    cluster_name = "Terraform_Upstash_Cluster"
    region       = "eu-west-1"
    multizone    = false
  }

  resource "upstash_kafka_topic" "exampleKafkaTopic" {
    topic_name       = "TerraformTopic"
    partitions       = 1
    retention_time   = 625135
    retention_size   = 725124
    max_message_size = 829213
    cleanup_policy   = "delete"

    # Here, you can use the newly created kafka_cluster resource (above) named exampleKafkaCluster.
    # And use its ID so that the topic binds to it.
    # Alternatively, provide the ID of an already created cluster.
    cluster_id = resource.upstash_kafka_cluster.exampleKafkaCluster.cluster_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="cleanup_policy" type="string" required>
  Cleanup policy will be used in the topic. `compact` or `delete`
</ParamField>

<ParamField query="cluster_id" type="string" required>
  ID of the cluster the topic will be deployed in
</ParamField>

<ParamField query="max_message_size" type="number" required>
  Max message size in the topic
</ParamField>

<ParamField query="partitions" type="number" required>
  The number of partitions the topic will have
</ParamField>

<ParamField query="retention_size" type="number" required>
  Retention size of the messages in the topic
</ParamField>

<ParamField query="retention_time" type="number" required>
  Retention time of messages in the topic
</ParamField>

<ParamField query="topic_name" type="string" required>
  Name of the topic
</ParamField>

### Read-Only

<ResponseField name="creation_time" type="number">
  Creation time of the topic
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="multizone" type="bool">
  Whether multizone replication is enabled
</ResponseField>

<ResponseField name="password" type="string">
  Password to be used in authenticating to the cluster
</ResponseField>

<ResponseField name="region" type="string">
  Region of the kafka topic
</ResponseField>

<ResponseField name="rest_endpoint" type="string">
  REST endpoint of the kafka topic
</ResponseField>

<ResponseField name="state" type="string">
  State of the credential. `active` or `deleted`
</ResponseField>

<ResponseField name="tcp_endpoint" type="string">
  TCP endpoint of the kafka topic
</ResponseField>

<ResponseField name="topic_id" type="string">
  Unique cluster ID for created topic
</ResponseField>

<ResponseField name="username" type="string">
  Base64 encoded username to be used in authenticating to the cluster
</ResponseField>


# upstash_qstash_endpoint

Create and manage QStash endpoints.

<RequestExample>
  ```hcl example.tf
  resource "upstash_qstash_endpoint" "exampleQStashEndpoint" {
    url      = "https://***.***"
    topic_id = resource.upstash_qstash_topic.exampleQstashTopic.topic_id
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="topic_id" type="string" required>
  Topic ID that the endpoint is added to
</ParamField>

<ParamField query="url" type="string" required>
  URL of the endpoint
</ParamField>

### Read-Only

<ResponseField name="endpoint_id" type="string">
  Unique QStash endpoint ID
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="topic_name" type="string">
  Unique QStash topic name for endpoint
</ResponseField>


# upstash_qstash_schedule

Create and manage QStash schedules.

<RequestExample>
  ```hcl example.tf
  resource "upstash_qstash_schedule" "exampleQStashSchedule" {
    destination = resource.upstash_qstash_topic.exampleQstashTopic.topic_id
    cron        = "* * * * */2"

    # or simply provide a link
    # destination = "https://***.***"
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="cron" type="string" required>
  Cron string for QStash Schedule
</ParamField>

<ParamField query="destination" type="string" required>
  Destination for QStash Schedule. Either Topic ID or valid URL
</ParamField>

### Optional

<ParamField query="body" type="string">
  Body to send for the POST request in string format. Needs escaping () double
  quotes.
</ParamField>

<ParamField query="callback" type="string">
  Callback URL for QStash Schedule.
</ParamField>

<ParamField query="content_based_deduplication" type="bool">
  Content based deduplication for QStash Scheduling.
</ParamField>

<ParamField query="content_type" type="string">
  Content type for QStash Scheduling.
</ParamField>

<ParamField query="deduplicatoin_id" type="string">
  Deduplication ID for QStash Scheduling.
</ParamField>

<ParamField query="delay" type="string">
  Delay for QStash Schedule.
</ParamField>

<ParamField query="forward_headers" type="map(string)">
  Forward headers to your API
</ParamField>

<ParamField query="not_before" type="number">
  Start time for QStash Scheduling.
</ParamField>

<ParamField query="retries" type="number">
  Retries for QStash Schedule requests.
</ParamField>

### Read-Only

<ResponseField name="created_at" type="number">
  Creation time for QStash Schedule.
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="schedule_id" type="string">
  Unique QStash Schedule ID for requested schedule
</ResponseField>


# upstash_qstash_topic

Create and manage QStash topics

<RequestExample>
  ```hcl example.tf
  resource "upstash_qstash_topic" "exampleQStashTopic" {
    name = "exampleQStashTopicName"
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="name" type="string" required>
  Name of the QStash topic
</ParamField>

### Read-Only

<ResponseField name="endpoints" type="list(map(string))">
  Endpoints for the QStash topic
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="topic_id" type="string">
  Unique QStash topic ID for requested topic
</ResponseField>


# upstash_redis_database

Create and manage Upstash Redis databases.

<RequestExample>
  ```hcl example.tf
  resource "upstash_redis_database" "exampleDB" {
    database_name = "Terraform DB6"
    region        = "eu-west-1"
    tls           = "true"
    multizone     = "true"
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="database_name" type="string" required>
  Name of the database
</ParamField>

<ParamField query="region" type="string" required>
  Region of the database. Possible values are: `global`, `eu-west-1`,
  `us-east-1`, `us-west-1`, `ap-northeast-1` , `eu-central1`
</ParamField>

### Optional

<ParamField query="auto_scale" type="bool">
  Upgrade to higher plans automatically when it hits quotas
</ParamField>

<ParamField query="eviction" type="bool">
  Enable eviction, to evict keys when your database reaches the max size
</ParamField>

<ParamField query="primary_region" type="string">
  Primary region for the database (Only works if region='global'. Can be one of
  \[us-east-1, us-west-1, us-west-2, eu-central-1, eu-west-1, sa-east-1,
  ap-southeast-1, ap-southeast-2])
</ParamField>

<ParamField query="read_regions" type="set(string)">
  Read regions for the database (Only works if region='global' and
  primary\_region is set. Can be any combination of \[us-east-1, us-west-1,
  us-west-2, eu-central-1, eu-west-1, sa-east-1, ap-southeast-1,
  ap-southeast-2], excluding the one given as primary.)
</ParamField>

<ParamField query="tls" type="bool">
  When enabled, data is encrypted in transit. (If changed to false from true,
  results in deletion and recreation of the resource)
</ParamField>

### Read-Only

<ResponseField name="creation_time" type="number">
  Creation time of the database
</ResponseField>

<ResponseField name="database_id" type="string">
  Unique Database ID for created database
</ResponseField>

<ResponseField name="database_type" type="string">
  Type of the database
</ResponseField>

<ResponseField name="db_daily_bandwidth_limit" type="number">
  Daily bandwidth limit for the database
</ResponseField>

<ResponseField name="db_disk_threshold" type="number">
  Disk threshold for the database
</ResponseField>

<ResponseField name="db_max_clients" type="number">
  Max clients for the database
</ResponseField>

<ResponseField name="db_max_commands_per_second" type="number">
  Max commands per second for the database
</ResponseField>

<ResponseField name="db_max_entry_size" type="number">
  Max entry size for the database
</ResponseField>

<ResponseField name="db_max_request_size" type="number">
  Max request size for the database
</ResponseField>

<ResponseField name="db_memory_threshold" type="number">
  Memory threshold for the database
</ResponseField>

<ResponseField name="endpoint" type="string">
  Database URL for connection
</ResponseField>

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="password" type="string">
  Password of the database
</ResponseField>

<ResponseField name="port" type="number">
  Port of the endpoint
</ResponseField>

<ResponseField name="read_only_rest_token" type="string">
  Rest Token for the database.
</ResponseField>

<ResponseField name="rest_token" type="string">
  Rest Token for the database.
</ResponseField>

<ResponseField name="state" type="string">
  State of the database
</ResponseField>

<ResponseField name="user_email" type="string">
  User email for the database
</ResponseField>


# upstash_team

Create and manage teams on Upstash.

<RequestExample>
  ```hcl example.tf
  resource "upstash_team" "exampleTeam" {
    team_name    = "TerraformTeam"
    copy_cc      = false
    team_members = {
        # Owner is the owner of the api_key.
        "X@Y.Z": "owner",
        "A@B.C": "dev",
        "E@E.F": "finance",
      }
  }
  ```
</RequestExample>

## Schema

### Required

<ParamField query="copy_cc" type="bool" required>
  Whether Credit Card is copied
</ParamField>

<ParamField query="team_members" type="map(string)" required>
  Members of the team. (Owner must be specified, which is the owner of the api
  key.)
</ParamField>

<ParamField query="team_name" type="string" required>
  Name of the team
</ParamField>

### Read-Only

<ResponseField name="id" type="string">
  The ID of this resource.
</ResponseField>

<ResponseField name="team_id" type="string">
  Unique Cluster ID for created cluster
</ResponseField>


# Get Started



<CardGroup cols={2}>
  <Card title="Serverless Redis" href="/redis/overall/getstarted">
    Create a Redis Database within seconds
  </Card>

  <Card title="Serverless Vector" href="/vector/overall/getstarted">
    Create a Vector Database for AI & LLMs
  </Card>

  <Card title="QStash" href="/qstash/overall/getstarted">
    Publish your first message
  </Card>

  <Card title="Upstash Workflow" href="/workflow/getstarted">
    Write durable serverless functions
  </Card>
</CardGroup>

## Concepts

<CardGroup cols={2}>
  <Card title="Serverless" href="/common/concepts/serverless">
    Upstash is serverless. You don't need to provision any infrastructure. Just
    create a database and start using it.
  </Card>

  <Card title="Scale to Zero" href="/common/concepts/scale-to-zero">
    Price scales to zero. You don't pay for idle or unused resources. You pay
    only for what you use.
  </Card>

  <Card title="Global Replication" href="/common/concepts/global-replication">
    Upstash Redis replicates your data for the best latency all over the world.
  </Card>

  <Card title="Access Anywhere" href="/common/concepts/access-anywhere">
    Upstash REST APIs enable access from all types of runtimes.
  </Card>
</CardGroup>

## Get In touch

<CardGroup cols={3}>
  <Card title="X" icon="X" href="https://x.com/upstash">
    Follow us on X for the latest news and updates.
  </Card>

  <Card title="Discord" icon="discord" href="https://upstash.com/discord">
    Join our Discord Community and ask your questions to the team and other
    developers.
  </Card>

  <Card title="GitHub" icon="github" href="https://github.com/upstash/issues/issues/new">
    Raise an issue on GitHub.
  </Card>
</CardGroup>


# Aiven Http Sink Connector



<Snippet file="kafka/connector-deprecation.mdx" />

Aiven Http Sink Connector calls a given http endpoint for each item published to
your Kafka Topics.

In this guide, we will walk you through creating an Aiven Http Sink Connector.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Prepare the Test Environment

If you already have an HTTP endpoint that you will call, you can skip this step and continue from the [Create The Connector](#create-the-connector) section.

We will use [webhook.site](https://webhook.site/) to verify if the connector is
working. Go to [webhook.site](https://webhook.site/) and copy the unique url to
pass it in the connector config later.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/http/1url.png" width="100%" />
</Frame>

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose your connector as **Aiven Http Sink Connector**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/http/sink/1connector.png" />
</Frame>

Enter the required properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/http/sink/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/http/sink/3advanced.png" />
</Frame>

Congratulations! You have created an Aiven Http Sink Connector.

As you put data into your selected topics, the requests should be visible in
[webhook.site](https://webhook.site/)

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/http/sink/4result.png" />
</Frame>


# Supported Drivers



<Snippet file="kafka/connector-deprecation.mdx" />

These are the currently supported databases and JDBC drivers:

| Database      | JDBC Driver                   |
| ------------- | ----------------------------- |
| PostgreSQL    | postgresql-42.3.3             |
| MySQL         | mysql-connector-java-8.0.28   |
| MS SQL Server | mssql-jdbc-10.2.0             |
| Snowflake     | snowflake-jdbc-3.13.16        |
| ClickHouse    | clickhouse-jdbc-0.3.2-patch11 |
| SQLite        | sqlite-jdbc-3.36.0.3          |


# Aiven JDBC Sink Connector



<Snippet file="kafka/connector-deprecation.mdx" />

Aiven JDBC Sink Connector allows you to continuously store the data from your
Kafka Topics in any sql dialect relational database like Mysql,PostgreSql etc.
In this guide, we will walk you through creating an Aiven JDBC Sink Connector.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose your connector as **Aiven JDBC Connector Sink**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/jdbc/source/1connector.png" />
</Frame>

Enter the required properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/jdbc/sink/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/jdbc/sink/3advanced.png" />
</Frame>

Congratulations! You have created an Aiven JDBC Sink Connector.

As you put data into your selected topics, the data will be written into your
relational database.

## Supported Databases

<Snippet file="kafka/aivenjdbcdrivers.mdx" />


# Aiven JDBC Source Connector



<Snippet file="kafka/connector-deprecation.mdx" />

Aiven JDBC Source Connector allows you to capture any changes in your SQL
dialect relational databases and store them as messages on your Kafka topics. In
this guide, we will walk you through creating the Aiven JDBC Source Connector.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose your connector as **Aiven JDBC Connector Source**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/jdbc/source/1connector.png" />
</Frame>

Enter the required properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/jdbc/source/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/jdbc/source/3advanced.png" />
</Frame>

Congratulations! You have created an Aiven JDBC Source Connector. As you put
data into your relational database, your topics will be created and populated
with new data.

You can go to the **Messages** section to see latest events as they are coming
from Kafka.

## Supported Databases

<Snippet file="kafka/aivenjdbcdrivers.mdx" />


# Aiven OpenSearch Sink Connector



<Snippet file="kafka/connector-deprecation.mdx" />

Aiven OpenSearch Sink Connector allows you to continuously store the data from
your Kafka Topics in any OpenSearch compatible product like Amazon OpenSearch,
Elasticsearch, etc.

In this guide, we will walk you through creating an Aiven OpenSearch Sink
Connector with Elasticsearch.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Prepare the Elasticsearch Environment

If you already have an Elasticsearch environment with the following information,
skip this step and continue from the
[Create The Connector](#create-the-connector) section.

*   `connection.url`
*   `connection.username`
*   `connection.password`

Go to [Elastic Cloud](https://cloud.elastic.co/deployments) Create or a
deployment. Aside from the name, default configurations should be fine for this
guide.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/opensearch/elasticsearch/1deployment.png" />
</Frame>

Don't forget to save the deployment credentials. We need them to create the
connector later.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/opensearch/elasticsearch/2credentials.png" />
</Frame>

Lastly, we need the connection endpoint. Click on your deployment to see the
details and click to the "Copy Endpoint" of Elasticsearch in Applications
section.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/opensearch/elasticsearch/3endpoint.png" />
</Frame>

These three(username, password, and endpoint) should be enough to create the
connector.

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose your connector as **Aiven OpenSearch Sink Connector**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/opensearch/sink/1connector.png" />
</Frame>

Enter the required properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/opensearch/sink/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/opensearch/sink/3advanced.png" />
</Frame>

Congratulations! You have created an OpenSearch Sink Connector.

As you put data into your selected topics, the data will be written into
ElasticSearch.


# Aiven Amazon S3 Sink Connector



<Snippet file="kafka/connector-deprecation.mdx" />

Aiven Amazon S3 Sink Connector allows you to continuously store the data from
your Kafka Topics in Amazon S3. In this guide, we will walk you through creating
a Amazon S3 Sink Connector.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Prepare the Amazon S3 Environment

If you already have a Amazon S3 environment with the following information, skip
this step and continue from the [Create The Connector](#create-the-connector)
section. Note that the user with the given access keys, should have permission
to modify the given bucket.

*   `aws.access.key.id`
*   `aws.secret.access.key`
*   `aws.s3.bucket.name`
*   `aws.s3.region`

Go to [AWS S3 Console](https://s3.console.aws.amazon.com/s3/) Create or select a
bucket. Note that this bucket name will be used later to configure the
connector.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/01bucket.png" />
</Frame>

To make this guide simple, we will allow public access to this bucket(not
recommended in production).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/02bucket.png" />
</Frame>

You can disable public access and allow only following IP's coming from Upstash
:

```
52.48.149.7
52.213.40.91
174.129.75.41
34.195.190.47
52.58.175.235
18.158.44.120
63.34.151.162
54.247.137.96
3.78.151.126
3.124.80.204
34.236.200.33
44.195.74.73
```

Aside from bucket name and public access changes, default configurations should
be fine for this guide.

Next, we will create a user account with permissions to modify S3 buckets Go to
[AWS IAM](https://console.aws.amazon.com/iam) , then "Access Management" and
"Users". Click on "Add Users".

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/03user.png" width="100%" />
</Frame>

Give a name to the user and continue with the next screen.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/04user.png" width="100%" />
</Frame>

On the "Set Permissions" screen, we will give the "AmazonFullS3Access" to this
user.

This gives more permissions than needed. You can create a custom policy with
following json for more restrictive policy.

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucketMultipartUploads",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": "*"
        }
    ]
}
```

After creating the user, we will go into the details of that user to create a
key. Click on the user, then go to the "Security Credentials". An the "Access
Keys" section, click on the "Create access key" button.

<img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/06accesskey.png" width="100%" />

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/07accesskey.png" width="100%" />
</Frame>

We will choose "Application running outside AWS" and create the access key.
Don't forget to store access key id and secret key. We will use these two when
creating the connector.

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose your connector as **Aiven Amazon S3 Connector**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/sink/1connector.png" />
</Frame>

Enter the required properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/sink/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/sink/3advanced.png" />
</Frame>

Congratulations! You have created an Aiven Amazon S3 Sink Connector.

As you put data into your selected topics, the data will be written into Amazon
S3. You can see the data coming from your related bucket in the Amazon Console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/aiven/amazonS3/sink/4result.png" width="100%" />
</Frame>


# Google BigQuery Sink Connector



<Snippet file="kafka/connector-deprecation.mdx" />

Google BigQuery Sink Connector allows you to continuously store the data from
your Kafka Topics in Google BigQuery. In this guide, we will walk you through
creating a Google BigQuery Sink Connector.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Prepare the Google BigQuery Environment

If you already have a Google BigQuery environment with the following
information, skip this step and continue from the
[Create The Connector](#create-the-connector) section.

*   project name
*   a data set
*   an associated google service account with permission to modify the google big
    query dataset.

Go to [Google Cloud BigQuery](https://console.cloud.google.com/bigquery). Create
or select a project. Note that this project name will be used later to configure
the connector.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/01project.png" />
</Frame>

Create a dataset for the project. Note that this dataset name will be used later
to configure the connector.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/02dataset.png" />
</Frame>

Default configurations should be fine for this guide.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/03dataset.png" />
</Frame>

Next, we will create a service account which later we will connect to this
project. Go to [Google Cloud Console](https://console.cloud.google.com/), then
"IAM & admin" and "Service accounts"

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/04serviceaccount.png" />
</Frame>

Click on "Create Service Account".

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/05serviceaccount.png" />
</Frame>

Give a name to your service account.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/06serviceaccount.png" />
</Frame>

Configure permissions for the service account. To keep it simple, we will make
this service account "Owner" to allow everything. You may want to be more
specific.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/07serviceaccount.png" />
</Frame>

The rest of the config can be left empty. After creating the service account, we
will go to its settings to attach a key to it. Go to the "Actions" tab, and
select "Manage keys".

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/08managekeys.png" />
</Frame>

Then create a new key, if you don't have one already. We will select the "JSON"
key type as recommended.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/09createkey.png" />
</Frame>

We will use the content of this JSON file when creating the connector. For
reference it should look something like this:

```json
{
  "type": "service_account",
  "project_id": "bigquerysinkproject",
  "private_key_id": "b5e8b29ed62171aaaa2b5045f04826635bcf78c4",
  "private_key": "-----BEGIN PRIVATE A_LONG_PRIVATE_KEY_WILL_BE_HERE PRIVATE KEY-----\n",
  "client_email": "serviceforbigquerysink@bigquerysinkproject.iam.gserviceaccount.com",
  "client_id": "109444138898162952667",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/serviceforbigquerysink%40bigquerysinkproject.iam.gserviceaccount.com"
}
```

Then we need to give permission to this service account from the dataset that we
created. From [BigQuery Console](https://console.cloud.google.com/bigquery) go
to your dataset settings and click share.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/10share.png" />
</Frame>

The "Dataset Permissions" view will open. Click to "Add Principal" We will add
the service account we have created as a principal here. And we will assign the
"Owner" role to it to make this example simple. You may want to be more specific
here.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/11share.png" />
</Frame>

With this step, the BigQuery dataset should be ready to use with the connector.

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose your connector as **Google BigQuery Sink Connector**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/sink/1connector.png" />
</Frame>

Enter the required properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/sink/2config.png" />
</Frame>

Note that the Google BigQuery Connector expects the data to have a schema. That
is why we choose JsonConvertor with schema included. Alternatively AvroConvertor
with SchemaRegistry can be used as well.

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/sink/3advanced.png" />
</Frame>

Congratulations! You have created a Google BigQuery Sink Connector.

As you put data into your selected topics, the data will be written into Google
BigQuery. You can view it from the Google BigQuery Console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/bigquery/sink/4result.png" />
</Frame>


# Supported Connect Plugins



<Snippet file="kafka/connector-deprecation.mdx" />

You can use several types of plugins together with your connectors.
Here are all the supported plugins supported.

# Common Plugins supported by all connectors

## Transforms

Related documentation: [https://kafka.apache.org/documentation/#connect\_transforms](https://kafka.apache.org/documentation/#connect_transforms)

org.apache.kafka.connect.transforms.Cast\$Key
org.apache.kafka.connect.transforms.Cast\$Value
org.apache.kafka.connect.transforms.DropHeaders
org.apache.kafka.connect.transforms.ExtractField\$Key
org.apache.kafka.connect.transforms.ExtractField\$Value
org.apache.kafka.connect.transforms.Filter
org.apache.kafka.connect.transforms.Flatten\$Key
org.apache.kafka.connect.transforms.Flatten\$Value
org.apache.kafka.connect.transforms.HeaderFrom\$Key
org.apache.kafka.connect.transforms.HeaderFrom\$Value
org.apache.kafka.connect.transforms.HoistField\$Key
org.apache.kafka.connect.transforms.HoistField\$Value
org.apache.kafka.connect.transforms.InsertField\$Key
org.apache.kafka.connect.transforms.InsertField\$Value
org.apache.kafka.connect.transforms.InsertHeader
org.apache.kafka.connect.transforms.MaskField\$Key
org.apache.kafka.connect.transforms.MaskField\$Value
org.apache.kafka.connect.transforms.RegexRouter
org.apache.kafka.connect.transforms.ReplaceField\$Key
org.apache.kafka.connect.transforms.ReplaceField\$Value
org.apache.kafka.connect.transforms.SetSchemaMetadata\$Key
org.apache.kafka.connect.transforms.SetSchemaMetadata\$Value
org.apache.kafka.connect.transforms.TimestampConverter\$Key
org.apache.kafka.connect.transforms.TimestampConverter\$Value
org.apache.kafka.connect.transforms.TimestampRouter
org.apache.kafka.connect.transforms.ValueToKey

## Predicates

Related documentation: [https://kafka.apache.org/documentation/#connect\_predicates](https://kafka.apache.org/documentation/#connect_predicates)

org.apache.kafka.connect.transforms.predicates.HasHeaderKey
org.apache.kafka.connect.transforms.predicates.RecordIsTombstone
org.apache.kafka.connect.transforms.predicates.TopicNameMatches

## Converters

org.apache.kafka.connect.converters.ByteArrayConverter
org.apache.kafka.connect.converters.DoubleConverter
org.apache.kafka.connect.converters.FloatConverter
org.apache.kafka.connect.converters.IntegerConverter
org.apache.kafka.connect.converters.LongConverter
org.apache.kafka.connect.converters.ShortConverter
org.apache.kafka.connect.json.JsonConverter
org.apache.kafka.connect.storage.StringConverter
org.apache.kafka.connect.storage.SimpleHeaderConverter
io.confluent.connect.avro.AvroConverter

# Plugins Supported By Only Debezium Connectors

## Transforms

Related documentation: [https://debezium.io/documentation/reference/stable/transformations/index.html](https://debezium.io/documentation/reference/stable/transformations/index.html)

io.debezium.connector.mongodb.transforms.ExtractNewDocumentState
io.debezium.connector.mongodb.transforms.outbox.MongoEventRouter
io.debezium.connector.mysql.transforms.ReadToInsertEvent
io.debezium.transforms.ByLogicalTableRouter
io.debezium.transforms.ExtractChangedRecordState
io.debezium.transforms.ExtractNewRecordState
io.debezium.transforms.HeaderToValue
io.debezium.transforms.UnwrapFromEnvelope
io.debezium.transforms.outbox.EventRouter
io.debezium.transforms.partitions.ComputePartition
io.debezium.transforms.partitions.PartitionRouting

## Converters

io.debezium.converters.BinaryDataConverter
io.debezium.converters.ByteArrayConverter
io.debezium.converters.ByteBufferConverter
io.debezium.converters.CloudEventsConverter

# Plugins Supported By Only Debezium Mongo Connector

## Transforms

Related documentation: [https://debezium.io/documentation/reference/stable/transformations/index.html](https://debezium.io/documentation/reference/stable/transformations/index.html)

io.debezium.connector.mongodb.transforms.ExtractNewDocumentState
io.debezium.connector.mongodb.transforms.outbox.MongoEventRouter

# Plugins Supported By Only Snowflake Sink Connector

## Converters

com.snowflake.kafka.connector.records.SnowflakeAvroConverter
com.snowflake.kafka.connector.records.SnowflakeAvroConverterWithoutSchemaRegistry
com.snowflake.kafka.connector.records.SnowflakeJsonConverter


# Troubleshooting



<Snippet file="kafka/connector-deprecation.mdx" />

# Allowlist (whitelist) Upstash IP addresses

For security purposes, some external services may require adding the upstash IP addresses to be listed in their systems.

Here is the complete IP list that Upstash will send traffic from:

```
52.48.149.7
52.213.40.91
174.129.75.41
34.195.190.47
52.58.175.235
18.158.44.120
63.34.151.162
54.247.137.96
3.78.151.126
3.124.80.204
34.236.200.33
44.195.74.73
```


# Debezium MongoDB Source Connector



<Snippet file="kafka/connector-deprecation.mdx" />

Debezium MongoDB Source Connector allows you to capture any changes in your
MongoDB database and store them as messages in your Kafka topics. In this guide,
we will walk you through creating a Debezium MongoDB Source Connector with
MongoDB database to Upstash Kafka.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose **Debezium MongoDB Connector**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/debezium/mongo/1connector.png" />
</Frame>

Enter a connector name and MongoDB URI(connection string). Other configurations
are optional. We will skip them for now.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/debezium/mongo/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation.

<Warning>
  If your MongoDB database is SSL enabled, don't forget to add
  `"mongodb.ssl.enabled": true` at this step. For example, MongoDB Atlas is
  always SSL enabled.
</Warning>

After that we can continue by clicking **Connect**.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/debezium/mongo/3advanced.png" />
</Frame>

Congratulations! You have created a Debezium MongoDB Source Connector to Kafka.
Note that no topics will be created until some data is available on the MongoDB
database.

You can go to **Messages** section of the related topic to see latest events as
they are coming from Kafka.


# Debezium Mysql Source Connector



<Snippet file="kafka/connector-deprecation.mdx" />

Debezium Mysql Source Connector allows you to capture any changes on your Mysql
DB and store them as messages on your Kafka topics. In this guide, we will walk
you through creating Debezium Mysql Source Connector.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose your connector as **Debezium Mysql Connector** for this example

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/debezium/mysql/1connector.png" />
</Frame>

Enter the required properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/debezium/mysql/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/debezium/mysql/3advanced.png" />
</Frame>

Congratulations! You have created a Debezium Mysql Source Connector. As you put
data into your Mysql DB, you will see that topics prefixed with given **Server
Name** will be created and populated with new data.

You can go to **Messages** section to see latest events as they are coming from
Kafka.


# Debezium PostgreSQL Source Connector



<Snippet file="kafka/connector-deprecation.mdx" />

Debezium PostgreSQL Source Connector allows you to capture any changes on your
PostgreSQL DB and store them as messages on your Kafka topics. In this guide, we
will walk you through creating Debezium PostgreSQL Source Connector.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose your connector as **Debezium PostgreSQL Connector** for this example

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/debezium/psql/1connector.png" />
</Frame>

Enter the required properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/debezium/psql/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/debezium/psql/3advanced.png" />
</Frame>

Congratulations! You have created a Debezium PostgreSQL Source Connector. As you
put data into your PostgreSQL DB, you will see that related topics will be
created and populated with new data.

You can go to **Messages** section to see latest events as they are coming from
Kafka.


# Deprecation Notice



As of April 2024, Kafka Connectors are deprecated and will be removed in October, 1st 2024. Please check our [blog post](https://upstash.com/blog/kafka-connectors-deprecation) for more information.

If you were previously using Kafka Connect provided by Upstash, please follow [this guide](https://github.com/upstash/kafka-connectors?tab=readme-ov-file#migration-guide-from-upstash-kafka-connect) to migrate to your own self-hosted Kafka Connect.

If you have any questions or need further assistance, reach out to us at [support@upstash.com](mailto:support@upstash.com) or join our community on [Discord](https://upstash.com/discord).


# Introduction



<Snippet file="kafka/connector-deprecation.mdx" />

Kafka Connect is a tool for streaming data between Apache Kafka and other
systems without writing a single line of code. Via Kafka Sink Connectors, you
can export your data into any other storage. Via Kafka Source Connectors, you
can pull data to your Kafka topics from other systems.

Kafka Connectors can be self hosted but it requires you to setup and maintain
extra processes/machines. Upstash provides hosted versions of connectors for
your Kafka cluster. This will remove the burden of maintaining an extra system and also improve performance, as it will be closer to your cluster.

## Pricing

Connectors are **free** to use. We don't charge anything extra for connectors
other than per message pricing of Kafka topics. Check out
[Pricing](https://upstash.com/pricing/kafka) for details on our per message pricing.

## Get Started

We will create a MongoDB source connector as an example. You can find examples
for all supported connectors on the left side bar under `Connectors` section.

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Create a MongoDB Database

Let's prepare our MongoDB Atlas Database. Go to
[MongoDB Atlas](https://www.mongodb.com/cloud/atlas/register) and register.
Select `Build Database` and choose the `Free Shared` option for this example.
Proceed with `Create Cluster` as the defaults should be fine. If this is
your first time, you will see `Security Quickstart` screen.

Choose a username and password. You will need these later to put it in
connection string to MongoDB.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/sec1.png" width="520" />
</Frame>

<Warning>
  You will be allowing Upstash to connect to your MongoDB database in the next
  screen. So be careful in this step.
</Warning>

Select Cloud Environment and then IP Access List. Enter following static Upstash
IP addresses to IP Access List.

```
52.48.149.7
52.213.40.91
174.129.75.41
34.195.190.47
52.58.175.235
18.158.44.120
63.34.151.162
54.247.137.96
3.78.151.126
3.124.80.204
34.236.200.33
44.195.74.73
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/sec2.png" width="520" />
</Frame>

From here, you will be redirected to Database Deployments screen. Go to
`Connect` and select `Connect your application` to find the MongoDB
URI(connection string). Copy this string to use later when creating our Kafka
Connector. Don't forget to replace the password that you selected earlier for
your MongoDB user.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/uri.png" width="520" />
</Frame>

### Create the Connector

Head over to [console.upstash.com](https://console.upstash.com) and select
your Kafka cluster. Go the Connectors tab, and create your first connector with
`New Connector` button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Then choose your connector as `MongoDB Connector Source` for this example.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/1connector.png" />
</Frame>

Choose a connector name and enter MongoDB URI(connection string) that we
prepared earlier in Config screen. Other configurations are optional. We will
skip them for now.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/2config.png" />
</Frame>

Advanced screen is for any other configuration that selected Connector supports.
At the top of this screen, you can find a link to related documentation. For
this example, we can proceed with what we have and click `Connect` button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/3advanced.png" />
</Frame>

Congratulations you have created your first source connector to Kafka. Note that
no topics will be created until some data is available on the MongoDB.

### See It In Action

With this setup, anything that you have introduced in your MongoDB will be
available on your Kafka topic immediately.

Lets go to MongoDB and populate it with some data.

From main `Database` screen, choose `Browse Collections` , and then click
`Add My Own Data`. Create your database in the next screen.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/createDatabase.png" width="100%" />
</Frame>

Select `Insert Document` on the right.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/insert1.png" width="100%" />
</Frame>

And lets put some data here.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/insert2.png" />
</Frame>

Shortly, we should see a topic created in Upstash Console Kafka with
`DATABASE_NAME.COLLECTION_NAME` in MongoDB database.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/topic.png" width="100%" />
</Frame>

After selecting the topic, you can go to `Messages` section to see latest events
as they are coming from Kafka.

## Next

Check our list of available connectors and how to use them from following links:

*   [MongoDB Source Connector](./mongosource)
*   [MongoDB Sink Connector](./mongosink)
*   [Debezium MongoDB Source Connector](./debeziummongo)
*   [Debezium MysqlDB Source Connector](./debeziummysql)
*   [Debezium PostgreSql Source Connector](./debeziumpsql)
*   [Aiven JDBC Source Connector](./aivenjdbcsource)
*   [Aiven JDBC Sink Connector](./aivenjdbcsink)
*   [Google BigQuery Sink Connector](./bigquerysink)
*   [Aiven Amazon S3 Sink Connector](./aivens3sink)
*   [Aiven OpenSearch(Elasticsearch) Sink Connector](./aivenopensearchsink)
*   [Aiven Http Sink Connector](./aivenhttpsink)
*   [Snowflake Sink Connector](./snowflakesink)

If the connector that you need is not in this list, please add a request to our
[Road Map](https://roadmap.upstash.com/)


# MongoDB Sink Connector



<Snippet file="kafka/connector-deprecation.mdx" />

MongoDB Sink Connector allows you to continuously store the data that appears in your
Kafka Topics to MongoDB database. In this guide, we will walk you through
creating DB Sink Connector with MongoDB database to Upstash Kafka.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose **MongoDB Connector Sink**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/sink/1connector.png" />
</Frame>

Enter a connector name and MongoDB URI(connection string). Select single or
multiple topics from existing topics to read data from.

Enter Database and Collection that the selected topics are written into. We
entered "new" as Database and "test" as Collection. It is not required for this
database and collection to exist on MongoDB database. They will be created
automatically.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/sink/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/sink/3advanced.png" />
</Frame>

Congratulations! You have created your MongoDB Sink Connector. As you put data
into your selected topics, the data will be written into your MongoDB database.


# MongoDB Source Connector



<Snippet file="kafka/connector-deprecation.mdx" />

MongoDB Source Connector allows you to capture any changes in your MongoDB and
store them as messages on your Kafka topics. In this guide, we will walk you
through creating MongoDB Source Connector with MongoDB database to Upstash
Kafka.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose **MongoDB Connector Source**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/1connector.png" />
</Frame>

Enter a connector name and MongoDB URI(connection string). Other configurations
are optional. We will skip them for now.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/mongo/source/3advanced.png" />
</Frame>

Congratulations! You have created your MongoDB Source Connector to Kafka. Note
that no topics will be created until some data is available on the MongoDB
database.

You can go to **Messages** section of your topic to see latest events as they
are coming from Kafka.


# Snowflake Sink Connector



<Snippet file="kafka/connector-deprecation.mdx" />

The Snowflake Sink Connector allows you to continuously store the data from your
Kafka Topics to Snowflake.

In this guide, we will walk you through creating a Snowflake Sink Connector.

## Get Started

### Create a Kafka Cluster

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

### Prepare the Snowflake Environment

From the snowflake console, the following configurations need to be obtained:

1.  `snowflake.url.name`
2.  `snowflake.user.name`
3.  `snowflake.private.key`
4.  `snowflake.database.name`
5.  `snowflake.schema.name`

If you already have these and configured the required roles and keys for the
database and the user, you can skip to the
[Create The Connector](#create-the-connector) section.

For more detailed configurations see
[the snowflake connector documentation](https://docs.snowflake.com/en/user-guide/kafka-connector-install#kafka-configuration-properties)

#### snowflake.url.name

`snowflake.url.name` can be found at the home page of
[the snowflake app](https://app.snowflake.com). Click on the account identifier
and navigate to `copy account URL` as shown below.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/snowflake/1url.png" width="100%" />
</Frame>

A URL similar to [https://mn93536.eu-central-1.snowflakecomputing.com](https://mn93536.eu-central-1.snowflakecomputing.com) will be
copied. We need to append port 443 while passing it to the connector. At the end
`snowflake.url.name` will look like the following.

```
https://mn93536.eu-central-1.snowflakecomputing.com:443
```

#### snowflake.user.name

`snowflake.user.name` can be seen on the profile view. To open the profile view,
go to the top left and click on the profile as shown below.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/snowflake/2user.png" width="520" />
</Frame>

#### snowflake.private.key

`snowflake.private.key` will be generated by you locally. A pair of private and
public keys need to be generated. `public.key` will be set to the user on the
snowflake and the private key will be set to the connector as
`snowflake.private.key`.

See
[the following document](https://docs.snowflake.com/en/user-guide/kafka-connector-install#using-key-pair-authentication-key-rotation)
to learn how to generate the keys and set the public key to snowflake.

#### snowflake.database.name & snowflake.schema.name

From [the snowflake app](https://app.snowflake.com), create a database and a
schema. To be able to use this schema and connector we need to create and assign
a custom role to the database and the schema. You can follow
[this document](https://docs.snowflake.com/en/user-guide/kafka-connector-install#creating-a-role-to-use-the-kafka-connector)
to see how to do it.

Make sure that the script described in the document above is running on the
desired database and schema by selecting them at the top of the script as
follows:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/snowflake/3role.png" width="100%" />
</Frame>

Now, everything should be ready on the snowflake side. We can move on the
creating the connector.

### Create the Connector

Go to the Connectors tab, and create your first connector by clicking the **New
Connector** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/connector.png" width="100%" />
</Frame>

Choose your connector as **Snowflake Connector**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/snowflake/sink/1connector.png" />
</Frame>

Enter the required properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/snowflake/sink/2config.png" />
</Frame>

The advanced screen is for any other configuration that the selected connector
supports. At the top of this screen, you can find a link to related
documentation. We can proceed with what we have and click the **Connect** button
directly.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/snowflake/sink/3advanced.png" />
</Frame>

Congratulations! You have created a Snowflake Sink Connector.

As you put data into your selected topics, the data will be written into
Snowflake. You should see the data in
[the snowflake app](https://app.snowflake.com) as follows:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/connect/snowflake/4data.png" />
</Frame>


# Compliance



## Upstash Legal & Security Documents

*   [Upstash Terms of Service](https://upstash.com/static/trust/terms.pdf)
*   [Upstash Privacy Policy](https://upstash.com/static/trust/privacy.pdf)
*   [Upstash Data Processing Agreement](https://upstash.com/static/trust/dpa.pdf)
*   [Upstash Technical and Organizational Security Measures](https://upstash.com/static/trust/security-measures.pdf)
*   [Upstash Subcontractors](https://upstash.com/static/trust/subprocessors.pdf)

## Is Upstash SOC2 Compliant?

As of July 2023, Upstash Redis and Kafka are SOC2 compliant. Check our [trust page](https://trust.upstash.com/) for details.

## Is Upstash ISO-27001 Compliant?

We are in process of getting this certification. Contact us
([support@upstash.com](mailto:support@upstash.com)) to learn about the expected
date.

## Is Upstash GDPR Compliant?

Yes. For more information, see our
[Privacy Policy](https://upstash.com/static/trust/privacy.pdf). We acquire DPAs
from each [subcontractor](https://upstash.com/static/trust/subprocessors.pdf)
that we work with.

## Is Upstash HIPAA Compliant?

Upstash is currently not HIPAA compliant. Contact us
([support@upstash.com](mailto:support@upstash.com)) if HIPAA is important for
you and we can share more details.

## Is Upstash PCI Compliant?

Upstash does not store personal credit card information. We use Stripe for
payment processing. Stripe is a certified PCI Service Provider Level 1, which is
the highest level of certification in the payments industry.

## Does Upstash conduct vulnerability scanning and penetration tests?

Yes, we use third party tools and work with pen testers. We share the results
with Enterprise customers. Contact us
([support@upstash.com](mailto:support@upstash.com)) for more information.

## Does Upstash take backups?

Yes, we take regular snapshots of the data cluster to the AWS S3 platform.

## Does Upstash encrypt data?

Customers can enable TLS while creating database/cluster, and we recommend it
for production databases/clusters. Also we encrypt data at rest at request of
customers.


# Integration with Third Parties & Partnerships



<Frame>
  <img height="100" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/integration/upstash-integration-diagram.png" />
</Frame>

## Introduction

In this guideline we will outline the steps to integrate Upstash into your platform (GUI or Web App) and allow your users to create and manage Upstash databases without leaving your interfaces. We will explain how to use OAuth2.0 as the underlying foundation to enable this access seamlessly.

If your product or service offering utilizes Redis, Kafka or QStash or if there is a common use case that your end users enable by leveraging these database resources, we invite you to be a partner with us. By integrating Upstash into your platform, you can offer a more complete package for your customers and become a one stop shop. This will also position yourself at the forefront of innovative cloud computing trends such as serverless and expand your customer base.

This is the most commonly used partnership integration model that can be easily implemented by following this guideline. Recently [Cloudflare workers integration](https://blog.cloudflare.com/cloudflare-workers-database-integration-with-upstash/) is implemented through this methodology. For any further questions or partnership discussions please send us an email at [partnerships@upstash.com](mailto:partnerships@upstash.com)

<Info>
  Before starting development to integrate Upstash into your product, please
  send an email to [partnerships@upstash.com](mailto:partnerships@upstash.com) for further assistance and guidance.
</Info>

**General Flow (High level user flow)**

1.  User clicks **`Connect Upstash`** button on your platform’s surface (GUI, Web App)
2.  This initiates the OAuth 2.0 flow, which opens a new browser page displaying the **`Upstash Login Page`**.
3.  If this is an existing user, user logins with their Upstash credentials otherwise they can directly sign up for a new Upstash account.
4.  Browser window redirects to **`Your account has been connected`** page and authentication window automatically closes.
5.  After the user returns to your interface, they see their Upstash Account is now connected.

## Technical Design (SPA - Regular Web Application)

1.  Users click `Connect Upstash` button from Web App.
2.  Web App initiate Upstash OAuth 2.0 flow. Web App can use
    [Auth0 native libraries](https://auth0.com/docs/libraries).

<Note>
  Please reach [partnerships@upstash.com](mailto:partnerships@upstash.com) to receive client id and callback url.
</Note>

3.  After user returns from OAuth 2.0 flow then web app will have JWT token. Web
    App can generate Developer Api key:

```bash
curl -XPOST https://api.upstash.com/apikey \
    -H "Authorization: Bearer JWT_KEY" \
    -H "Content-Type: application/json" \
    -d '{ "name": "APPNAME_API_KEY_TIMESTAMP" }'
```

4.  Web App need to save Developer Api Key to the backend.

## Technical Design ( GUI Apps )

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/integration/oauth2-integration.png" />
</Frame>

1.  User clicks **`Connect Upstash`** button from web app.
2.  Web app initiates Upstash OAuth 2.0 flow and it can use **[Auth0 native libraries](https://auth0.com/docs/libraries)**.
3.  App will open new browser:

```
https://auth.upstash.com/authorize?response_type=code&audience=upstash-api&scope=offline_access&client_id=XXXXXXXXXX&redirect_uri=http%3A%2F%2Flocalhost:3000
```

<Note>Please reach [partnerships@upstash.com](mailto:partnerships@upstash.com) to receive client id.</Note>

4.  After user authenticated Auth0 will redirect user to
    `localhost:3000/?code=XXXXXX`

5.  APP can return some nice html response when Auth0 returns to `localhost:3000`

6.  After getting `code` parameter from the URL query, GUI App will make http
    call to the Auth0 code exchange api. Example CURL request

```bash
curl -XPOST 'https://auth.upstash.com/oauth/token' \
  --header 'content-type: application/x-www-form-urlencoded' \
  --data 'grant_type=authorization_code --data audience=upstash-api' \
  --data 'client_id=XXXXXXXXXXX' \
  --data 'code=XXXXXXXXXXXX' \
  --data 'redirect_uri=localhost:3000'
```

Response:

```json
{
  "access_token": "XXXXXXXXXX",
  "refresh_token": "XXXXXXXXXXX",
  "scope": "offline_access",
  "expires_in": 172800,
  "token_type": "Bearer"
}
```

7.  After 6th Step the response will include `access_token`, it has 3 days TTL.
    GUI App will call Upstash API to get a developer api key:

```bash
curl https://api.upstash.com/apikey -H "Authorization: Bearer JWT_KEY" -d '{ "name" : "APPNAME_API_KEY_TIMESTAMP" }'
```

8.  GUI App will save Developer Api key locally. Then GUI App can call any
    Upstash Developer API [developer.upstash.com/](https://developer.upstash.com/)

## Managing Resources

After obtaining Upstash Developer Api key, your platform surface (web or GUI) can call Upstash API. For example **[Create Database](https://developer.upstash.com/#create-database-global)**, **[List Database](https://developer.upstash.com/#list-databases)**

In this flow, you can ask users for region information and name of the database then can call Create Database API to complete the task

Example CURL request:

```bash
curl -X POST \
  https://api.upstash.com/v2/redis/database \
  -u 'EMAIL:API_KEY' \
  -d '{"name":"myredis", "region":"global", "primary_region":"us-east-1", "read_regions":["us-west-1","us-west-2"], "tls": true}'
```


# Legal



## Upstash Legal Documents

*   [Upstash Terms of Service](https://upstash.com/trust/terms.pdf)
*   [Upstash Privacy Policy](https://upstash.com/trust/privacy.pdf)
*   [Upstash Subcontractors](https://upstash.com/trust/subprocessors.pdf)


# Enterprise Support



Enterprise Support is recommended for customers who use Upstash as part of
their production systems.

Enterprise Support includes the following services:

*   Response time SLA
*   Dedicated Slack/Discord Channels
*   Dedicated real time support: We reserve our engineers for you to help you for
    cases like architecture review, product launch or data migration. Max 10 hours
    per / month.

### Response Time SLA

*   General guidance: 24 hours
*   System impaired: \< 12 hours
*   Production system impaired: \< 4 hours
*   Production system down: \< 1 hour

### Pricing

To purchase or learn more about Enterprise Support, please contact us at [support@upstash.com](mailto:support@upstash.com)


# Uptime SLA



This Service Level Agreement ("SLA") applies to the use of the Upstash services,
offered under the terms of our Terms of Service or other agreement with us
governing your use of Upstash. This SLA does not apply to Upstash services in
the Upstash Free Tier. It is clarified that this SLA is subject to the terms of
the Agreement, and does not derogate therefrom (capitalized terms, unless
otherwise indicated herein, have the meaning specified in the Agreement).
Upstash reserves the right to change the terms of this SLA by publishing updated
terms on its website, such change to be effective as of the date of publication.

### Regional and Global Database SLA

Upstash will use commercially reasonable efforts to make regional and global
databases available with a Monthly Uptime Percentage of at least 99.99%.

In the event any of the services do not meet the SLA, you will be eligible to
receive a Service Credit as described below.

| Monthly Uptime Percentage                           | Service Credit Percentage |
| --------------------------------------------------- | ------------------------- |
| Less than 99.99% but equal to or greater than 99.0% | 10%                       |
| Less than 99.0% but equal to or greater than 95.0%  | 30%                       |
| Less than 95.0%                                     | 60%                       |

### SLA Credits

Service Credits are calculated as a percentage of the monthly bill (excluding
one-time payments such as upfront payments) for the service in the affected
region that did not meet the SLA.

Uptime percentages are recorded and published in the
[Upstash Status Page](https://status.upstash.com).

To receive a Service Credit, you should submit a claim by sending an email to
[support@upstash.com](mailto:support@upstash.com). Your credit request should be
received by us before the end of the second billing cycle after the incident
occurred.

We will apply any service credits against future payments for the applicable
services. At our discretion, we may issue the Service Credit to the credit card
you used. Service Credits will not entitle you to any refund or other payment. A
Service Credit will be applicable and issued only if the credit amount for the
applicable monthly billing cycle is greater than one dollar (\$1 USD). Service
Credits may not be transferred or applied to any other account.


# Support & Contact Us



## Community

The [Upstash Discord Channel](https://upstash.com/discord) is the best way to
interact with the community.

## Team

You can contact the team
via [support@upstash.com](mailto:support@upstash.com) for technical support as
well as for questions and feedback.

## Follow Us

Follow us on [X](https://x.com/upstash).

## Bugs & Issues

You can help us improve Upstash by reporting issues, suggesting new features, and
giving general feedback in
our [Community GitHub Repo](https://github.com/upstash/issues/issues/new).

## Enterprise Support

Get [Enterprise Support](/common/help/prosupport) from the Upstash team.


# Uptime Monitor



## Status Page

You can track the uptime status of Upstash databases in
[Upstash Status Page](https://status.upstash.com)

## Latency Monitor

You can see the average latencies for different regions in
[Upstash Latency Monitoring](https://latency.upstash.com) page


# Connect Using Kafka Clients



Connecting to Upstash Kafka using any Kafka client is very straightforward. If
you do not have a Kafka cluster and/or topic already, follow
[these steps](../overall/getstarted) to create one.

After creating a cluster and a topic, just go to cluster details page on the
[Upstash Console](https://console.upstash.com) and copy bootstrap endpoint,
username and password.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/cluster-detail.png" width="100%" />
</Frame>

Then replace following parameters in the code snippets of your favourite Kafka
client or language below.

*   `{{ BOOTSTRAP_ENDPOINT }}`
*   `{{ UPSTASH_KAFKA_USERNAME }}`
*   `{{ UPSTASH_KAFKA_PASSWORD }}`
*   `{{ TOPIC_NAME }}`

## Create a Topic

<CodeGroup>
  ```typescript TypeScript
  const { Kafka } = require("kafkajs");

  const kafka = new Kafka({ brokers: ["{{ BOOTSTRAP_ENDPOINT }}"], sasl: {
  mechanism: "scram-sha-512", username: "{{ UPSTASH_KAFKA_USERNAME }}", password:
  "{{ UPSTASH_KAFKA_PASSWORD }}", }, ssl: true, });

  const admin = kafka.admin();

  const createTopic = async () => { await admin.connect(); await
  admin.createTopics({ validateOnly: false, waitForLeaders: true, topics: [ {
  topic: "{{ TOPIC_NAME }}", numPartitions: partitions, replicationFactor:
  replicationFactor, }, ], }); await admin.disconnect(); }; createTopic();

  ```

  ```py Python
  from kafka import KafkaAdminClient
  from kafka.admin import NewTopic

  admin = KafkaAdminClient(
    bootstrap_servers=['{{ BOOTSTRAP_ENDPOINT }}'],
    sasl_mechanism='SCRAM-SHA-512',
    security_protocol='SASL_SSL',
    sasl_plain_username='{{ UPSTASH_KAFKA_USERNAME }}',
    sasl_plain_password='{{ UPSTASH_KAFKA_PASSWORD }}',
  )

  admin.create_topics([NewTopic(name='{{ TOPIC_NAME }}', num_partitions=partitions, replication_factor=replicationFactor)])
  admin.close()
  ```

  ```java Java
  class CreateTopic {
    public static void main(String[] args) throws Exception {
      var props = new Properties();
      props.put("bootstrap.servers", "{{ BOOTSTRAP_ENDPOINT }}");
      props.put("sasl.mechanism", "SCRAM-SHA-512");
      props.put("security.protocol", "SASL_SSL");
      props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required " +
                  "username=\"{{ UPSTASH_KAFKA_USERNAME }}\" " +
                  "password=\"{{ UPSTASH_KAFKA_PASSWORD }}\";");

      try (var admin = Admin.create(props)) {
          admin.createTopics(
              Set.of(new NewTopic("{{ TOPIC_NAME }}", partitions, replicationFactor))
            ).all().get();
      }
    }
  }
  ```

  ```go Go
  import (
  	"context"
  	"crypto/tls"
  	"log"

  	"github.com/segmentio/kafka-go"
  	"github.com/segmentio/kafka-go/sasl/scram"
  )

  func main() {
    mechanism, err := scram.Mechanism(scram.SHA512,
        "{{ UPSTASH_KAFKA_USERNAME }}", "{{ UPSTASH_KAFKA_PASSWORD }}")
    if err != nil {
      log.Fatalln(err)
    }

    dialer := &kafka.Dialer{
      SASLMechanism: mechanism,
      TLS:           &tls.Config{},
    }

    conn, err := dialer.Dial("tcp", "{{ BOOTSTRAP_ENDPOINT }}")
    if err != nil {
        log.Fatalln(err)
    }
    defer conn.Close()

    controller, err := conn.Controller()
    if err != nil {
        log.Fatalln(err)
    }

    controllerConn, err := dialer.Dial("tcp", net.JoinHostPort(controller.Host, strconv.Itoa(controller.Port)))
    if err != nil {
        log.Fatalln(err)
    }
    defer controllerConn.Close()

    err = controllerConn.CreateTopics(kafka.TopicConfig{
  		Topic:             "{{ TOPIC_NAME }}",
  		NumPartitions:     partitions,
  		ReplicationFactor: replicationFactor,
  	})
    if err != nil {
        log.Fatalln(err)
    }
  }
  ```
</CodeGroup>

## Produce a Message

<CodeGroup>
  ```typescript TypeScript
  const { Kafka } = require("kafkajs");

  const kafka = new Kafka({
    brokers: ["{{ BOOTSTRAP_ENDPOINT }}"],
    sasl: {
      mechanism: "scram-sha-512",
      username: "{{ UPSTASH_KAFKA_USERNAME }}",
      password: "{{ UPSTASH_KAFKA_PASSWORD }}",
    },
    ssl: true,
  });

  const producer = kafka.producer();

  const produce = async () => {
    await producer.connect();
    await producer.send({
      topic: "{{ TOPIC_NAME }}",
      messages: [{ value: "Hello Upstash!" }],
    });
    await producer.disconnect();
  };
  produce();
  ```

  ```py Python
  from kafka import KafkaProducer

  producer = KafkaProducer(
    bootstrap_servers=['{{ BOOTSTRAP_ENDPOINT }}'],
    sasl_mechanism='SCRAM-SHA-512',
    security_protocol='SASL_SSL',
    sasl_plain_username='{{ UPSTASH_KAFKA_USERNAME }}',
    sasl_plain_password='{{ UPSTASH_KAFKA_PASSWORD }}',
  )

  future = producer.send('{{ TOPIC_NAME }}', b'Hello Upstash!')
  record_metadata = future.get(timeout=10)
  print (record_metadata)
  producer.close()
  ```

  ```java Java
  class Produce {
    public static void main(String[] args) throws Exception {
      var props = new Properties();
      props.put("bootstrap.servers", "{{ BOOTSTRAP_ENDPOINT }}");
      props.put("sasl.mechanism", "SCRAM-SHA-512");
      props.put("security.protocol", "SASL_SSL");
      props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required " +
                  "username=\"{{ UPSTASH_KAFKA_USERNAME }}\" " +
                  "password=\"{{ UPSTASH_KAFKA_PASSWORD }}\";");
      props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
      props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

      try (var producer = new KafkaProducer<String, String>(props)) {
          producer.send(new ProducerRecord<CodeGroup>("{{ TOPIC_NAME }}", "Hello Upstash!"));
          producer.flush();
      }
    }
  }
  ```

  ```go Go
  import (
  	"context"
  	"crypto/tls"
  	"log"

  	"github.com/segmentio/kafka-go"
  	"github.com/segmentio/kafka-go/sasl/scram"
  )

  func main() {
    mechanism, err := scram.Mechanism(scram.SHA512,
        "{{ UPSTASH_KAFKA_USERNAME }}", "{{ UPSTASH_KAFKA_PASSWORD }}")
    if err != nil {
      log.Fatalln(err)
    }

    dialer := &kafka.Dialer{
      SASLMechanism: mechanism,
      TLS:           &tls.Config{},
    }

    w := kafka.NewWriter(kafka.WriterConfig{
      Brokers:  []string{"{{ BOOTSTRAP_ENDPOINT }}"},
      Topic:    "{{ TOPIC_NAME }}",
      Dialer:   dialer,
    })
    defer w.Close()

    err = w.WriteMessages(context.Background(),
      kafka.Message{
          Value: []byte("Hello Upstash!"),
      },
    )
    if err != nil {
      log.Fatalln("failed to write messages:", err)
    }
  }
  ```
</CodeGroup>

## Consume Messages

<CodeGroup>
  ```typescript TypeScript
  const { Kafka } = require("kafkajs");

  const kafka = new Kafka({
    brokers: ["{{ BOOTSTRAP_ENDPOINT }}"],
    sasl: {
      mechanism: "scram-sha-512",
      username: "{{ UPSTASH_KAFKA_USERNAME }}",
      password: "{{ UPSTASH_KAFKA_PASSWORD }}",
    },
    ssl: true,
  });

  const consumer = kafka.consumer({ groupId: "{{ GROUP_NAME }}" });
  const consume = async () => {
    await consumer.connect();
    await consumer.subscribe({
      topic: "{{ TOPIC_NAME }}",
      fromBeginning: true,
    });

    await consumer.run({
      eachMessage: async ({ topic, partition, message }) => {
        console.log({
          topic: topic,
          partition: partition,
          message: JSON.stringify(message),
        });
      },
    });
  };
  consume();
  ```

  ```py Python
  from kafka import KafkaConsumer

  consumer = KafkaConsumer(
    bootstrap_servers=['{{ BOOTSTRAP_ENDPOINT }}'],
    sasl_mechanism='SCRAM-SHA-512',
    security_protocol='SASL_SSL',
    sasl_plain_username='{{ UPSTASH_KAFKA_USERNAME }}',
    sasl_plain_password='{{ UPSTASH_KAFKA_PASSWORD }}',
    group_id='{{ GROUP_NAME }}',
    auto_offset_reset='earliest',
  )

  consumer.subscribe(['{{ TOPIC_NAME }}'])
  records = consumer.poll(timeout_ms=10000)
  print(records)
  consumer.close()
  ```

  ```java Java
  class Consume {
    public static void main(String[] args) throws Exception {
      var props = new Properties();
      props.put("bootstrap.servers", "{{ BOOTSTRAP_ENDPOINT }}");
      props.put("sasl.mechanism", "SCRAM-SHA-512");
      props.put("security.protocol", "SASL_SSL");
      props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required " +
                  "username=\"{{ UPSTASH_KAFKA_USERNAME }}\" " +
                  "password=\"{{ UPSTASH_KAFKA_PASSWORD }}\";");

      props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
      props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
      props.put("auto.offset.reset", "earliest");
      props.put("group.id", "{{ GROUP_NAME }}");

      try(var consumer = new KafkaConsumer<String, String>(props)) {
          consumer.subscribe(Collections.singleton("{{ TOPIC_NAME }}"));
          var records = consumer.poll(Duration.ofSeconds(10));
          for (var record : records) {
              System.out.println(record);
          }
      }
    }
  }
  ```

  ```go Go
  import (
  	"context"
  	"crypto/tls"
  	"log"
  	"time"

  	"github.com/segmentio/kafka-go"
  	"github.com/segmentio/kafka-go/sasl/scram"
  )

  func main() {
    mechanism, err := scram.Mechanism(scram.SHA512,
        "{{ UPSTASH_KAFKA_USERNAME }}", "{{ UPSTASH_KAFKA_PASSWORD }}")
    if err != nil {
      log.Fatalln(err)
    }

    dialer := &kafka.Dialer{
      SASLMechanism: mechanism,
      TLS:           &tls.Config{},
    }

    r := kafka.NewReader(kafka.ReaderConfig{
      Brokers:  []string{"{{ BOOTSTRAP_ENDPOINT }}"},
      GroupID: "{{ GROUP_NAME }}",
      Topic:   "{{ TOPIC_NAME }}",
      Dialer:  dialer,
    })
    defer r.Close()

    ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
    defer cancel()

    m, err := r.ReadMessage(ctx)
    if err != nil {
      log.Fatalln(err)
    }
    log.Printf("%+v\n", m)
  }
  ```
</CodeGroup>


# Connect Using kaf CLI



[kaf](https://github.com/birdayz/kaf) is a modern CLI for Apache Kafka. You can
connect to your Upstash Kafka cluster using `kaf`.

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

In the cluster details section of the
[Upstash Console](https://console.upstash.com) copy bootstrap endpoint, username
and password. Then replace following parameters in the code snippets below with
the actual values you copied earlier.

*   `$BOOTSTRAP_ENDPOINT`
*   `$UPSTASH_KAFKA_USERNAME`
*   `$UPSTASH_KAFKA_PASSWORD`
*   `$GROUP_ID`
*   `$TOPIC_NAME`

Initially we should add cluster configuration to `kaf`'s config file, which
should be located in `~/.kaf/config`. Open config file if it exists or create an
empty one and insert following config:

```yaml
clusters:
  - name: $CLUSTER_NAME
    brokers:
      - $BOOTSTRAP_ENDPOINT
    SASL:
      mechanism: SCRAM-SHA-512
      username: $UPSTASH_KAFKA_USERNAME
      password: $UPSTASH_KAFKA_PASSWORD
    security-protocol: SASL_SSL
```

<Note>
  `$CLUSTER_NAME` is a logical name, which is used to identify different Kafka
  cluster. You can use your Upstash cluster name.
</Note>

To select the cluster configuration to use, run:

```shell
kaf config use-cluster $CLUSTER_NAME
```

At this point you should be able to connect to your Kafka cluster using `kaf`.

**List Brokers and Topics:**

```shell
kaf nodes
```

```shell
kaf topics
```

**Produce a message:**

```shell
echo "Hello Upstash!" | kaf produce $TOPIC_NAME
```

**Fetch messages:**

```shell
kaf consume $TOPIC_NAME
```

**Consume messages using consumer groups:**

```shell
kaf consume $TOPIC_NAME -g $GROUP_ID --offset oldest
```

For more information see [kaf](https://github.com/birdayz/kaf) repository.


# Connect Using kcat CLI



[kcat](https://github.com/edenhill/kcat) is a generic command line non-JVM
producer and consumer for Apache Kafka. You can connect to your Upstash Kafka
cluster using `kcat`.

<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

In the cluster details section of the
[Upstash Console](https://console.upstash.com) copy bootstrap endpoint, username
and password. Then replace following parameters in the code snippets below with
the actual values you copied earlier.

*   `$BOOTSTRAP_ENDPOINT`
*   `$UPSTASH_KAFKA_USERNAME`
*   `$UPSTASH_KAFKA_PASSWORD`
*   `$GROUP_ID`
*   `$TOPIC_NAME`

**Query cluster metadata:**

```shell
kcat -b $BOOTSTRAP_ENDPOINT -X security.protocol=SASL_SSL \
    -X sasl.mechanisms=SCRAM-SHA-512 \
    -X sasl.username=$UPSTASH_KAFKA_USERNAME \
    -X sasl.password=$UPSTASH_KAFKA_PASSWORD \
    -L
```

**Produce a message:**

```shell
echo "Hello Upstash!" | kcat -b $BOOTSTRAP_ENDPOINT
    -X security.protocol=SASL_SSL \
    -X sasl.mechanisms=SCRAM-SHA-512 \
    -X sasl.username=$UPSTASH_KAFKA_USERNAME \
    -X sasl.password=$UPSTASH_KAFKA_PASSWORD \
    -P -t $TOPIC_NAME
```

**Fetch messages:**

```shell
kcat -b $BOOTSTRAP_ENDPOINT -X security.protocol=SASL_SSL \
    -X sasl.mechanisms=SCRAM-SHA-512 \
    -X sasl.username=$UPSTASH_KAFKA_USERNAME \
    -X sasl.password=$UPSTASH_KAFKA_PASSWORD \
    -C -t $TOPIC_NAME
```

**Consume messages using consumer groups:**

```shell
kcat -b $BOOTSTRAP_ENDPOINT -X security.protocol=SASL_SSL \
    -X sasl.mechanisms=SCRAM-SHA-512 \
    -X sasl.username=$UPSTASH_KAFKA_USERNAME \
    -X sasl.password=$UPSTASH_KAFKA_PASSWORD \
    -o beginning -G $GROUP_ID $TOPIC_NAME
```

For more information see [kcat](https://github.com/edenhill/kcat) repository.


# Connect with upstash-kafka



[upstash-kafka](https://github.com/upstash/upstash-kafka/blob/master/README.md)
is an HTTP/REST based Kafka client built on top of
[Upstash Kafka REST API](https://docs.upstash.com/kafka/rest).

It is the only connectionless (HTTP based) Kafka client and designed to work
with:

*   Serverless functions (AWS Lambda ...)
*   Cloudflare Workers (see
    [the example](https://github.com/upstash/upstash-kafka/tree/main/examples/cloudflare-workers))
*   Fastly Compute\@Edge
*   Next.js Edge, Remix, Nuxt ...
*   Client side web/mobile applications
*   WebAssembly
*   and other environments where HTTP is preferred over TCP.

## Quick Start

### Install

```bash
npm install @upstash/kafka
```

### Authenticate

Copy URL, username and password from
[Upstash Console](https://console.upstash.com)

```typescript
import { Kafka } from "@upstash/kafka";

const kafka = new Kafka({
  url: "<UPSTASH_KAFKA_REST_URL>",
  username: "<UPSTASH_KAFKA_REST_USERNAME>",
  password: "<UPSTASH_KAFKA_REST_PASSWORD>",
});
```

### Produce

```typescript
const p = kafka.producer();
const message = { hello: "world" }; // Objects will get serialized using `JSON.stringify`
const response = await p.produce("TOPIC", message);
const response2 = await p.produce("TOPIC", message, {
  partition: 1,
  timestamp: 4567,
  key: "KEY",
  headers: [{ key: "TRACE-ID", value: "32h67jk" }],
});
```

### Produce Many

```javascript
const p = kafka.producer();
const res = await p.produceMany([
  {
    topic: "TOPIC",
    value: "MESSAGE",
    // ...options
  },
  {
    topic: "TOPIC-2",
    value: "MESSAGE-2",
    // ...options
  },
]);
```

### Consume

<Note>
  When a new consumer instance is created, it may return empty messages until
  consumer group coordination is completed.
</Note>

```javascript
const c = kafka.consumer();
const messages = await c.consume({
  consumerGroupId: "group_1",
  instanceId: "instance_1",
  topics: ["test.topic"],
  autoOffsetReset: "earliest",
});
```

## Commit

While `consume` commits automatically, you can commit manually as below:

```typescript
const consumerGroupId = "mygroup";
const instanceId = "myinstance";
const topic = "my.topic";

const c = kafka.consumer();
const messages = await c.consume({
  consumerGroupId,
  instanceId,
  topics: [topic],
  autoCommit: false,
});

for (const message of messages) {
  // message handling logic

  await c.commit({
    consumerGroupId,
    instanceId,
    offset: {
      topic: message.topic,
      partition: message.partition,
      offset: message.offset,
    },
  });
}
```

## Fetch

```typescript
const c = kafka.consumer();
const messages = await c.fetch({
  topic: "greeting",
  partition: 3,
  offset: 42,
  timeout: 1000,
});
```

## Examples

See [here](https://github.com/upstash/upstash-kafka/tree/main/examples) for more
examples.


# Consume Messages Using REST API



<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

In the cluster details section of the
[Upstash Console](https://console.upstash.com), scroll down the **REST API**
section and and copy `UPSTASH_KAFKA_REST_URL`, `UPSTASH_KAFKA_REST_USERNAME` and
`UPSTASH_KAFKA_REST_PASSWORD` using the copy icons next to them.

We will use a `Node.js` sample code to show how to consume messages using the
REST API. Our sample will use a topic named `cities` and consume previously
produced city names from this topic using Kafka consumer groups and automatic
offset committing.

Replace following parameters in the code snippets below with your actual values.

```js
const address = "https://tops-stingray-7863-eu1-rest-kafka.upstash.io";
const user = "G9wcy1zdGluZ3JheS03ODYzJMUX";
const pass = "eUmYCkAlxEhihIc7Hooi2IA2pz2fw==";
const auth = Buffer.from(`${user}:${pass}`).toString("base64");
const topic = "cities";
```

Following code will consume city names using `mygroup` consumer group id and
`myconsumer` consumer id from the topic starting from the latest offset and
print the consumed messages and their offsets to the console:

```js
async function consumeTopic(groupId, consumerId, topic) {
  const response = await fetch(
    `${address}/consume/${groupId}/${consumerId}/${topic}`,
    {
      headers: { Authorization: `Basic ${auth}` },
    }
  );
  const messages = await response.json();
  messages.forEach((m) => {
    console.log(`Message: ${m.value}, Offset: ${m.offset}`);
  });
}

consumeTopic("mygroup", "myconsumer", topic);
```

By default consume API starts consuming from the latest offset. It's also
possible to start from the earliest offset by passing
`Kafka-Auto-Offset-Reset: earliest` request header:

```js
async function consumeTopic(groupId, consumerId, topic, offsetReset) {
  const response = await fetch(
    `${address}/consume/${groupId}/${consumerId}/${topic}`,
    {
      headers: {
        Authorization: `Basic ${auth}`,
        "Kafka-Auto-Offset-Reset": offsetReset,
      },
    }
  );
  const messages = await response.json();
  messages.forEach((m) => {
    console.log(`Message: ${m.value}, Offset: ${m.offset}`);
  });
}

consumeTopic("mygroup", "myconsumer", topic, "earliest");
```

We can also go deeper and turn off auto-commit behaviour of the consumer to
manually commit the offsets later. To turn off auto commit, we should send
`Kafka-Enable-Auto-Commit: false` header. This allows us to commit the offsets
only when all messages processed successfully.

```js
async function consumeTopicWithoutCommit(
  groupId,
  consumerId,
  topic,
  offsetReset
) {
  const response = await fetch(
    `${address}/consume/${groupId}/${consumerId}/${topic}`,
    {
      headers: {
        Authorization: `Basic ${auth}`,
        "Kafka-Auto-Offset-Reset": offsetReset,
        "Kafka-Enable-Auto-Commit": "false",
      },
    }
  );
  const messages = await response.json();
  messages.forEach((m) => {
    console.log(`Message: ${m.value}, Offset: ${m.offset}`);
  });
}

async function commitOffsetsFor(groupId, consumerId) {
  const response = await fetch(`${address}/commit/${groupId}/${consumerId}`, {
    headers: { Authorization: `Basic ${auth}` },
  });
  const resp = await response.json();
  console.log(
    `Result: ${resp.result}, Error: ${resp.error}, Status: ${resp.status}`
  );
}

consumeTopicWithoutCommit("mygroup", "myconsumer", topic, "earliest");
commitOffsetsFor("mygroup", "myconsumer");
```

For more info about using the REST API see
[Kafka REST Consume API](../rest/restconsumer#consume-api) section.


# Use Serverless Kafka as an Event Source For AWS Lambda



In this tutorial we will implement a serverless message processing pipeline
using Upstash Kafka and AWS Lambda. We will use Upstash Kafka as a source for an
AWS Lambda function. The produced messages will trigger AWS Lambda, so your
Lambda function will process the messages.

Because Upstash Kafka is a true serverless product, the whole pipeline will be
serverless. You pay only when your pipeline is actively processing messages.

### Create Upstash Kafka

First, create an Upstash Kafka cluster and topic following
[those steps.](../overall/getstarted) You will need the endpoint, username and
password in the following steps.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/howto/awslambda/cluster_detail.png" width="100%" />
</Frame>

### Create AWS Lambda Function

Now let’s create an AWS Lambda function. For the best performance, select the
same region with Upstash Kafka cluster. We will use Node.js runtime.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/howto/awslambda/create_lambda.png" width="100%" />
</Frame>

<Note>You can use Serverless Framework or AWS SAM for this step.</Note>

### Lambda Function Code

Update your function's code as below:

```javascript
exports.handler = async (event) => {
  if (!event.records) {
    return { response: "no kafka event" };
  }

  for (let messages of Object.values(event.records)) {
    for (let msg of messages) {
      let buff = Buffer.from(msg.value, "base64");
      let text = buff.toString("ascii");
      // process the message
      console.log(text);
    }
  }

  return { response: "success" };
};
```

The above code parses the Kafka message from the event parameter. AWS encodes
the message using `base64` so we decode the message and log it to the console.

### Create AWS Secret

AWS Lambda trigger needs the Kafka credentials to be bundled as a secret. So we
will create a secret in
[AWS Secrets Manager](https://console.aws.amazon.com/secretsmanager/home?region=us-east-1#!/newSecret?step=selectSecret).
Select `Other type of secret`. Enter your Kafka cluster's username and password
as key/value pairs as below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/howto/awslambda/new_secret.png" width="100%" />
</Frame>

In the next screen give a name to your secret.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/howto/awslambda/new_secret2.png" width="100%" />
</Frame>

### Edit AWS Lambda Role

Now we need to configure the Lambda function’s role to access the secrets.

On the AWS Lambda function’s page, click on `Configuration` tab and
`Permissions`. Click to the link just below the `Role name` label.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/howto/awslambda/lambda_config.png" width="100%" />
</Frame>

The IAM management console will be opened in a new tab. On the `Permissions` tab
click on the link which starts with `AWSLambdaBasicExecutionRole-....`

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/howto/awslambda/lambda_config2.png" width="100%" />
</Frame>

Click on the `Edit Policy` button and add this configuration in the JSON tab:

```json
{
  "Effect": "Allow",
  "Action": ["secretsmanager:GetSecretValue"],
  "Resource": ["REPLACE_THE_ARN_OF_THE_SECRET"]
}
```

You need to replace the ARN of the secret that you created in the previous step.

### Create the Trigger

Go back to your Lambda functions page and click the `Add trigger` button. Select
`Apache Kafka` from the menu and fill in the inputs.

Bootstrap servers: copy/paste endpoint from Upstash console.

Topic name: enter your topic’s name

Click on the `Add` button under Authentication. Select `SASL_SCRAM_256_AUTH` and
select the secret that you created in the previous step.

Check the `Enable trigger` checkbox and you can leave the remaining inputs as
they are.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/howto/awslambda/trigger_config.png" width="100%" />
</Frame>

### Testing

Now let’s produce messages and see if AWS Lambda is processing the messages.
Copy the curl URL to produce a message from
[Upstash Console](https://console.upstash.com).

```shell
➜ curl https://full-mantis-14289-us1-rest-kafka.upstash.io/produce/newtopic/newmessage -u  ZnVsbC1tYW50aXMtMTQyODkkimaEsuUsiT9TGk3OFdjveYHBV9Jjzow03SnUtRQ:4-R-fmtoalXnoeu9TjQBOOL4njfSKwEsE10YvHMiW63hFljqUrrq5_yAq4TPGd9c6JbqfQ==

{
  "topic" : "newtopic",
  "partition" : 0,
  "offset" : 48,
  "timestamp" : 1639522675505
}
```

Check the cloudwatch **(Lambda > Monitor > View logs in CloudWatch)**. You
should see the messages you produced are logged by Lambda function.


# Fetch Messages Using REST API



<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

In the cluster details section of the
[Upstash Console](https://console.upstash.com), scroll down the **REST API**
section and and copy `UPSTASH_KAFKA_REST_URL`, `UPSTASH_KAFKA_REST_USERNAME` and
`UPSTASH_KAFKA_REST_PASSWORD` using the copy icons next to them.

We will use a `Node.js` sample code to show how to fetch messages using the REST
API. Our sample will use a topic named `cities` and fetch previously produced
city names from this topic without using Kafka consumer groups.

Replace following parameters in the code snippets below with your actual values.

```js
const address = "https://tops-stingray-7863-eu1-rest-kafka.upstash.io";
const user = "G9wcy1zdGluZ3JheS03ODYzJMUX";
const pass = "eUmYCkAlxEhihIc7Hooi2IA2pz2fw==";
const auth = Buffer.from(`${user}:${pass}`).toString("base64");
const topic = "cities";
```

Following code will fetch city names from `0th` partition of the topic starting
from `1st` offset and print the fetched messages and their offsets to the
console:

```js
async function fetchTopic(topic, partition, offset) {
  const request = {
    topic: topic,
    partition: partition,
    offset: offset,
  };
  const response = await fetch(`${address}/fetch/`, {
    headers: { Authorization: `Basic ${auth}` },
    method: "POST",
    body: JSON.stringify(request),
  });
  const messages = await response.json();
  messages.forEach((m) => {
    console.log(`Message: ${m.value}, Offset: ${m.offset}`);
  });
}

fetchTopic(topic, 0, 1);
```

For more info about using the REST API see
[Kafka REST Fetch API](../rest/restconsumer#consume-api) section.


# Use Serverless Kafka to Produce Events in Cloudflare Workers



In this tutorial, we will produce events to Upstash Kafka from a Cloudflare
Workers function.

### Create Kafka

First, create an Upstash Kafka cluster and topic following
[those steps.](https://docs.upstash.com/kafka) You will need the endpoint,
username and password in the following steps.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/howto/awslambda/cluster_detail.png" width="100%" />
</Frame>

### Create Project

We will use
[Wrangler](https://developers.cloudflare.com/workers/get-started/guide) to
create the application. After installing and configuring wrangler, create a
folder for your project inside the folder run: `wrangler init`

It will create `wrangler.toml`. Paste your account id to the toml which is
logged by wrangler.

Copy and paste the Upstash Kafka URL, topic name, username and password to the
toml.

```toml
name = "produce-in-cloudflare-workers"
type = 'webpack'
account_id = 'REPLACE_HERE'
route = ''
zone_id = ''
usage_model = ''
workers_dev = true
target_type = "webpack"

[vars]
UPSTASH_KAFKA_REST_URL = "REPLACE_HERE"
UPSTASH_KAFKA_REST_USERNAME = "REPLACE_HERE"
UPSTASH_KAFKA_REST_PASSWORD = "REPLACE_HERE"
```

### Implement the Function

Init a node project and install @upstash/kafka:

```
npm init
npm install @upstash/kafka
```

Add the index.js as below:

```javascript
import { Kafka } from "@upstash/kafka";

addEventListener("fetch", (event) => {
  event.respondWith(handleRequest(event.request));
});

async function handleRequest(request) {
  console.log("START", request);

  const kafka = new Kafka({
    url: UPSTASH_KAFKA_REST_URL,
    username: UPSTASH_KAFKA_REST_USERNAME,
    password: UPSTASH_KAFKA_REST_PASSWORD,
  });

  const { pathname } = new URL(request.url);
  if (pathname.startsWith("/favicon")) {
    return fetch(request);
  }
  const p = kafka.producer();
  const message = { hello: "world" }; // Objects will get serialized using `JSON.stringify`
  const response = await p.produce("mytopic", message);

  return new Response(JSON.stringify(response));
}
```

The above code simply sends the message to Kafka. If your message is more
complicated then you can send it in the request body as explained
[here](./producewithrest).

### Run and Deploy the Function

Run the function locally: `wrangler dev`

Deploy your function to Cloudflare by running:

```
wrangler publish
```

This command will output your URL. The output of the URL should be something
like this:

```json
{
  "topic": "newtopic",
  "partition": 0,
  "offset": 278,
  "timestamp": 1640728294879
}
```

### Test the Function

Now let’s validate that the messages are pushed to Kafka. We can consume the
Kafka topic using the REST API. You can copy the curl code to consume from the
Upstash Console.

```
produce-in-lambda git:(master) ✗ curl https://full-mantis-14289-us1-rest-kafka.upstash.io/consume/GROUP_NAME/GROUP_INSTANCE_NAME/mytopic -u REPLACE_USER_NAME:REPLACE_PASSWORD

[ {
  "topic" : "newtopic",
  "partition" : 0,
  "offset" : 282,
  "timestamp" : 1639610767445,
  "key" : "",
  "value" : "hello",
  "headers" : [ ]
} ]%
```

### upstash-kafka vs other Kafka Clients

Upstash also supports native Kafka clients (e.g. KafkaJS). But Cloudflare
Workers runtime does not allow TCP connections.
[upstash-kafka](https://github.com/upstash/upstash-kafka) is HTTP based. That's
why we use [upstash-kafka](https://github.com/upstash/upstash-kafka) in our
Cloudflare examples.


# Use Serverless Kafka to Produce Events in AWS Lambda



In this tutorial, we will produce events to Upstash Kafka from an AWS Lambda
function.

### Create Kafka

First, create an Upstash Kafka cluster and topic following
[those steps.](../overall/getstarted) You will need the endpoint, username and
password in the following steps.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/howto/awslambda/cluster_detail.png" width="100%" />
</Frame>

### Create Project

We will use Serverless Framework to create the application.

```shell
kafka-examples git:(master) serverless
 What do you want to make? AWS - Node.js - HTTP API
 What do you want to call this project? produce-in-lambda
Downloading "aws-node-http-api" template...
Project successfully created in produce-in-lambda folder
```

Then we will initialize a node project and install axios dependency.

```shell
npm init
npm install axios
```

### Implement the Lambda Function

Open the handler.js and update as below:

```javascript
const fetch = require("axios").default;

module.exports.hello = async (event) => {
  const msg = "Hello";
  const address = "https://REPLACE_YOUR_ENDPOINT";
  const user = "REPLACE YOUR USERNAME";
  const pass = "REPLACE YOUR PASSWORD";
  const auth = Buffer.from(`${user}:${pass}`).toString("base64");
  const response = await fetch(`${address}/produce/newtopic/${msg}}`, {
    headers: {
      Authorization: `Basic ${auth}`,
    },
  });
  const res = response.data;

  return {
    statusCode: 200,
    body: JSON.stringify(
      {
        header: "Pushed this message to Upstash Kafka with REST API!",
        message: msg,
        response: res,
      },
      null,
      2
    ),
  };
};
```

You need to replace the endpoint, username and password above with the values
that you copy from the [Upstash Console](https://console.upstash.com).

The above code simply creates a producer and sends the message to Kafka.

### Deploy the Lambda Function

You can deploy your function to AWS by running:

```
serverless deploy
```

This command will output your URL. The output should be something like this:

```json
{
  "header": "Pushed this message to Upstash Kafka!",
  "message": {
    "value": "Hello message"
  }
}
```

### Test the Function

Now let’s validate that the messages are pushed to Kafka. We can consume the
Kafka topic using the REST API. You can copy the curl code to consume from the
Upstash Console.

```
produce-in-lambda git:(master) ✗ curl https://full-mantis-14289-us1-rest-kafka.upstash.io/consume/GROUP_NAME/GROUP_INSTANCE_NAME/newtopic -u REPLACE_USER_NAME:REPLACE_PASSWORD

[ {
  "topic" : "newtopic",
  "partition" : 0,
  "offset" : 98,
  "timestamp" : 1639610767445,
  "key" : "",
  "value" : "Hello message",
  "headers" : [ ]
} ]%
```

### REST vs Kafka Client

We can also use a native Kafka client (e.g. KafkaJS) to access our Kafka
cluster. See
[the repo](https://github.com/upstash/kafka-examples/tree/master/produce-in-lambda)
for both examples. But there is a latency overhead if connecting (and
disconnecting) to the Kafka with each function invocation. In our tests, the
latency of the function with REST is about 10ms whereas it goes up to 50ms when
KafkaJS is used. Kafka client's performance could be improved by caching the
client outside the function but it can cause other problems as explained
[here](https://blog.upstash.com/serverless-database-connections).

**Troubleshooting:** If Lambda function outputs `internal error`, check the
cloudwatch log **(Lambda > Monitor > View logs in CloudWatch)**.


# Monitoring Upstash Kafka Cluster with AKHQ



[AKHQ](https://akhq.io) is a GUI for monitoring & managing Apache Kafka topics,
topics data, consumers group etc. You can connect and monitor your Upstash Kafka
cluster using [AKHQ](https://akhq.io).

To be able to use [AKHQ](https://akhq.io), first you should create a yaml
configuration file:

```yaml
akhq:
  connections:
    my-cluster:
      properties:
        bootstrap.servers: "tops-stingray-7863-eu1-rest-kafka.upstash.io:9092"
        sasl.mechanism: SCRAM-SHA-512
        security.protocol: SASL_SSL
        sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="ZmlycG9iZXJtYW4ZHtSXVwmyJQ" password="J6ocnQfe25vUsI8AX-XxA==";
```

<Note>
  You should replace `bootstrap.servers` and `sasl.jaas.config` attributes with
  your cluster endpoint and credentials.
</Note>

You can start [AKHQ](https://akhq.io) application directly using `jar` file.
First download the latest release from
[releases page](https://github.com/tchiotludo/akhq/releases). Then launch the
application using following command:

```shell
java -Dmicronaut.config.files=application.yml -jar akhq.jar
```

Alternatively you can start using Docker:

```shell
docker run -p 8080:8080 -v ~/akhq/application.yml:/app/application.yml tchiotludo/akhq
```

After launching the [AKHQ](https://akhq.io) app, just go to
[http://localhost:8080](http://localhost:8080) to access UI.

For more information see
[AKHQ documentation](https://akhq.io/docs/#installation).


# Monitoring Upstash Kafka Cluster with Conduktor



[Conduktor](https://www.conduktor.io/) is a quite powerful application to
monitor and manage Apache Kafka clusters. You can connect and monitor your
Upstash Kafka cluster using [Conduktor](https://www.conduktor.io/). Conduktor
has a free for development and testing.

### Install Conduktor

Conduktor is a desktop application. So you need to
[download](https://www.conduktor.io/download/) it first. If you are using a Mac,
you can install it using `brew` too.

```shell
brew tap conduktor/brew
brew install conduktor
```

### Connect Your Cluster

Once you install Conduktor and
[create an Upstash Kafka cluster and topic](../overall/getstarted), you can
connect your cluster to Conduktor. Open Conduktor and click on
`New Kafka Cluster` button.

*   You can set any name as `Cluster Name`.
*   Copy Kafka endpoint from [Upstash console](https://console.upstash.com) and
    paste to `Bootstrap Servers` field.
*   In Upstash console, copy the properties from the `Properties` tab. Paste it to
    the `Additional Properties` field on Conduktor.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/monitoring/conduktor1.png" />
</Frame>

Once you connected to the cluster, now you can produce and consume to your
topics using Conduktor.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/monitoring/conduktor2.png" />
</Frame>


# Monitoring Upstash Kafka Cluster with kafka-ui



[kafka-ui](https://github.com/provectus/kafka-ui) is a GUI for monitoring Apache
Kafka. From their description:

> Kafka UI for Apache Kafka is a simple tool that makes your data flows
> observable, helps find and troubleshoot issues faster and deliver optimal
> performance. Its lightweight dashboard makes it easy to track key metrics of
> your Kafka clusters - Brokers, Topics, Partitions, Production, and
> Consumption.

You can connect and monitor your Upstash Kafka cluster using
[kafka-ui](https://github.com/provectus/kafka-ui).

To be able to use [kafka-ui](https://github.com/provectus/kafka-ui), first you
should create a yaml configuration file:

```yaml
kafka:
  clusters:
    - name: my-cluster
      bootstrapServers: "tops-stingray-7863-eu1-rest-kafka.upstash.io:9092"
      properties:
        sasl.mechanism: SCRAM-SHA-512
        security.protocol: SASL_SSL
        sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="ZmlycG9iZXJtYW4ZHtSXVwmyJQ" password="J6ocnQfe25vUsI8AX-XxA==";
```

<Note>
  You should replace `bootstrap.servers` and `sasl.jaas.config` attributes with
  your cluster endpoint and credentials.
</Note>

You can start [kafka-ui](https://github.com/provectus/kafka-ui) application
directly using `jar` file. First download the latest release from
[releases page](https://github.com/provectus/kafka-ui/releases). Then launch the
application using following command in the same directory with `application.yml`
file:

```shell
java -jar kafka-ui-api-X.Y.Z.jar
```

Alternatively you can start using Docker:

```shell
docker run -p 8080:8080 -v ~/kafka-ui/application.yml:/application.yml provectuslabs/kafka-ui:latest
```

After launching the [kafka-ui](https://github.com/provectus/kafka-ui) app, just
go to [http://localhost:8080](http://localhost:8080) to access UI.

For more information see
[kafka-ui documentation](https://github.com/provectus/kafka-ui/blob/master/README.md).


# Produce Messages Using REST API



<Info>
  If you do not have a Kafka cluster and/or topic already, follow [these
  steps](../overall/getstarted) to create one.
</Info>

In the cluster details section of the
[Upstash Console](https://console.upstash.com), scroll down the **REST API**
section and and copy `UPSTASH_KAFKA_REST_URL`, `UPSTASH_KAFKA_REST_USERNAME` and
`UPSTASH_KAFKA_REST_PASSWORD` using the copy icons next to them.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/connect-rest.png" />
</Frame>

We will use a `Node.js` sample code to show how to produce message(s) using the
REST API. Our sample will use a topic named `cities` and send a few city names
to this topic.

Replace following parameters in the code snippets below with your actual values.

```js
const address = "https://tops-stingray-7863-eu1-rest-kafka.upstash.io";
const user = "G9wcy1zdGluZ3JheS03ODYzJMUX";
const pass = "eUmYCkAlxEhihIc7Hooi2IA2pz2fw==";
const auth = Buffer.from(`${user}:${pass}`).toString("base64");
const topic = "cities";
```

Following code will produce three city names to a topic:

```js
async function produce(topic, msg) {
  const response = await fetch(`${address}/produce/${topic}/${msg}`, {
    headers: { Authorization: `Basic ${auth}` },
  });
  const metadata = await response.json();
  console.log(
    `Topic: ${metadata.topic}, Partition: ${metadata.partition}, Offset: ${metadata.offset}`
  );
}

produce(topic, "Tokyo");
produce(topic, "Istanbul");
produce(topic, "London");
```

Alternatively we can post all cities using a single request, instead of
producing them one-by-one. Note that in this case, URL does not have the message
argument but instead all messages are posted in the request body.

```js
async function produceMulti(topic, ...messages) {
  let data = messages.map((msg) => {
    return { value: msg };
  });
  const response = await fetch(`${address}/produce/${topic}`, {
    headers: { Authorization: `Basic ${auth}` },
    method: "POST",
    body: JSON.stringify(data),
  });
  const metadata = await response.json();
  metadata.forEach((m) => {
    console.log(
      `Topic: ${m.topic}, Partition: ${m.partition}, Offset: ${m.offset}`
    );
  });
}

produceMulti(topic, "Tokyo", "Istanbul", "London");
```

For more info about using the REST API see
[Kafka REST Produce API](../rest/restproducer) section.


# Clickhouse

This tutorial shows how to set up a pipeline to stream traffic events to Upstash Kafka and analyse with Clickhouse

In this tutorial series, we will show how to build an end to end real time
analytics system. We will stream the traffic (click) events from our web
application to Upstash Kafka then we will analyse it on real time. We will
implement one simply query with different stream processing tools:

```sql
SELECT city, count() FROM page_views where event_time > now() - INTERVAL 15 MINUTE group by city
```

Namely, we will query the number of page views from different cities in last 15
minutes. We keep the query and scenario intentionally simple to make the series
easy to understand. But you can easily extend the model for your more complex
realtime analytics scenarios.

If you do not have already set up Kafka pipeline, see
[the first part of series](./cloudflare_workers) where we
did the set up our pipeline including Upstash Kafka and Cloudflare Workers (or
Vercel).

In this part of the series, we will showcase how to use ClickHouse to run a
query on a Kafka topic.

## Clickhouse Setup

You can create a managed service from
[Clickhouse Cloud](https://clickhouse.cloud/) with a 30 days free trial.
Select your region and enter a name for your service. For simplicity, you can
allow access to the service from anywhere. If you want to restrict to the IP
addresses here is the list of Upstash addresses that needs permission:

```text
52.48.149.7
52.213.40.91
174.129.75.41
34.195.190.47
52.58.175.235
18.158.44.120
63.34.151.162
54.247.137.96
3.78.151.126
3.124.80.204
34.236.200.33
44.195.74.73
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/clickhouse/ss1.png" />
</Frame>

### Create a table

On Clickhouse service screen click on `Open SQL console`. Click on `+` to open a
new query window and run the following query to create a table:

```sql
CREATE TABLE page_views
(
    country String,
    city String,
    region String,
    url String,
    ip String,
    event_time DateTime DEFAULT now()
)
ORDER BY (event_time)
```

 

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/clickhouse/ss7.png" />
</Frame>

## Kafka Setup

We will create an [Upstash Kafka cluster](https://console.upstash.com/kafka).
Upstash offers serverless Kafka cluster with per message pricing. Select the
same (or nearest) region with region of Clickhouse for the best performance.
 

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/clickhouse/ss8.png" />
</Frame>

Also create a topic whose messages will be streamed to Clickhouse.  

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/clickhouse/ss9.png" />
</Frame>

## Connector Setup

We will create a connector on
[Upstash console](https://console.upstash.com/kafka). Select your cluster and
click on `Connectors` tab. Select `Aiven JDBC Connector - Sink`

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/clickhouse/ss2.png" />
</Frame>

Click next to skip the Config step as we will enter the configuration manually
at the third (Advanced) step.

In the third step. copy paste the below config to the text editor:

```json
{
  "name": "kafka-clickhouse",
  "properties": {
    "auto.create": false,
    "auto.evolve": false,
    "batch.size": 10,
    "connection.password": "KqVQvD4HWMng",
    "connection.url": "jdbc:clickhouse://a8mo654iq4e.eu-central-1.aws.clickhouse.cloud:8443/default?ssl=true",
    "connection.user": "default",
    "connector.class": "io.aiven.connect.jdbc.JdbcSinkConnector",
    "errors.deadletterqueue.topic.name": "dlqtopic",
    "insert.mode": "insert",
    "key.converter": "org.apache.kafka.connect.storage.StringConverter",
    "key.converter.schemas.enable": false,
    "pk.mode": "none",
    "table.name.format": "page_views",
    "topics": "mytopic",
    "value.converter": "org.apache.kafka.connect.json.JsonConverter",
    "value.converter.schemas.enable": true
  }
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/clickhouse/ss4.png" />
</Frame>

Replace the following attributes:

*   "name" : Name your connector.
*   "connection.password": Copy this from your Clickhouse dashboard. (`Connect` >
    `View connection string`)
*   "connection.url": Copy this from your Clickhouse dashboard. (`Connect` >
    `View connection string`)
*   "connection.user": Copy this from your Clickhouse dashboard. (`Connect` >
    `View connection string`)
*   "errors.deadletterqueue.topic.name": Give a name for your dead letter topic.
    It will be auto created.
*   "topics": Enter the name of the topic that you have created.

Note that there should be `?ssl=true` as a parameter for the connection.url.

Click the `Connect` button to create the connector.

## Test and Run

Clickhouse expects a schema together with the message payload. We need to go
back to [the set up step](./cloudflare_workers) and update
the message object to include schema as below:

```js
const message = {
  schema: {
    type: "struct",
    optional: false,
    version: 1,
    fields: [
      {
        field: "country",
        type: "string",
        optional: false,
      },
      {
        field: "city",
        type: "string",
        optional: false,
      },
      {
        field: "region",
        type: "string",
        optional: false,
      },
      {
        field: "url",
        type: "string",
        optional: false,
      },
      {
        field: "ip",
        type: "string",
        optional: false,
      },
    ],
  },
  payload: {
    country: req.geo?.country,
    city: req.geo?.city,
    region: req.geo?.region,
    url: req.url,
    ip: req.headers.get("x-real-ip"),
    mobile: req.headers.get("sec-ch-ua-mobile"),
    platform: req.headers.get("sec-ch-ua-platform"),
    useragent: req.headers.get("user-agent"),
  },
};
```

<Note>
  It is not ideal to send the schema together with payload. Schema registry is a
  solution. Upstash will launch managed schema registry service soon.
</Note>

After deploying the changes (Cloudflare Workers or Vercel function), visit your
web app to generate traffic to Kafka.

Now, go to the Clickhouse console. `Connect` > `Open SQL console`. Click on
`page_views` (your table's name) on the left menu. You should see the table is
populated like below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/clickhouse/ss6.png" />
</Frame>

Also run the following query to get most popular cities in last 15 minutes:

```shell
SELECT city, count() FROM page_views where event_time > now() - INTERVAL 15 MINUTE group by city
```

It should return something like below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/clickhouse/ss10.png" />
</Frame>


# Cloudflare Workers



As a tutorial for this integration, we'll implement a real time analytics system. We'll stream the traffic (click) events from our web application to Upstash Kafka. Here's the implementation for this simple query:

```sql
SELECT city, count() FROM kafka_topic_page_views where  timestamp > now() - INTERVAL 15 MINUTE group by city
```

Namely, we will query the number of page views from different cities in last 15 minutes. We keep the query and scenario intentionally simple to make the series easy to understand. But you can easily extend the model for your more complex realtime analytics scenarios.

We'll use Clouflare Workers to intercept incoming requests to the website, and run a serverless function.

### Kafka Setup

Create an Upstash Kafka cluster and a topic as explained
[here](https://docs.upstash.com/kafka).

### Project Setup

We will use **C3 (create-cloudflare-cli)** command-line tool to create our application. You can open a new terminal window and run C3 using the prompt below.

<CodeGroup>
  ```shell npm
  npm create cloudflare@latest
  ```

  ```shell yarn
  yarn create cloudflare@latest
  ```
</CodeGroup>

This will install the `create-cloudflare` package, and lead you through setup. C3 will also install Wrangler in projects by default, which helps us testing and deploying the application.

```text
➜  npm create cloudflare@latest
Need to install the following packages:
  create-cloudflare@2.1.0
Ok to proceed? (y) y

using create-cloudflare version 2.1.0

╭ Create an application with Cloudflare Step 1 of 3
│
├ In which directory do you want to create your application?
│ dir ./cloudflare_starter
│
├ What type of application do you want to create?
│ type "Hello World" Worker
│
├ Do you want to use TypeScript?
│ yes typescript
│
├ Copying files from "hello-world" template
│
├ Do you want to use TypeScript?
│ yes typescript
│
├ Retrieving current workerd compatibility date
│ compatibility date 2023-08-07
│
├ Do you want to use git for version control?
│ yes git
│
╰ Application created
```

We will also install the **Upstash Kafka SDK** to connect to Kafka.

```bash
npm install @upstash/kafka
```

### The Code

You can update the `src/index.ts` file with the code below:

<CodeGroup>
  ```ts src/index.ts
  import { Kafka } from "@upstash/kafka";

  export interface Env {
    UPSTASH_KAFKA_REST_URL: string;
    UPSTASH_KAFKA_REST_USERNAME: string;
    UPSTASH_KAFKA_REST_PASSWORD: string;
  }

  export default {
    async fetch(
      request: Request,
      env: Env,
      ctx: ExecutionContext
    ): Promise<Response> {
      if (new URL(request.url).pathname == "/favicon.ico") {
        return new Response(null, { status: 200 });
      }

      let message = {
        country: request.cf?.country,
        city: request.cf?.city,
        region: request.cf?.region,
        url: request.url,
        ip: request.headers.get("x-real-ip"),
        mobile: request.headers.get("sec-ch-ua-mobile"),
        platform: request.headers.get("sec-ch-ua-platform"),
        useragent: request.headers.get("user-agent"),
      };

      const kafka = new Kafka({
        url: env.UPSTASH_KAFKA_REST_URL,
        username: env.UPSTASH_KAFKA_REST_USERNAME,
        password: env.UPSTASH_KAFKA_REST_PASSWORD,
      });

      const p = kafka.producer();
      // Please update the topic according to your configuration
      const topic = "mytopic";

      ctx.waitUntil(p.produce(topic, JSON.stringify(message)));
      // if you use CF Workers to intercept your existing site, uncomment below
      // return await fetch(request);
      return new Response("My website");
    },
  };
  ```

  ```js src/index.js
  import { Kafka } from "@upstash/kafka";

  export default {
    async fetch(request, env, ctx) {
      if (new URL(request.url).pathname == "/favicon.ico") {
        return new Response(null, { status: 200 });
      }

      let message = {
        country: request.cf?.country,
        city: request.cf?.city,
        region: request.cf?.region,
        url: request.url,
        ip: request.headers.get("x-real-ip"),
        mobile: request.headers.get("sec-ch-ua-mobile"),
        platform: request.headers.get("sec-ch-ua-platform"),
        useragent: request.headers.get("user-agent"),
      };

      const kafka = new Kafka({
        url: env.UPSTASH_KAFKA_REST_URL,
        username: env.UPSTASH_KAFKA_REST_USERNAME,
        password: env.UPSTASH_KAFKA_REST_PASSWORD,
      });

      const p = kafka.producer();
      // Please update the topic according to your configuration
      const topic = "mytopic";

      ctx.waitUntil(p.produce(topic, JSON.stringify(message)));
      // if you use CF Workers to intercept your existing site, uncomment below
      // return await fetch(request);
      return new Response("My website");
    },
  };
  ```
</CodeGroup>

Above, we simply parse the request object and send the useful information to Upstash Kafka. You may add/remove information depending on your own requirements.

### Configure Credentials

There are two methods for setting up the credentials for Upstash Kafka client. The recommended way is to use Cloudflare Upstash Integration. Alternatively, you can add the credentials manually.

#### Using the Cloudflare Integration

Access to the [Cloudflare Dashboard](https://dash.cloudflare.com) and login with the same account that you've used while setting up the Worker application. Then, navigate to **Workers & Pages > Overview** section on the sidebar. Here, you'll find your application listed.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/cloudflare-integration/overview.png" />
</Frame>

Clicking on the application will direct you to the application details page, where you can perform the integration process. Switch to the **Settings** tab in the application details, and proceed to **Integrations** section. You will see various Worker integrations listed. To proceed, click the **Add Integration** button associated with the Upstash Kafka.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/cloudflare-integration/kafka-add-integration.png" />
</Frame>

On the Integration page, connect to your Upstash account. Then, select the related cluster from the dropdown menu. Finalize the process by pressing **Add Integration** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/cloudflare-integration/kafka-credentials.png" />
</Frame>

#### Setting up Manually

Navigate to [Upstash Console](https://console.upstash.com) and copy/paste your `UPSTASH_KAFKA_REST_URL`, `UPSTASH_KAFKA_REST_USERNAME` and `UPSTASH_KAFKA_REST_PASSWORD` credentials to your `wrangler.toml` as below.

```yaml
[vars]
UPSTASH_KAFKA_REST_URL="REPLACE_HERE"
UPSTASH_KAFKA_REST_USERNAME="REPLACE_HERE"
UPSTASH_KAFKA_REST_PASSWORD="REPLACE_HERE"
```

### Test and Deploy

You can test the function locally with `npx wrangler dev`

Deploy your function to Cloudflare with `npx wrangler deploy`

Once the deployment is done, the endpoint of the function will be provided to you.

You can check if logs are collected in Kafka by copying the `curl` expression from the console:

```shell
curl https://<UPSTASH_KAFKA_REST_URL>/consume/GROUP_NAME/GROUP_INSTANCE_NAME/TOPIC \
  -H "Kafka-Auto-Offset-Reset: earliest" -u \
  REPLACE_HERE
```


# Decodable

This tutorial shows how to integrate Upstash Kafka with Decodable

[Decodable](https://www.decodable.co/product?utm_source=upstash) is a platform
which enables developers to build data pipelines using SQL. It is built on
Apache Flink under the hood to provide a seamless experience, while abstracting
away the underlying complexity. In this post, we will show how to connect an
Upstash Kafka topic to Decodable to streamline messages from Kafka to Decodable.

## Upstash Kafka Setup

Create a Kafka cluster using
[Upstash Console](https://console.upstash.com/kafka) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

## Decodable Setup

Just like Upstash, Decodable is a managed service that means you do not need to
host or provision anything. You can easily register for free and start using it.

After creating your account, click on `Connections` and `New Connection`. Select
`Apache Kafka`. Then:

*   Select Source as connection type.
*   Select **SASL\_SSL** as security protocol and **SCRAM-SHA-256** as SASL
    mechanism.
*   Enter your topic, SASL username, SASL password. You can find all those from
    Upstash console.
*   Value format should be JSON.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec1.png" />
</Frame>

In the next step, click on `New Stream` and give a name to it.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec2.png" />
</Frame>

In the schema screen, add `country`, `city`, `region` and `url` with `string`
type.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec3.png" />
</Frame>

Give a name to your connection and click `Create Connection`. In the next screen
click on Start.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec4.png" />
</Frame>

## Test the Setup

Now, let's some events to our Kafka topic. Go to Upstash console, click on your
cluster then `Topics`, click `mytopic`. Select `Messages` tab then click
`Produce a new message`. Send a message in JSON format like the below:

```json
{
  "country": "US",
  "city": "San Jose",
  "region": "CA",
  "url": "https://upstash.com"
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec41.png" />
</Frame>

Now, go back to Decodable console, click Streams and select the one you have
created. Then click `Run Preview`. You should see something like:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec5.png" />
</Frame>

## Links

[Decodable documentation](https://docs.decodable.co/docs)

[Decodable console](https://app.decodable.co/)

[Upstash console](https://console.upstash.com/kafka)


# EMQX Cloud

This tutorial shows how to integrate Upstash Kafka with EMQX Cloud

EMQX, a robust open-source MQTT message broker, is engineered for scalable, distributed environments, prioritizing high availability, throughput, and minimal latency. As a preferred protocol in the IoT landscape, MQTT (Message Queuing Telemetry Transport) excels in enabling devices to effectively publish and subscribe to messages.

Offered by EMQ, EMQX Cloud is a comprehensively managed MQTT service in the cloud, inherently scalable and secure. Its design is particularly advantageous for IoT applications, providing dependable MQTT messaging services.

This guide elaborates on streaming MQTT data to Upstash by establishing data integration. This process allows clients to route temperature and humidity metrics to EMQX Cloud using MQTT protocol, and subsequently channel these data streams into a Kafka topic within Upstash.

## Initiating Kafka Clusters on Upstash

Begin your journey with Upstash by visiting [Upstash](https://upstash.com/) and registering for an account.

### Kafka Cluster Creation

1.  After logging in, initiate the creation of a Kafka cluster by selecting the **Create Cluster** button.

2.  Input an appropriate name and select your desired deployment region, ideally close to your EMQX Cloud deployment for optimized performance.

3.  Choose your cluster type: opt for a single replica for development/testing or a multi-replica setup for production scenarios.

4.  Click **Create Cluster** to establish your serverless Kafka cluster.

![UPSTASH](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_kafka_01.png)

### Topic Configuration

1.  Inside the Cluster console, navigate to **Topics** and proceed with **Create Topic**.
2.  Enter `emqx` in the **Topic name** field, maintaining default settings, then finalize with **Create**.

![UPSTASH](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_kafka_02.png)

### Setting Up Credentials

1.  Go to **Credentials** in the navigation menu and choose **New Credentials**.

2.  Here, you can customize the topic and permissions for the credential. Default settings will be used in this tutorial.

![UPSTASH](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_kafka_03.png)

With these steps, we have laid the groundwork for Upstash.

## Establishing Data Integration with Upstash

### Enabling EMQX Cloud's NAT Gateway

1.  Sign in to the EMQX Cloud console and visit the deployment overview page.

2.  Click on the **NAT Gateway** section at the bottom of the page and opt for **Subscribe Now**.

![NAT](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/public_nat.png)

### Data Integration Setup

1.  In the EMQX Cloud console, under your deployment, go to **Data Integrations** and select **Upstash for Kafka**.

    ![create resource](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_kafka_04.png)

2.  Fill in the **Endpoints** details from the Upstash Cluster details into the **Kafka Server** fields. Insert the username and password created in Create Credentials into the respective fields and click **Test** to confirm the connection.

    ![create resource](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_kafka_05.png)

3.  Opt for **New** to add a Kafka resource. You'll see your newly created Upstash for Kafka listed under **Configured Resources**.

4.  Formulate a new SQL rule. Input the following SQL command in the **SQL** field. This rule will process messages from the `temp_hum/emqx` topic and append details like client\_id, topic, and timestamp.

```sql
SELECT
timestamp as up_timestamp,
clientid as client_id,
payload.temp as temp,
payload.hum as hum
FROM
"temp_hum/emqx"
```

![rule sql](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/kafka_create_sql.png)

5.  Conduct an SQL test by inputting the test payload, topic, and client data. Success is indicated by results similar to the example below.

    ![rule sql](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/kafka_create_sql_test.png)

6.  Advance to **Next** to append an action to the rule. Specify the Kafka topic and message format, then confirm.

```bash
# kafka topic
emqx

# kafka message template
{"up_timestamp": ${up_timestamp}, "client_id": ${client_id}, "temp": ${temp}, "hum": ${hum}}
```

![rule sql](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/kafka_action.png)

7.  View the rule SQL statement and bound actions by clicking **View Details** after successfully adding the action.

8.  To review created rules, click **View Created Rules** on the Data Integrations


# Apache Flink

This tutorial shows how to integrate Upstash Kafka with Apache Flink

[Apache Flink](https://flink.apache.org/) is a distributed processing engine
which can process streaming data.

### Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create two topics by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name first topic
“input”, since we are going to stream this topic to other one, which we can name
it as “output”.

### Project Setup

<Info>
  If you already have a project and want to implement Upstash Kafka and Apache
  Flink integration into it, you can skip this section and continue with [Add
  Apache Flink and Kafka into the
  Project](#add-apache-flink-and-kafka-into-the-project).
</Info>

Install Maven to your machine by following [Maven Installation Guide](https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html).

Run `mvn –version` in a terminal or in a command prompt to make sure you have
Maven downloaded.

It should print out the version of the Maven you have:

```
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3\apache-maven\bin\..
Java version: 1.8.0_232, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
```

To create the Maven project;

Go into the folder that you want to create the project in your terminal or
command prompt by running `cd <folder path>`

Run the following command:

```
mvn archetype:generate -DgroupId=com.kafkaflinkinteg.app -DartifactId=kafkaflinkinteg-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
```

### Add Apache Flink and Kafka into the Project

Open the project folder by using an IDE which has maven plugin such as Intellij,
Visual Studio, Eclipse etc. Add following Apache Flink dependencies into the
dependencies tag in `pom.xml` file.

```xml
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-connector-kafka -->
<dependency>
 <groupId>org.apache.flink</groupId>
 <artifactId>flink-connector-kafka</artifactId>
 <version>1.16.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-connector-base -->
<dependency>
 <groupId>org.apache.flink</groupId>
 <artifactId>flink-connector-base</artifactId>
 <version>1.16.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-java -->
<dependency>
 <groupId>org.apache.flink</groupId>
 <artifactId>flink-streaming-java</artifactId>
 <version>1.16.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-clients -->
<dependency>
 <groupId>org.apache.flink</groupId>
 <artifactId>flink-clients</artifactId>
 <version>1.16.0</version>
</dependency>
```

### Streaming From One Topic to Another Topic

You need to create 2 more classes (LineSplitter, CustomSerializationSchema) for
word count example.

#### LineSplitter

This class will be custom implementation of FlatMapFunction from Apache Flink
client library. It takes a sentence, splits into words and returns a
two-dimensional Tuple in format: `(<word>, 1)`.

Create LineSplitter class as following.

```java
package com.kafkaflinkinteg.app;

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;

/**
 * Implements the string tokenizer that splits sentences into words as a user-defined * FlatMapFunction. The function takes a line (String) and splits it into * multiple pairs in the form of "(word,1)" (Tuple2<String, Integer>).
 */
public class LineSplitter  implements FlatMapFunction<String, Tuple2<String, Integer>> {

	@Override
	public void flatMap(String value, Collector<Tuple2<String, Integer>> out) {
		// normalize and split the line
		String[] tokens = value.toLowerCase().split("\\W+");

		// emit the pairs
		for (String token : tokens) {
	        if (token.length() > 0) {
	            out.collect(new Tuple2<String, Integer>(token, 1));
	        }
        }
    }
}
```

#### CustomSerializationSchema

This class will be a custom implementation of KafkaRecordSerializationSchema
from Apache Flink Kafka connector library. It will provide a schema for
serializing and converting data from two-dimensional Tuple, which will be the
output of word counting process, to Kafka record format.

Create CustomSerializationSchema class as following:

```java
package com.kafkaflinkinteg.app;

import org.apache.flink.api.common.serialization.SerializationSchema;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.connector.kafka.sink.KafkaRecordSerializationSchema;
import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonProcessingException;
import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.kafka.clients.producer.ProducerRecord;

public class CustomSerializationSchema<T> implements KafkaRecordSerializationSchema<Tuple2<String, Integer>> {
	private String topic;
	private ObjectMapper mapper;

	public CustomSerializationSchema(String topic) {
		this.topic = topic;
	}

    @Override
    public void open(SerializationSchema.InitializationContext context, KafkaSinkContext sinkContext) throws Exception {
	    KafkaRecordSerializationSchema.super.open(context, sinkContext);
	}

    @Override
	public ProducerRecord<byte[], byte[]> serialize(Tuple2<String, Integer> stringIntegerTuple2, KafkaSinkContext kafkaSinkContext, Long aLong) {
        byte[] k = null;
        byte[] v = null;
        if (mapper == null) {
	        mapper = new ObjectMapper();
	    }
        try {
	        k = mapper.writeValueAsBytes(stringIntegerTuple2.f0);
	        v = mapper.writeValueAsBytes(stringIntegerTuple2.f1);
	    } catch ( JsonProcessingException e) {
		    // error
		}
        return new ProducerRecord<>(topic, k,v);
	}
}
```

#### Integration

Import the following packages first:

```java
package com.kafkaflinkinteg.app;

import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.connector.base.DeliveryGuarantee;
import org.apache.flink.connector.kafka.sink.KafkaSink;
import org.apache.flink.connector.kafka.source.KafkaSource;
import org.apache.flink.connector.kafka.source.enumerator.initializer.OffsetsInitializer;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows;
import org.apache.flink.streaming.api.windowing.time.Time;

import java.util.Properties;
```

Define the names of the topics you are going to work on:

```java
String inputTopic = "input";
String outputTopic = "output";
```

Create the following properties for Apache Flink Kafka connector and replace
`UPSTASH-KAFKA-*` placeholders with your cluster information.

```java
Properties props = new Properties();
props.put("transaction.timeout.ms", "90000"); // e.g., 2 hours
props.put("bootstrap.servers", "UPSTASH-KAFKA-ENDPOINT:9092");
props.put("sasl.mechanism", "SCRAM-SHA-256");
props.put("security.protocol", "SASL_SSL");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";");
```

Get the stream execution environment to create and execute the pipeline in it.

```java
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
```

Create the Kafka consumer.

```java
KafkaSource<String> source = KafkaSource.<String>builder()
        .setStartingOffsets(OffsetsInitializer.earliest())
        .setProperties(props)
        .setTopics(inputTopic)
        .setGroupId("my-group")
        .setValueOnlyDeserializer(new SimpleStringSchema())
        .build();
```

Implement the stream processing part, which will take the input sentence from
source and count words.

```java
DataStream<Tuple2<String, Integer>> stream = env.fromSource(source, WatermarkStrategy.noWatermarks(), "Kafka Source")
        .flatMap(new LineSplitter())
        .keyBy(value -> value.f0)
        .window(TumblingProcessingTimeWindows.of(Time.seconds(5)))
        .sum(1);
```

You can see the output by printing the data stream.

```java
stream.print();
```

If you produce message to the input topic from your
[console](https://console.upstash.com), you will see the output like this:

```
2> (This,1)
1> (an,1)
3> (is,1)
2> (sentence,1)
4> (example,1)
```

Next, create a Kafka producer to sink the data stream to output Kafka topic.

```java
KafkaSink sink = KafkaSink.<String>builder()
        .setKafkaProducerConfig(props)
        .setDeliveryGuarantee(DeliveryGuarantee.EXACTLY_ONCE)
        .setTransactionalIdPrefix("integ")
        .setRecordSerializer(new CustomSerializationSchema(outputTopic))
        .build();
stream.sinkTo(sink);
```

Finally, execute the Stream execution environment that was retrieved and run it.

```java
env.execute();
```


# Upstash Kafka with Decodable

This tutorial shows how to integrate Upstash Kafka with Decodable

[Decodable](https://www.decodable.co/product?utm_source=upstash) is a platform
which enables developers to build data pipelines using SQL. It is built on
Apache Flink under the hood to provide a seamless experience, while abstracting
away the underlying complexity. In this post, we will show how to connect an
Upstash Kafka topic to Decodable to streamline messages from Kafka to Decodable.

## Upstash Kafka Setup

Create a Kafka cluster using
[Upstash Console](https://console.upstash.com/kafka) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

## Decodable Setup

Just like Upstash, Decodable is a managed service that means you do not need to
host or provision anything. You can easily register for free and start using it.

After creating your account, click on `Connections` and `New Connection`. Select
`Apache Kafka`. Then:

*   Select Source as connection type.
*   Select **SASL\_SSL** as security protocol and **SCRAM-SHA-256** as SASL
    mechanism.
*   Enter your topic, SASL username, SASL password. You can find all those from
    Upstash console.
*   Value format should be JSON.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec1.png" />
</Frame>

In the next step, click on `New Stream` and give a name to it.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec2.png" />
</Frame>

In the schema screen, add `country`, `city`, `region` and `url` with `string`
type.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec3.png" />
</Frame>

Give a name to your connection and click `Create Connection`. In the next screen
click on Start.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec4.png" />
</Frame>

## Test the Setup

Now, let's some events to our Kafka topic. Go to Upstash console, click on your
cluster then `Topics`, click `mytopic`. Select `Messages` tab then click
`Produce a new message`. Send a message in JSON format like the below:

```json
{
  "country": "US",
  "city": "San Jose",
  "region": "CA",
  "url": "https://upstash.com"
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec41.png" />
</Frame>

Now, go back to Decodable console, click Streams and select the one you have
created. Then click `Run Preview`. You should see something like:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/decodable/dec5.png" />
</Frame>

## Links

[Decodable documentation](https://docs.decodable.co/docs)

[Decodable console](https://app.decodable.co/)

[Upstash console](https://console.upstash.com/kafka)


# Upstash Kafka with Apache Flink

This tutorial shows how to integrate Upstash Kafka with Apache Flink

[Apache Flink](https://flink.apache.org/) is a distributed processing engine
which can process streaming data.

### Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create two topics by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name first topic
“input”, since we are going to stream this topic to other one, which we can name
it as “output”.

### Project Setup

<Info>
  If you already have a project and want to implement Upstash Kafka and Apache
  Flink integration into it, you can skip this section and continue with [Add
  Apache Flink and Kafka into the
  Project](#add-apache-flink-and-kafka-into-the-project).
</Info>

Install Maven to your machine by following [Maven Installation Guide](https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html).

Run `mvn –version` in a terminal or in a command prompt to make sure you have
Maven downloaded.

It should print out the version of the Maven you have:

```
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3\apache-maven\bin\..
Java version: 1.8.0_232, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
```

To create the Maven project;

Go into the folder that you want to create the project in your terminal or
command prompt by running `cd <folder path>`

Run the following command:

```
mvn archetype:generate -DgroupId=com.kafkaflinkinteg.app -DartifactId=kafkaflinkinteg-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
```

### Add Apache Flink and Kafka into the Project

Open the project folder by using an IDE which has maven plugin such as Intellij,
Visual Studio, Eclipse etc. Add following Apache Flink dependencies into the
dependencies tag in `pom.xml` file.

```xml
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-connector-kafka -->
<dependency>
 <groupId>org.apache.flink</groupId>
 <artifactId>flink-connector-kafka</artifactId>
 <version>1.16.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-connector-base -->
<dependency>
 <groupId>org.apache.flink</groupId>
 <artifactId>flink-connector-base</artifactId>
 <version>1.16.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-java -->
<dependency>
 <groupId>org.apache.flink</groupId>
 <artifactId>flink-streaming-java</artifactId>
 <version>1.16.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-clients -->
<dependency>
 <groupId>org.apache.flink</groupId>
 <artifactId>flink-clients</artifactId>
 <version>1.16.0</version>
</dependency>
```

### Streaming From One Topic to Another Topic

You need to create 2 more classes (LineSplitter, CustomSerializationSchema) for
word count example.

#### LineSplitter

This class will be custom implementation of FlatMapFunction from Apache Flink
client library. It takes a sentence, splits into words and returns a
two-dimensional Tuple in format: `(<word>, 1)`.

Create LineSplitter class as following.

```java
package com.kafkaflinkinteg.app;

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.util.Collector;

/**
 * Implements the string tokenizer that splits sentences into words as a user-defined * FlatMapFunction. The function takes a line (String) and splits it into * multiple pairs in the form of "(word,1)" (Tuple2<String, int>).
 */
public class LineSplitter  implements FlatMapFunction<String, Tuple2<String, int>> {

	@Override
	public void flatMap(String value, Collector<Tuple2<String, int>> out) {
		// normalize and split the line
		String[] tokens = value.toLowerCase().split("\\W+");

		// emit the pairs
		for (String token : tokens) {
	        if (token.length() > 0) {
	            out.collect(new Tuple2<String, int>(token, 1));
	        }
        }
    }
}
```

#### CustomSerializationSchema

This class will be a custom implementation of KafkaRecordSerializationSchema
from Apache Flink Kafka connector library. It will provide a schema for
serializing and converting data from two-dimensional Tuple, which will be the
output of word counting process, to Kafka record format.

Create CustomSerializationSchema class as following:

```java
package com.kafkaflinkinteg.app;

import org.apache.flink.api.common.serialization.SerializationSchema;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.connector.kafka.sink.KafkaRecordSerializationSchema;
import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.core.JsonProcessingException;
import org.apache.flink.shaded.jackson2.com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.kafka.clients.producer.ProducerRecord;

public class CustomSerializationSchema<T> implements KafkaRecordSerializationSchema<Tuple2<String, int>> {
	private String topic;
	private ObjectMapper mapper;

	public CustomSerializationSchema(String topic) {
		this.topic = topic;
	}

    @Override
    public void open(SerializationSchema.InitializationContext context, KafkaSinkContext sinkContext) throws Exception {
	    KafkaRecordSerializationSchema.super.open(context, sinkContext);
	}

    @Override
	public ProducerRecord<byte[], byte[]> serialize(Tuple2<String, int> stringintTuple2, KafkaSinkContext kafkaSinkContext, Long aLong) {
        byte[] k = null;
        byte[] v = null;
        if (mapper == null) {
	        mapper = new ObjectMapper();
	    }
        try {
	        k = mapper.writeValueAsBytes(stringintTuple2.f0);
	        v = mapper.writeValueAsBytes(stringintTuple2.f1);
	    } catch ( JsonProcessingException e) {
		    // error
		}
        return new ProducerRecord<>(topic, k,v);
	}
}
```

#### Integration

Import the following packages first:

```java
package com.kafkaflinkinteg.app;

import org.apache.flink.api.common.eventtime.WatermarkStrategy;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.connector.base.DeliveryGuarantee;
import org.apache.flink.connector.kafka.sink.KafkaSink;
import org.apache.flink.connector.kafka.source.KafkaSource;
import org.apache.flink.connector.kafka.source.enumerator.initializer.OffsetsInitializer;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows;
import org.apache.flink.streaming.api.windowing.time.Time;

import java.util.Properties;
```

Define the names of the topics you are going to work on:

```java
String inputTopic = "input";
String outputTopic = "output";
```

Create the following properties for Apache Flink Kafka connector and replace
`UPSTASH-KAFKA-*` placeholders with your cluster information.

```java
Properties props = new Properties();
props.put("transaction.timeout.ms", "90000"); // e.g., 2 hours
props.put("bootstrap.servers", "UPSTASH-KAFKA-ENDPOINT:9092");
props.put("sasl.mechanism", "SCRAM-SHA-256");
props.put("security.protocol", "SASL_SSL");
props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";");
```

Get the stream execution environment to create and execute the pipeline in it.

```java
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
```

Create the Kafka consumer.

```java
KafkaSource<String> source = KafkaSource.<String>builder()
        .setStartingOffsets(OffsetsInitializer.earliest())
        .setProperties(props)
        .setTopics(inputTopic)
        .setGroupId("my-group")
        .setValueOnlyDeserializer(new SimpleStringSchema())
        .build();
```

Implement the stream processing part, which will take the input sentence from
source and count words.

```java
DataStream<Tuple2<String, int>> stream = env.fromSource(source, WatermarkStrategy.noWatermarks(), "Kafka Source")
        .flatMap(new LineSplitter())
        .keyBy(value -> value.f0)
        .window(TumblingProcessingTimeWindows.of(Time.seconds(5)))
        .sum(1);
```

You can see the output by printing the data stream.

```java
stream.print();
```

If you produce message to the input topic from your
[console](https://console.upstash.com), you will see the output like this:

```
2> (This,1)
1> (an,1)
3> (is,1)
2> (sentence,1)
4> (example,1)
```

Next, create a Kafka producer to sink the data stream to output Kafka topic.

```java
KafkaSink sink = KafkaSink.<String>builder()
        .setKafkaProducerConfig(props)
        .setDeliveryGuarantee(DeliveryGuarantee.EXACTLY_ONCE)
        .setTransactionalIdPrefix("integ")
        .setRecordSerializer(new CustomSerializationSchema(outputTopic))
        .build();
stream.sinkTo(sink);
```

Finally, execute the Stream execution environment that was retrieved and run it.

```java
env.execute();
```


# Upstash Kafka with ksqlDB

This tutorial shows how to integrate Upstash Kafka with ksqlDB

[ksqlDB](https://www.confluent.io/product/ksqldb) is a SQL interface for
performing stream processing over the Kafka environment.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

## ksqlDB Setup

Upstash does not have a managed ksqlDB. Therefore, set up ksqlDB on a docker
container and replace UPSTASH-KAFKA-\* placeholders with your cluster
information.

First, download and install [Docker](https://www.docker.com/).

Create a `docker-compose.yml` file as below:

```yml
version: "2"

services:
  ksqldb-server:
    image: confluentinc/ksqldb-server:0.28.2
    hostname: ksqldb-server
    container_name: ksqldb-server
    ports:
      - "8088:8088"
    environment:
      KSQL_LISTENERS: "http://0.0.0.0:8088"
      KSQL_BOOTSTRAP_SERVERS: "UPSTASH_KAFKA_ENDPOINT"
      KSQL_SASL_MECHANISM: "SCRAM-SHA-256"
      KSQL_SECURITY_PROTOCOL: "SASL_SSL"
      KSQL_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.scram.ScramLoginModule required username="UPSTASH_KAFKA_USERNAME" password="UPSTASH_KAFKA_PASSWORD";'
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
  ksqldb-cli:
    image: confluentinc/ksqldb-cli:0.28.2
    container_name: ksqldb-cli
    depends_on:
      - ksqldb-server
    entrypoint: /bin/sh
    tty: true
```

Open your CLI, navigate to the folder that includes the docker-compose.yml file
you created and start ksqlDB by running `docker-compose up`.

When you check your Kafka cluster from
[console](https://console.upstash.com/kafka), you will see new topics created
after you start ksqlDB.

## Streaming From One Topic to Another Topic

Implementing a word count example project can be done with both ksqlDB CLI and
Java client. In both ways, it is going to be done by consecutive streams. The
operations of the process will be as
`receive input > split into array > convert to rows > count occurrences`.

### Using ksqlDB CLI

Start the ksqlDB CLI by running the following command:

```
docker exec -it ksqldb-cli ksql http://ksqldb-server:8088
```

Create the first stream, which reads from the "input" topic:

```
ksql> CREATE STREAM source_stream (sentence VARCHAR) WITH (kafka_topic='input', value_format='json', partitions=1);
```

Create the second stream, which reads from source\_stream, splits the string to
an array, and writes to the split\_stream topic.

```
ksql> CREATE STREAM split_stream AS SELECT regexp_split_to_array(sentence, ' ') as word_array FROM source_stream EMIT CHANGES;
```

Next, create the third stream, which reads from split\_stream created above,
converts word\_array to rows, and writes to explode\_stream.

```
ksql> CREATE STREAM explode_stream AS SELECT explode(word_array) as words FROM split_stream EMIT CHANGES;
```

Lastly, create a table, which will count the words’ occurrences and write it to
the "OUTPUT" topic.

```
ksql> CREATE TABLE output AS SELECT words as word, count(words) as occurrence FROM explode_stream GROUP BY words EMIT CHANGES;
```

You can check what you have created so far by running the following commands on
ksqlDB CLI.

```
ksql> show tables;

Table Name  | Kafka Topic	| Key Format	| Value Format	| Windowed
--------------------------------------------------------------------
OUTPUT  	| OUTPUT		| KAFKA  		| JSON			| false
--------------------------------------------------------------------

ksql> show streams;

Stream Name			| Kafka Topic					| Key Format	| Value Format	| Windowed
------------------------------------------------------------------------------------------
EXPLODE_STREAM		| EXPLODE_STREAM				| KAFKA			| JSON	| false
KSQL_PROCESSING_LOG | default_ksql_processing_log	| KAFKA			| JSON  | false
SOURCE_STREAM  		| input  						| KAFKA			| JSON  | false
SPLIT_STREAM  		| SPLIT_STREAM  				| KAFKA			| JSON  | false
------------------------------------------------------------------------------------------
```

### Using Java Client

#### Project Setup

> :pushpin: **Note** If you already have a project and want to implement Upstash
> Kafka and ksqlDB integration into it, you can skip this section and continue
> with
> [Add Ksqldb and Kafka into the Project](#add-ksqldb-and-kafka-into-the-project).

Install Maven to your machine by following
[Maven Installation Guide](https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html)

Run `mvn –version` in a terminal or a command prompt to make sure you have Maven
downloaded.

It should print out the version of the Maven you have:

```
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3\apache-maven\bin\..
Java version: 1.8.0_232, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
```

To create the Maven project;

Go into the folder that you want to create the project in your terminal or
command prompt by running `cd <folder path>`

Run the following command:

```
mvn archetype:generate -DgroupId=com.kafkaksqldbinteg.app -DartifactId=kafkaksqldbinteg-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
```

#### Add ksqlDB and Kafka into the Project

Open the project folder using an IDE with maven plugins such as Intellij, Visual
Studio, Eclipse, etc. Add ksqlDB into the `pom.xml` file.

```xml
<repositories>
  <repository>
  <id>confluent</id>
  <name>confluent-repo</name>
  <url>http://packages.confluent.io/maven/</url>
  </repository>
</repositories>
<dependencies>
  <dependency>
  <groupId>io.confluent.ksql</groupId>
  <artifactId>ksqldb-api-client</artifactId>
  <version>7.3.0</version>
  </dependency>
</dependencies>
```

#### Streaming

Import the following packages.

```java
import io.confluent.ksql.api.client.Client;
import io.confluent.ksql.api.client.ClientOptions;
import io.confluent.ksql.api.client.ExecuteStatementResult;

import java.util.concurrent.CompletableFuture;
```

Create a ksqlDB client first.

```java
String KSQLDB_SERVER_HOST = "localhost";
int KSQLDB_SERVER_HOST_PORT = 8088;
ClientOptions options = ClientOptions.create()
  .setHost(KSQLDB_SERVER_HOST)
  .setPort(KSQLDB_SERVER_HOST_PORT);
Client client = Client.create(options);
```

Create the first stream, which reads from "input" topic:

```java
String SOURCE_STREAM = "CREATE STREAM IF NOT EXISTS source_stream (sentence VARCHAR)" +
        " WITH (kafka_topic='input', value_format='json', partitions=1);";
CompletableFuture<ExecuteStatementResult> result =
        client.executeStatement(SOURCE_STREAM);
System.out.println(result);
```

Create the second stream, which reads from source\_stream, split the string to an
array, and writes to the split\_stream topic.

```java
String SPLIT_STREAM = "CREATE STREAM IF NOT EXISTS split_stream " +
  "AS SELECT regexp_split_to_array(sentence, ' ') " +
  "as word_array FROM source_stream EMIT CHANGES;";
CompletableFuture<ExecuteStatementResult> result1 =
  client.executeStatement(SPLIT_STREAM);System.out.println(result1);
```

Next, create the third stream, which reads from split\_stream created above,
converts word\_array to rows, and writes to explode\_stream.

```java
String EXPLODE_STREAM = "CREATE STREAM IF NOT EXISTS explode_stream " +
  "AS SELECT explode(word_array) " +
  "as words FROM split_stream EMIT CHANGES;";
CompletableFuture<ExecuteStatementResult> result2 =
  client.executeStatement(EXPLODE_STREAM);System.out.println(result2);
```

Lastly, create a table, which will count the words’ occurrences and write it to
the "OUTPUT" topic.

```java
String OUTPUT_TABLE = "CREATE TABLE output " +
  "AS SELECT words as word, count(words) " +
  "as occurrence FROM explode_stream GROUP BY words EMIT CHANGES;";
CompletableFuture<ExecuteStatementResult> result3 =
  client.executeStatement(OUTPUT_TABLE);System.out.println(result3);
```

## Results

The word count stream we created above is taking input sentences in JSON format
from the "input" topic and sends word count results to the "OUTPUT" topic.

You can both send input and observe the output on
[console](https://console.upstash.com/kafka).

Send the input sentence to the "input" topic. The key can be a random string,
but since we defined "sentence" as a field while creating the `source_stream`,
the value must be a JSON that includes “sentence” as a key for this use case:

```json
{
“sentence”: “This is an example sentence”
}
```

Once you send this message to "input" topic, you can observe the result at
"OUTPUT" topic as following:

```
Timestamp				Key			Value
2022-12-06 23:39:56		This		{"OCCURRENCE":1}
2022-12-06 23:39:56 	is			{"OCCURRENCE":1}
2022-12-06 23:39:56 	an			{"OCCURRENCE":1}
2022-12-06 23:39:56		example 	{"OCCURRENCE":1}
2022-12-06 23:39:56		sentence	{"OCCURRENCE":1}
```


# Upstash Kafka with Materialize

This tutorial shows how to integrate Upstash Kafka with Materialize

[Materialize](https://materialize.com/docs/get-started/) is a PostgreSQL
wire-compatible stream database for low latency applications.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create two topics by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name first topic
`materialize_input`, since we are going to stream from this topic to Materialize
database. Name of the second topic can be `materialize_output`. This one is
going to receive stream from Materialize.

## Materialize Setup

Materialize is `wire-compatible` with PostgreSQL, that’s why it can be used with
most of the SQL clients.

[Sign up](https://materialize.com/register) and complete activation of your
Materialize account first.

Once you completed your activation, you can sign in and enable the region to run
Materialize database. It can provide better performance if you enable the same
region with location of your Upstash Kafka cluster.

Region setup takes a few minutes. During that time, create a new app password
from `Connect` tab for your project. This step will generate a password and
display it just once. You should copy that password to somewhere safe before it
disappears.

To interact with your Materialize database, you need to download one of the
PostgreSQL installers mentioned
[here](https://materialize.com/docs/get-started/quickstart/#before-you-begin).

After installing a PostgreSQL on your machine, open SQL shell, run the command
appeared on Connect tab to connect SQL Shell to Materialize database. You will
need to enter the app password to log in.

Now you are connected to your Materialize!

## Connect Materialize to Upstash Kafka

You first need to save Upstash username and password to Materialize’s secret
management system to be able to connect Materialize to Upstash Kafka.

To do this, run the following command from the psql terminal by replacing
`<upstash-username>` and `<upstash-password>` with the username and password you
see on your Upstash Kafka cluster:

```sql
CREATE SECRET upstash_username AS '<upstash-username>';

CREATE SECRET upstash_password AS '<upstash-password>';
```

`CREATE SECRET` command stores a sensitive value with the name assigned to it as
identifier. Once you define name and corresponding value with this command, you
will then be able to use the sensitive value by calling its name.

As the next step, we need to create a connection between Materialize and Upstash
Kafka by running following command from the psql terminal:

```sql
CREATE CONNECTION <connection-name> TO KAFKA (
  BROKER '<upstash-endpoint>',
  SASL MECHANISMS = 'SCRAM-SHA-256',
  SASL USERNAME = SECRET upstash_username,
  SASL PASSWORD = SECRET upstash_password
);
```

`<connection-name>` is the going to be used as the name of the connection. You
can name it as you wish.

`<upstash-endpoint>` is the endpoint of your Kafka. You can copy it from your
Upstash console.

Your connection is now established between Upstash Kafka and Materialize!

## Create Source

Source means streaming from external data source or pipeline to Materialize
database. By creating source, the message you add to the topic is going to be
streamed from Upstash Kafka to Materialize source.

You can create a source from SQL Shell first by running the following command:

```sql
CREATE SOURCE <source-name>
  FROM KAFKA CONNECTION <connection_name>  (TOPIC '<source-topic-name>')
  FORMAT BYTES
  WITH (SIZE = '3xsmall');
```

In this tutorial, we are going to use connection we established in the previous
section and use “materialized\_input” as source topic.

Once you created source, you can see it:

```sql
materialize=> SHOW SOURCES;
name                    |  type     |  size
------------------------+-----------+---------
upstash_source          | kafka     | 3xsmall
upstash_source_progress | subsource |
(2 rows)
```

To test this source, go to your
[Upstash console](https://console.upstash.com/kafka), open `materialize_input`
topic in your Kafka cluster.

Produce a message in this topic.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/materialize/send-message.png" />
</Frame>

The message you sent to this topic should be streamed to Materialize source.

Query the Materialize source from SQL Shell by converting it to a readable form
since we defined the source format as “BYTE” while creating the source.

```sql
materialize=> SELECT convert_from(data, 'utf8') as data from upstash_source;
data
-----------------------------
"This is my test sentence."
(1 row)
```

## Create Sink

Sink means streaming from Materialize database to external data stores or
pipelines. By creating a sink, the data you inserted to Materialize table or
source will be streamed to the Upstash Kafka topic.

For testing purposes, let's create a new table. This table will be streamed to
the Upstash Kafka sink topic.

```sql
materialize=> CREATE TABLE mytable (name text, age int);
CREATE TABLE

materialize=> SELECT * FROM mytable;
name | age
-----+-----
(0 rows)
```

Create a sink from SQL Shell by running the following command:

```sql
CREATE SINK <sink-name>
  FROM <source, table or mview>
  INTO KAFKA CONNECTION <connection-name> (TOPIC '<sink-topic-name>')
  FORMAT JSON
  ENVELOPE DEBEZIUM
  WITH (SIZE = '3xsmall');
```

We are going to use the connection we created and “materialize\_output” as sink
topic. We can also use the table named “mytable” we have just created.

Once you created sink, you can see it:

```sql
materialize=> SHOW SINKS;
name         | type  |  size
-------------+-------+---------
upstash_sink | kafka | 3xsmall
(1 row)
```

To test this sink, go to your
[Upstash console](https://console.upstash.com/kafka), open the output topic in
your Kafka cluster. Open messages tab to see incoming messages.

Now insert a new row to the table to be streamed:

```sql
materialize=> INSERT INTO mytable VALUES ('Noah', 1);
INSERT 0 1
materialize=> SELECT * FROM mytable;
name | age
-----+-----
Noah |  1
(1 row)
```

You can see this row streamed to the Upstash Kafka output topic on your Upstash
console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/materialize/receive-message.png" />
</Frame>


# Upstash Kafka with Apache Pinot

This tutorial shows how to integrate Upstash Kafka with Apache Pinot

[Apache Pinot](https://pinot.apache.org/) is a real-time distributed OLAP
(Online Analytical Processing) data store. It aims to make users able to execute
OLAP queries with low latency. It can consume the data from batch data sources
or streaming sources, which can be Upstash Kafka.

## Upstash Kafka Setup

Create a Kafka cluster using
[Upstash Console](https://console.upstash.com/kafka) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create one topic by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). This topic is going to
be source for Apache Pinot table. Let’s name it “transcript” for this example
tutorial.

## Apache Pinot Setup

You need a host to run Apache Pinot. For this quick setup, you can run it on
your local machine.

First, download [Docker](https://www.docker.com/). Running in docker container
is much better option for running Apache Pinot than running it locally.

Once you have docker on your machine, you can follow the steps on
[Getting Started](https://docs.pinot.apache.org/basics/getting-started/running-pinot-in-docker)
run Apache Pinot in docker.

In short, you will need to pull the Apache Pinot image by running following
command.

```
docker pull apachepinot/pinot:latest
```

Create a file named docker-compose.yml with the following content.

```yml
version: "3.7"
services:
  pinot-zookeeper:
    image: zookeeper:3.5.6
    container_name: pinot-zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
  pinot-controller:
    image: apachepinot/pinot:0.12.0
    command: "StartController -zkAddress pinot-zookeeper:2181"
    container_name: pinot-controller
    restart: unless-stopped
    ports:
      - "9000:9000"
    environment:
      JAVA_OPTS: "-Dplugins.dir=/opt/pinot/plugins -Xms1G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-controller.log"
    depends_on:
      - pinot-zookeeper
  pinot-broker:
    image: apachepinot/pinot:0.12.0
    command: "StartBroker -zkAddress pinot-zookeeper:2181"
    restart: unless-stopped
    container_name: "pinot-broker"
    ports:
      - "8099:8099"
    environment:
      JAVA_OPTS: "-Dplugins.dir=/opt/pinot/plugins -Xms4G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-broker.log"
    depends_on:
      - pinot-controller
  pinot-server:
    image: apachepinot/pinot:0.12.0
    command: "StartServer -zkAddress pinot-zookeeper:2181"
    restart: unless-stopped
    container_name: "pinot-server"
    ports:
      - "8098:8098"
    environment:
      JAVA_OPTS: "-Dplugins.dir=/opt/pinot/plugins -Xms4G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-server.log"
    depends_on:
      - pinot-broker
```

Go into the directory from your terminal and run the following command to start
Pinot.

```
docker-compose --project-name pinot-demo up
```

Now, Apache Pinot should be up and running. You can check it by running:

```
docker container ls
```

You should see the output like this:

```
CONTAINER ID   IMAGE                     COMMAND                  CREATED              STATUS              PORTS                                                                     NAMES
ba5cb0868350   apachepinot/pinot:0.9.3   "./bin/pinot-admin.s…"   About a minute ago   Up About a minute   8096-8099/tcp, 9000/tcp                                                   pinot-server
698f160852f9   apachepinot/pinot:0.9.3   "./bin/pinot-admin.s…"   About a minute ago   Up About a minute   8096-8098/tcp, 9000/tcp, 0.0.0.0:8099->8099/tcp, :::8099->8099/tcp        pinot-broker
b1ba8cf60d69   apachepinot/pinot:0.9.3   "./bin/pinot-admin.s…"   About a minute ago   Up About a minute   8096-8099/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp                  pinot-controller
54e7e114cd53   zookeeper:3.5.6           "/docker-entrypoint.…"   About a minute ago   Up About a minute   2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, :::2181->2181/tcp, 8080/tcp   pinot-zookeeper
```

Now, you should add table to your Pinot to store the data streamed from Kafka
topic.

You need to open [http://localhost:9000/](http://localhost:9000/) on your
browser.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/pinot-opened.png" />
</Frame>

Click on “Tables” section.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/table-page-opened.png" />
</Frame>

First, click on “Add Schema” and fill it until you see the following JSON as
your schema config.

```json
{
  "schemaName": "transcript",
  "dimensionFieldSpecs": [
    {
      "name": "studentID",
      "dataType": "INT"
    },
    {
      "name": "firstName",
      "dataType": "STRING"
    },
    {
      "name": "lastName",
      "dataType": "STRING"
    },
    {
      "name": "gender",
      "dataType": "STRING"
    },
    {
      "name": "subject",
      "dataType": "STRING"
    }
  ],
  "metricFieldSpecs": [
    {
      "name": "score",
      "dataType": "FLOAT"
    }
  ],
  "dateTimeFieldSpecs": [
    {
      "name": "timestamp",
      "dataType": "LONG",
      "format": "1:MILLISECONDS:EPOCH",
      "granularity": "1:MILLISECONDS"
    }
  ]
}
```

Click save and click to “Add Realtime Table” since we will stream the data
real-time.

On this page, table name must be the same name with the schema name, which is
“transcript” in this case.

Then, go below on this page and replace “segmentsConfig” and “tableIndexConfig”
sections in the table config on your browser with the following JSON. Do not
forget to replace UPSTASH-KAFKA-\* placeholders with your cluster information.

```json
{
  "segmentsConfig": {
    "timeColumnName": "timestampInEpoch",
    "timeType": "MILLISECONDS",
    "schemaName": "transcript",
    "replicasPerPartition": "1",
    "replication": "1"
  },
  "tableIndexConfig": {
    "loadMode": "MMAP",
    "streamConfigs": {
      "streamType": "kafka",
      "stream.kafka.consumer.type": "lowlevel",
      "stream.kafka.topic.name": "transcript",
      "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
      "stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
      "stream.kafka.broker.list": "UPSTASH-KAFKA-ENDPOINT:9092",
      "security.protocol": "SASL_SSL",
      "sasl.jaas.config": "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";",
      "sasl.mechanism": "SCRAM-SHA-256",
      "realtime.segment.flush.threshold.rows": "0",
      "realtime.segment.flush.threshold.time": "24h",
      "realtime.segment.flush.threshold.segment.size": "50M",
      "stream.kafka.consumer.prop.auto.offset.reset": "smallest"
    }
  }
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/table-created.png" />
</Frame>

## Test the Setup

Now, let's create some events to our Kafka topic. Go to Upstash console, click
on your cluster then Topics, click “transcript”. Select Messages tab then click
Produce a new message. Send a message in JSON format like the below:

```json
{
  "studentID": 205,
  "firstName": "Natalie",
  "lastName": "Jones",
  "gender": "Female",
  "subject": "Maths",
  "score": 3.8,
  "timestampInEpoch": 1571900400000
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/send-message.png" />
</Frame>

Now, go back to your Pinot console on your browser. Navigate to “Query Console”
from the left side bar. When you click on “transcript” table, you will see the
result of the following query automatically.

```sql
select * from transcript limit 10
```

The query result should be as following:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/query-result.png" />
</Frame>

## Links

[Running Pinot in Docker](https://docs.pinot.apache.org/basics/getting-started/running-pinot-in-docker)

[Apache Pinot Stream Ingestion](https://docs.pinot.apache.org/basics/data-import/pinot-stream-ingestion)


# Upstash Kafka with Apache Spark

This tutorial shows how to integrate Upstash Kafka with Apache Spark

[Apache Spark](https://spark.apache.org/) is a multi-language engine for
executing data engineering, data science, and machine learning on single-node
machines or clusters.

### Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com/) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create a topic by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name the topic
“sentence”.

### Project Setup

<Info>
  If you already have a project and want to implement Upstash Kafka and Apache
  Spark integration into it, you can skip this section and continue with [Add
  Spark and Kafka into the Project](#add-spark-and-kafka-into-the-project).
</Info>

Install Maven to your machine by following [Maven Installation Guide](https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html).

Run `mvn –version` in a terminal or in a command prompt to make sure you have
Maven downloaded.

It should print out the version of the Maven you have:

```
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3\apache-maven\bin\..
Java version: 1.8.0_232, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
```

To create the Maven project;

Go into the folder that you want to create the project in your terminal or
command prompt by running `cd <folder path>`

Run the following command:

```
mvn archetype:generate -DgroupId=com.kafkasparkinteg.app -DartifactId=kafkasparkinteg-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
```

### Add Spark and Kafka into the Project

Open the project folder by using an IDE which has maven plugin such as Intellij,
Visual Studio, Eclipse etc. Add following Spark dependencies into the
dependencies tag in `pom.xml` file.

```xml
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.12</artifactId>
<version>3.3.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.12</artifactId>
<version>3.3.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.12</artifactId>
<version>3.3.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka-0-10 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-10_2.12</artifactId>
<version>3.3.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql-kafka-0-10 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql-kafka-0-10_2.12</artifactId>
<version>3.3.1</version>
</dependency>
```

### Using Apache Spark as Producer

Import the following packages first:

```java
import org.apache.spark.api.java.function.FlatMapFunction;
import org.apache.spark.sql.*;
import org.apache.spark.sql.types.DataTypes;
import org.apache.spark.sql.types.StructType;
import java.util.*;
```

To send messages to Kafka from Spark, use the following code after replacing the
`UPSTASH-KAFKA-*` placeholders with your cluster information:

```java
SparkSession spark = SparkSession.builder()
  .appName("quickstart")
  .config("spark.master", "local")
  .getOrCreate();

StructType structType = new StructType();
structType = structType.add("key", DataTypes.StringType, false);
structType = structType.add("value", DataTypes.StringType, false);

List<Row> rows = new ArrayList<Row>();
rows.add(RowFactory.create("test key", "This is an example sentence"));

Dataset<Row> sentenceDF = spark.createDataFrame(rows, structType);

sentenceDF.selectExpr("CAST(key AS STRING)", "CAST(value AS STRING)")
.write()
.format("kafka")
.option("kafka.bootstrap.servers", "UPSTASH-KAFKA-ENDPOINT:9092")
.option("kafka.sasl.mechanism", "SCRAM-SHA-256")
.option("kafka.security.protocol", "SASL_SSL")
.option("kafka.sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";")
.option("topic", "sentence")
.save();
```

Before running the project, open the messages of the topic from
[console](https://console.upstash.com).

You can observe new message coming to the topic on Upstash console when you run
your project.

### Using Apache Spark as Consumer

If the following packages are not imported, import them first:

```java
import org.apache.spark.api.java.function.FlatMapFunction;
import org.apache.spark.sql.*;
import org.apache.spark.sql.types.DataTypes;
import org.apache.spark.sql.types.StructType;

import java.util.*;
```

To receive the messages from Kafka topic by Apache Spark and to process, use the
following code after replacing the UPSTASH-KAFKA-\* placeholders with your
cluster information:

```java
SparkSession spark = SparkSession.builder()
  .appName("quickstart")
  .config("spark.master", "local")
  .getOrCreate();

Dataset<Row> lines = spark
  .read()
  .format("kafka")
  .option("kafka.bootstrap.servers", "UPSTASH-KAFKA-ENDPOINT:9092")
  .option("kafka.sasl.mechanism", "SCRAM-SHA-256")
  .option("kafka.security.protocol", "SASL_SSL")
  .option("kafka.sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";")
  .option("startingOffsets", "earliest")
  .option("subscribe", "sentence")
  .load()
  .selectExpr("CAST(key AS STRING)", "CAST(value AS STRING)");
// PROCESS RECEIVED MESSAGE - Word counting part
Dataset<String> words = lines.select("value")
  .as(Encoders.STRING())
  .flatMap( new FlatMapFunction<String, String>() { @Override public Iterator<String> call(String x) { return Arrays.asList(x.split(" ")).iterator(); }
  }, Encoders.STRING()); Dataset<Row> wordCounts = words.groupBy("value").count(); wordCounts.show();
```

You can verify that you can see the sentence, which you sent, on your console
with number of word occurrences:

```
+--------+-----+
|   value|count|
+--------+-----+
| example|    1|
|      is|    1|
|sentence|    1|
|      an|    1|
|    This|    1|
+--------+-----+
```


# Upstash Kafka with StarTree

This tutorial shows how to integrate Upstash Kafka with StarTree

[StarTree](https://startree.ai/) provides a fully managed, Apache Pinot based
real-time analytics database on its cloud environment.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create one topic by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). This topic will be the
source for the Apache Pinot table running on StarTree. Let’s name it
“transcript” for this example tutorial.

## StarTree Setup

To be able to use StarTree cloud, you first need to
[create an account](https://startree.ai/saas-signup).

There are two steps to initialize the cloud environment on StarTree. First, you
need to create an organization. Next, you need to create a workspace under this
new organization.

For these setup steps, you can also follow
[StarTree quickstart](https://dev.startree.ai/docs/startree-enterprise-edition/startree-cloud/getting-started/saas/how-to-guide).

## Connect StarTree Cloud to Upstash Kafka

Once you created your workspace, open Data Manager under the `Services` section
in your workspace. Data Manager is where we will connect Upstash Kafka and work
on the Pinot table.

To connect Upstash Kafka with StarTree, create a new connection in Data Manager.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/create-a-connection.png" />
</Frame>

As the connection type, select Kafka.

In Kafka connection settings, fill the following options:

*   Connection Name: It can be anything. It is up to you.

*   Broker Url: This should be the endpoint of your Upstash Kafka cluster. You can
    find it in the details section in your
    [Upstash Kafka cluster](https://console.upstash.com/kafka).

*   Authentication Type: `SASL`

*   Security Protocol: `SASL_SSL`

*   SASL Mechanism: `SCRAM-SHA-256`

*   Username: This should be the username given in the details section in your
    [Upstash Kafka cluster](https://console.upstash.com/kafka).

*   Password: This should be the password given in the details section in your
    [Upstash Kafka cluster](https://console.upstash.com/kafka).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/create-kafka-connection.png" />
</Frame>

To proceed, you need to test the connection first. Once the test connection is
successful, then you can create the connection.

Now you have a connection between Upstash Kafka and StarTree Cloud! The next
step is to create a dataset to store data streamed from Upstash Kafka.

Let’s return to the Data Manager overview page and create a new dataset.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/create-a-dataset.png" />
</Frame>

As the connection type, select Kafka again.

Now you can select the Kafka connection you created for connecting Upstash
Kafka.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/select-kafka-connection.png" />
</Frame>

In the next step, you need to name your dataset, provide the Kafka topic to be
the source of this new dataset and define the data format. We can give
“transcript” as the topic and select JSON as the data format.

To proceed to the next step, we must first produce a message in our Kafka topic.
StarTree doesn’t allow us to go to the next step before it validates the
connection is working, and data is being streamed correctly.

To make StarTree validate our connection, let’s turn back to the Upstash console
and create some events for our Kafka topic. To do this, click on your Kafka
cluster on Upstash console and go to the “Topics” section. Open the source
topic, which is “transcript” in this case. Select the Messages tab, then click
Produce a new message. Send a message in JSON format like the one below:

```json
{
  "studentID": 205,
  "firstName": "Natalie",
  "lastName": "Jones",
  "gender": "Female",
  "subject": "Maths",
  "score": 3.8,
  "timestampInEpoch": 1571900400000
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/produce-message.png" />
</Frame>

Now go back to the dataset details steps on StarTree Data Manager.

After you click next, StarTree will consume the message in the source Kafka
topic to verify the connection. Once it consumes the message, the message will
be displayed.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/sample-data.png" />
</Frame>

In the next step, StarTree extracts the data model from the message you sent.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/data-modeling.png" />
</Frame>

If there is any additional configuration about the model of the data coming from
the source topic, you can add it here.

To keep things simple, we will click next without changing anything.

The last step is for more configuration of your dataset. We will click next
again and proceed to review. Click “Create Dataset” to finalize the dataset.

## Query Data

Open the dataset you created on StarTree Data Manager and navigate to the query
console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/open-query-console.png" />
</Frame>

You will be redirected to Pinot Query Console running on StarTree cloud.

When you run the following SQL Query, you will see the data that came from
Upstash Kafka into your dataset.

```sql
select * from <Dataset-name> limit 10
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/query-result.png" />
</Frame>


# Upstash Kafka with Kafka Streams

This tutorial shows how to integrate Upstash Kafka with Kafka Streams

[Kafka Streams](https://kafka.apache.org/documentation/streams/) is a client
library, which streams data from one Kafka topic to another.

### Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create two topics by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name first topic
“input”, since we are going to stream this topic to other one, which we can name
it as “output”.

### Project Setup

<Note>
  If you already have a project and want to use Kafka Streams with Upstash Kafka
  in it, you can skip this section and continue with [Add Kafka Streams into the
  Project](#add-kafka-streams-into-the-project).
</Note>

Install Maven to your machine by following [Maven Installation Guide](https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html).

Run `mvn –version` in a terminal or in a command prompt to make sure you have
Maven downloaded.

It should print out the version of the Maven you have:

```
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3\apache-maven\bin\..
Java version: 1.8.0_232, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
```

To create the Maven project;

Go into the folder that you want to create the project in your terminal or
command prompt by running `cd <folder path>`

Run the following command:

```
mvn archetype:generate -DgroupId=com.kafkastreamsinteg.app -DartifactId=kafkastreamsinteg-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
```

### Add Kafka Streams into the Project

Open the project folder by using an IDE which has maven plugin such as Intellij,
Visual Studio, Eclipse etc. Add following dependencies into the dependencies tag
in `pom.xml` file.

```xml
<dependency>
  <groupId>org.apache.kafka</groupId>
  <artifactId>kafka-streams</artifactId>
  <version>3.3.1</version>
</dependency>

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-reload4j</artifactId>
  <version>2.0.3</version>
</dependency>
```

### Streaming From One Topic to Another Topic

Import the following packages first:

```java
import org.apache.kafka.common.config.SaslConfigs;
import org.apache.kafka.common.config.TopicConfig;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.common.utils.Bytes;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.Topology;
import org.apache.kafka.streams.kstream.Grouped;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.Materialized;
import org.apache.kafka.streams.kstream.Produced;
import org.apache.kafka.streams.state.KeyValueStore;

import java.util.Arrays;
import java.util.Properties;
import java.util.concurrent.CountDownLatch;
import java.util.regex.Pattern;
```

Define the names of the topics you are going to work on:

```java
String inputTopic = "input";
String outputTopic = "output";
```

Create the following properties for Kafka Streams and replace UPSTASH-KAFKA-\*
placeholders with your cluster information.

```java
final Properties props = new Properties();
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "UPSTASH-KAFKA-ENDPOINT:9092");
props.put(SaslConfigs.SASL_MECHANISM, "SCRAM-SHA-256");
props.put(StreamsConfig.SECURITY_PROTOCOL_CONFIG, "SASL_SSL");
props.put(StreamsConfig.APPLICATION_ID_CONFIG,"myLastNewProject");
props.put(SaslConfigs.SASL_JAAS_CONFIG, "org.apache.kafka.common.security.scram.ScramLoginModule username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";");
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.topicPrefix(TopicConfig.RETENTION_MS_CONFIG), 604800000); // 7 days for internal repartition topic retention period
props.put(StreamsConfig.topicPrefix(TopicConfig.CLEANUP_POLICY_CONFIG), TopicConfig.CLEANUP_POLICY_DELETE); // delete cleanup policy for internal repartition topic
props.put(StreamsConfig.topicPrefix(TopicConfig.RETENTION_BYTES_CONFIG), 268435456); // 256 MB for internal repartition topic retention size
```

Start the builder for streaming and assign input topic as the source:

```java
StreamsBuilder builder = new StreamsBuilder();
KStream<String, String> source = builder.stream(inputTopic);
```

Apply the following steps to count the words in the sentence sent to input topic
and stream the results to the output topic:

```java
Pattern pattern = Pattern.compile("\\W+", Pattern.UNICODE_CHARACTER_CLASS);
Materialized<String, Long, KeyValueStore<Bytes, byte[]>> materialized = Materialized.as("countMapping");
materialized.withLoggingDisabled();
source.flatMapValues(value -> Arrays.asList(pattern.split(value.toLowerCase())))
        .groupBy((key, word) -> word, Grouped.as("groupMapping"))
        .count(materialized).toStream().mapValues(Object::toString)
        .to(outputTopic, Produced.with(Serdes.String(), Serdes.String()));
```

Since “groupby” function causing repartition and creation of a new internal
topic to store the groups intermediately, be sure that there is enough partition
capacity on your Upstash Kafka. For detailed information about the max partition
capacity of Kafka cluster, check [plans](https://upstash.com/#section-pricing).

Just to be sure, you can check from topic section on
[console](https://console.upstash.com) if the internal repartition topic created
successfully when you run your application and send data to input topic. For
reference, naming convention for internal repartition topics:

```
<application id><group name><repartition>
```

Next, finalize and build the streams builder. Create a topology of your process.
It can be viewed by printing.

```java
final Topology topology = builder.build();
System.out.println(topology.describe());
```

Here is the example topology in this scenario:

```
Topologies:

Sub-topology: 0
Source: KSTREAM-SOURCE-0000000000 (topics: [input])
--> KSTREAM-FLATMAPVALUES-0000000001
Processor: KSTREAM-FLATMAPVALUES-0000000001 (stores: [])
--> groupMapping
<-- KSTREAM-SOURCE-0000000000
Processor: groupMapping (stores: [])
--> groupMapping-repartition-filter
<-- KSTREAM-FLATMAPVALUES-0000000001
Processor: groupMapping-repartition-filter (stores: [])
--> groupMapping-repartition-sink
<-- groupMapping
Sink: groupMapping-repartition-sink (topic: groupMapping-repartition)
<-- groupMapping-repartition-filter

Sub-topology: 1
Source: groupMapping-repartition-source (topics: [groupMapping-repartition])
--> KSTREAM-AGGREGATE-0000000003
Processor: KSTREAM-AGGREGATE-0000000003 (stores: [countMapping])
--> KTABLE-TOSTREAM-0000000007
<-- groupMapping-repartition-source
Processor: KTABLE-TOSTREAM-0000000007 (stores: [])
--> KSTREAM-MAPVALUES-0000000008
<-- KSTREAM-AGGREGATE-0000000003
Processor: KSTREAM-MAPVALUES-0000000008 (stores: [])
--> KSTREAM-SINK-0000000009
<-- KTABLE-TOSTREAM-0000000007
Sink: KSTREAM-SINK-0000000009 (topic: output)
<-- KSTREAM-MAPVALUES-0000000008
```

Finally, start the Kafka Streams that was built and run it.

```java
final KafkaStreams streams = new KafkaStreams(topology, props);
final CountDownLatch latch = new CountDownLatch(1);
try {
	streams.start();
	System.out.println("streams started");
	latch.await();
} catch (final Throwable e) {
    System.exit(1);
}

Runtime.getRuntime().addShutdownHook(new Thread("streams-word-count") {
    @Override
	public void run() {
	    streams.close();
		latch.countDown();
	}
});
```


# ksqlDB

This tutorial shows how to integrate Upstash Kafka with ksqlDB

[ksqlDB](https://www.confluent.io/product/ksqldb) is a SQL interface for
performing stream processing over the Kafka environment.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

## ksqlDB Setup

Upstash does not have a managed ksqlDB. Therefore, set up ksqlDB on a docker
container and replace UPSTASH-KAFKA-\* placeholders with your cluster
information.

First, download and install [Docker](https://www.docker.com/).

Create a `docker-compose.yml` file as below:

```yml
version: "2"

services:
  ksqldb-server:
    image: confluentinc/ksqldb-server:0.28.2
    hostname: ksqldb-server
    container_name: ksqldb-server
    ports:
      - "8088:8088"
    environment:
      KSQL_LISTENERS: "http://0.0.0.0:8088"
      KSQL_BOOTSTRAP_SERVERS: "UPSTASH_KAFKA_ENDPOINT"
      KSQL_SASL_MECHANISM: "SCRAM-SHA-256"
      KSQL_SECURITY_PROTOCOL: "SASL_SSL"
      KSQL_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.scram.ScramLoginModule required username="UPSTASH_KAFKA_USERNAME" password="UPSTASH_KAFKA_PASSWORD";'
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
  ksqldb-cli:
    image: confluentinc/ksqldb-cli:0.28.2
    container_name: ksqldb-cli
    depends_on:
      - ksqldb-server
    entrypoint: /bin/sh
    tty: true
```

Open your CLI, navigate to the folder that includes the docker-compose.yml file
you created and start ksqlDB by running `docker-compose up`.

When you check your Kafka cluster from
[console](https://console.upstash.com/kafka), you will see new topics created
after you start ksqlDB.

## Streaming From One Topic to Another Topic

Implementing a word count example project can be done with both ksqlDB CLI and
Java client. In both ways, it is going to be done by consecutive streams. The
operations of the process will be as
`receive input > split into array > convert to rows > count occurrences`.

### Using ksqlDB CLI

Start the ksqlDB CLI by running the following command:

```
docker exec -it ksqldb-cli ksql http://ksqldb-server:8088
```

Create the first stream, which reads from the "input" topic:

```
ksql> CREATE STREAM source_stream (sentence VARCHAR) WITH (kafka_topic='input', value_format='json', partitions=1);
```

Create the second stream, which reads from source\_stream, splits the string to
an array, and writes to the split\_stream topic.

```
ksql> CREATE STREAM split_stream AS SELECT regexp_split_to_array(sentence, ' ') as word_array FROM source_stream EMIT CHANGES;
```

Next, create the third stream, which reads from split\_stream created above,
converts word\_array to rows, and writes to explode\_stream.

```
ksql> CREATE STREAM explode_stream AS SELECT explode(word_array) as words FROM split_stream EMIT CHANGES;
```

Lastly, create a table, which will count the words’ occurrences and write it to
the "OUTPUT" topic.

```
ksql> CREATE TABLE output AS SELECT words as word, count(words) as occurrence FROM explode_stream GROUP BY words EMIT CHANGES;
```

You can check what you have created so far by running the following commands on
ksqlDB CLI.

```
ksql> show tables;

Table Name  | Kafka Topic	| Key Format	| Value Format	| Windowed
--------------------------------------------------------------------
OUTPUT  	| OUTPUT		| KAFKA  		| JSON			| false
--------------------------------------------------------------------

ksql> show streams;

Stream Name			| Kafka Topic					| Key Format	| Value Format	| Windowed
------------------------------------------------------------------------------------------
EXPLODE_STREAM		| EXPLODE_STREAM				| KAFKA			| JSON	| false
KSQL_PROCESSING_LOG | default_ksql_processing_log	| KAFKA			| JSON  | false
SOURCE_STREAM  		| input  						| KAFKA			| JSON  | false
SPLIT_STREAM  		| SPLIT_STREAM  				| KAFKA			| JSON  | false
------------------------------------------------------------------------------------------
```

### Using Java Client

#### Project Setup

> :pushpin: **Note** If you already have a project and want to implement Upstash
> Kafka and ksqlDB integration into it, you can skip this section and continue
> with
> [Add Ksqldb and Kafka into the Project](#add-ksqldb-and-kafka-into-the-project).

Install Maven to your machine by following
[Maven Installation Guide](https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html)

Run `mvn –version` in a terminal or a command prompt to make sure you have Maven
downloaded.

It should print out the version of the Maven you have:

```
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3\apache-maven\bin\..
Java version: 1.8.0_232, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
```

To create the Maven project;

Go into the folder that you want to create the project in your terminal or
command prompt by running `cd <folder path>`

Run the following command:

```
mvn archetype:generate -DgroupId=com.kafkaksqldbinteg.app -DartifactId=kafkaksqldbinteg-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
```

#### Add ksqlDB and Kafka into the Project

Open the project folder using an IDE with maven plugins such as Intellij, Visual
Studio, Eclipse, etc. Add ksqlDB into the `pom.xml` file.

```xml
<repositories>
  <repository>
  <id>confluent</id>
  <name>confluent-repo</name>
  <url>http://packages.confluent.io/maven/</url>
  </repository>
</repositories>
<dependencies>
  <dependency>
  <groupId>io.confluent.ksql</groupId>
  <artifactId>ksqldb-api-client</artifactId>
  <version>7.3.0</version>
  </dependency>
</dependencies>
```

#### Streaming

Import the following packages.

```java
import io.confluent.ksql.api.client.Client;
import io.confluent.ksql.api.client.ClientOptions;
import io.confluent.ksql.api.client.ExecuteStatementResult;

import java.util.concurrent.CompletableFuture;
```

Create a ksqlDB client first.

```java
String KSQLDB_SERVER_HOST = "localhost";
int KSQLDB_SERVER_HOST_PORT = 8088;
ClientOptions options = ClientOptions.create()
  .setHost(KSQLDB_SERVER_HOST)
  .setPort(KSQLDB_SERVER_HOST_PORT);
Client client = Client.create(options);
```

Create the first stream, which reads from "input" topic:

```java
String SOURCE_STREAM = "CREATE STREAM IF NOT EXISTS source_stream (sentence VARCHAR)" +
        " WITH (kafka_topic='input', value_format='json', partitions=1);";
CompletableFuture<ExecuteStatementResult> result =
        client.executeStatement(SOURCE_STREAM);
System.out.println(result);
```

Create the second stream, which reads from source\_stream, split the string to an
array, and writes to the split\_stream topic.

```java
String SPLIT_STREAM = "CREATE STREAM IF NOT EXISTS split_stream " +
  "AS SELECT regexp_split_to_array(sentence, ' ') " +
  "as word_array FROM source_stream EMIT CHANGES;";
CompletableFuture<ExecuteStatementResult> result1 =
  client.executeStatement(SPLIT_STREAM);System.out.println(result1);
```

Next, create the third stream, which reads from split\_stream created above,
converts word\_array to rows, and writes to explode\_stream.

```java
String EXPLODE_STREAM = "CREATE STREAM IF NOT EXISTS explode_stream " +
  "AS SELECT explode(word_array) " +
  "as words FROM split_stream EMIT CHANGES;";
CompletableFuture<ExecuteStatementResult> result2 =
  client.executeStatement(EXPLODE_STREAM);System.out.println(result2);
```

Lastly, create a table, which will count the words’ occurrences and write it to
the "OUTPUT" topic.

```java
String OUTPUT_TABLE = "CREATE TABLE output " +
  "AS SELECT words as word, count(words) " +
  "as occurrence FROM explode_stream GROUP BY words EMIT CHANGES;";
CompletableFuture<ExecuteStatementResult> result3 =
  client.executeStatement(OUTPUT_TABLE);System.out.println(result3);
```

## Results

The word count stream we created above is taking input sentences in JSON format
from the "input" topic and sends word count results to the "OUTPUT" topic.

You can both send input and observe the output on
[console](https://console.upstash.com/kafka).

Send the input sentence to the "input" topic. The key can be a random string,
but since we defined "sentence" as a field while creating the `source_stream`,
the value must be a JSON that includes “sentence” as a key for this use case:

```json
{
“sentence”: “This is an example sentence”
}
```

Once you send this message to "input" topic, you can observe the result at
"OUTPUT" topic as following:

```
Timestamp				Key			Value
2022-12-06 23:39:56		This		{"OCCURRENCE":1}
2022-12-06 23:39:56 	is			{"OCCURRENCE":1}
2022-12-06 23:39:56 	an			{"OCCURRENCE":1}
2022-12-06 23:39:56		example 	{"OCCURRENCE":1}
2022-12-06 23:39:56		sentence	{"OCCURRENCE":1}
```


# Materialize

This tutorial shows how to integrate Upstash Kafka with Materialize

[Materialize](https://materialize.com/docs/get-started/) is a PostgreSQL
wire-compatible stream database for low latency applications.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create two topics by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name first topic
`materialize_input`, since we are going to stream from this topic to Materialize
database. Name of the second topic can be `materialize_output`. This one is
going to receive stream from Materialize.

## Materialize Setup

Materialize is `wire-compatible` with PostgreSQL, that’s why it can be used with
most of the SQL clients.

[Sign up](https://materialize.com/register) and complete activation of your
Materialize account first.

Once you completed your activation, you can sign in and enable the region to run
Materialize database. It can provide better performance if you enable the same
region with location of your Upstash Kafka cluster.

Region setup takes a few minutes. During that time, create a new app password
from `Connect` tab for your project. This step will generate a password and
display it just once. You should copy that password to somewhere safe before it
disappears.

To interact with your Materialize database, you need to download one of the
PostgreSQL installers mentioned
[here](https://materialize.com/docs/get-started/quickstart/#before-you-begin).

After installing a PostgreSQL on your machine, open SQL shell, run the command
appeared on Connect tab to connect SQL Shell to Materialize database. You will
need to enter the app password to log in.

Now you are connected to your Materialize!

## Connect Materialize to Upstash Kafka

You first need to save Upstash username and password to Materialize’s secret
management system to be able to connect Materialize to Upstash Kafka.

To do this, run the following command from the psql terminal by replacing
`<upstash-username>` and `<upstash-password>` with the username and password you
see on your Upstash Kafka cluster:

```sql
CREATE SECRET upstash_username AS '<upstash-username>';

CREATE SECRET upstash_password AS '<upstash-password>';
```

`CREATE SECRET` command stores a sensitive value with the name assigned to it as
identifier. Once you define name and corresponding value with this command, you
will then be able to use the sensitive value by calling its name.

As the next step, we need to create a connection between Materialize and Upstash
Kafka by running following command from the psql terminal:

```sql
CREATE CONNECTION <connection-name> TO KAFKA (
  BROKER '<upstash-endpoint>',
  SASL MECHANISMS = 'SCRAM-SHA-256',
  SASL USERNAME = SECRET upstash_username,
  SASL PASSWORD = SECRET upstash_password
);
```

`<connection-name>` is the going to be used as the name of the connection. You
can name it as you wish.

`<upstash-endpoint>` is the endpoint of your Kafka. You can copy it from your
Upstash console.

Your connection is now established between Upstash Kafka and Materialize!

## Create Source

Source means streaming from external data source or pipeline to Materialize
database. By creating source, the message you add to the topic is going to be
streamed from Upstash Kafka to Materialize source.

You can create a source from SQL Shell first by running the following command:

```sql
CREATE SOURCE <source-name>
  FROM KAFKA CONNECTION <connection_name>  (TOPIC '<source-topic-name>')
  FORMAT BYTES
  WITH (SIZE = '3xsmall');
```

In this tutorial, we are going to use connection we established in the previous
section and use “materialized\_input” as source topic.

Once you created source, you can see it:

```sql
materialize=> SHOW SOURCES;
name                    |  type     |  size
------------------------+-----------+---------
upstash_source          | kafka     | 3xsmall
upstash_source_progress | subsource |
(2 rows)
```

To test this source, go to your
[Upstash console](https://console.upstash.com/kafka), open `materialize_input`
topic in your Kafka cluster.

Produce a message in this topic.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/materialize/send-message.png" />
</Frame>

The message you sent to this topic should be streamed to Materialize source.

Query the Materialize source from SQL Shell by converting it to a readable form
since we defined the source format as “BYTE” while creating the source.

```sql
materialize=> SELECT convert_from(data, 'utf8') as data from upstash_source;
data
-----------------------------
"This is my test sentence."
(1 row)
```

## Create Sink

Sink means streaming from Materialize database to external data stores or
pipelines. By creating a sink, the data you inserted to Materialize table or
source will be streamed to the Upstash Kafka topic.

For testing purposes, let's create a new table. This table will be streamed to
the Upstash Kafka sink topic.

```sql
materialize=> CREATE TABLE mytable (name text, age int);
CREATE TABLE

materialize=> SELECT * FROM mytable;
name | age
-----+-----
(0 rows)
```

Create a sink from SQL Shell by running the following command:

```sql
CREATE SINK <sink-name>
  FROM <source, table or mview>
  INTO KAFKA CONNECTION <connection-name> (TOPIC '<sink-topic-name>')
  FORMAT JSON
  ENVELOPE DEBEZIUM
  WITH (SIZE = '3xsmall');
```

We are going to use the connection we created and “materialize\_output” as sink
topic. We can also use the table named “mytable” we have just created.

Once you created sink, you can see it:

```sql
materialize=> SHOW SINKS;
name         | type  |  size
-------------+-------+---------
upstash_sink | kafka | 3xsmall
(1 row)
```

To test this sink, go to your
[Upstash console](https://console.upstash.com/kafka), open the output topic in
your Kafka cluster. Open messages tab to see incoming messages.

Now insert a new row to the table to be streamed:

```sql
materialize=> INSERT INTO mytable VALUES ('Noah', 1);
INSERT 0 1
materialize=> SELECT * FROM mytable;
name | age
-----+-----
Noah |  1
(1 row)
```

You can see this row streamed to the Upstash Kafka output topic on your Upstash
console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/materialize/receive-message.png" />
</Frame>


# Apache Pinot

This tutorial shows how to integrate Upstash Kafka with Apache Pinot

[Apache Pinot](https://pinot.apache.org/) is a real-time distributed OLAP
(Online Analytical Processing) data store. It aims to make users able to execute
OLAP queries with low latency. It can consume the data from batch data sources
or streaming sources, which can be Upstash Kafka.

## Upstash Kafka Setup

Create a Kafka cluster using
[Upstash Console](https://console.upstash.com/kafka) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create one topic by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). This topic is going to
be source for Apache Pinot table. Let’s name it “transcript” for this example
tutorial.

## Apache Pinot Setup

You need a host to run Apache Pinot. For this quick setup, you can run it on
your local machine.

First, download [Docker](https://www.docker.com/). Running in docker container
is much better option for running Apache Pinot than running it locally.

Once you have docker on your machine, you can follow the steps on
[Getting Started](https://docs.pinot.apache.org/basics/getting-started/running-pinot-in-docker)
run Apache Pinot in docker.

In short, you will need to pull the Apache Pinot image by running following
command.

```
docker pull apachepinot/pinot:latest
```

Create a file named docker-compose.yml with the following content.

```yml
version: "3.7"
services:
  pinot-zookeeper:
    image: zookeeper:3.5.6
    container_name: pinot-zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
  pinot-controller:
    image: apachepinot/pinot:0.12.0
    command: "StartController -zkAddress pinot-zookeeper:2181"
    container_name: pinot-controller
    restart: unless-stopped
    ports:
      - "9000:9000"
    environment:
      JAVA_OPTS: "-Dplugins.dir=/opt/pinot/plugins -Xms1G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-controller.log"
    depends_on:
      - pinot-zookeeper
  pinot-broker:
    image: apachepinot/pinot:0.12.0
    command: "StartBroker -zkAddress pinot-zookeeper:2181"
    restart: unless-stopped
    container_name: "pinot-broker"
    ports:
      - "8099:8099"
    environment:
      JAVA_OPTS: "-Dplugins.dir=/opt/pinot/plugins -Xms4G -Xmx4G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-broker.log"
    depends_on:
      - pinot-controller
  pinot-server:
    image: apachepinot/pinot:0.12.0
    command: "StartServer -zkAddress pinot-zookeeper:2181"
    restart: unless-stopped
    container_name: "pinot-server"
    ports:
      - "8098:8098"
    environment:
      JAVA_OPTS: "-Dplugins.dir=/opt/pinot/plugins -Xms4G -Xmx16G -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Xloggc:gc-pinot-server.log"
    depends_on:
      - pinot-broker
```

Go into the directory from your terminal and run the following command to start
Pinot.

```
docker-compose --project-name pinot-demo up
```

Now, Apache Pinot should be up and running. You can check it by running:

```
docker container ls
```

You should see the output like this:

```
CONTAINER ID   IMAGE                     COMMAND                  CREATED              STATUS              PORTS                                                                     NAMES
ba5cb0868350   apachepinot/pinot:0.9.3   "./bin/pinot-admin.s…"   About a minute ago   Up About a minute   8096-8099/tcp, 9000/tcp                                                   pinot-server
698f160852f9   apachepinot/pinot:0.9.3   "./bin/pinot-admin.s…"   About a minute ago   Up About a minute   8096-8098/tcp, 9000/tcp, 0.0.0.0:8099->8099/tcp, :::8099->8099/tcp        pinot-broker
b1ba8cf60d69   apachepinot/pinot:0.9.3   "./bin/pinot-admin.s…"   About a minute ago   Up About a minute   8096-8099/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp                  pinot-controller
54e7e114cd53   zookeeper:3.5.6           "/docker-entrypoint.…"   About a minute ago   Up About a minute   2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp, :::2181->2181/tcp, 8080/tcp   pinot-zookeeper
```

Now, you should add table to your Pinot to store the data streamed from Kafka
topic.

You need to open [http://localhost:9000/](http://localhost:9000/) on your
browser.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/pinot-opened.png" />
</Frame>

Click on “Tables” section.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/table-page-opened.png" />
</Frame>

First, click on “Add Schema” and fill it until you see the following JSON as
your schema config.

```json
{
  "schemaName": "transcript",
  "dimensionFieldSpecs": [
    {
      "name": "studentID",
      "dataType": "INT"
    },
    {
      "name": "firstName",
      "dataType": "STRING"
    },
    {
      "name": "lastName",
      "dataType": "STRING"
    },
    {
      "name": "gender",
      "dataType": "STRING"
    },
    {
      "name": "subject",
      "dataType": "STRING"
    }
  ],
  "metricFieldSpecs": [
    {
      "name": "score",
      "dataType": "FLOAT"
    }
  ],
  "dateTimeFieldSpecs": [
    {
      "name": "timestamp",
      "dataType": "LONG",
      "format": "1:MILLISECONDS:EPOCH",
      "granularity": "1:MILLISECONDS"
    }
  ]
}
```

Click save and click to “Add Realtime Table” since we will stream the data
real-time.

On this page, table name must be the same name with the schema name, which is
“transcript” in this case.

Then, go below on this page and replace “segmentsConfig” and “tableIndexConfig”
sections in the table config on your browser with the following JSON. Do not
forget to replace UPSTASH-KAFKA-\* placeholders with your cluster information.

```json
{
  "segmentsConfig": {
    "timeColumnName": "timestampInEpoch",
    "timeType": "MILLISECONDS",
    "schemaName": "transcript",
    "replicasPerPartition": "1",
    "replication": "1"
  },
  "tableIndexConfig": {
    "loadMode": "MMAP",
    "streamConfigs": {
      "streamType": "kafka",
      "stream.kafka.consumer.type": "lowlevel",
      "stream.kafka.topic.name": "transcript",
      "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
      "stream.kafka.consumer.factory.class.name": "org.apache.pinot.plugin.stream.kafka20.KafkaConsumerFactory",
      "stream.kafka.broker.list": "UPSTASH-KAFKA-ENDPOINT:9092",
      "security.protocol": "SASL_SSL",
      "sasl.jaas.config": "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";",
      "sasl.mechanism": "SCRAM-SHA-256",
      "realtime.segment.flush.threshold.rows": "0",
      "realtime.segment.flush.threshold.time": "24h",
      "realtime.segment.flush.threshold.segment.size": "50M",
      "stream.kafka.consumer.prop.auto.offset.reset": "smallest"
    }
  }
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/table-created.png" />
</Frame>

## Test the Setup

Now, let's create some events to our Kafka topic. Go to Upstash console, click
on your cluster then Topics, click “transcript”. Select Messages tab then click
Produce a new message. Send a message in JSON format like the below:

```json
{
  "studentID": 205,
  "firstName": "Natalie",
  "lastName": "Jones",
  "gender": "Female",
  "subject": "Maths",
  "score": 3.8,
  "timestampInEpoch": 1571900400000
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/send-message.png" />
</Frame>

Now, go back to your Pinot console on your browser. Navigate to “Query Console”
from the left side bar. When you click on “transcript” table, you will see the
result of the following query automatically.

```sql
select * from transcript limit 10
```

The query result should be as following:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/pinot/query-result.png" />
</Frame>

## Links

[Running Pinot in Docker](https://docs.pinot.apache.org/basics/getting-started/running-pinot-in-docker)

[Apache Pinot Stream Ingestion](https://docs.pinot.apache.org/basics/data-import/pinot-stream-ingestion)


# Proton

This tutorial shows how to integrate Upstash Kafka with Proton

[Proton](https://github.com/timeplus-io/proton) is a unified streaming SQL processing engine which can connect to historical data processing in one single binary. It helps data engineers and platform engineers solve complex real-time analytics use cases, and powers the [Timeplus](https://timeplus.com) streaming analytics platform.

Both Timeplus and Proton can be integrated with Upstash Kafka. Timeplus provides intuitive web UI to minimize the SQL typing and clicks. Proton provides SQL interface to read/write data for Upstash.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) by following [Getting Started](https://docs.upstash.com/kafka).

Create two topics by following the creating topic [steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name the first topic `input`, since we are going to stream from this topic to Proton. The name of the second topic can be `output`. This one is going to receive the stream from Proton.

## Setup Proton

Proton is a single binary for Linux/Mac, also available as a Docker image. You can download/install it via various options:

*   ghcr.io/timeplus-io/proton:latest
*   brew tap timeplus-io/timeplus; brew install proton
*   curl -sSf [https://raw.githubusercontent.com/timeplus-io/proton/develop/install.sh](https://raw.githubusercontent.com/timeplus-io/proton/develop/install.sh) | sh
*   or download the binary for Linux/Mac via [https://github.com/timeplus-io/proton/releases/tag/v1.3.31](https://github.com/timeplus-io/proton/releases/tag/v1.3.31)

With Docker engine installed on your local machine, pull and run the latest version of the Proton Docker image.

```shell
docker run -d --pull always --name proton ghcr.io/timeplus-io/proton:latest
```

Connect to your proton container and run the proton-client tool to connect to the local Proton server:

```shell
docker exec -it proton proton-client -n
```

## Create an External Stream to read Kafka data

[External Stream](https://docs.timeplus.com/proton-kafka#connect-upstash) is the key way for Proton to connect to Kafka cluster and read/write data.

```sql
CREATE EXTERNAL STREAM input(
    requestedUrl string,
    method string,
    ipAddress string,
    requestDuration int)
SETTINGS type='kafka', 
         brokers='grizzly-1234-us1-kafka.upstash.io:9092',
         topic='input',
         data_format='JSONEachRow',
         security_protocol='SASL_SSL', 
         sasl_mechanism='SCRAM-SHA-256',
         username='..', 
         password='..'
```

## Run Streaming SQL

Then you can run the following streaming SQL:

```sql
select * from input where _tp_time>earliest_ts()
```

Let's go to Upstash UI and post a JSON message in `input` topic:

```json
{
  "requestedUrl": "http://www.internationalinteractive.name/end-to-end",
  "method": "PUT",
  "ipAddress": "186.58.241.7",
  "requestDuration": 678
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/post-message.png" />
</Frame>

Right after the message is posted, you should be able to see it in the Proton query result.

## Apply Streaming ETL and Write Data to Upstash Kafka

Cancel the previous streaming SQL and use the following one to mask the IP addresses.

```sql
select now64() AS time, requestedUrl,method,lower(hex(md5(ipAddress))) AS ip
from input where _tp_time > earliest_ts()
```

You will see results as below:

```
┌────────────────────time─┬─requestedUrl────────────────────────────────────────┬─method─┬─ip───────────────────────────────┐
│ 2024-01-10 03:43:16.997 │ http://www.internationalinteractive.name/end-to-end │ PUT    │ d5b267be9018abbe87c1357723f2520c │
│ 2024-01-10 03:43:16.997 │ http://www.internationalinteractive.name/end-to-end │ PUT    │ d5b267be9018abbe87c1357723f2520c │
└─────────────────────────┴─────────────────────────────────────────────────────┴────────┴──────────────────────────────────┘
```

To write the data back to Kafka, you need to create a new external stream (with `output` as topic name) and use a Materialized View as the background job to write data continuously to the output stream.

```sql
CREATE EXTERNAL STREAM target(
    _tp_time datetime64(3), 
    time datetime64(3), 
    requestedUrl string, 
    method string, 
    ip string) 
    SETTINGS type='kafka', 
         brokers='grizzly-1234-us1-kafka.upstash.io:9092',
         topic='output',
         data_format='JSONEachRow',
         security_protocol='SASL_SSL', 
         sasl_mechanism='SCRAM-SHA-256',
         username='..', 
         password='..';

-- setup the ETL pipeline via a materialized view
CREATE MATERIALIZED VIEW mv INTO target AS 
    SELECT now64() AS _tp_time, now64() AS time, requestedUrl, method, lower(hex(md5(ipAddress))) AS ip FROM input;
```

Go back to the Upstash UI. Create a few more messages in `input` topic and you should get them available in `output` topic with raw IP addresses masked.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/output-topic.png" />
</Frame>

Congratulations! You just setup a streaming ETL with Proton, without any JVM components. Check out [https://github.com/timeplus-io/proton](https://github.com/timeplus-io/proton) for more details or join [https://timeplus.com/slack](https://timeplus.com/slack)


# Upstash Kafka with Quix

This tutorial shows how to integrate Upstash Kafka with Quix

[Quix](https://quix.io?utm_source=upstash) is a complete platform for developing, deploying, and monitoring stream processing pipelines. You use the Quix Streams Python library to develop modular stream processing applications, and deploy them to containers managed in Quix with a single click. You can develop and manage applications on the command line or manage them in Quix Cloud and visualize them as a end-to-end pipeline.

## Upstash Kafka Setup

Create a Kafka cluster using
[Upstash Console](https://console.upstash.com/kafka) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

## Quix Setup

Like Upstash, Quix is a managed service—this means that you don't need to provision any servers or compute resources. You can [register for a free trial account](https://portal.platform.quix.io/self-sign-up/) and started in minutes.

# Configure the broker settings

When you [create your first project](https://quix.io/docs/create/create-project.html) in Quix, you'll be asked to configure a message broker. You have the option to configure an external broker (instead the default Quix managed broker).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/quix/quix-upstash-broker-option.png" />
</Frame>

To use Upstash as your message broker, select the Upstash option and configure the settings shown in the following screenshot:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/quix/quix-upstash-broker-settings.png" />
</Frame>

## Create your first pipeline

To help you get started, the Quix platform includes several pipeline templates that you can deploy in a few clicks.

To test the Upstash connection, you can use the ["Hello Quix" template](https://quix.io/templates/hello-quix) which is a simple three-step pipeline:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/quix/helloquix-template.png" />
</Frame>

*   Click [**Clone this project** ](https://portal.platform.quix.io/signup?projectName=Hello%20Quix\&httpsUrl=https://github.com/quixio/template-hello-quix\&branchName=tutorial).
*   On the **Import Project** screen, select **Quix advanced configuration**  (this option ensures that you'll get the option to configure own broker settings).
*   Follow the project creation wizard and configure your Upstash connection details when prompted.
*   Click **Sync your pipeline**

## Test the Setup

In the Quix portal, wait for the services to deploy and show up as "Running".

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/quix/helloquix-pipeline.png" />
</Frame>

Check that the required topics ("*csv-data*" and "*counted-names*") show up in both Quix and Upstash. In Upstash, topics that originate from Quix show up with the Quix workspace and project name as a prefix (e.g. "*quixdemo-helloquix-csv-data*").

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/quix/helloquix-pipeline.png" />
</Frame>

## Links

[Quix documentation](https://quix.io/docs/get-started/welcome.html)

[Quix guide to creating projects](https://quix.io/blog/how-to-create-a-project-from-a-template#cloning-a-project-template-into-github)

[Quix portal](https://portal.platform.quix.io/workspaces)

[Upstash console](https://console.upstash.com/kafka)


# RisingWave

This tutorial shows how to integrate Upstash Kafka with RisingWave

[RisingWave](https://risingwave.com) is a distributed SQL streaming database that enables simple, efficient, and reliable processing of streaming data.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) by following [Getting Started](https://docs.upstash.com/kafka).

Create two topics by following the creating topic [steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name the first topic `risingwave_input`, since we are going to stream from this topic to RisingWave. The name of the second topic can be `risingwave_output`. This one is going to receive the stream from RisingWave.

## RisingWave Setup

RisingWave provides RisingWave Cloud, a fully managed and scalable stream processing platform.

To use the RisingWave Cloud, [create an account](https://cloud.risingwave.com/auth/signup) first.

After creating the account, navigate to the `Clusters` in the right bar. Click on `Create Cluster` and select your plan and cluster configuration.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/risingwave/create-cluster.png" />
</Frame>

Creation of the cluster takes a few minutes.

Once the cluster is created, open it and navigate to the `Query` page.

You need to create a user to log  in to the cluster on Cloud first. The user will be a superuser by default.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/risingwave/create-user.png" />
</Frame>

Now, you have the required RisingWave setup to connect to the Upstash Kafka.

## Create Source

Source means streaming from an external database or pipeline to the RisingWave.

By creating a new source on RisingWave Cloud, the message you add to the Upstash Kafka topic is going to be streamed to the RisingWave database.

Create a source from [RisingWave Cloud console](https://cloud.risingwave.com/console) by running the following command:

```sql
CREATE  SOURCE <source-name> (
	name VARCHAR,
	city VARCHAR,
)
WITH(
	connector = 'kafka',
	topic = 'risingwave_input',
	properties.bootstrap.server = '<UPSTASH-KAFKA-ENDPOINT>',
	scan.startup.mode = 'latest',
	properties.sasl.mechanism = 'SCRAM-SHA-512',
	properties.security.protocol = 'SASL_SSL',
	properties.sasl.username = '<UPSTASH-KAFKA-USERNAME>',
	properties.sasl.password  = '<UPSTASH-KAFKA-PASSWORD>'
)  FORMAT PLAIN ENCODE JSON;
```

You should replace the `UPSTASH-KAFKA-*` placeholders with the credentials from Upstash Kafka console.

This query will create a source on RisingWave. The source can be seen on the left in the console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/risingwave/source-view.png" />
</Frame>

You can also see it in the [Sources](https://cloud.risingwave.com/source/) tab.

To test, go to your [Upstash console](https://console.upstash.com/kafka) and open the `risingwave_input` topic in your Kafka cluster.

Produce a message in this topic in a JSON format. The message should include the fields we defined in the source creation query.

```json
{
	"name": "Noah",
	"city": "London"
}
```

After producing the message, go back to the RisingWave console and run the following query to see the streamed data.

```sql
SELECT * FROM <source-name>;
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/risingwave/list-rows.png" />
</Frame>

## Create Sink

Sink means streaming from RisingWave database to the external data stores or pipelines.

By creating a sink, the data you inserted into the  RisingWave table or the data streamed through the source will be streamed to the Upstash Kafka topic.

For testing purposes, let’s create a new table by running the following query. This table will be streamed to the Upstash Kafka sink topic.

```sql
CREATE  TABLE  <table-name> (name VARCHAR, city VARCHAR);
```

Create a sink from [RisingWave Cloud console](https://cloud.risingwave.com/console) by running the following command:

```sql
CREATE  SINK <sink-name> FROM  <table-name>
WITH  (
	connector = 'kafka',
	properties.bootstrap.server = '<UPSTASH-KAFKA-ENDPOINT>',
	properties.sasl.mechanism = 'SCRAM-SHA-512',
	properties.security.protocol = 'SASL_SSL',
	properties.sasl.username = '<UPSTASH-KAFKA-USERNAME>',
	properties.sasl.password = '<UPSTASH-KAFKA-PASSWORD>',
	topic = 'risingwave_output',
	properties.message.max.bytes = 2000
)
FORMAT PLAIN ENCODE JSON  (
	force_append_only = 'true'
);
```

You should replace the `UPSTASH-KAFKA-*` placeholders with the credentials from Upstash Kafka console.

To test this sink, go to your [Upstash console](https://console.upstash.com/kafka), open the `risingwave_output` topic in your Kafka cluster. Open the messages tab to see incoming messages.

Now insert a new row to the table to be streamed:

```sql
INSERT  INTO <table-name> VALUES  ('Noah',  'Manchester');
```

You can see this row streamed to the Upstash Kafka output topic on your Upstash console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/risingwave/output.png" />
</Frame>


# Rockset

This tutorial shows how to integrate Upstash Kafka with Rockset

[Rockset](https://rockset.com) is a real-time search and analytics database designed to serve millisecond-latency analytical queries on event streams, CDC streams, and vectors.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) by following [Getting Started](https://docs.upstash.com/kafka).

Create one topic by following the creating topic [steps](https://docs.upstash.com/kafka#create-a-topic). This topic will be the source for the Rockset. Let’s name it “transcript” for this example tutorial.

## Rockset Setup

To be able to use the Rockset, you first need to [create an account](https://rockset.com/create).

There are a couple of steps to create your organisation. After completing them, you can see your [Rockset dashboard](https://console.rockset.com) is created.

## Connect Rockset to Upstash Kafka

To ingest data from Upstash Kafka to Rockset, open [Integrations](https://console.rockset.com/integrations) and click to `Create your first Integration`.

Select `Kafka` as the external service and click `Start`. In the next step, name your integration and select Apache Kafka. In the data format section, select the data format and give the name of the topic you created.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/rockset/create-integration.png" />
</Frame>

When you continue, you will see 5 new steps to create and configure the Kafka Connector for Rockset. Kafka – Rockset connection can be established using the plugin Rockset provided only. Therefore, we have to create self-managed Kafka connector.

Since this tutorial explains the first connection, select `New — no pre-existing Kafka Connect cluster`.

To create the required Kafka connector, you must first download [Apache Kafka Connect](https://kafka.apache.org/downloads).

In the next step, you can give the endpoint of the Kafka Cluster as the `Address of Apache Kafka Broker` and then download the provided Kafka Connect properties.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/rockset/kafka-connect.png" />
</Frame>

The `connect-standalone.properties` file should be located in the same folder as Kafka.

Open `connect-standalone.properties` and add the following properties.

```properties
consumer.sasl.mechanism=SCRAM-SHA-256
consumer.security.protocol=SASL_SSL
consumer.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="<UPSTASH-KAFKA-USERNAME>" \
  password="<UPSTASH-KAFKA-PASSWORD>";
sasl.mechanism=SCRAM-SHA-256
security.protocol=SASL_SSL
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="<UPSTASH-KAFKA-USERNAME>" \
  password="<UPSTASH-KAFKA-PASSWORD>";
```

These additional properties will allow your Kafka connector access to your Kafka cluster and consume the topics.

In the next step, download the `Rockset Sink Connector` and `Rockset Sink Connector Properties`. Locate these files in the same folder with Kafka as well.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/rockset/sink-connector.png" />
</Frame>

Now, navigate to the folder that contains all these files and execute the following command to run a standalone Apache Kafka Connect with Rockset Sink Connector.

```bash
./kafka_2.13-3.6.1/bin/connect-standalone.sh ./connect-standalone.properties ./connect-rockset-sink.properties
```

Before completing the integration, we can check if the data is coming to the Rockset. Let’s return to the Upstash console, click on your Kafka cluster and go to the “Topics” section. Open the source topic, which is `transcript` in this case. Select the Messages tab, then click Produce a new message. Send a message in JSON format like the one below:

```json
{
	"studentID": 205,
	"firstName": "Natalie",
	"lastName": "Jones",
	"gender": "Female",
	"subject": "Maths",
	"score": 3.8
}
```

You should see the integration `Active`.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/rockset/integration-active.png" />
</Frame>

## Query Data

When you complete the integration setup, click `Create Collection from Integration`. This will allow you to collect the data from the Uptash Kafka topic and query that data.

Type your Kafka topic in the data source selection step.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/rockset/source-selection.png" />
</Frame>

We can leave the Ingest Transformation Query as it is in the `Transform Data` step.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/rockset/ingest-transformation.png" />
</Frame>

Lastly, name the collection and create.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/rockset/configure-collection.png" />
</Frame>

Now, select `Query This Collection`. Go back to the Upstash console and produce a new message in the source topic.

```json
{
	"studentID": 201,
	"firstName": "John",
	"lastName": "Doe",
	"gender": "Male",
	"subject": "Physics",
	"score": 4.4
}
```

Let’s go to the [Rockset query editor](https://console.rockset.com/query) and run the following query.

```sql
SELECT * FROM commons.transcipts LIMIT 10
```

You can see the last message you sent to the source topic returned.


# Apache Spark

This tutorial shows how to integrate Upstash Kafka with Apache Spark

[Apache Spark](https://spark.apache.org/) is a multi-language engine for
executing data engineering, data science, and machine learning on single-node
machines or clusters.

### Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com/) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create a topic by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name the topic
“sentence”.

### Project Setup

<Info>
  If you already have a project and want to implement Upstash Kafka and Apache
  Spark integration into it, you can skip this section and continue with [Add
  Spark and Kafka into the Project](#add-spark-and-kafka-into-the-project).
</Info>

Install Maven to your machine by following [Maven Installation Guide](https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html).

Run `mvn –version` in a terminal or in a command prompt to make sure you have
Maven downloaded.

It should print out the version of the Maven you have:

```
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3\apache-maven\bin\..
Java version: 1.8.0_232, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
```

To create the Maven project;

Go into the folder that you want to create the project in your terminal or
command prompt by running `cd <folder path>`

Run the following command:

```
mvn archetype:generate -DgroupId=com.kafkasparkinteg.app -DartifactId=kafkasparkinteg-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
```

### Add Spark and Kafka into the Project

Open the project folder by using an IDE which has maven plugin such as Intellij,
Visual Studio, Eclipse etc. Add following Spark dependencies into the
dependencies tag in `pom.xml` file.

```xml
<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.12</artifactId>
<version>3.3.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.12</artifactId>
<version>3.3.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql_2.12</artifactId>
<version>3.3.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka-0-10 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming-kafka-0-10_2.12</artifactId>
<version>3.3.1</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql-kafka-0-10 -->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-sql-kafka-0-10_2.12</artifactId>
<version>3.3.1</version>
</dependency>
```

### Using Apache Spark as Producer

Import the following packages first:

```java
import org.apache.spark.api.java.function.FlatMapFunction;
import org.apache.spark.sql.*;
import org.apache.spark.sql.types.DataTypes;
import org.apache.spark.sql.types.StructType;
import java.util.*;
```

To send messages to Kafka from Spark, use the following code after replacing the
`UPSTASH-KAFKA-*` placeholders with your cluster information:

```java
SparkSession spark = SparkSession.builder()
  .appName("quickstart")
  .config("spark.master", "local")
  .getOrCreate();

StructType structType = new StructType();
structType = structType.add("key", DataTypes.StringType, false);
structType = structType.add("value", DataTypes.StringType, false);

List<Row> rows = new ArrayList<Row>();
rows.add(RowFactory.create("test key", "This is an example sentence"));

Dataset<Row> sentenceDF = spark.createDataFrame(rows, structType);

sentenceDF.selectExpr("CAST(key AS STRING)", "CAST(value AS STRING)")
.write()
.format("kafka")
.option("kafka.bootstrap.servers", "UPSTASH-KAFKA-ENDPOINT:9092")
.option("kafka.sasl.mechanism", "SCRAM-SHA-256")
.option("kafka.security.protocol", "SASL_SSL")
.option("kafka.sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";")
.option("topic", "sentence")
.save();
```

Before running the project, open the messages of the topic from
[console](https://console.upstash.com).

You can observe new message coming to the topic on Upstash console when you run
your project.

### Using Apache Spark as Consumer

If the following packages are not imported, import them first:

```java
import org.apache.spark.api.java.function.FlatMapFunction;
import org.apache.spark.sql.*;
import org.apache.spark.sql.types.DataTypes;
import org.apache.spark.sql.types.StructType;

import java.util.*;
```

To receive the messages from Kafka topic by Apache Spark and to process, use the
following code after replacing the UPSTASH-KAFKA-\* placeholders with your
cluster information:

```java
SparkSession spark = SparkSession.builder()
  .appName("quickstart")
  .config("spark.master", "local")
  .getOrCreate();

Dataset<Row> lines = spark
  .read()
  .format("kafka")
  .option("kafka.bootstrap.servers", "UPSTASH-KAFKA-ENDPOINT:9092")
  .option("kafka.sasl.mechanism", "SCRAM-SHA-256")
  .option("kafka.security.protocol", "SASL_SSL")
  .option("kafka.sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";")
  .option("startingOffsets", "earliest")
  .option("subscribe", "sentence")
  .load()
  .selectExpr("CAST(key AS STRING)", "CAST(value AS STRING)");
// PROCESS RECEIVED MESSAGE - Word counting part
Dataset<String> words = lines.select("value")
  .as(Encoders.STRING())
  .flatMap( new FlatMapFunction<String, String>() { @Override public Iterator<String> call(String x) { return Arrays.asList(x.split(" ")).iterator(); }
  }, Encoders.STRING()); Dataset<Row> wordCounts = words.groupBy("value").count(); wordCounts.show();
```

You can verify that you can see the sentence, which you sent, on your console
with number of word occurrences:

```
+--------+-----+
|   value|count|
+--------+-----+
| example|    1|
|      is|    1|
|sentence|    1|
|      an|    1|
|    This|    1|
+--------+-----+
```


# StarTree

This tutorial shows how to integrate Upstash Kafka with StarTree

[StarTree](https://startree.ai/) provides a fully managed, Apache Pinot based
real-time analytics database on its cloud environment.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create one topic by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). This topic will be the
source for the Apache Pinot table running on StarTree. Let’s name it
“transcript” for this example tutorial.

## StarTree Setup

To be able to use StarTree cloud, you first need to
[create an account](https://startree.ai/saas-signup).

There are two steps to initialize the cloud environment on StarTree. First, you
need to create an organization. Next, you need to create a workspace under this
new organization.

For these setup steps, you can also follow
[StarTree quickstart](https://dev.startree.ai/docs/startree-enterprise-edition/startree-cloud/getting-started/saas/how-to-guide).

## Connect StarTree Cloud to Upstash Kafka

Once you created your workspace, open Data Manager under the `Services` section
in your workspace. Data Manager is where we will connect Upstash Kafka and work
on the Pinot table.

To connect Upstash Kafka with StarTree, create a new connection in Data Manager.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/create-a-connection.png" />
</Frame>

As the connection type, select Kafka.

In Kafka connection settings, fill the following options:

*   Connection Name: It can be anything. It is up to you.

*   Broker Url: This should be the endpoint of your Upstash Kafka cluster. You can
    find it in the details section in your
    [Upstash Kafka cluster](https://console.upstash.com/kafka).

*   Authentication Type: `SASL`

*   Security Protocol: `SASL_SSL`

*   SASL Mechanism: `SCRAM-SHA-256`

*   Username: This should be the username given in the details section in your
    [Upstash Kafka cluster](https://console.upstash.com/kafka).

*   Password: This should be the password given in the details section in your
    [Upstash Kafka cluster](https://console.upstash.com/kafka).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/create-kafka-connection.png" />
</Frame>

To proceed, you need to test the connection first. Once the test connection is
successful, then you can create the connection.

Now you have a connection between Upstash Kafka and StarTree Cloud! The next
step is to create a dataset to store data streamed from Upstash Kafka.

Let’s return to the Data Manager overview page and create a new dataset.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/create-a-dataset.png" />
</Frame>

As the connection type, select Kafka again.

Now you can select the Kafka connection you created for connecting Upstash
Kafka.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/select-kafka-connection.png" />
</Frame>

In the next step, you need to name your dataset, provide the Kafka topic to be
the source of this new dataset and define the data format. We can give
“transcript” as the topic and select JSON as the data format.

To proceed to the next step, we must first produce a message in our Kafka topic.
StarTree doesn’t allow us to go to the next step before it validates the
connection is working, and data is being streamed correctly.

To make StarTree validate our connection, let’s turn back to the Upstash console
and create some events for our Kafka topic. To do this, click on your Kafka
cluster on Upstash console and go to the “Topics” section. Open the source
topic, which is “transcript” in this case. Select the Messages tab, then click
Produce a new message. Send a message in JSON format like the one below:

```json
{
  "studentID": 205,
  "firstName": "Natalie",
  "lastName": "Jones",
  "gender": "Female",
  "subject": "Maths",
  "score": 3.8,
  "timestampInEpoch": 1571900400000
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/produce-message.png" />
</Frame>

Now go back to the dataset details steps on StarTree Data Manager.

After you click next, StarTree will consume the message in the source Kafka
topic to verify the connection. Once it consumes the message, the message will
be displayed.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/sample-data.png" />
</Frame>

In the next step, StarTree extracts the data model from the message you sent.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/data-modeling.png" />
</Frame>

If there is any additional configuration about the model of the data coming from
the source topic, you can add it here.

To keep things simple, we will click next without changing anything.

The last step is for more configuration of your dataset. We will click next
again and proceed to review. Click “Create Dataset” to finalize the dataset.

## Query Data

Open the dataset you created on StarTree Data Manager and navigate to the query
console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/open-query-console.png" />
</Frame>

You will be redirected to Pinot Query Console running on StarTree cloud.

When you run the following SQL Query, you will see the data that came from
Upstash Kafka into your dataset.

```sql
select * from <Dataset-name> limit 10
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/startree/query-result.png" />
</Frame>


# Kafka Streams

This tutorial shows how to integrate Upstash Kafka with Kafka Streams

[Kafka Streams](https://kafka.apache.org/documentation/streams/) is a client
library, which streams data from one Kafka topic to another.

### Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli) by following
[Getting Started](https://docs.upstash.com/kafka).

Create two topics by following the creating topic
[steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name first topic
“input”, since we are going to stream this topic to other one, which we can name
it as “output”.

### Project Setup

<Note>
  If you already have a project and want to use Kafka Streams with Upstash Kafka
  in it, you can skip this section and continue with [Add Kafka Streams into the
  Project](#add-kafka-streams-into-the-project).
</Note>

Install Maven to your machine by following [Maven Installation Guide](https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html).

Run `mvn –version` in a terminal or in a command prompt to make sure you have
Maven downloaded.

It should print out the version of the Maven you have:

```
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: D:\apache-maven-3.6.3\apache-maven\bin\..
Java version: 1.8.0_232, vendor: AdoptOpenJDK, runtime: C:\Program Files\AdoptOpenJDK\jdk-8.0.232.09-hotspot\jre
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
```

To create the Maven project;

Go into the folder that you want to create the project in your terminal or
command prompt by running `cd <folder path>`

Run the following command:

```
mvn archetype:generate -DgroupId=com.kafkastreamsinteg.app -DartifactId=kafkastreamsinteg-app -DarchetypeArtifactId=maven-archetype-quickstart -DarchetypeVersion=1.4 -DinteractiveMode=false
```

### Add Kafka Streams into the Project

Open the project folder by using an IDE which has maven plugin such as Intellij,
Visual Studio, Eclipse etc. Add following dependencies into the dependencies tag
in `pom.xml` file.

```xml
<dependency>
  <groupId>org.apache.kafka</groupId>
  <artifactId>kafka-streams</artifactId>
  <version>3.3.1</version>
</dependency>

<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-reload4j</artifactId>
  <version>2.0.3</version>
</dependency>
```

### Streaming From One Topic to Another Topic

Import the following packages first:

```java
import org.apache.kafka.common.config.SaslConfigs;
import org.apache.kafka.common.config.TopicConfig;
import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.common.utils.Bytes;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.Topology;
import org.apache.kafka.streams.kstream.Grouped;
import org.apache.kafka.streams.kstream.KStream;
import org.apache.kafka.streams.kstream.Materialized;
import org.apache.kafka.streams.kstream.Produced;
import org.apache.kafka.streams.state.KeyValueStore;

import java.util.Arrays;
import java.util.Properties;
import java.util.concurrent.CountDownLatch;
import java.util.regex.Pattern;
```

Define the names of the topics you are going to work on:

```java
String inputTopic = "input";
String outputTopic = "output";
```

Create the following properties for Kafka Streams and replace UPSTASH-KAFKA-\*
placeholders with your cluster information.

```java
final Properties props = new Properties();
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "UPSTASH-KAFKA-ENDPOINT:9092");
props.put(SaslConfigs.SASL_MECHANISM, "SCRAM-SHA-256");
props.put(StreamsConfig.SECURITY_PROTOCOL_CONFIG, "SASL_SSL");
props.put(StreamsConfig.APPLICATION_ID_CONFIG,"myLastNewProject");
props.put(SaslConfigs.SASL_JAAS_CONFIG, "org.apache.kafka.common.security.scram.ScramLoginModule username=\"UPSTASH-KAFKA-USERNAME\" password=\"UPSTASH-KAFKA-PASSWORD\";");
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.topicPrefix(TopicConfig.RETENTION_MS_CONFIG), 604800000); // 7 days for internal repartition topic retention period
props.put(StreamsConfig.topicPrefix(TopicConfig.CLEANUP_POLICY_CONFIG), TopicConfig.CLEANUP_POLICY_DELETE); // delete cleanup policy for internal repartition topic
props.put(StreamsConfig.topicPrefix(TopicConfig.RETENTION_BYTES_CONFIG), 268435456); // 256 MB for internal repartition topic retention size
```

Start the builder for streaming and assign input topic as the source:

```java
StreamsBuilder builder = new StreamsBuilder();
KStream<String, String> source = builder.stream(inputTopic);
```

Apply the following steps to count the words in the sentence sent to input topic
and stream the results to the output topic:

```java
Pattern pattern = Pattern.compile("\\W+", Pattern.UNICODE_CHARACTER_CLASS);
Materialized<String, Long, KeyValueStore<Bytes, byte[]>> materialized = Materialized.as("countMapping");
materialized.withLoggingDisabled();
source.flatMapValues(value -> Arrays.asList(pattern.split(value.toLowerCase())))
        .groupBy((key, word) -> word, Grouped.as("groupMapping"))
        .count(materialized).toStream().mapValues(Object::toString)
        .to(outputTopic, Produced.with(Serdes.String(), Serdes.String()));
```

Since “groupby” function causing repartition and creation of a new internal
topic to store the groups intermediately, be sure that there is enough partition
capacity on your Upstash Kafka. For detailed information about the max partition
capacity of Kafka cluster, check [plans](https://upstash.com/#section-pricing).

Just to be sure, you can check from topic section on
[console](https://console.upstash.com) if the internal repartition topic created
successfully when you run your application and send data to input topic. For
reference, naming convention for internal repartition topics:

```
<application id><group name><repartition>
```

Next, finalize and build the streams builder. Create a topology of your process.
It can be viewed by printing.

```java
final Topology topology = builder.build();
System.out.println(topology.describe());
```

Here is the example topology in this scenario:

```
Topologies:

Sub-topology: 0
Source: KSTREAM-SOURCE-0000000000 (topics: [input])
--> KSTREAM-FLATMAPVALUES-0000000001
Processor: KSTREAM-FLATMAPVALUES-0000000001 (stores: [])
--> groupMapping
<-- KSTREAM-SOURCE-0000000000
Processor: groupMapping (stores: [])
--> groupMapping-repartition-filter
<-- KSTREAM-FLATMAPVALUES-0000000001
Processor: groupMapping-repartition-filter (stores: [])
--> groupMapping-repartition-sink
<-- groupMapping
Sink: groupMapping-repartition-sink (topic: groupMapping-repartition)
<-- groupMapping-repartition-filter

Sub-topology: 1
Source: groupMapping-repartition-source (topics: [groupMapping-repartition])
--> KSTREAM-AGGREGATE-0000000003
Processor: KSTREAM-AGGREGATE-0000000003 (stores: [countMapping])
--> KTABLE-TOSTREAM-0000000007
<-- groupMapping-repartition-source
Processor: KTABLE-TOSTREAM-0000000007 (stores: [])
--> KSTREAM-MAPVALUES-0000000008
<-- KSTREAM-AGGREGATE-0000000003
Processor: KSTREAM-MAPVALUES-0000000008 (stores: [])
--> KSTREAM-SINK-0000000009
<-- KTABLE-TOSTREAM-0000000007
Sink: KSTREAM-SINK-0000000009 (topic: output)
<-- KSTREAM-MAPVALUES-0000000008
```

Finally, start the Kafka Streams that was built and run it.

```java
final KafkaStreams streams = new KafkaStreams(topology, props);
final CountDownLatch latch = new CountDownLatch(1);
try {
	streams.start();
	System.out.println("streams started");
	latch.await();
} catch (final Throwable e) {
    System.exit(1);
}

Runtime.getRuntime().addShutdownHook(new Thread("streams-word-count") {
    @Override
	public void run() {
	    streams.close();
		latch.countDown();
	}
});
```


# Timeplus

This tutorial shows how to integrate Upstash Kafka with Timeplus

[Timeplus](https://timeplus.com) is a streaming-first data analytics platform. It provides powerful end-to-end capabilities, leveraging the open source streaming engine [Proton](https://github.com/timeplus-io/proton), to help teams process streaming and historical data quickly and intuitively, accessible for organizations of all sizes and industries. It enables data engineers and platform engineers to unlock streaming data value using SQL.

## Upstash Kafka Setup

Create a Kafka cluster using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) by following [Getting Started](https://docs.upstash.com/kafka).

Create two topics by following the creating topic [steps](https://docs.upstash.com/kafka#create-a-topic). Let’s name the first topic `input`, since we are going to stream from this topic to Timeplus. The name of the second topic can be `output`. This one is going to receive the stream from Timeplus.

## Create a Kafka Source in Timeplus

Besides the Open Source engine [Proton](https://github.com/timeplus-io/proton), Timeplus also offers Timeplus Cloud, a fully managed cloud service with SOC2 Type 1 Security Compliance.

To use the Timeplus Cloud, [create an account](https://us.timeplus.cloud) and setup a new workspace.

After creating the workspace, click the `Data Ingestion` in the menu bar. Click on `Add Data`

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/0-add-data.png" />
</Frame>

Choose the `Apache Kafka` source.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/1-choose-kafka-source.png" />
</Frame>

In the wizard, specify the Kafka Brokers as `<name>-<region>-kafka.upstash.io:9092`. Enable all security options and choose `SASL SCRAM 256` and type the username and password.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/2-source-step-1.png" />
</Frame>

Click the `Next` button. In the dropdown list, you should be able to see all available Kafka topics. Choose the `input` topic. Leave the `JSON` as the `Read As` option. Choose `Earliest` if you already create messages in the topic. Otherwise use the default value `Latest`.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/3-source-step-2.png" />
</Frame>

Click the `Next` button, it will start loading messages in the `input` topic.

Let's go to Upstash UI and post a JSON message in `input` topic:

```json
{
  "requestedUrl": "http://www.internationalinteractive.name/end-to-end",
  "method": "PUT",
  "ipAddress": "186.58.241.7",
  "requestDuration": 678
}
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/post-message.png" />
</Frame>

Right after the message is posted, you should be able to see it in the Timeplus UI. Since you specify JSON format, those 4 key/value pairs are read as 4 columns. Choose a name for the data stream, say `input` and accept the default options.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/3-source-step-preview.png" />
</Frame>

Click the `Next` button to review the settings. Finally, click the `Create the source` button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/4-source-step-4.png" />
</Frame>

There will be a green notification message informing you the source has been created.

## Run Streaming SQL

Click the `Query` menu on the left and type the streaming SQL as:

```sql
select * from input
```

Go back to the Upstash UI to post a few more messages to the topic and you can see those live events in the query result.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/query-stream.png" />
</Frame>

## Apply Streaming ETL and Write Data to Upstash Kafka

Cancel the previous streaming SQL and use the following one to mask the IP addresses.

```sql
select now64() AS time, requestedUrl,method,lower(hex(md5(ipAddress))) AS ip
from input where _tp_time > earliest_ts()
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/streaming-etl-sql.png" />
</Frame>

Click the `Send as Sink` button. Use the default `Kafka` output type and specify the broker, topic name(`output`), user name and password.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/create-sink.png" />
</Frame>

Click the `Send` button to create the sink.

Go back to the Upstash UI. Create a few more messages in `input` topic and you should get them available in `output` topic with raw IP addresses masked.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/timeplus/output-topic.png" />
</Frame>

Congratulations! You just setup a streaming ETL with a single line of SQL in Timeplus Cloud. Learn more about Timeplus by visiting [https://docs.timeplus.com/](https://docs.timeplus.com/) or join [https://timeplus.com/slack](https://timeplus.com/slack)


# Tinybird

This tutorial shows how to set up a pipeline to stream traffic events to Upstash Kafka and analyse with Tinybird

In this tutorial series, we will show how to build an end to end real time
analytics system. We will stream the traffic (click) events from our web
application to Upstash Kafka then we will analyse it on real time. We will
implement one simply query with different stream processing tools:

```sql
SELECT city, count() FROM kafka_topic_page_views where  timestamp > now() - INTERVAL 15 MINUTE group by city
```

Namely, we will query the number of page views from different cities in last 15
minutes. We keep the query and scenario intentionally simple to make the series
easy to understand. But you can easily extend the model for your more complex
realtime analytics scenarios.

If you do not have already set up Kafka pipeline, see
[the first part of series](./cloudflare_workers) where we
did the set up our pipeline including Upstash Kafka and Cloudflare Workers (or
Vercel).

In this part of the series, we will showcase how to use Tinybird to run a query
on a Kafka topic.

### Tinybird Setup

Create a [Tinybird](https://tinybird.co) account and select a region. Create an
empty workspace. On the wizard screen click `Add Data` button.

![tiny1.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel-edge-tinybird/tiny1.png)

In the next screen click on Kafka tab and fill the fields with the credentials
copied from Upstash Kafka console. Key is `username` , secret is `password` .
Select `SCRAM-SHA-256`. Then click on `Connect` button.

![tiny2.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel-edge-tinybird/tiny2.png)

If the connection is successful, then you should see the topic you have just
created. Select it and click `Continue` .

![tiny3.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel-edge-tinybird/tiny3.png)

In the next screen, you should see data is populated from your Kafka topic. It
should look like the below. Select `Latest` and click `Create Data Source`

![tiny4.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel-edge-tinybird/tiny4.png)

Click `Create Pipe` on the next screen.

![tiny5.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel-edge-tinybird/tiny5.png)

In the next page, you will see the query editor where you can execute queries on
your data pipe. You can rename the views. Update the query as (replace the
datasource):

```sql
SELECT city, count() FROM  kafka_ds_534681 where  timestamp > now() - INTERVAL 15 MINUTE group by city
```

![tiny6.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel-edge-tinybird/tiny6.png)

You should see the number of page view from cities in last 15 minutes. The good
thing with TinyBird is you can chain queries (new transformation node) also you
can add multiple data sources (e.g. Kafka topics) and join them in a single
query.

If you are happy with your query, click on `Create API Endpoint` at top right.
It will create an API endpoint which returns the result for your query.

![tiny7.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel-edge-tinybird/tiny7.png)

Copy the curl command and try, you should see the result like below:

```bash
curl --compressed -H 'Authorization: Bearer p.eyJ1IjogIjMyMDM1YzdjLTRkOGYtNDA3CJpZCI6ICJlNTY4ZjVjYS1hNjNiLTRlZTItYTFhMi03MjRiNzhiNmE5MWEifQ.3KgyhWoohcr_0HCt6J7y-kt71ZmWOlrjhYyFa7TaUlA'  https://api.us-east.tinybird.co/v0/pipes/kafka_ds_534681_pipe_2444.json
{
	"meta":
	[
		{
			"name": "city",
			"type": "String"
		},
		{
			"name": "count()",
			"type": "UInt64"
		}
	],

	"data":
	[
		{
			"city": "San Jose",
			"count()": 8
		}
	],

	"rows": 1,

	"statistics":
	{
		"elapsed": 0.000562736,
		"rows_read": 8,
		"bytes_read": 168
	}
}
```

## Conclusion

We have built a simple data pipeline which collect data from edge to Kafka then
create real time reports using SQL. Thanks to serverless technologies (Vercel,
Upstash, Tinybird), we achieved this without dealing any server or
infrastructural configurations. You can easily extend and adapt this example for
much more complex architectures and queries.


# Vercel Edge



Vercel Edge middleware allows you to intercept the requests to your application
served by Vercel platform. We will create a simple Next.js application and send
the traffic events to Upstash Kafka using the Vercel Edge functions.

Create a Next.js application:

```shell
npx create-next-app@latest --typescript
```

Create a middleware.ts (or .js) file in your app directory.
Update the file as below:

```js app/middleware.ts
import { NextResponse } from "next/server";
import type { NextRequest, NextFetchEvent } from "next/server";
import { Kafka } from "@upstash/kafka";

// Trigger this middleware to run on the `/secret-page` route
export const config = {
  matcher: "/",
};

export async function middleware(req: NextRequest, event: NextFetchEvent) {
  // Extract country. Default to US if not found.
  console.log(req.url);

  const kafka = new Kafka({
    url: process.env.UPSTASH_KAFKA_REST_URL,
    username: process.env.UPSTASH_KAFKA_REST_USERNAME,
    password: process.env.UPSTASH_KAFKA_REST_PASSWORD,
  });

  let message = {
    country: req.geo?.country,
    city: req.geo?.city,
    region: req.geo?.region,
    url: req.url,
    ip: req.headers.get("x-real-ip"),
    mobile: req.headers.get("sec-ch-ua-mobile"),
    platform: req.headers.get("sec-ch-ua-platform"),
    useragent: req.headers.get("user-agent"),
  };
  const p = kafka.producer();
  const topic = "mytopic";

  event.waitUntil(p.produce(topic, JSON.stringify(message)));

  // Rewrite to URL
  return NextResponse.next();
}
```

Above, we simply parse the request object and send useful information to Kafka.
You may add/remove information depending on your own requirements.

### Configure Credentials

We're going to store our credentials in `.env` file. You can copy/paste the necessary credentials from the Upstash Console.

```text .env
UPSTASH_KAFKA_REST_URL=<YOUR_URL>
UPSTASH_KAFKA_REST_USERNAME=<YOUR_USERNAME>
UPSTASH_KAFKA_REST_PASSWORD=<YOUR_PASSWORD>
```

You can test the function locally with `npm run dev`. Deploy your function to
Vercel with `vercel --prod`

The endpoint of the function will be printed. You can check if logs are
collected in Kafka by copying the `curl` expression from the console:

```shell
curl https://real-goldfish-14081-us1-rest-kafka.upstash.io/consume/GROUP_NAME/GROUP_INSTANCE_NAME/mytopic -H "Kafka-Auto-Offset-Reset: earliest" -u \
  REPLACE_HERE
```

### Conclusion

We have successfully built to the pipeline to collect the traffic data from our
web application to Upstash Kafka. In the remaining of the series, we will
analyze the data in Kafka with different realtime analytics tool which are
capable to read from Kafka.


# AKHQ

How to use AKHQ with Upstash Kafka

[AKHQ](https://akhq.io) is a GUI for monitoring & managing Apache Kafka topics,
topics data, consumers group etc. You can connect and monitor your Upstash Kafka
cluster using [AKHQ](https://akhq.io).

To be able to use [AKHQ](https://akhq.io), first you should create a yaml
configuration file:

```yaml
akhq:
  connections:
    my-cluster:
      properties:
        bootstrap.servers: "tops-stingray-7863-eu1-kafka.upstash.io:9092"
        sasl.mechanism: SCRAM-SHA-512
        security.protocol: SASL_SSL
        sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="ZmlycG9iZXJtYW4ZHtSXVwmyJQ" password="J6ocnQfe25vUsI8AX-XxA==";
      schema-registry:
        url: UPSTASH_KAFKA_REST_URL/schema-registry
        basic-auth-username: UPSTASH_KAFKA_REST_USERNAME
        basic-auth-password: UPSTASH_KAFKA_REST_PASSWORD
```

<Note>
  You should replace `bootstrap.servers` and `sasl.jaas.config` attributes with your cluster endpoint and credentials.

  You can find the endpoint, username and password from the cluster page at [Upstash Console](https://console.upstash.com).
  Scroll down to the `REST API` section to find schema-registry related configs:

  *   `UPSTASH_KAFKA_REST_URL`
  *   `UPSTASH_KAFKA_REST_USERNAME`
  *   `UPSTASH_KAFKA_REST_PASSWORD`
</Note>

You can start [AKHQ](https://akhq.io) application directly using `jar` file.
First download the latest release from
[releases page](https://github.com/tchiotludo/akhq/releases). Then launch the
application using following command:

```shell
java -Dmicronaut.config.files=application.yml -jar akhq.jar
```

Alternatively you can start using Docker:

```shell
docker run -p 8080:8080 -v ~/akhq/application.yml:/app/application.yml tchiotludo/akhq
```

After launching the [AKHQ](https://akhq.io) app, just go to
[http://localhost:8080](http://localhost:8080) to access UI.

For more information see
[AKHQ documentation](https://akhq.io/docs/#installation).


# Conduktor

How to monitor and manage Upstash Kafka clusters using Conduktor

[Conduktor](https://www.conduktor.io/) is a quite powerful application to
monitor and manage Apache Kafka clusters. You can connect and monitor your
Upstash Kafka cluster using [Conduktor](https://www.conduktor.io/). Conduktor
has a free for development and testing.

### Install Conduktor

Conduktor is a desktop application. So you need to
[download](https://www.conduktor.io/download/) it first. If you are using a Mac,
you can install it using `brew` too.

```shell
brew tap conduktor/brew
brew install conduktor
```

### Connect Your Cluster

Once you install Conduktor and
[create an Upstash Kafka cluster and topic](../overall/getstarted), you can
connect your cluster to Conduktor. Open Conduktor and click on
`New Kafka Cluster` button.

*   You can set any name as `Cluster Name`.
*   Copy Kafka endpoint from [Upstash console](https://console.upstash.com) and
    paste to `Bootstrap Servers` field.
*   In Upstash console, copy the properties from the `Properties` tab. Paste it to
    the `Additional Properties` field on Conduktor.

<img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/monitoring/conduktor1.png" width="100%" />

Once you connected to the cluster, now you can produce and consume to your
topics using Conduktor.

<img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/monitoring/conduktor2.png" width="100%" />


# kafka-ui

Connect and monitor your Upstash Kafka cluster using kafka-ui.

[kafka-ui](https://github.com/provectus/kafka-ui) is a GUI for monitoring Apache
Kafka. From their description:

> Kafka UI for Apache Kafka is a simple tool that makes your data flows
> observable, helps find and troubleshoot issues faster and deliver optimal
> performance. Its lightweight dashboard makes it easy to track key metrics of
> your Kafka clusters - Brokers, Topics, Partitions, Production, and
> Consumption.

You can connect and monitor your Upstash Kafka cluster using
[kafka-ui](https://github.com/provectus/kafka-ui).

To be able to use [kafka-ui](https://github.com/provectus/kafka-ui), first you
should create a yaml configuration file:

```yaml
kafka:
  clusters:
    - name: my-cluster
      bootstrapServers: "tops-stingray-7863-eu1-kafka.upstash.io:9092"
      properties:
        sasl.mechanism: SCRAM-SHA-512
        security.protocol: SASL_SSL
        sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="ZmlycG9iZXJtYW4ZHtSXVwmyJQ" password="J6ocnQfe25vUsI8AX-XxA==";
      schemaRegistry: UPSTASH_KAFKA_REST_URL/schema-registry
      schemaRegistryAuth:
          username: UPSTASH_KAFKA_REST_USERNAME
          password: UPSTASH_KAFKA_REST_PASSWORD
```

<Note>
  You should replace `bootstrap.servers` and `sasl.jaas.config` attributes with your cluster endpoint and credentials.

  You can find the endpoint, username and password from the cluster page at [Upstash Console](https://console.upstash.com).
  Scroll down to the `REST API` section to find schema-registry related configs:

  *   `UPSTASH_KAFKA_REST_URL`
  *   `UPSTASH_KAFKA_REST_USERNAME`
  *   `UPSTASH_KAFKA_REST_PASSWORD`
</Note>

You can start [kafka-ui](https://github.com/provectus/kafka-ui) application
directly using `jar` file. First download the latest release from
[releases page](https://github.com/provectus/kafka-ui/releases). Then launch the
application using following command in the same directory with `application.yml`
file:

```shell
java -jar kafka-ui-api-X.Y.Z.jar
```

Alternatively you can start using Docker:

```shell
docker run -p 8080:8080 -v ~/kafka-ui/application.yml:/application.yml provectuslabs/kafka-ui:latest
```

After launching the [kafka-ui](https://github.com/provectus/kafka-ui) app, just
go to [http://localhost:8080](http://localhost:8080) to access UI.

For more information see
[kafka-ui documentation](https://github.com/provectus/kafka-ui/blob/master/README.md).


# Compare



kafkacompare


# Credentials



<Warning>
  The default Kafka credential limit is set at **10**.

  If you require an extension beyond this limit, we kindly request that you submit a formal request to [support@upstash.com](mailto:support@upstash.com).
</Warning>

When you create a Kafka cluster on the
[Upstash Console](https://console.upstash.com), a *default* user credentials is
created automatically. *Default* user credentials has *full access* rights to
the Kafka cluster, which can produce to & consume from any topic.

You can create additional credentials which can have limited access to topics.
Credentials can be limited to *produce-only* or *consume-only* and also for a
certain topic or a topic prefix. When a topic prefix is used, a user will have
access to any topic whose name starts with the prefix. Newly created credentials
can be used with both Kafka clients and Upstash REST API similar to the
*default* credentials.

To create new credentials, switch to **Credentials** tab in the cluster details
page, and click the "*New Credentials*" button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/credentials/new-credentials.png" width="520" />
</Frame>

There are three inputs in the **New Credentials** view:

*   **Name**: Name of the credentials. This name is shown on the Console only to
    identify the credentials.
*   **Topic**: A specific topic name or a topic prefix. This can be a literal
    topic name (such as `users.events`, `product-orders`), or a topic prefix (such
    as `users.*`, `products-*`). Wildcard character `*` can only be used at the
    end of the topic name. `*` means any topic.
*   **Permissions**: Access permissions for the credentials. It has three options:
    `Full Access`, `Produce Only` and `Consume Only`.

You can see all existing credentials, copy username & password by clicking "+"
button and delete a credentials inside the "Credentials" tab.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/credentials/credentials.png" />
</Frame>

Additionally you can change the credentials shown on the console and used in the
code snippets by clicking "Credentials" box at the top right on the Console and
selecting one of the credentials.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/credentials/credentials-switch.png" />
</Frame>


# Pro and Enterprise Plans



<Snippet file="kafka/enterprise.mdx" />


# Getting Started

Create a Kafka cluster in seconds

<Warning>
  **Deprecated**

  Starting September 11, 2024, Upstash Kafka has been transitioned into deprecation period, which will last for 6 months. It will eventually be discontinued on March 11, 2025.

  During deprecation period, existing Upstash Kafka clusters and all of it's features will be fully supported. New Upstash Kafka cluster creation will not be possible.
  You can find further details in our blog post [here](https://upstash.com/blog/workflow-kafka).
</Warning>

## Create a Kafka Cluster

Once you logged in, you can create a Kafka cluster by clicking on the **Create
Cluster** button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/create-cluster.png" />
</Frame>

**Name:** Type a name for the Kafka cluster.

**Region:** Please select the region in which you would like your cluster to be
deployed. To optimize performance, it is recommended to choose the region that
is closest to your applications. We have plans to expand our support to other
regions and cloud providers in the future. Please send your requests to
[feedback@upstash.com](mailto:feedback@upstash.com) to expedite it.

**Type:** Select the cluster type. Currently there are only two options, choose
single replica for testing/development, multi replica for production use cases:

*   **Single Replica:** Topics created in the single replica cluster will only
    have single replica.
*   **Multi Replica:** Topics created in the multi replica cluster will have three
    replicas.

Upon clicking the **Create** button, you will be presented with a list of your
clusters as shown below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/cluster-list.png" />
</Frame>

You can click on a cluster to navigate to the details page of related cluster.
Details section contains the necessary configurations and credentials.

<Frame caption="Cluster details page, which shows the cluster bootstrap endpoint, username and password.">
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/cluster-detail.png" />
</Frame>

## Create a Topic

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/tapic-tab.png" />
</Frame>

To create a new topic, switch to the **Topics** tab in cluster details page, and
click the **Create topic** button:

<Frame>
  {" "}

  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/create-topic.png" />
</Frame>

Type a name for your topic and pick a partition count according to your
application's needs.

<Warning>
  Note that Kafka topic names can contain only alphanumerics, underscore (\_),
  hyphen (-) and dot (.) characters.
</Warning>

You can also change some advanced properties such as

*   <Tooltip tip="Retention time in Kafka determines how long messages are stored in a topic before they are eligible for deletion. It allows you to control how long data remains available for consumption.">
      {" "}

      Retention Time
    </Tooltip>

*   <Tooltip tip="Retention size in Kafka determines the maximum data capacity of a topic. It allows you to manage storage space by automatically discarding older messages when the size threshold is reached.">
      {" "}

      Retention Size
    </Tooltip>

*   <Tooltip tip="Cleanup policy in Kafka defines how obsolete messages are managed within a topic. It specifies the rules for retaining or removing messages based on specific criteria, such as time or space.">
      {" "}

      Cleanup Policy
    </Tooltip>

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/topic-advanced.png" />
</Frame>

Upon creating the topic, you will have access to a topics list that provides an
overview of the topic's configuration and usage details. This list allows you to
view and modify the topic configuration, as well as delete the topic if needed.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/topics-list.png" />
</Frame>

## Connect to Cluster

To establish a connection with the Kafka cluster, you have the flexibility to
utilize any Kafka clients of your choice. In the cluster details section, you
will find code snippets specifically designed for several popular Kafka clients.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/connect-clients.png" />
</Frame>

Normally username and password will be shown as `{{ UPSTASH_KAFKA_USERNAME }}`
and `{{ UPSTASH_KAFKA_PASSWORD }}`. By turning on "Show secrets" toggle, these
secrets will be replaced with the actual values of your cluster. For more info
about using Kafka clients see [Kafka API](./kafkaapi/) section.

Alternatively, you can use the REST API to connect the cluster. For more info
about using the REST API see [Kafka REST API](../rest/restintro).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/connect-rest.png" />
</Frame>


# Kafka API



Upstash uses [Apache Kafka](https://kafka.apache.org/) for deployments and
provides a serverless Kafka cluster access using both native Kafka clients (over
TCP) and REST API (over HTTP). As a consequence of this flexible model, there
are some restrictions when using
[Kafka protocol](https://kafka.apache.org/protocol), mainly for administrative
Kafka APIs.

Currently following [Kafka Protocol APIs](https://kafka.apache.org/protocol) are
supported by Upstash:

| NAME            | KEY | NAME                 | KEY | NAME                    | KEY |
| --------------- | :-: | -------------------- | :-: | ----------------------- | :-: |
| Produce         |  0  | DescribeGroups       |  15 | EndTxn                  |  26 |
| Fetch           |  1  | ListGroups           |  16 | TxnOffsetCommit         |  28 |
| ListOffsets     |  2  | SaslHandshake        |  17 | DescribeConfigs         |  32 |
| Metadata        |  3  | ApiVersions          |  18 | AlterConfigs            |  33 |
| OffsetCommit    |  8  | CreateTopics         |  19 | DescribeLogDirs         |  35 |
| OffsetFetch     |  9  | DeleteTopics         |  20 | SaslAuthenticate        |  36 |
| FindCoordinator |  10 | DeleteRecords        |  21 | CreatePartitions        |  37 |
| JoinGroup       |  11 | InitProducerId       |  22 | DeleteGroups            |  42 |
| Heartbeat       |  12 | OffsetForLeaderEpoch |  23 | IncrementalAlterConfigs |  44 |
| LeaveGroup      |  13 | AddPartitionsToTxn   |  24 | OffsetDelete            |  47 |
| SyncGroup       |  14 | AddOffsetsToTxn      |  25 | DescribeCluster         |  60 |

<Note>
  Some of the unsupported Kafka APIs are in our roadmap to make them available.
  If you need an API that we do not support at the moment, please drop a note to
  [support@upstash.com](mailto:support@upstash.com). So we can inform you when
  we are planning to support it.
</Note>

## Connect Using Kafka Clients

Connecting to Upstash Kafka using any Kafka client is very straightforward. If
you do not have a Kafka cluster and/or topic already, follow
[these steps](../overall/getstarted) to create one.

After creating a cluster and a topic, just go to cluster details page on the
[Upstash Console](https://console.upstash.com) and copy bootstrap endpoint,
username and password.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/getting_started/cluster-detail.png" />
</Frame>

Then replace following parameters in the code snippets of your favourite Kafka
client or language below.

*   `{{ BOOTSTRAP_ENDPOINT }}`
*   `{{ UPSTASH_KAFKA_USERNAME }}`
*   `{{ UPSTASH_KAFKA_PASSWORD }}`
*   `{{ TOPIC_NAME }}`

## Create a Topic

<Tabs>
  <Tab title="Java">
    ```java
    class CreateTopic {
      public static void main(String[] args) throws Exception {
        var props = new Properties();
        props.put("bootstrap.servers", "{{ BOOTSTRAP_ENDPOINT }}");
        props.put("sasl.mechanism", "SCRAM-SHA-512");
        props.put("security.protocol", "SASL_SSL");
        props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required " +
                    "username=\"{{ UPSTASH_KAFKA_USERNAME }}\" " +
                    "password=\"{{ UPSTASH_KAFKA_PASSWORD }}\";");

        try (var admin = Admin.create(props)) {
            admin.createTopics(
                Set.of(new NewTopic("{{ TOPIC_NAME }}", partitions, replicationFactor))
              ).all().get();
        }
      }
    }
    ```
  </Tab>

  <Tab title="Node.js">
    ```typescript
    const { Kafka } = require("kafkajs");

    const kafka = new Kafka({
      brokers: ["{{ BOOTSTRAP_ENDPOINT }}"],
      sasl: {
        mechanism: "scram-sha-512",
        username: "{{ UPSTASH_KAFKA_USERNAME }}",
        password: "{{ UPSTASH_KAFKA_PASSWORD }}",
      },
      ssl: true,
    });

    const admin = kafka.admin();

    const createTopic = async () => {
      await admin.connect();
      await admin.createTopics({
        validateOnly: false,
        waitForLeaders: true,
        topics: [
          {
            topic: "{{ TOPIC_NAME }}",
            numPartitions: partitions,
            replicationFactor: replicationFactor,
          },
        ],
      });
      await admin.disconnect();
    };
    createTopic();
    ```
  </Tab>

  <Tab title="Python">
    ```py
    from kafka import KafkaAdminClient
    from kafka.admin import NewTopic

    admin = KafkaAdminClient(
      bootstrap_servers=['{{ BOOTSTRAP_ENDPOINT }}'],
      sasl_mechanism='SCRAM-SHA-512',
      security_protocol='SASL_SSL',
      sasl_plain_username='{{ UPSTASH_KAFKA_USERNAME }}',
      sasl_plain_password='{{ UPSTASH_KAFKA_PASSWORD }}',
    )

    admin.create_topics([NewTopic(name='{{ TOPIC_NAME }}', num_partitions=partitions, replication_factor=replicationFactor)])
    admin.close()
    ```
  </Tab>

  <Tab title="Go">
    ```go
    import (
            "context"
            "crypto/tls"
            "log"

            "github.com/segmentio/kafka-go"
            "github.com/segmentio/kafka-go/sasl/scram"
    )

    func main() {
      mechanism, err := scram.Mechanism(scram.SHA512,
          "{{ UPSTASH_KAFKA_USERNAME }}", "{{ UPSTASH_KAFKA_PASSWORD }}")
      if err != nil {
        log.Fatalln(err)
      }

      dialer := &kafka.Dialer{
        SASLMechanism: mechanism,
        TLS:           &tls.Config{},
      }

      conn, err := dialer.Dial("tcp", "{{ BOOTSTRAP_ENDPOINT }}")
      if err != nil {
          log.Fatalln(err)
      }
      defer conn.Close()

      controller, err := conn.Controller()
      if err != nil {
          log.Fatalln(err)
      }

      controllerConn, err := dialer.Dial("tcp", net.JoinHostPort(controller.Host, strconv.Itoa(controller.Port)))
      if err != nil {
          log.Fatalln(err)
      }
      defer controllerConn.Close()

      err = controllerConn.CreateTopics(kafka.TopicConfig{
                    Topic:             "{{ TOPIC_NAME }}",
                    NumPartitions:     partitions,
                    ReplicationFactor: replicationFactor,
            })
      if err != nil {
          log.Fatalln(err)
      }
    }
    ```
  </Tab>
</Tabs>

## Produce a Message

<Tabs groupId="kafka-clients">
  <Tab title="Java">
    ```java
    class Produce {
      public static void main(String[] args) throws Exception {
        var props = new Properties();
        props.put("bootstrap.servers", "{{ BOOTSTRAP_ENDPOINT }}");
        props.put("sasl.mechanism", "SCRAM-SHA-512");
        props.put("security.protocol", "SASL_SSL");
        props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required " +
                    "username=\"{{ UPSTASH_KAFKA_USERNAME }}\" " +
                    "password=\"{{ UPSTASH_KAFKA_PASSWORD }}\";");
        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
        props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        try (var producer = new KafkaProducer<String, String>(props)) {
            producer.send(new ProducerRecord<>("{{ TOPIC_NAME }}", "Hello Upstash!"));
            producer.flush();
        }
      }
    }
    ```
  </Tab>

  <Tab title="Node.js">
    ```typescript
    const { Kafka } = require("kafkajs");

    const kafka = new Kafka({
      brokers: ["{{ BOOTSTRAP_ENDPOINT }}"],
      sasl: {
        mechanism: "scram-sha-512",
        username: "{{ UPSTASH_KAFKA_USERNAME }}",
        password: "{{ UPSTASH_KAFKA_PASSWORD }}",
      },
      ssl: true,
    });

    const producer = kafka.producer();

    const produce = async () => {
      await producer.connect();
      await producer.send({
        topic: "{{ TOPIC_NAME }}",
        messages: [{ value: "Hello Upstash!" }],
      });
      await producer.disconnect();
    };
    produce();
    ```
  </Tab>

  <Tab title="Python">
    ```py
    from kafka import KafkaProducer

    producer = KafkaProducer(
      bootstrap_servers=['{{ BOOTSTRAP_ENDPOINT }}'],
      sasl_mechanism='SCRAM-SHA-512',
      security_protocol='SASL_SSL',
      sasl_plain_username='{{ UPSTASH_KAFKA_USERNAME }}',
      sasl_plain_password='{{ UPSTASH_KAFKA_PASSWORD }}',
    )

    future = producer.send('{{ TOPIC_NAME }}', b'Hello Upstash!')
    record_metadata = future.get(timeout=10)
    print (record_metadata)
    producer.close()
    ```
  </Tab>

  <Tab title="Go">
    ```go
    import (
            "context"
            "crypto/tls"
            "log"

            "github.com/segmentio/kafka-go"
            "github.com/segmentio/kafka-go/sasl/scram"
    )

    func main() {
      mechanism, err := scram.Mechanism(scram.SHA512,
          "{{ UPSTASH_KAFKA_USERNAME }}", "{{ UPSTASH_KAFKA_PASSWORD }}")
      if err != nil {
        log.Fatalln(err)
      }

      dialer := &kafka.Dialer{
        SASLMechanism: mechanism,
        TLS:           &tls.Config{},
      }

      w := kafka.NewWriter(kafka.WriterConfig{
        Brokers:  []string{"{{ BOOTSTRAP_ENDPOINT }}"},
        Topic:    "{{ TOPIC_NAME }}",
        Dialer:   dialer,
      })
      defer w.Close()

      err = w.WriteMessages(context.Background(),
        kafka.Message{
            Value: []byte("Hello Upstash!"),
        },
      )
      if err != nil {
        log.Fatalln("failed to write messages:", err)
      }
    }
    ```
  </Tab>
</Tabs>

## Consume Messages

<Tabs groupId="kafka-clients">
  <Tab title="Java">
    ```java
    class Consume {
      public static void main(String[] args) throws Exception {
        var props = new Properties();
        props.put("bootstrap.servers", "{{ BOOTSTRAP_ENDPOINT }}");
        props.put("sasl.mechanism", "SCRAM-SHA-512");
        props.put("security.protocol", "SASL_SSL");
        props.put("sasl.jaas.config", "org.apache.kafka.common.security.scram.ScramLoginModule required " +
                    "username=\"{{ UPSTASH_KAFKA_USERNAME }}\" " +
                    "password=\"{{ UPSTASH_KAFKA_PASSWORD }}\";");

        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("auto.offset.reset", "earliest");
        props.put("group.id", "{{ GROUP_NAME }}");

        try(var consumer = new KafkaConsumer<String, String>(props)) {
            consumer.subscribe(Collections.singleton("{{ TOPIC_NAME }}"));
            var records = consumer.poll(Duration.ofSeconds(10));
            for (var record : records) {
                System.out.println(record);
            }
        }
      }
    }
    ```
  </Tab>

  <Tab title="Node.js">
    ```typescript
    const { Kafka } = require("kafkajs");

    const kafka = new Kafka({
      brokers: ["{{ BOOTSTRAP_ENDPOINT }}"],
      sasl: {
        mechanism: "scram-sha-512",
        username: "{{ UPSTASH_KAFKA_USERNAME }}",
        password: "{{ UPSTASH_KAFKA_PASSWORD }}",
      },
      ssl: true,
    });

    const consumer = kafka.consumer({ groupId: "{{ GROUP_NAME }}" });
    const consume = async () => {
      await consumer.connect();
      await consumer.subscribe({ topic: "{{ TOPIC_NAME }}", fromBeginning: true });

      await consumer.run({
        eachMessage: async ({ topic, partition, message }) => {
          console.log({
            topic: topic,
            partition: partition,
            message: JSON.stringify(message),
          });
        },
      });
    };
    consume();
    ```
  </Tab>

  <Tab title="Python">
    ```py
    from kafka import KafkaConsumer

    consumer = KafkaConsumer(
      bootstrap_servers=['{{ BOOTSTRAP_ENDPOINT }}'],
      sasl_mechanism='SCRAM-SHA-512',
      security_protocol='SASL_SSL',
      sasl_plain_username='{{ UPSTASH_KAFKA_USERNAME }}',
      sasl_plain_password='{{ UPSTASH_KAFKA_PASSWORD }}',
      group_id='{{ GROUP_NAME }}',
      auto_offset_reset='earliest',
    )

    consumer.subscribe(['{{ TOPIC_NAME }}'])
    records = consumer.poll(timeout_ms=10000)
    print(records)
    consumer.close()
    ```
  </Tab>

  <Tab title="Go">
    ```go
    import (
            "context"
            "crypto/tls"
            "log"
            "time"

            "github.com/segmentio/kafka-go"
            "github.com/segmentio/kafka-go/sasl/scram"
    )

    func main() {
      mechanism, err := scram.Mechanism(scram.SHA512,
          "{{ UPSTASH_KAFKA_USERNAME }}", "{{ UPSTASH_KAFKA_PASSWORD }}")
      if err != nil {
        log.Fatalln(err)
      }

      dialer := &kafka.Dialer{
        SASLMechanism: mechanism,
        TLS:           &tls.Config{},
      }

      r := kafka.NewReader(kafka.ReaderConfig{
        Brokers:  []string{"{{ BOOTSTRAP_ENDPOINT }}"},
        GroupID: "{{ GROUP_NAME }}",
        Topic:   "{{ TOPIC_NAME }}",
        Dialer:  dialer,
      })
      defer r.Close()

      ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
      defer cancel()

      m, err := r.ReadMessage(ctx)
      if err != nil {
        log.Fatalln(err)
      }
      log.Printf("%+v\n", m)
    }
    ```
  </Tab>
</Tabs>


# Pricing & Limits





# Use Cases



kafkausecases


# Using ksqlDB with Kafka

This tutorial shows how to set up a pipeline to stream traffic events to Upstash Kafka and analyse with ksqlDB

In this tutorial series, we will show how to build an end to end real time
analytics system. We will stream the traffic (click) events from our web
application to Upstash Kafka then we will analyse it on real time. We will
implement one simply query with different stream processing tools:

```sql
SELECT city, count() FROM kafka_topic_page_views where  timestamp > now() - INTERVAL 15 MINUTE group by city
```

Namely, we will query the number of page views from different cities in last 15
minutes. We keep the query and scenario intentionally simple to make the series
easy to understand. But you can easily extend the model for your more complex
realtime analytics scenarios.

If you do not have already set up Kafka pipeline, see
[the first part of series](../integrations/cloudflare_workers) where we
did the set up our pipeline including Upstash Kafka and Cloudflare Workers (or
Vercel).

In this part of the series, we will showcase how to use ksqlDB to run a query on
a Kafka topic.

### ksqlDB Setup

Upstash does not have a managed ksqlDB. In this article we will set up ksqlDB
using Docker Compose.

Create a `docker-compose.yml` file as below:

```yaml
version: "2"

services:
  ksqldb-server:
    image: confluentinc/ksqldb-server:0.28.2
    hostname: ksqldb-server
    container_name: ksqldb-server
    ports:
      - "8088:8088"
    environment:
      KSQL_LISTENERS: "http://0.0.0.0:8088"
      KSQL_BOOTSTRAP_SERVERS: "REPLACE_YOUR_ENDPOINT"
      KSQL_SASL_MECHANISM: "SCRAM-SHA-256"
      KSQL_SECURITY_PROTOCOL: "SASL_SSL"
      KSQL_SASL_JAAS_CONFIG: 'org.apache.kafka.common.security.scram.ScramLoginModule required username="REPLACE_YOUR_USERNAME" password="REPLACE_YOUR_PASSWORD";'
      KSQL_KSQL_LOGGING_PROCESSING_STREAM_AUTO_CREATE: "true"
      KSQL_KSQL_LOGGING_PROCESSING_TOPIC_AUTO_CREATE: "true"
  ksqldb-cli:
    image: confluentinc/ksqldb-cli:0.28.2
    container_name: ksqldb-cli
    depends_on:
      - ksqldb-server
    entrypoint: /bin/sh
    tty: true
```

Don't forget to replace your endpoint, username and password above. Now start
the ksqlDB by running:

```shell
docker-compose up
```

Check your Kafka cluster in Upstash console, you will see new topics auto
created by ksqlDB.

### Start ksqlDB CLI

We will use ksqlDB cli to create streams and run queries. Start the CLI by
running:

```shell
docker exec -it ksqldb-cli ksql http://ksqldb-server:8088
```

### Create a stream

You can think of a stream as a schema on top of a Kafka topic. You can query a
stream and it will not return until it's terminated. New updates are pushed to
the stream. This type of queries is called `push query`.

Let's create a stream:

```sql
CREATE STREAM pageViews (country VARCHAR, city VARCHAR, region VARCHAR, url VARCHAR, ip VARCHAR, mobile VARCHAR, platform VARCHAR, useragent VARCHAR )
WITH (kafka_topic='mytopic', value_format='json', partitions=1);
```

You need to set the same topic which you have created in the
[the first part of series](../integrations/cloudflare_workers).

### Query the stream (push query)

You can query the stream to get the new updates to your Kafka topic:

```sql
SELECT * FROM pageViews EMIT CHANGES;
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/kafka/real-time-analytics/img/ksqldb/streamquery.png" />
</Frame>

The query will continue display the updates until you terminate it.

### Create a table (materialized view)

Now let's create a table to query the cities with the highest number of page
views in last 10 minutes.

```sql
CREATE TABLE topCities AS
  SELECT city, COUNT(*) AS views FROM pageViews
    WINDOW TUMBLING (SIZE 10 MINUTE)
    GROUP BY city
    EMIT CHANGES;
```

We have used tumbling window to count the views. Check
[here](https://docs.ksqldb.io/en/latest/concepts/time-and-windows-in-ksqldb-queries/#window-types)
to learn about the other options.

### Query the table (pull query)

We can simply query the table. This is a pull query, it will return the current
result and terminate.

```sql
select * from topCities
```

!\[ksqldb1.png]\(/img/ksqldb/<Frame><img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/kafka/real-time-analytics/img/ksqldb/tablequery.png" /></Frame>
.png)

You see the results with the same city but different intervals. If you just need
the latest interval (last 10 minutes) then run a query like this:

```sql
select * from topCities where WINDOWSTART > (UNIX_TIMESTAMP() - (10*60*1000+1));
```

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/kafka/real-time-analytics/img/ksqldb/tablequery2.png" />
</Frame>

In this query, we get the results with a starting window of last 10 minutes.

### Resources

[Upstash Kafka setup](../integrations/cloudflare_workers)

[ksqlDB setup](https://ksqldb.io/quickstart.html#quickstart-content)

[ksqlDB concepts](https://docs.ksqldb.io/en/latest/concepts/)

### Conclusion

We have built a simple data pipeline which collect data from edge to Kafka then
create real time reports using SQL. You can easily extend and adapt this example
for much more complex architectures and queries.


# Consumer APIs



Consumer APIs in Kafka are used for fetching and consuming messages from Kafka
topics. Similar to Kafka clients, there are two mechanisms for consuming
messages: manual offset seeking and the use of consumer groups.

Manual offset seeking allows consumers to specify the desired offset from which
they want to consume messages, providing precise control over the consumption
process.

Consumer groups, on the other hand, manage offsets automatically within a
dedicated Kafka topic. They enable multiple consumers to work together in a
coordinated manner, where each consumer within the group is assigned a subset of
partitions from the Kafka topic. This automatic offset management simplifies the
consumption process and facilitates efficient and parallel message processing
across the consumer group.

We call the first one as **Fetch API** and the second one as **Consume API**.
Consume API has some additional methods if you wish to commit offsets manually.

Both Fetch API and Consume API return array of messages as JSON. Message
structure is as following:

```ts
Message {
    topic: String,
    partition: Int,
    offset: Long,
    timestamp: Long,
    key: String,
    value: String,
    headers: Array<Header>
}
```

## Fetch API

`[GET | POST] /fetch`:

Fetches the message(s) starting with a given offset inside the partition. This
API doesn't use consumer groups. A `FetchRequest` should be sent via request
body as JSON. Structure of the `FetchRequest` is:

```ts
FetchRequest{
    topic: String,
    partition: Int,
    offset: Long,
    topicPartitionOffsets: Set<TopicPartitionOffset>,
    timeout: Long
}

TopicPartitionOffset{
    topic: String,
    partition: Int,
    offset: Long
}
```

It's possible to send a fetch request for only a single
`<topic, partition, offset>` or a set of them using `topicPartitionOffsets`.

`timeout` field defines the time to wait at most for the fetch request in
milliseconds. It's optional and its default value 1000.

*   Fetch from a single `<topic, partition, offset>`:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/fetch -u myuser:mypass \
    -d '{"topic": "greetings", "partition": 3, "offset": 11, "timeout": 1000}'
```

*   Fetch from multiple `<topic, partition, offset>` triples:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/fetch -u myuser:mypass \
    -d '{
        "topicPartitionOffsets": [
            {"topic": "greetings", "partition": 1, "offset": 1},
            {"topic": "greetings", "partition": 2, "offset": 1},
            {"topic": "greetings", "partition": 3, "offset": 1},
            {"topic": "cities", "partition": 1, "offset": 10},
            {"topic": "cities", "partition": 2, "offset": 20}
        ],
        "timeout": 5000
    }'
```

*   You can even combine both:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/fetch -u myuser:mypass \
    -d '{
        "topic": "words", "partition": 0, "offset": 0,
        "topicPartitionOffsets": [
            {"topic": "cities", "partition": 1, "offset": 10},
            {"topic": "cities", "partition": 2, "offset": 20}
        ],
        "timeout": 5000
    }'
```

## Consume API

Fetches the message(s) using Kafka consumer group mechanism and may commit the
offsets automatically. Consume API has two variants:

*   `GET /consume/$CONSUMER_GROUP/$INSTANCE_ID/$TOPIC?timeout=$TIMEOUT`
*   `[GET | POST] /consume/$CONSUMER_GROUP/$INSTANCE_ID`

`$CONSUMER_GROUP` is the name of the consumer group which is used as
[Kafka consumer group id](https://kafka.apache.org/documentation/#consumerconfigs_group.id).
`$INSTANCE_ID` is used identify Kafka consumer instances in the same consumer
group. It's used as
[Kafka consumer instance id](https://kafka.apache.org/documentation/#consumerconfigs_group.instance.id).
Each consumer instance is handled by a separate consumer client.

In the second variant, a `ConsumeRequest` should be sent via request body as
JSON. Structure of the `ConsumeRequest` is:

```typescript
ConsumeRequest{
    topic: String,
    topics: Set<String>,
    timeout: Long
}
```

It's possible to send a consume request for only a single `topic` or multiple
`topics`. `timeout` field defines the time to wait at most for the consume
request in milliseconds. It's optional and its default value is `1000`.

Consumer group instances will be closed after some idle time. So consume
requests should be sent periodically to keep them alive.

#### Request Headers

Kafka consumer instance can be configured with request headers. These
headers are only required for the very first request which creates and
initializes the consumer but it's fine to send them with every request and has
no further effect.

*   `Kafka-Enable-Auto-Commit`: If true, the consumer's offset will be
    periodically committed in the background. Valid values are `<true, false>`.
    Default is `true`.
*   `Kafka-Auto-Commit-Interval`: The frequency in milliseconds that the consumer
    offsets are auto-committed to Kafka if auto commit is enabled. Default is
    `5000`.
*   `Kafka-Auto-Offset-Reset`: What to do when there is no initial offset in Kafka
    or if the current offset does not exist any more on the server. Default value
    is `latest`.
    *   `earliest`: Automatically reset the offset to the earliest offset
    *   `latest`: Automatically reset the offset to the latest offset
    *   `none`: Throw exception to the consumer if no previous offset is found for
        the consumer's group.
*   `Kafka-Session-Timeout-Ms`: The timeout used to detect client failures.
    The client sends periodic heartbeats to the broker.  If no heartbeats are received
    by the broker before the expiration of this session timeout, the client will be
    removed from the group and initiate a rebalance. Default is `120000`(2 minutes)

<Tip>
  If all or some of these headers are missing in the consume request, default
  values will be used.
</Tip>

<Warning>
  The first time a consumer is created, it needs to figure out the group
  coordinator by asking the Kafka brokers and joins the consumer group. This
  process takes some time to complete. That's why when a consumer instance is
  created first time, it may return empty messages until consumer group
  coordination is completed.
</Warning>

*   Consume from a single topic:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/consume/mygroup/myconsumer/greetings -u myuser:mypass
```

*   Consume from a single topic with timeout:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/consume/mygroup/myconsumer/greetings?timeout=10000 \
    -u myuser:mypass
```

*   Consume from a single topic via request body:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/consume/mygroup/myconsumer -u myuser:mypass \
    -d '{"topic": "greetings", "timeout": 1000}'
```

*   Consume from multiple topics:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/consume/mygroup/myconsumer -u myuser:mypass \
    -d '{"topics": ["greetings", "cities", "words"], "timeout": 1000}'
```

*   Consume from topics without auto commit:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/consume/mygroup/myconsumer -u myuser:mypass \
    -H "Kafka-Enable-Auto-Commit: false" \
    -d '{"topics": ["greetings", "cities", "words"], "timeout": 1000}'
```

*   Consume from topics starting from the earliest message:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/consume/mygroup/myconsumer -u myuser:mypass \
    -H "Kafka-Auto-Offset-Reset: earliest" \
    -d '{"topics": ["greetings", "cities", "words"], "timeout": 1000}'
```

*   Consume from topics with custom auto commit interval:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/consume/mygroup/myconsumer -u myuser:mypass \
    -H "Kafka-Enable-Auto-Commit: true" \
    -H "Kafka-Auto-Commit-Interval: 3000" \
    -d '{"topics": ["greetings", "cities", "words"], "timeout": 1000}'
```

<Info>
  Maximum number of consumer group instances is limited to total partition count
  of all topics.
</Info>

## Commit Consumer API

`[GET | POST] /commit/$CONSUMER_GROUP/$INSTANCE_ID`

Commits the fetched message offsets. Commit API should be used alongside
**Consume API**, especially when auto commit is disabled. Request body should be
a single `TopicPartitionOffset` object or an array of `TopicPartitionOffset`s as
JSON.

```typescript
TopicPartitionOffset{topic: String, partition: Int, offset: Long}
```

When the body is empty (or an empty array), then the consumer will commit the
last consumed messages.

*   Commit single topic partition offset:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/commit/mygroup/myconsumer -u myuser:mypass \
    -d '{"topic": "cities", "partition": 1, "offset": 10}'
```

*   Commit multiple topic partition offsets:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/commit/mygroup/myconsumer -u myuser:mypass \
    -d '[
        {"topic": "cities", "partition": 0, "offset": 13},
        {"topic": "cities", "partition": 1, "offset": 37},
        {"topic": "greetings", "partition": 0, "offset": 19}
    ]'
```

*   Commit all latest consumed message offsets:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/commit/mygroup/myconsumer -u myuser:mypass
```

**Response:**

When commit is completed, a success JSON result will be returned:

```json
{ "result": "Success", "status": 200 }
```

## Remove Consumer Instance

`[POST|DELETE] /delete-consumer/$CONSUMER_GROUP/$INSTANCE_ID`

Stops and removes a previously created consumer group instance.

**Response:**

When deletion is completed, a success JSON result will be returned:

```json
{ "result": "Success", "status": 200 }
```


# Introduction



Upstash offers a REST API alongside TCP-based Kafka clients, enabling access to
Kafka topics over HTTP. The REST API is particularly valuable in restricted
environments, such as mobile or edge devices, as it provides a lightweight
alternative to native Kafka clients. By utilizing the REST API, you can
eliminate the need for manual management of Kafka clients and connections. It
offers convenience and simplicity for interacting with Kafka topics without the
complexities associated with native client implementations.

## Get Started

If you do not have a Kafka cluster and/or topic already, follow
[these steps](../overall/getstarted) to create one.

In the cluster details section of the
[Upstash Console](https://console.upstash.com), scroll down the `REST API`
section. You will see two basic REST API snippets there; the first one is to
produce a message to a topic and the second one is to consume messages from a
topic using Kafka consumer group mechanism.

*   Producer

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/$TOPIC/$MESSAGE \
  -u {{ UPSTASH_KAFKA_REST_USERNAME }}:{{ UPSTASH_KAFKA_REST_PASSWORD }}
```

*   Consumer

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/consume/$GROUP_NAME/$GROUP_INSTANCE_NAME/$TOPIC \
  -u {{ UPSTASH_KAFKA_REST_USERNAME }}:{{ UPSTASH_KAFKA_REST_PASSWORD }}
```

Upstash Kafka REST API uses HTTP Basic Authentication scheme. You should copy
the `UPSTASH_KAFKA_REST_USERNAME` and `UPSTASH_KAFKA_REST_PASSWORD` from the
console and replace then in the code snippets shown above.

## Produce

To produce a message just replace the `$TOPIC` variable with a topic name which
you've created before and replace the `$MESSAGE` with the message you want to
send to the Kafka topic.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/mytopic/hello_kafka -u myuser:mypass
```

This will send the message to the `mytopic` Kafka topic and return the metadata
related to the message as a JSON, like:

```json
{
  "topic": "mytopic",
  "partition": 0,
  "offset": 0,
  "timestamp": 1637743323016
}
```

<Note>
  For more info and options about producer API please see [REST Producer
  API](./restproducer) section.
</Note>

## Consume

To consume messages from the topic, replace the `$TOPIC` variable with a topic
name which you've created before, replace the `$GROUP_NAME` with a meaningful
name to be used as the
[Kafka consumer group id](https://kafka.apache.org/documentation/#consumerconfigs_group.id),
and `$GROUP_INSTANCE_NAME` with a name for
[Kafka consumer instance id](https://kafka.apache.org/documentation/#consumerconfigs_group.instance.id).

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/consume/mygroup/mygroup_instance0/mytopic -u myuser:mypass
```

This will consume some messages from the `mytopic` using Kafka consumer group
mechanism and return the messages as a JSON array, like:

```json
[
  {
    "topic": "mytopic",
    "partition": 0,
    "offset": 5,
    "timestamp": 1637745824883,
    "key": "",
    "value": "hello-world",
    "headers": []
  },
  {
    "topic": "mytopic",
    "partition": 0,
    "offset": 6,
    "timestamp": 1637745829327,
    "key": "",
    "value": "hello-kafka",
    "headers": []
  },
  {
    "topic": "mytopic",
    "partition": 0,
    "offset": 7,
    "timestamp": 1637745834756,
    "key": "",
    "value": "hello-upstash",
    "headers": []
  }
]
```

<Note>
  For more info and options about consumer API please see [REST Consumer
  APIs](./restconsumer) section.
</Note>

## Responses

Each API returns a JSON response and they have their own specific structures.
When the API call fails for a reason (illegal argument, unauthorized access,
invalid API etc), a common error JSON message returned. Its structure is as
following:

```typescript
{error: String, status: Int}
```

`error` field contains the error message which explains the cause and `status`
field shows the HTTP status code for the error.

## Next

Apart from the basic usages explained in this section, there are three
categories for the REST API:

*   [Producer API](./restproducer)

*   [Consumer APIs](./restconsumer)

*   [Metadata APIs](./restmetadata)


# Metadata API



## List Topics

`GET /topics`

Lists all topics belonging to the user.

**Response:**

JSON Map of topic name to topic's partition:

```json
{
  "cities": 12,
  "greetings": 1,
  "words": 137
}
```

## List Consumers

*   `GET /consumers`

Lists consumers belonging to the user known by the REST server.

**Response:**

JSON array of `GroupAssignments`.

```typescript
TopicAssignments{
    topic: String,
    partitions: Array<Int>
}

InstanceAssignments{
    name: String,
    topics: Array<TopicAssignments>
}

GroupAssignments{
    name: String,
    instances: Array<InstanceAssignments>
}
```

```json
[
  {
    "name": "mygroup",
    "instances": [
      {
        "name": "instance-1",
        "topics": [
          {
            "topic": "cities",
            "partitions": [0, 1, 2]
          },
          {
            "topic": "words",
            "partitions": [10, 21, 32]
          }
        ]
      },
      {
        "name": "instance-2",
        "topics": [
          {
            "topic": "cities",
            "partitions": [3, 4, 5]
          },
          {
            "topic": "words",
            "partitions": [1, 3, 5, 7]
          }
        ]
      }
    ]
  }
]
```

## List Committed Offsets

`[GET | POST] /committed/$CONSUMER_GROUP/$INSTANCE_ID?timeout=$TIMEOUT`

Returns the last committed offsets for the topic partitions inside the group.
Can be used alongside **Commit Consumer API**. Request body should be a single
`TopicPartition` object or an array of `TopicPartition`s:

```typescript
TopicPartition{topic: String, partition: Int}
```

`timeout` parameter defines the time to wait at most for the offsets in
milliseconds. It's optional and its default value is 10 seconds (`10000`).

*   List committed offset for a single topic partition:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/committed/mygroup/myconsumer -u myuser:mypass \
    -d '{"topic": "cities", "partition": 0}'
```

*   List committed offsets for multiple topic partitions:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/committed/mygroup/myconsumer -u myuser:mypass \
    -d '[
        {"topic": "cities", "partition": 0},
        {"topic": "cities", "partition": 1},
        {"topic": "greetings", "partition": 0}
    ]'
```

*   List committed offsets with a `1 second` timeout:

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/committed/mygroup/myconsumer?timeout=1000 -u myuser:mypass \
    -d '{"topic": "cities", "partition": 1}'
```

**Response:**

Returns a JSON array of `TopicPartitionOffset`s:

```json
[
  { "topic": "cities", "partition": 0, "offset": 11 },
  { "topic": "cities", "partition": 1, "offset": 21 },
  { "topic": "greetings", "partition": 0, "offset": 1 }
]
```

## Get Topic Partition Offsets

`[GET | POST] /offsets/$TIMESTAMP?timeout=$TIMEOUT`

Returns the offsets for the given partitions by timestamp. The returned offset
for each partition is the earliest offset whose timestamp is greater than or
equal to the given timestamp in the corresponding partition.

Request body should be a single `TopicPartition` object or an array of
`TopicPartition`s:

```typescript
TopicPartition{topic: String, partition: Int}
```

`timeout` parameter defines the time to wait at most for the offsets in
milliseconds. It's optional and its default value is 10 seconds (`10000`).

*   Offset for a single topic partition:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/1642591892 -u myuser:mypass \
        -d '{"topic": "cities", "partition": 0}'
    ```

*   Offsets for multiple topic partitions:
    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/1642591892 -u myuser:mypass \
        -d '[
            {"topic": "cities", "partition": 0},
            {"topic": "cities", "partition": 1},
            {"topic": "greetings", "partition": 0}
        ]'
    ```

To find out the first and the end offsets for the partitions, `earliest` and
`latest` strings should be used as timestamp values.

*   Beginning offsets for topic partitions:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/earliest -u myuser:mypass \
        -d '[
            {"topic": "cities", "partition": 0},
            {"topic": "cities", "partition": 1},
            {"topic": "greetings", "partition": 0}
        ]'
    ```

*   End offsets for topic partitions:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/latest -u myuser:mypass \
        -d '[
            {"topic": "cities", "partition": 0},
            {"topic": "cities", "partition": 1},
            {"topic": "greetings", "partition": 0}
        ]'
    ```

*   Offsets with a `1 second` timeout:
    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/offsets/latest?timeout=1000 -u myuser:mypass \
        -d '{"topic": "cities", "partition": 1}'
    ```

**Response:**

Returns a JSON array of `TopicPartitionOffset`s:

```json
[
  { "topic": "cities", "partition": 0, "offset": 11 },
  { "topic": "cities", "partition": 1, "offset": 21 },
  { "topic": "greetings", "partition": 0, "offset": 1 }
]
```


# Producer API



Producer API is used to send one or more messages to the same or multiple Kafka
topics.

There are three variants of the Producer API:

1.  `GET /produce/$TOPIC/$MESSAGE?key=$KEY`:

    Sends a single message (`$MESSAGE`) to a topic (`$TOPIC`) using HTTP GET.
    Optionally message key can be appended with a query parameter`?key=$KEY`.

    Without message key:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/greetings/hello_kafka -u myuser:mypass
    ```

    With a message key:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/cities/Istanbul?key=city -u myuser:mypass
    ```

    **Response:**

    A single metadata response is returned as JSON. Its structure is:

    ```typescript
    Metadata{
        topic: String,
        partition: Int,
        offset: Long,
        timestamp: Long
    }
    ```

2.  `[GET | POST] /produce/$TOPIC`:

    Produces one or more messages to a single topic (`$TOPIC`). Messages are sent
    using request body as JSON. Structure of the message JSON is:

    ```typescript
    Header {key: String, value: String}

    Message{
        partition?: Int,
        timestamp?: Long,
        key?: String,
        value: String,
        headers?: Array<Header>
    }
    ```

    Only `value` field is required. `partition`, `timestamp`, `key` and `headers`
    fields are optional. When left blank, `partition` and `timestamp` fields will
    be assigned by Kafka brokers. It's valid to send a single message or array of
    messages as JSON.

    *   Single message with only value:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/greetings -u myuser:mypass \
        -d '{"value": "hello_kafka"}'
    ```

    *   Single message with multiple attributes:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/cities -u myuser:mypass \
        -d '{"partition": 1, "key": "city", "value": "Istanbul", "headers": [{"key": "expire", "value": "1637745834756"}] }'
    ```

    *   Multiple messages with only values:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/greetings -u myuser:mypass \
        -d '[
            {"value": "hello_world"},
            {"value": "hello_upstash"},
            {"value": "hello_kafka"}
        ]'
    ```

    *   Multiple messages with attributes:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce/cities -u myuser:mypass \
        -d '[
            {"partition": 1, "timestamp": 1637745834744, "key": "city", "value": "Istanbul"},
            {"partition": 2, "timestamp": 1637745823147, "key": "city", "value": "London"},
            {"partition": 3, "timestamp": 1637455583829, "key": "city", "value": "Tokyo"},
        ]'
    ```

    **Response:**

    A metadata array response is returned as JSON. Its structure is same as
    above. Each metadata in the response array belongs the message with the same
    order in the request.

3.  `[GET | POST] /produce`:

    Produces one or more messages to a single or multiple topics. Messages are
    sent using request body as JSON. Structure of the message JSON is the same as
    the above, there's only an additional `topic` field:

    ```typescript
    Message{
        topic: String,
        partition?: Int,
        timestamp?: Long,
        key?: String,
        value: String,
        headers?: Array<Header>
    }
    ```

    Only `topic` and `value` fields are required. `partition`, `timestamp`, `key`
    and `headers` fields are optional. When left blank, `partition` and
    `timestamp` fields will be assigned by Kafka brokers. It's valid to send a
    single message or array of messages as JSON.

    *   Single message:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce -u myuser:mypass \
        -d '{"topic": "greetings", "value": "hello_kafka"}'
    ```

    *   Multiple messages for a single topic:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce -u myuser:mypass \
        -d '[
            {"topic": "greetings", "value": "hello_world"},
            {"topic": "greetings", "value": "hello_upstash"},
            {"topic": "greetings", "value": "hello_kafka"}
        ]'
    ```

    *   Multiple messages to multiple topics:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce -u myuser:mypass \
        -d '[
            {"topic": "greetings", "value": "hello_world"},
            {"topic": "greetings", "value": "hello_upstash"},
            {"topic": "cities", "partition": 1, "value": "Istanbul"},
            {"topic": "cities", "partition": 2, "value": "London"},
            {"topic": "cities", "partition": 3, "value": "Tokyo"},
        ]'
    ```

    **Response:**

    A metadata array response is returned as JSON. Its structure is same as
    above. Each metadata in the response array belongs the message with the same
    order in the request.

<Info>
  Producer API works with `acks=all` [producer
  configuration](https://kafka.apache.org/28/documentation.html#producerconfigs_acks),
  which means the topic partition's leader will wait for the all in-sync
  replicas to acknowledge the produced messages.
</Info>


# Webhook API for Kafka



A webhook is a custom *HTTP callback*, which can be triggered by some event from
another service, such as:

*   pushing code to a git repository (e.g.
    [GitHub](https://docs.github.com/en/developers/webhooks-and-events/webhooks/about-webhooks))

*   an app is built and deployed to production (e.g.
    [CircleCI](https://circleci.com/docs/2.0/webhooks/))

*   a new user signed-up to a website (e.g.
    [Auth0](https://auth0.com/docs/customize/hooks))

*   a payment failed (e.g. [Stripe](https://stripe.com/docs/webhooks) )

*   a new order is submitted on an e-commerce app (e.g.
    [Shopify](https://shopify.dev/api/admin-rest/2022-01/resources/webhook#top) )

*   an app fires a failure event on a logging system (e.g.
    [Datadog](https://docs.datadoghq.com/integrations/webhooks/) )

When one of these events occurs, the source service notifies the webhook API by
making a call using an HTTP request. Because webhook APIs are pure HTTP, they
can be added to the existing flows without using another layer, such as
serverless functions, to call the target API.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/rest/webhook.png" className="ss" width="100%" />
</Frame>

Upstash Kafka Webhook API allows to publish these events directly to a user
defined topic without using a third-party infrastructure or service.

Signature of the Webhook API is:

```js
[GET | POST] /webhook?topic=$TOPIC_NAME
```

`topic` parameter is the target Kafka topic name to store events. Request body
is used as the message value and request headers (excluding standard HTTP
headers) are converted to message headers.

Webhook API supports both Basic HTTP Authentication and passing credentials as
query params when the source service does not support HTTP Auth. When Basic HTTP
Auth is not available, `user` and `pass` query parameters should be used to send
Upstash Kafka REST credentials.

*   Usage with Basic HTTP Auth:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/webhook?topic=my-app-events -u myuser:mypass \
        -d 'some event data'
    ```

*   Usage without HTTP Auth:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/webhook?topic=my-app-events&user=myuser&pass=mypass \
        -d 'some event data'
    ```

*   With HTTP Headers:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/webhook?topic=my-app-events -u myuser:mypass \
        -d 'some event data' \
        -H "event-timestamp: 1642628041" \
        -H "event-origin: my-app"
    ```

    Above webhook call is equivalent to:

    ```shell
    curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/produce -u myuser:mypass \
        -d '{
            "topic": "my-app-events",
            "value": "some event data",
            "headers": [
                {"key": "event-timestamp", "value": "1642628041"},
                {"key": "event-origin", "value": "my-app"},
              ]
            }'
    ```


# Compatibility



One of the following compatibility rules can be selected when using the Schema Registry:

*   `BACKWARD`: consumers using the new schema can read data written by producers using the latest registered schema.
*   `BACKWARD_TRANSITIVE`(default): consumers using the new schema can read data written by producers using all previously registered schemas.
*   `FORWARD`: consumers using the latest registered schema can read data written by producers using the new schema.
*   `FORWARD_TRANSITIVE`: consumers using all previously registered schemas can read data written by producers using the new schema.
*   `FULL`: the new schema is forward and backward compatible with the latest registered schema.
*   `FULL_TRANSITIVE`: the new schema is forward and backward compatible with all previously registered schemas.
*   `NONE`: schema compatibility checks are disabled.

To enable the semantics above, you are allowed to make the following changes in each option.
If you choose `TRANSITIVE`, it means that schema will be compared against all versions before,
not only the last one.
The table shows which applications `consumers` / `producers` should be upgraded first.

| Compatibility Type      | What is allowed                             | Upgrade first  |
| ----------------------- | ------------------------------------------- | -------------- |
| `BACKWARD[_TRANSITIVE]` | Delete fields. Add optional fields.         | Consumers      |
| `FORWARD[_TRANSITIVE]`  | Add fields. Delete optional fields.         | Producers      |
| `FULL[_TRANSITIVE]`     | Add optional fields.Delete optional fields. | Any order      |
| `NONE`                  | All changes are accepted.                   | Not Applicable |


# How to



Schema registry can be used in various scenarios. In this page, the configurations for different use-cases are listed.

You can find the related parameters that you need the use in the configurations from [Upstash Console](https://console.upstash.com).
Scroll down to the `REST API` section to find the values you need:

*   `UPSTASH_KAFKA_REST_URL`
*   `UPSTASH_KAFKA_REST_USERNAME`
*   `UPSTASH_KAFKA_REST_PASSWORD`

## Producer/Consumer

### Producer

If you need to configure your producers to use the schema registry, add the following properties to producer properties in addition to the broker configurations. Note that the selected deserializer needs to be schema-registry aware.

```java
Properties props = new Properties();
// ... other configurations like broker.url and broker authentication are skipped
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer, "io.confluent.kafka.serializers.KafkaAvroSerializer");
props.put("schema.registry.url", UPSTASH_KAFKA_REST_URL + "/schema-registry");
props.put("basic.auth.credentials.source", "USER_INFO");
props.put("basic.auth.user.info", UPSTASH_KAFKA_REST_USERNAME + ":" + UPSTASH_KAFKA_REST_PASSWORD);

try (var producer = new KafkaProducer<String, org.apache.avro.GenericRecord>(props)) {
    // ...
  }
```

### Consumer

If you need to configure your consumers to use the schema registry, add the following
properties to consumer properties in addition to the broker configurations.
Note that the selected deserializer needs to be schema-registry aware.

```java
Properties props = new Properties();
// ... other configurations like broker.url and broker authentication are skipped
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer, "io.confluent.kafka.serializers.KafkaAvroDeserializer");
props.put("schema.registry.url", "$UPSTASH_KAFKA_REST_URL/schema-registry");
props.put("basic.auth.credentials.source", "USER_INFO");
props.put("basic.auth.user.info", UPSTASH_KAFKA_REST_USERNAME + ":" + UPSTASH_KAFKA_REST_PASSWORD);

try(var consumer = new KafkaConsumer<String, org.apache.avro.GenericRecord>(props)) {
    // ...
}
```

## Connectors

Some connectors forces you to use a STRUCT as key/value which means that you need a schema and a schema aware convertor to use with the connector.
For this case, you can add the following configurations to your connector.

```json
{
    "name": "myConnector",
    "properties": {
        "connector.class": "io.debezium.connector.mysql.MySqlConnector",
        // other configurations are skipped.

        "key.converter": "io.confluent.connect.avro.AvroConverter",
        "key.converter.upstash.schema.registry.enable": "true",
        "value.converter": "io.confluent.connect.avro.AvroConverter",
        "value.converter.upstash.schema.registry.enable": "true"
    }
}
```

The config above is the shorther version of the following.

```json
{
    "name": "myConnector",
    "properties": {
        "connector.class": "io.debezium.connector.mysql.MySqlConnector",
        // other configurations are skipped.

        "key.converter": "io.confluent.connect.avro.AvroConverter",
        "key.converter.basic.auth.credentials.source": "USER_INFO",
        "key.converter.basic.auth.user.info": "UPSTASH_KAFKA_REST_USERNAME:UPSTASH_KAFKA_REST_PASSWORD",
        "key.converter.schema.registry.url": "UPSTASH_KAFKA_REST_URL/schema-registry",

        "value.converter": "io.confluent.connect.avro.AvroConverter",
        "value.converter.basic.auth.credentials.source": "USER_INFO",
        "value.converter.basic.auth.user.info": "UPSTASH_KAFKA_REST_USERNAME:UPSTASH_KAFKA_REST_PASSWORD",
        "value.converter.schema.registry.url": "UPSTASH_KAFKA_REST_URL/schema-registry"
    }
}
```

## Thirdy party UI tools

You can use third-party UI tools to use the schema registry with.
We have schema registry configuration examples in our `Monitoring` section.

*   [AKHQ](../monitoring/akhq)
*   [kafka-ui](../monitoring/kafka-ui)

## SchemaRegistryClient

[SchemaRegistryClient](https://github.com/confluentinc/schema-registry/blob/master/client/src/main/java/io/confluent/kafka/schemaregistry/client/SchemaRegistryClient.java)
can be used to access the schema registry programmatically. In this case, you can configure it as following:

```java
Map<String, String> configs = new HashMap<>();
configs.put(SchemaRegistryClientConfig.BASIC_AUTH_CREDENTIALS_SOURCE, "USER_INFO");
configs.put(SchemaRegistryClientConfig.USER_INFO_CONFIG, UPSTASH_KAFKA_REST_USERNAME + ":" + UPSTASH_KAFKA_REST_PASSWORD);
var client = new CachedSchemaRegistryClient(UPSTASH_KAFKA_REST_URL + "/schema-registry", 100, configs);
```


# Introduction



Schema Registry serves as a central hub to handle and validate schemas for message data related to Kafka topics.
It also manages serialization and deserialization of data over the network.
This aids producers and consumers in maintaining data consistency and compatibility as schemas change.

Schema Registry by Upstash is API compatible with the Confluent Schema Registry. That means you can use
it with:

*   io.confluent.kafka.serializers.KafkaAvroSerializer/Deserializer
*   io.confluent.connect.avro.AvroConverter
*   Any UI tool that supports Confluent Schema Registry.

See [How to](./schemahowto) to learn how to configure the schema registry to use in various scenarios.

See [Compatibility](./schemacompatibility) page for details of compatibility settings.

See [Rest API](./schemarest) in case you want to directly access the schema registry.


# Rest API



## Register Schema

`POST /subjects/$SUBJECT/versions?normalize=[true/false]`

Registers the schema under given `$SUBJECT` only if the new schema is [compatible](./schemacompatibility).

`normalize` is false by default. If passed, the schema will be normalized.
Normalization enables semantically same but syntactically different schemas to be accounted as the same schema.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/subjects/$SUBJECT/versions -u myuser:mypass -d '
{
  "schema": "{
    \"type\": \"record\",
    \"name\": \"myRecord\",
    \"fields\":
        [
        {\"type\": \"string\",\"name\": \"field1\"},
        {\"type\": \"int\"   ,\"name\": \"field2\"}
        ]
  }",
  "schemaType": "AVRO"
}'
```

**Success Response:**

The schema ID returned as the response:

```json
{"id" : 2}
```

**Fail Response:**

```
- 409 Conflict – Incompatible schema
- 422 Unprocessable Entity
  - Error code 42201 – Invalid schema
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

## Check schema

`POST /subjects/$SUBJECT?normalize=[true/false]&deleted=[true/false]`

Check if the given schema is registered under the `$SUBJECT`. Returns the schema along with subject, version, and schema type.

`normalize` is false by default. If set to `true`, the schema will be normalized.
Normalization enables semantically same but syntactically different schemas to be accounted as the same schema.

`deleted` is false by default. If set to `true`, the soft-deleted schemas under the subject will also be taken into account.
See [Delete Subject](#delete-subject) or [Delete Schema](#delete-schema) for details.

````shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/my-subject/$SUBJECT -u myuser:mypass -d '
{
  "schema": "{
    \"type\": \"record\",
    \"name\": \"myRecord\",
    \"fields\":
        [
        {\"type\": \"string\",\"name\": \"field1\"},
        {\"type\": \"int\"   ,\"name\": \"field2\"}
        ]
  }",
  "schemaType": "AVRO"
}'

**Success Response:**

```json
{
      "subject": "my-subject",
      "id": 2
      "version": 3
      "schema":
         "
          {
            "schema": "{
              \"type\": \"record\",
              \"name\": \"myRecord\",
              \"fields\":
                  [
                  {\"type\": \"string\",\"name\": \"field1\"},
                  {\"type\": \"int\"   ,\"name\": \"field2\"}
                  ]
            }",
            "schemaType": "AVRO"
          }
         "
    }
````

**Fail Response**

```
- 404 Not Found
- Error code 40401 – Subject not found
- Error code 40403 – Schema not found
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

## Set Config

`PUT /config`

Sets the global compatibility. Global compatibility is effective if a subject is not assigned a compatibility by default.
See [Compatibility](./schemacompatibility) for options.

```shell
curl -X PUT https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/config -u myuser:mypass -d '
{
  "compatibility" : "FULL"
}'
```

`PUT /config/$SUBJECT`

Sets the compatibility of a subject.
See [Compatibility](./schemacompatibility) for options.

```shell
curl -X PUT https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/config/my-subject -u myuser:mypass -d '
{
  "compatibility" : "FULL"
}'
```

**Success Response:**

```json
{
  "compatibilityLevel" : "FULL"
}
```

**Fail Response**

```
- 500 Internal Server Error
    - Error code 50001 – Error in the backend data store
    - Error code 50002 – Operation timed out
    - Error code 50003 – Error while forwarding the request to the primary
```

## Get Config

`GET /config`

Retrieves the global config. Note that the default global config is `BACKWARD_TRANSITIVE` by default if not set.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/config -u myuser:mypass
```

`GET /config/$SUBJECT?defaultToGlobal=[true/false]`

Retrieves the config of the given `$SUBJECT`.

`defaultToGlobal` is false by default. When set to `true`, this endpoint will show the effective compatibility on a register
operation. When set to `false`, it may return 404 Not found if a subject level compatibility is not set.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/config/my-subject -u myuser:mypass
```

**Success Response:**

```json
{
  "compatibilityLevel" : "FULL"
}
```

**Fail Response**

```
- 404 Not Found – Subject not found
- 500 Internal Server Error
    - Error code 50001 – Error in the backend data store
    - Error code 50002 – Operation timed out
    - Error code 50003 – Error while forwarding the request to the primary
```

## Get All Schemas

`GET /schemas?deleted=[false/true]`

Returns all schemas registered under subjects. Note that this endpoint will return the same schema multiple times if the same schema registered
under different subjects.

`deleted` is false by default. If set to `true`, the soft-deleted schemas under the subject will also be taken into account.
See [Delete Subject](#delete-subject) or [Delete Schema](#delete-schema) for details.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/schemas -u myuser:mypass
```

**Success Response:**

```json
[
  {
    "subject": "subject1",
    "version": 1,
    "id": 1,
    "schemaType": "AVRO",
    "schema": "{\"type\":\"record\",\"name\":\"test\",\"fields\":[{\"name\":\"field1\",\"type\":\"string\"}]}"
  },
  {
    "subject": "subject2",
    "version": 1,
    "id": 2,
    "schemaType": "AVRO",
    "schema": "{\"type\":\"record\",\"name\":\"test\",\"fields\":[{\"name\":\"field1\",\"type\":\"string\"},{\"name\":\"field2\",\"type\":\"string\",\"default\":\"x\"}]}"
  }
]
```

**Fail Response**

```
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

## Get Schema With SchemaId

`GET /schemas/ids/$SCHEMA_ID`

Returns the schema corresponding to the given `$SCHEMA_ID`.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/schema/ids/1 -u myuser:mypass
```

**Success Response:**

```json
{
  "schema": "{\"type\":\"record\",\"name\":\"test\",\"fields\":[{\"name\":\"field1\",\"type\":\"string\"}]}"
}
```

**Fail Response**

```
- 404 Not Found
  - Error code 40403 – Schema not found
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

`GET /schemas/ids/$SCHEMA_ID/schema`

Returns the schema corresponding to the given `$SCHEMA_ID`. Additionally unwraps the inner schema field.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/schema/ids/1/schema -u myuser:mypass
```

**Success Response:**

```json
{
  "type": "record",
  "name": "test",
  "fields": [
    {
      "name": "field1",
      "type": "string"
    }
  ]
}
```

**Fail Response**

```
- 404 Not Found
  - Error code 40403 – Schema not found
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

## Get Schema With Subject And Version

`GET /subjects/$SUBJECT/versions/$VERSION?deleted=[true/false]`

Returns the schema with its metadata corresponding to the given `$SUBJECT` and `$VERSION`.
VERSION could be an int or string `latest`.

`deleted` is false by default. If set to `true`, the soft-deleted schemas under the subject will also be taken into account.
See [Delete Subject](#delete-subject) or [Delete Schema](#delete-schema) for details.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/subjects/subject1/versions/1 -u myuser:mypass
```

**Success Response:**

```json
{
  "subject": "s1",
  "version": 1,
  "id": 1,
  "schemaType": "AVRO",
  "schema": "{\"type\":\"record\",\"name\":\"test\",\"fields\":[{\"name\":\"field1\",\"type\":\"string\"}]}"
}
```

**Fail Response**

```
- 404 Not Found
  - Error code 40403 – Schema not found
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

`GET /subjects/$SUBJECT/versions/$VERSION?deleted=[true/false]`

Returns only the schema corresponding to the given `$SUBJECT` and `$VERSION`.
VERSION could be an int or string `latest`.

`deleted` is false by default. If set to `true`, the soft-deleted schemas under the subject will also be taken into account.
See [Delete Subject](#delete-subject) or [Delete Schema](#delete-schema) for details.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/subjects/subject1/versions/1/schema -u myuser:mypass
```

**Success Response:**

```json
{
  "type": "record",
  "name": "test",
  "fields": [
    {
      "name": "field1",
      "type": "string"
    }
  ]
}
```

**Fail Response**

```
- 404 Not Found
  - Error code 40401 – Subject not found
  - Error code 40402 – Version not found
- 422 Unprocessable Entity
  - Error code 42202 – Invalid version
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

## Get All Subjects

`GET /subjects?deleted=[false/true]`

Returns all subjects.

`deleted` is false by default. If set to `true`, the soft-deleted subjects will also be taken into account.
See [Delete Subject](#delete-subject) or [Delete Schema](#delete-schema) for details.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/subjects -u myuser:mypass
```

**Success Response:**

```json
["subject1","subject2"]
```

**Fail Response**

```
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

## Get Subject Versions

`GET /subjects/$SUBJECT/versions?deleted=[false/true]`

Returns the versions of the given `$SUBJECT`.

`deleted` is false by default. If set to `true`, the soft-deleted subject versions will also be taken into account.
See [Delete Subject](#delete-subject) or [Delete Schema](#delete-schema) for details.

```shell
curl https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/subjects/subject1/versions -u myuser:mypass
```

**Success Response:**

```json
[ 1, 2 ]
```

**Fail Response**

```
- 404 Not Found
  - Error code 40401 – Subject not found
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

## Delete Subject

`DELETE /subjects/$SUBJECT?permanent=[true/false]`

Deletes the given `$SUBJECT`. Returns the deleted versions as the response.

<Warning>
  This endpoint should rarely be needed in production and needs to be used with caution.
  The use-case for this endpoint is mostly cleaning up the resources after testing in development environments.
</Warning>

`permanent` is false by default. In this case, the subject only will be soft-deleted. The corresponding schemas and schema-ids
will not be deleted. Any serializer/deserializer needing these schemas still will be able to use them.

if `permanent` is set to true. The schemas and schema-ids will also be deleted from the system only if these schemas are not
registered under any other subjects. The schemas will be deleted only after the last related subject is permanently deleted.

Note that a subject can not be permanently deleted before it is soft-deleted.

```shell
curl -X DELETE https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/subjects/my-subject -u myuser:mypass
```

**Success Response:**

Returns the deleted versions as the response.

```json
[1, 2, 3]
```

**Fail Response:**

```
- 404 Not Found
  - Error code 40401 – Subject not found
  - Error code 40404 - Subject '$SUBJECT' was soft deleted. Set permanent=true to delete permanently
  - Error code 40405 - Subject '$SUBJECT' was not deleted first before being permanently deleted
- 422 Unprocessable Entity
  - Error code 42202 – Invalid version
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```

## Delete Schema

`DELETE /subjects/$SUBJECT/versions/$VERSION?permanent=[true/false]`

Deletes the schema corresponding to the `$SUBJECT` and `$VERSION`. Returns the version as the response.
VERSION could be an int or string `latest`.

<Warning>
  This endpoint should rarely be needed in production and needs to be used with caution.
  The use-case for this endpoint is mostly cleaning up the resources after testing in development environments.
</Warning>

`permanent` is false by default. In this case, the subject-version only will be soft-deleted. The corresponding schema and schema-id
will not be deleted. Any serializer/deserializer needing this schema still will be able to use them.

if `permanent` is set to true. The schema and schema-id will also be deleted from the system only if this schema is not
registered under any other subjects. The schemas will be deleted only after the last related subject is permanently deleted.

Note that a subject-version can not be permanently deleted before it is soft-deleted.

```shell
curl -X DELETE https://tops-stingray-7863-eu1-rest-kafka.upstash.io/schema-registry/subjects/my-subject/versions/2 -u myuser:mypass
```

**Success Response:**

Returns the deleted version as the response.

```json
1
```

**Fail Response:**

```
- 404 Not Found
  - Error code 40401 – Subject not found
  - Error code 40402 – Version not found
  - Error code 40406 - Subject '$SUBJECT' Version $VERSION was soft deleted. Set permanent=true to delete permanently
  - Error code 40407 - Subject '$SUBJECT' Version $VERSION was not deleted first before being permanently deleted
- 422 Unprocessable Entity
  - Error code 42202 – Invalid version
- 500 Internal Server Error
  - Error code 50001 – Error in the backend data store
  - Error code 50002 – Operation timed out
  - Error code 50003 – Error while forwarding the request to the primary
```


# Typescript SDK



You can find the Github Repository [here](https://github.com/upstash/upstash-kafka).

## Installation

```bash
npm install @upstash/kafka
```

## Quickstart

1.  Go to [upstash](https://console.upstash.com/kafka) and select your cluster.
2.  Copy the `REST API` secrets at the bottom of the page

```typescript
import { Kafka } from "@upstash/kafka";

const kafka = new Kafka({
  url: "<UPSTASH_KAFKA_REST_URL>",
  username: "<UPSTASH_KAFKA_REST_USERNAME>",
  password: "<UPSTASH_KAFKA_REST_PASSWORD>",
});
```

## Produce a single message

```typescript
const p = kafka.producer();
const message = { hello: "world" }; // Objects will get serialized using `JSON.stringify`
const res = await p.produce("<my.topic>", message);
const res = await p.produce("<my.topic>", message, {
  partition: 1,
  timestamp: 12345,
  key: "<custom key>",
  headers: [{ key: "traceId", value: "85a9f12" }],
});
```

## Produce multiple messages.

The same options from the example above can be set for every message.

```typescript
const p = kafka.producer();
const res = await p.produceMany([
  {
    topic: "my.topic",
    value: { hello: "world" },
    // ...options
  },
  {
    topic: "another.topic",
    value: "another message",
    // ...options
  },
]);
```

## Consume

The first time a consumer is created, it needs to figure out the group
coordinator by asking the Kafka brokers and joins the consumer group. This
process takes some time to complete. That's why when a consumer instance is
created first time, it may return empty messages until consumer group
coordination is completed.

```typescript
const c = kafka.consumer();
const messages = await c.consume({
  consumerGroupId: "group_1",
  instanceId: "instance_1",
  topics: ["test.topic"],
  autoOffsetReset: "earliest",
});
```

More examples can be found in the
[docstring](https://github.com/upstash/upstash-kafka/blob/main/pkg/consumer.ts#L265)

## Commit manually

While `consume` can handle committing automatically, you can also use
`Consumer.commit` to manually commit.

```typescript
const consumerGroupId = "mygroup";
const instanceId = "myinstance";
const topic = "my.topic";

const c = kafka.consumer();
const messages = await c.consume({
  consumerGroupId,
  instanceId,
  topics: [topic],
  autoCommit: false,
});

for (const message of messages) {
  // message handling logic

  await c.commit({
    consumerGroupId,
    instanceId,
    offset: {
      topic: message.topic,
      partition: message.partition,
      offset: message.offset,
    },
  });
}
```

## Fetch

You can also manage offsets manually by using `Consumer.fetch`

```typescript
const c = kafka.consumer();
const messages = await c.fetch({
  topic: "greeting",
  partition: 3,
  offset: 42,
  timeout: 1000,
});
```

## Examples

See [examples](https://github.com/upstash/upstash-kafka/tree/main/examples) as
well as various examples in the docstrings of each method.


# Send Datadog Events to Kafka

This tutorial shows how to send Datadog Events to Upstash Kafka using webhook API

In this post, we will show how to connect Datadog to Upstash Kafka so Datadog
events will be sent to Kafka. We will use the [Webhook API](../rest/webhook)
provided by Upstash.

### Kafka Setup

Create an Upstash Kafka cluster and a topic as explained
[here](https://docs.upstash.com/kafka). In the cluster page, under the Webhook
API section, copy the webhook URL. It should be something like this:

```shell
https://definite-goldfish-14080-us1-rest-kafka.upstash.io/webhook?topic=datadog&user=ZGVmaW5pdGzQy2VXOja9Lkj35hhj&pass=v02ibEOSBgo42TwSZ0BPcIl2ziBk3eg7ITxCmkHwjm
```

You can change the topic parameter depending on which topic you want to send the
Datadog events.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/tutorials/datadog/d1.png" width="100%" />
</Frame>

### Webhook Setup

Login to your Datadog dashboard and click on
[Webhook Integration](https://app.datadoghq.com/account/settings?#integrations/webhooks).

Enter a name for your webhook and paste the webhook URL that you copied from the
Upstash Console. You can also change the payload template and add some custom
headers as described [here](https://docs.datadoghq.com/integrations/webhooks/).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/tutorials/datadog/d2.png" width="100%" />
</Frame>

### Monitor Setup

Now you need to select which events to sent to Kafka. You can either
[create a new monitor](https://app.datadoghq.com/monitors/create) or
[update existing monitors](https://app.datadoghq.com/monitors/manage). At the
`Notify your team` section you need to add your webhook, so the monitor will
start sending new events to Kafka via webhook API.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/kafka/tutorials/datadog/d3.png" width="100%" />
</Frame>

Now you can manually trigger an event and check your Kafka topic to see the
events are coming. Copy/paste the curl consume expression from \[Upstash Console]
to check the new events.

```shell
curl https://definite-goldfish-14224-us1-rest-kafka.upstash.io/consume/GROUP_NAME/GROUP_INSTANCE_NAME/datadogtopic -u \
 ZGVmaW5pdGUtZ29swSZ0BPcfdgfdfg45543tIl2ziBk3eg7ITxCmkHwjmRdN
```


# Get Started with AWS Lambda and Kafka



<Note>
  This tutorial shows how to produce Kafka messages in AWS Lambda. If you want
  to consume Kafka messages in AWS Lambda then check [this
  one](../howto/kafkaproduceinlambda)
</Note>

<Snippet file="kafka/awslambda.mdx" />


# Get Started with Cloudflare Workers and Kafka



TODO: andreas

import CloudflareWorkers from "../howto/kafkaproduceincloudflareworkers";

<CloudflareWorkers />


# Get Started with Next.js and Kafka

This tutorial shows how to use Upstash Kafka with Next.js.

In this post, we will implement the most simple application where we will
publish messages to Kafka from a Next.js application.

### Project Setup

First create a Next project with:

```
➜  kafka-examples git:(master) ✗ npx create-next-app@latest
✔ What is your project named? … getstarted-nextjs
Creating a new Next.js app in /Users/enes/dev/kafka-examples/getstarted-nextjs.
```

Then create an Upstash Kafka cluster and a topic as explained
[here](https://docs.upstash.com/kafka). In the cluster page, under the REST API
section, copy the producer code under the tab `Javascript (fetch)`.

### Implementation

Paste the producer code to the `pages/api/hello.js` as below:

```javascript
export default function handler(req, res) {
  fetch(
    "https://full-mantis-14079-us1-rest-kafka.upstash.io/produce/newtopic/MESSAGE",
    {
      headers: {
        Authorization:
          "Basic Wm5Wc2JDMzUwYVhNdE1UUXlPVDlUR2szT0ZkanZlWUhCVjlKanpvdzAzU25VdFJROjQtUi1mbXRvYWxYbm9ldTlUalFG5qZlNLd0VzRTEwWXZITWlXNjNoRmxqcVVycnE1X3lBcTRUUEdkOWM2SmJxZlE9PQ==",
      },
    }
  )
    .then((response) => response.json())
    .then((data) => {
      console.log(data);
    });

  res.status(200).json({ name: "John Doe" });
}
```

### Run and Deploy

Now you can test your code by running:

```
npm run dev
```

Check:

[http://localhost:3000/api/hello](http://localhost:3000/api/hello)

In the logs you should see the output of Kafka like below:

```json
{
  "topic": "newtopic",
  "partition": 0,
  "offset": 281,
  "timestamp": 1640993860432
}
```


# API Rate Limit Response

This page documents the rate limiting behavior of our API and explains how to handle different types of rate limit errors.

## Overview

Our API implements rate limiting to ensure fair usage and maintain service stability. There are three types of rate limits:

1.  **Daily rate limit**
2.  **Burst rate limit**
3.  **Chat-based rate limit**

When a rate limit is exceeded, the API returns a 429 status code along with specific headers that provide information about the limit, remaining requests/tokens, and reset time.

You can learn more about QStash plans and their limits on the [QStash pricing page](https://upstash.com/pricing/qstash).

### Daily Rate Limit

This is a **daily** limit applied to publish-related API endpoints (new message requests like publish, enqueue, or batch). Other API requests, such as fetching events or messages, do not count toward this limit.

**Headers**:

*   `RateLimit-Limit`: Maximum number of requests allowed per day
*   `RateLimit-Remaining`: Remaining number of requests for the day
*   `RateLimit-Reset`: Time (in unix timestamp) when the daily limit will reset

### Burst Rate Limit

This is a short-term limit **per second** to prevent rapid bursts of requests. This limit applies to all API endpoints.

**Headers**:

*   `Burst-RateLimit-Limit`: Maximum number of requests allowed in the burst window (1 second)
*   `Burst-RateLimit-Remaining`: Remaining number of requests in the burst window (1 second)
*   `Burst-RateLimit-Reset`: Time (in unix timestamp) when the burst limit will reset

### Chat-based Rate Limit

This limit is applied to chat-related API endpoints.

**Headers**:

*   `x-ratelimit-limit-requests`: Maximum number of requests allowed per day
*   `x-ratelimit-limit-tokens`: Maximum number of tokens allowed per day
*   `x-ratelimit-remaining-requests`: Remaining number of requests for the day
*   `x-ratelimit-remaining-tokens`: Remaining number of tokens for the day
*   `x-ratelimit-reset-requests`: Time (in unix timestamp) until the request limit resets
*   `x-ratelimit-reset-tokens`: Time (in unix timestamp) when the token limit will reset

### Example Rate Limit Error Handling

```typescript Handling Daily Rate Limit Error
import { QstashDailyRatelimitError } from "@upstash/qstash";

try {
  // Example of a publish request that could hit the daily rate limit
  const result = await client.publishJSON({
    url: "https://my-api...",
    // or urlGroup: "the name or id of a url group"
    body: {
      hello: "world",
    },
  });
} catch (error) {
  if (error instanceof QstashDailyRatelimitError) {
    console.log("Daily rate limit exceeded. Retry after:", error.reset);
    // Implement retry logic or notify the user
  } else {
    console.error("An unexpected error occurred:", error);
  }
}
```

```typescript Handling Burst Rate Limit Error
import { QstashRatelimitError } from "@upstash/qstash";

try {
  // Example of a request that could hit the burst rate limit
  const result = await client.publishJSON({
    url: "https://my-api...",
    // or urlGroup: "the name or id of a url group"
    body: {
      hello: "world",
    },
  });
} catch (error) {
  if (error instanceof QstashRatelimitError) {
    console.log("Burst rate limit exceeded. Retry after:", error.reset);
    // Implement exponential backoff or delay before retrying
  } else {
    console.error("An unexpected error occurred:", error);
  }
}
```

```typescript Handling Chat-based Rate Limit Error
import { QstashChatRatelimitError, Client, openai } from "@upstash/qstash";

try {
  // Example of a chat-related request that could hit the chat rate limit
  const client = new Client({
    token: "<QSTASH_TOKEN>",
  });

  const result = await client.publishJSON({
    api: {
      name: "llm",
      provider: openai({ token: process.env.OPENAI_API_KEY! }),
    },
    body: {
      model: "gpt-3.5-turbo",
      messages: [
        {
          role: "user",
          content: "Where is the capital of Turkey?",
        },
      ],
    },
    callback: "https://oz.requestcatcher.com/",
  });
} catch (error) {
  if (error instanceof QstashChatRatelimitError) {
    console.log("Chat rate limit exceeded. Retry after:", error.resetRequests);
    // Handle chat-specific rate limiting, perhaps by queueing requests
  } else {
    console.error("An unexpected error occurred:", error);
  }
}
```


# Authentication

Authentication for the QStash API

You'll need to authenticate your requests to access any of the endpoints in the
QStash API. In this guide, we'll look at how authentication works.

## Bearer Token

When making requests to QStash, you will need your `QSTASH_TOKEN` — you will
find it in the [console](https://console.upstash.com/qstash). Here's how to add
the token to the request header using cURL:

```bash
curl https://qstash.upstash.io/v2/publish/... \
  -H "Authorization: Bearer <QSTASH_TOKEN>"
```

## Query Parameter

In environments where setting the header is not possible, you can use the `qstash_token` query parameter instead.

```bash
curl https://qstash.upstash.io/v2/publish/...?qstash_token=<QSTASH_TOKEN>
```

Always keep your token safe and reset it if you suspect it has been compromised.


# Delete a message from the DLQ

DELETE https://qstash.upstash.io/v2/dlq/{dlqId}
Manually remove a message

Delete a message from the DLQ.

## Request

<ParamField path="dlqId" type="string">
  The dlq id of the message you want to remove. You will see this id when
  listing all messages in the dlq with the [/v2/dlq](/qstash/api/dlq/listMessages) endpoint.
</ParamField>

## Response

The endpoint doesn't return anything, a status code of 200 means the message is removed from the DLQ.
If the message is not found in the DLQ, (either is has been removed by you, or automatically), the endpoint returns a 404 status code.

<RequestExample>
  ```sh
  curl -X DELETE https://qstash.upstash.io/v2/dlq/my-dlq-id \
    -H "Authorization: Bearer <token>"
  ```
</RequestExample>


# Delete multiple messages from the DLQ

DELETE https://qstash.upstash.io/v2/dlq
Manually remove messages

Delete multiple messages from the DLQ.

<Info>
  You can get the `dlqId` from the [list DLQs endpoint](/qstash/api/dlq/listMessages).
</Info>

## Request

<ParamField body="dlqIds" type="string[]" required>
  The list of DLQ message IDs to remove.
</ParamField>

## Response

A deleted object with the number of deleted messages.

```JSON
{
  "deleted": number
}
```

<ResponseExample>
  ```json 200 OK
  {
    "deleted": 3
  }
  ```
</ResponseExample>

<RequestExample>
  ```sh curl
  curl -XDELETE https://qstash.upstash.io/v2/dlq \
    -H "Authorization: Bearer <token>" \
    -H "Content-Type: application/json" \
    -d '{
       "dlqIds": ["11111-0", "22222-0", "33333-0"]
      }'
  ```

  ```js Node
  const response = await fetch("https://qstash.upstash.io/v2/dlq", {
    method: "DELETE",
    headers: {
      Authorization: "Bearer <token>",
      "Content-Type": "application/json",
    },
    body: {
      dlqIds: [
        "11111-0",
        "22222-0",
        "33333-0",
      ],
    },
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json',
  }

  data = {
    "dlqIds": [
      "11111-0",
      "22222-0",
      "33333-0"
    ]
  }

  response = requests.delete(
    'https://qstash.upstash.io/v2/dlq',
    headers=headers,
    data=data
  )
  ```

  ```go Go
  var data = strings.NewReader(`{
    "dlqIds": [
      "11111-0",
      "22222-0",
      "33333-0"
    ]
  }`)
  req, err := http.NewRequest("DELETE", "https://qstash.upstash.io/v2/dlq", data)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  req.Header.Set("Content-Type", "application/json")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>


# Get a message from the DLQ

GET https://qstash.upstash.io/v2/dlq/{dlqId}
Get a message from the DLQ

Get a message from DLQ.

## Request

<ParamField path="dlqId" type="string">
  The dlq id of the message you want to retrieve. You will see this id when
  listing all messages in the dlq with the [/v2/dlq](/qstash/api/dlq/listMessages) endpoint,
  as well as in the content of [the failure callback](https://docs.upstash.com/qstash/features/callbacks#what-is-a-failure-callback)
</ParamField>

## Response

If the message is not found in the DLQ, (either is has been removed by you, or automatically), the endpoint returns a 404 status code.

<Snippet file="qstash-dlq-message-type.mdx" />

<RequestExample>
  ```sh
  curl -X GET https://qstash.upstash.io/v2/dlq/my-dlq-id \
    -H "Authorization: Bearer <token>"
  ```
</RequestExample>


# List messages in the DLQ

GET https://qstash.upstash.io/v2/dlq
List and paginate through all messages currently inside the DLQ

List all messages currently inside the DLQ

## Request

<ParamField query="cursor" type="string">
  By providing a cursor you can paginate through all of the messages in the DLQ
</ParamField>

<ParamField query="messageId" type="string">
  Filter DLQ messages by message id.
</ParamField>

<ParamField query="url" type="string">
  Filter DLQ messages by url.
</ParamField>

<ParamField query="topicName" type="string">
  Filter DLQ messages by url group.
</ParamField>

<ParamField query="scheduleId" type="string">
  Filter DLQ messages by schedule id.
</ParamField>

<ParamField query="queueName" type="string">
  Filter DLQ messages by queue name.
</ParamField>

<ParamField query="api" type="string">
  Filter DLQ messages by API name.
</ParamField>

<ParamField query="fromDate" type="number">
  Filter DLQ messages by starting date, in milliseconds (Unix timestamp). This is inclusive.
</ParamField>

<ParamField query="toDate" type="number">
  Filter DLQ messages by ending date, in milliseconds (Unix timestamp). This is inclusive.
</ParamField>

<ParamField query="responseStatus" type="number">
  Filter DLQ messages by HTTP response status code.
</ParamField>

<ParamField query="callerIp" type="string">
  Filter DLQ messages by IP address of the publisher.
</ParamField>

<ParamField query="count" type="number">
  The number of messages to return. Default and maximum is 100.
</ParamField>

<ParamField query="order" type="string">
  The sorting order of DLQ messages by timestamp. Valid values are "earliestFirst" and "latestFirst". The default is "earliestFirst".
</ParamField>

## Response

<ResponseField name="cursor" type="string">
  A cursor which you can use in subsequent requests to paginate through all
  events. If no cursor is returned, you have reached the end of the events.
</ResponseField>

<ResponseField name="messages" type="Array">
  <Expandable defaultOpen title="message">
    <Snippet file="qstash-dlq-message-type.mdx" />
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh
  curl https://qstash.upstash.io/v2/dlq \
    -H "Authorization: Bearer <token>"
  ```

  ```sh with cursor
  curl https://qstash.upstash.io/v2/dlq?cursor=xxx \
    -H "Authorization: Bearer <token>"
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  { 
    "messages": [
      {
        "messageId": "msg_123",
        "topicId": "tpc_123",
        "url":"https://example.com",
        "method": "POST",
        "header": {
          "My-Header": ["my-value"]
        },
        "body": "{\"foo\":\"bar\"}",
        "createdAt": 1620000000000,
        "state": "failed"
      }
    ]
  }
  ```
</ResponseExample>


# Enqueue a Message

POST https://qstash.upstash.io/v2/enqueue/{queueName}/{destination}
Enqueue a message

## Request

<ParamField path="queueName" type="string" required>
  The name of the queue that message will be enqueued on.
  If doesn't exist, it will be created automatically.
</ParamField>

<ParamField path="destination" type="string" required>
  Destination can either be a topic name or id that you configured in the
  Upstash console, a valid url where the message gets sent to, or a valid
  QStash API name like `api/llm`. If the destination is a URL, make sure
  the URL is prefixed with a valid protocol (`http://` or `https://`)
</ParamField>

<Snippet file="qstash-common-request.mdx" />

<ParamField header="Upstash-Deduplication-Id" type="string">
  Id to use while deduplicating messages, so that only one message with
  the given deduplication id is published.
</ParamField>

<ParamField header="Upstash-Content-Based-Deduplication" type="boolean">
  When set to true, automatically deduplicates messages based on their content,
  so that only one message with the same content is published.

  Content based deduplication creates unique deduplication ids based on the
  following message fields:

  *   Destination
  *   Body
  *   Headers
</ParamField>

## Response

<Snippet file="qstash-publish-response.mdx" />

<RequestExample>
  ```sh curl
  curl -X POST "https://qstash.upstash.io/v2/enqueue/myQueue/https://www.example.com" \
    -H "Authorization: Bearer <token>" \
    -H "Content-Type: application/json" \
    -H "Upstash-Method: POST" \
    -H "Upstash-Retries: 3" \
    -H "Upstash-Forward-Custom-Header: custom-value" \
    -d '{"message":"Hello, World!"}'
  ```

  ```js Node
  const response = await fetch(
    "https://qstash.upstash.io/v2/enqueue/myQueue/https://www.example.com",
    {
      method: "POST",
      headers: {
        Authorization: "Bearer <token>",
        "Content-Type": "application/json",
        "Upstash-Method": "POST",
        "Upstash-Retries": "3",
        "Upstash-Forward-Custom-Header": "custom-value",
      },
      body: JSON.stringify({
        message: "Hello, World!",
      }),
    }
  );
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json',
      'Upstash-Method': 'POST',
      'Upstash-Retries': '3',
      'Upstash-Forward-Custom-Header': 'custom-value',
  }

  json_data = {
      'message': 'Hello, World!',
  }

  response = requests.post(
    'https://qstash.upstash.io/v2/enqueue/myQueue/https://www.example.com',
     headers=headers,
     json=json_data
  )
  ```

  ```go Go
  var data = strings.NewReader(`{"message":"Hello, World!"}`)
  req, err := http.NewRequest("POST", "https://qstash.upstash.io/v2/enqueue/myQueue/https://www.example.com", data)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  req.Header.Set("Content-Type", "application/json")
  req.Header.Set("Upstash-Method", "POST")
  req.Header.Set("Upstash-Retries", "3")
  req.Header.Set("Upstash-Forward-Custom-Header", "custom-value")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json URL
  {
    "messageId": "msd_1234",
    "url": "https://www.example.com"
  }
  ```

  ```json URL Group
  [
    {
      "messageId": "msd_1234",
      "url": "https://www.example.com"
    },
    {
      "messageId": "msd_5678",
      "url": "https://www.somewhere-else.com",
      "deduplicated": true
    }
  ]
  ```
</ResponseExample>


# List Events

GET https://qstash.upstash.io/v2/events
List all events that happened, such as message creation or delivery

## Request

<ParamField query="cursor" type="string">
  By providing a cursor you can paginate through all of the events.
</ParamField>

<ParamField query="messageId" type="string">
  Filter events by message id.
</ParamField>

<ParamField query="state" type="string">
  Filter events by [state](/qstash/howto/debug-logs)
</ParamField>

<ParamField query="url" type="string">
  Filter events by url.
</ParamField>

<ParamField query="topicName" type="string">
  Filter events by URL Group (topic) name.
</ParamField>

<ParamField query="scheduleId" type="string">
  Filter events by schedule id.
</ParamField>

<ParamField query="queueName" type="string">
  Filter events by queue name.
</ParamField>

<ParamField query="fromDate" type="number">
  Filter events by starting date, in milliseconds (Unix timestamp). This is inclusive.
</ParamField>

<ParamField query="toDate" type="number">
  Filter events by ending date, in milliseconds (Unix timestamp). This is inclusive.
</ParamField>

<ParamField query="count" type="number">
  The number of events to return. Default and max is 1000.
</ParamField>

<ParamField query="order" type="string">
  The sorting order of events by timestamp. Valid values are "earliestFirst" and "latestFirst". The default is "latestFirst".
</ParamField>

## Response

<ResponseField name="cursor" type="string">
  A cursor which you can use in subsequent requests to paginate through all events.
  If no cursor is returned, you have reached the end of the events.
</ResponseField>

<ResponseField name="events" type="Array">
  <Expandable defaultOpen>
    <ResponseField name="time" type="int" required>
      Timestamp of this log entry, in milliseconds
    </ResponseField>

    <ResponseField name="messageId" type="string" required>
      The associated message id
    </ResponseField>

    <ResponseField name="header" type="Record<string, string[]>" required>
      The headers of the message.
    </ResponseField>

    <ResponseField name="body" type="string" required>
      Base64 encoded body of the message.
    </ResponseField>

    <ResponseField name="state" type="string" required>
      The current state of the message at this point in time.

      | Value              | Description                                                                              |
      | ------------------ | ---------------------------------------------------------------------------------------- |
      | `CREATED`          | The message has been accepted and stored in QStash                                       |
      | `ACTIVE`           | The task is currently being processed by a worker.                                       |
      | `RETRY`            | The task has been scheduled to retry.                                                    |
      | `ERROR`            | The execution threw an error and the task is waiting to be retried or failed.            |
      | `DELIVERED`        | The message was successfully delivered.                                                  |
      | `FAILED`           | The task has errored too many times or encountered an error that it cannot recover from. |
      | `CANCEL_REQUESTED` | The cancel request from the user is recorded.                                            |
      | `CANCELLED`        | The cancel request from the user is honored.                                             |
    </ResponseField>

    <ResponseField name="error" type="string" optional>
      An explanation what went wrong
    </ResponseField>

    <ResponseField name="nextDeliveryTime" type="int">
      The next scheduled time of the message.
      (Unix timestamp in milliseconds)
    </ResponseField>

    <ResponseField name="url" type="string">
      The destination url
    </ResponseField>

    <ResponseField name="topicName" type="string">
      The name of the URL Group (topic) if this message was sent through a topic
    </ResponseField>

    <ResponseField name="endpointName" type="int">
      The name of the endpoint if this message was sent through a URL Group
    </ResponseField>

    <ResponseField name="scheduleId" type="string">
      The scheduleId of the message if the message is triggered by a schedule
    </ResponseField>

    <ResponseField name="queueName" type="string">
      The name of the queue if this message is enqueued on a queue
    </ResponseField>

    <ResponseField name="header" type="string">
      The headers that are forwarded to the users endpoint
    </ResponseField>

    <ResponseField name="body" type="string">
      Base64 encoded body of the message
    </ResponseField>

    <ResponseField name="responseStatus" type="int">
      The status code of the response. Only set if the state is `ERROR`
    </ResponseField>

    <ResponseField name="responseBody" type="string">
      The base64 encoded body of the response. Only set if the state is `ERROR`
    </ResponseField>

    <ResponseField name="responseHeaders" type="Record<string, string[]>">
      The headers of the response. Only set if the state is `ERROR`
    </ResponseField>

    <ResponseField name="timeout" type="int">
      The timeout(in milliseconds) of the outgoing http requests, after which Qstash cancels the request
    </ResponseField>

    <ResponseField name="method" type="string">
      Method is the HTTP method of the message for outgoing request
    </ResponseField>

    <ResponseField name="callback" type="string">
      Callback is the URL address where QStash sends the response of a publish
    </ResponseField>

    <ResponseField name="callbackHeaders" type="Record<string, string[]>">
      The headers that are passed to the callback url
    </ResponseField>

    <ResponseField name="failureCallback" type="string">
      Failure Callback is the URL address where QStash sends the response of a publish
    </ResponseField>

    <ResponseField name="failureCallbackHeaders" type="Record<string, string[]>">
      The headers that are passed to the failure callback url
    </ResponseField>

    <ResponseField name="maxRetries" type="int">
      The number of retries that should be attempted in case of delivery failure
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/events \
    -H "Authorization: Bearer <token>"
  ```

  ```javascript Node
  const response = await fetch("https://qstash.upstash.io/v2/events", {
    headers: {
      Authorization: "Bearer <token>",
    },
  });
  ```

  ```python Python
  import requests
  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.get(
    'https://qstash.upstash.io/v2/events',
     headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/events", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "cursor": "1686652644442-12",
    "events":[
      {
        "time": "1686652644442",
        "messageId": "msg_123",
        "state": "delivered",
        "url": "https://example.com",
        "header": { "Content-Type": [ "application/x-www-form-urlencoded" ] },
        "body": "bWVyaGFiYSBiZW5pbSBhZGltIHNhbmNhcg=="
      }
    ] 
  }
  ```
</ResponseExample>


# Create Chat Completion

POST https://qstash.upstash.io/llm/v1/chat/completions
Creates a chat completion of one or more messages

Creates a chat completion that generates a textual response
for one or more messages using a large language model.

## Request

<ParamField body="model" type="string" required>
  Name of the model.
</ParamField>

<ParamField body="messages" type="Object[]" required>
  One or more chat messages.

  <Expandable defaultOpen="true">
    <ParamField body="role" type="string" required>
      The role of the message author. One of `system`, `assistant`, or `user`.
    </ParamField>

    <ParamField body="content" type="string" required>
      The content of the message.
    </ParamField>

    <ParamField body="name" type="string">
      An optional name for the participant.
      Provides the model information to differentiate between participants of the same role.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField body="frequency_penalty" type="number">
  Number between `-2.0` and `2.0`. Positive values penalize new tokens based on their existing
  frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim.
</ParamField>

<ParamField body="logit_bias" type="Object">
  Modify the likelihood of specified tokens appearing in the completion.

  Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer)
  to an associated bias value from `-100` to `100`. Mathematically, the bias is added to
  the logits generated by the model prior to sampling. The exact effect will vary
  per model, but values between `-1` and `1` should decrease or increase likelihood
  of selection; values like `-100` or `100` should result in a ban or exclusive
  selection of the relevant token.
</ParamField>

<ParamField body="logprobs" type="boolean">
  Whether to return log probabilities of the output tokens or not. If true, returns
  the log probabilities of each output token returned in the content of message.
</ParamField>

<ParamField body="top_logprobs" type="number">
  An integer between `0` and `20` specifying the number of most likely tokens to return at
  each token position, each with an associated log probability. logprobs must be set
  to true if this parameter is used.
</ParamField>

<ParamField body="max_tokens" type="number">
  The maximum number of tokens that can be generated in the chat completion.
</ParamField>

<ParamField body="n" type="number">
  How many chat completion choices to generate for each input message.

  Note that you will be charged based on the number of generated tokens
  across all of the choices. Keep `n` as `1` to minimize costs.
</ParamField>

<ParamField body="presence_penalty" type="number">
  Number between `-2.0` and `2.0`. Positive values penalize new tokens
  based on whether they appear in the text so far, increasing the
  model's likelihood to talk about new topics.
</ParamField>

<ParamField body="response_format" type="Object">
  An object specifying the format that the model must output.

  Setting to `{ "type": "json_object" }` enables JSON mode,
  which guarantees the message the model generates is valid JSON.

  **Important**: when using JSON mode, you must also instruct the model
  to produce JSON yourself via a system or user message. Without this,
  the model may generate an unending stream of whitespace until the
  generation reaches the token limit, resulting in a long-running and
  seemingly "stuck" request. Also note that the message content may
  be partially cut off if `finish_reason="length"`, which indicates the
  generation exceeded max\_tokens or the conversation exceeded the max context length.

  <Expandable defaultOpen="true">
    <ParamField body="type" type="string" required>
      Must be one of `text` or `json_object`.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField body="seed" type="number">
  This feature is in Beta. If specified, our system will make a best effort to sample
  deterministically, such that repeated requests with the same seed and parameters
  should return the same result. Determinism is not guaranteed, and you should
  refer to the `system_fingerprint` response parameter to monitor changes in the backend.
</ParamField>

<ParamField body="stop" type="string[]">
  Up to 4 sequences where the API will stop generating further tokens.
</ParamField>

<ParamField body="stream" type="boolean">
  If set, partial message deltas will be sent. Tokens will be sent as
  data-only server-sent events as they become available, with the stream
  terminated by a `data: [DONE]` message.
</ParamField>

<ParamField body="temperature" type="number">
  What sampling temperature to use, between `0` and `2`. Higher values
  like `0.8` will make the output more random, while lower values
  like `0.2` will make it more focused and deterministic.

  We generally recommend altering this or `top_p` but not both.
</ParamField>

<ParamField body="top_p" type="number">
  An alternative to sampling with temperature, called nucleus sampling,
  where the model considers the results of the tokens with `top_p`
  probability mass. So `0.1` means only the tokens comprising the top
  \`10%\`\` probability mass are considered.

  We generally recommend altering this or `temperature` but not both.
</ParamField>

## Response

Returned when `stream` is `false` or not set.

<ResponseField name="id" type="string">
  A unique identifier for the chat completion.
</ResponseField>

<ResponseField name="choices" type="Object[]">
  A list of chat completion choices. Can be more than one if `n` is greater than `1`.

  <Expandable defaultOpen="true">
    <ResponseField name="message" type="Object">
      A chat completion message generated by the model.

      <Expandable defaultOpen="true">
        <ResponseField name="role" type="string">
          The role of the author of this message.
        </ResponseField>

        <ResponseField name="content" type="string">
          The contents of the message.
        </ResponseField>
      </Expandable>
    </ResponseField>

    <ResponseField name="finish_reason" type="string">
      The reason the model stopped generating tokens. This will be `stop` if the
      model hit a natural stop point or a provided stop sequence, `length` if
      the maximum number of tokens specified in the request was reached.
    </ResponseField>

    <ResponseField name="stop_reason" type="string">
      The stop string or token id that caused the completion to stop,
      null if the completion finished for some other reason including
      encountering the EOS token
    </ResponseField>

    <ResponseField name="index" type="number">
      The index of the choice in the list of choices.
    </ResponseField>

    <ResponseField name="logprobs" type="Object">
      Log probability information for the choice.

      <Expandable defaultOpen="false">
        <ResponseField name="content" type="Object[]">
          A list of message content tokens with log probability information.

          <Expandable defaultOpen="false">
            <ResponseField name="token" type="string">
              The token.
            </ResponseField>

            <ResponseField name="logprob" type="number">
              The log probability of this token, if it is within the top 20 most likely tokens.
              Otherwise, the value `-9999.0` is used to signify that the token is very unlikely.
            </ResponseField>

            <ResponseField name="bytes" type="number[]">
              A list of integers representing the UTF-8 bytes representation of the token.
              Useful in instances where characters are represented by multiple tokens and
              their byte representations must be combined to generate the correct text
              representation. Can be null if there is no bytes representation for the token.
            </ResponseField>

            <ResponseField name="top_logprobs" type="Object[]">
              List of the most likely tokens and their log probability, at this token position.
              In rare cases, there may be fewer than the number of requested `top_logprobs` returned.

              <Expandable defaultOpen="false">
                <ResponseField name="token" type="string">
                  The token.
                </ResponseField>

                <ResponseField name="logprob" type="number">
                  The log probability of this token, if it is within the top 20 most likely tokens.
                  Otherwise, the value `-9999.0` is used to signify that the token is very unlikely.
                </ResponseField>

                <ResponseField name="bytes" type="number[]">
                  A list of integers representing the UTF-8 bytes representation of the token.
                  Useful in instances where characters are represented by multiple tokens and
                  their byte representations must be combined to generate the correct text
                  representation. Can be null if there is no bytes representation for the token.
                </ResponseField>
              </Expandable>
            </ResponseField>
          </Expandable>
        </ResponseField>
      </Expandable>
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="created" type="number">
  The Unix timestamp (in seconds) of when the chat completion was created.
</ResponseField>

<ResponseField name="model" type="string">
  The model used for the chat completion.
</ResponseField>

<ResponseField name="system_fingerprint" type="string">
  This fingerprint represents the backend configuration that the model runs with.

  Can be used in conjunction with the `seed` request parameter to understand
  when backend changes have been made that might impact determinism.
</ResponseField>

<ResponseField name="object" type="string">
  The object type, which is always `chat.completion`.
</ResponseField>

<ResponseField name="usage" type="Object">
  Usage statistics for the completion request.

  <Expandable defaultOpen="true">
    <ResponseField name="completion_tokens" type="number">
      Number of tokens in the generated completion.
    </ResponseField>

    <ResponseField name="prompt_tokens" type="number">
      Number of tokens in the prompt.
    </ResponseField>

    <ResponseField name="total_tokens" type="number">
      Total number of tokens used in the request (prompt + completion).
    </ResponseField>
  </Expandable>
</ResponseField>

## Stream Response

Returned when `stream` is `true`.

<ResponseField name="id" type="string">
  A unique identifier for the chat completion. Each chunk has the same ID.
</ResponseField>

<ResponseField name="choices" type="Object[]">
  A list of chat completion choices. Can be more than one if `n` is greater than `1`.
  Can also be empty for the last chunk.

  <Expandable defaultOpen="true">
    <ResponseField name="delta" type="Object">
      A chat completion delta generated by streamed model responses.

      <Expandable defaultOpen="true">
        <ResponseField name="role" type="string">
          The role of the author of this message.
        </ResponseField>

        <ResponseField name="content" type="string">
          The contents of the chunk message.
        </ResponseField>
      </Expandable>
    </ResponseField>

    <ResponseField name="finish_reason" type="string">
      The reason the model stopped generating tokens. This will be `stop` if the
      model hit a natural stop point or a provided stop sequence, `length` if
      the maximum number of tokens specified in the request was reached.
    </ResponseField>

    <ResponseField name="index" type="number">
      The index of the choice in the list of choices.
    </ResponseField>

    <ResponseField name="logprobs" type="Object">
      Log probability information for the choice.

      <Expandable defaultOpen="false">
        <ResponseField name="content" type="Object[]">
          A list of message content tokens with log probability information.

          <Expandable defaultOpen="false">
            <ResponseField name="token" type="string">
              The token.
            </ResponseField>

            <ResponseField name="logprob" type="number">
              The log probability of this token, if it is within the top 20 most likely tokens.
              Otherwise, the value `-9999.0` is used to signify that the token is very unlikely.
            </ResponseField>

            <ResponseField name="bytes" type="number[]">
              A list of integers representing the UTF-8 bytes representation of the token.
              Useful in instances where characters are represented by multiple tokens and
              their byte representations must be combined to generate the correct text
              representation. Can be null if there is no bytes representation for the token.
            </ResponseField>

            <ResponseField name="top_logprobs" type="Object[]">
              List of the most likely tokens and their log probability, at this token position.
              In rare cases, there may be fewer than the number of requested `top_logprobs` returned.

              <Expandable defaultOpen="false">
                <ResponseField name="token" type="string">
                  The token.
                </ResponseField>

                <ResponseField name="logprob" type="number">
                  The log probability of this token, if it is within the top 20 most likely tokens.
                  Otherwise, the value `-9999.0` is used to signify that the token is very unlikely.
                </ResponseField>

                <ResponseField name="bytes" type="number[]">
                  A list of integers representing the UTF-8 bytes representation of the token.
                  Useful in instances where characters are represented by multiple tokens and
                  their byte representations must be combined to generate the correct text
                  representation. Can be null if there is no bytes representation for the token.
                </ResponseField>
              </Expandable>
            </ResponseField>
          </Expandable>
        </ResponseField>
      </Expandable>
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="created" type="number">
  The Unix timestamp (in seconds) of when the chat completion was created. Each chunk has the same timestamp.
</ResponseField>

<ResponseField name="model" type="string">
  The model used for the chat completion.
</ResponseField>

<ResponseField name="system_fingerprint" type="string">
  This fingerprint represents the backend configuration that the model runs with.

  Can be used in conjunction with the `seed` request parameter to understand
  when backend changes have been made that might impact determinism.
</ResponseField>

<ResponseField name="object" type="string">
  The object type, which is always `chat.completion.chunk`.
</ResponseField>

<ResponseField name="usage" type="Object">
  it contains a null value except for the last chunk which contains the token usage statistics for the entire request.

  <Expandable defaultOpen="true">
    <ResponseField name="completion_tokens" type="number">
      Number of tokens in the generated completion.
    </ResponseField>

    <ResponseField name="prompt_tokens" type="number">
      Number of tokens in the prompt.
    </ResponseField>

    <ResponseField name="total_tokens" type="number">
      Total number of tokens used in the request (prompt + completion).
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl "https://qstash.upstash.io/llm/v1/chat/completions" \
      -X POST \
      -H "Authorization: Bearer QSTASH_TOKEN" \
      -H "Content-Type: application/json" \
      -d '{
          "model": "meta-llama/Meta-Llama-3-8B-Instruct",
          "messages": [
              {
                  "role": "user",
                  "content": "What is the capital of Turkey?"
              }
          ]
      }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "id": "cmpl-abefcf66fae945b384e334e36c7fdc97",
    "object": "chat.completion",
    "created": 1717483987,
    "model": "meta-llama/Meta-Llama-3-8B-Instruct",
    "choices": [
      {
        "index": 0,
        "message": {
          "role": "assistant",
          "content": "The capital of Turkey is Ankara."
        },
        "logprobs": null,
        "finish_reason": "stop",
        "stop_reason": null
      }
    ],
    "usage": {
      "prompt_tokens": 18,
      "total_tokens": 26,
      "completion_tokens": 8
    }
  }
  ```

  ```json 200 OK - Stream
  data: {"id":"cmpl-dfc1ad80d0254c2aaf3e7775d1830c9d","object":"chat.completion.chunk","created":1717484084,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"role":"assistant"},"logprobs":null,"finish_reason":null}]}

  data: {"id":"cmpl-dfc1ad80d0254c2aaf3e7775d1830c9d","object":"chat.completion.chunk","created":1717484084,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"content":"The"},"logprobs":null,"finish_reason":null}]}

  data: {"id":"cmpl-dfc1ad80d0254c2aaf3e7775d1830c9d","object":"chat.completion.chunk","created":1717484084,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"content":" capital"},"logprobs":null,"finish_reason":null}]}

  data: {"id":"cmpl-dfc1ad80d0254c2aaf3e7775d1830c9d","object":"chat.completion.chunk","created":1717484084,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"content":" of"},"logprobs":null,"finish_reason":null}]}

  data: {"id":"cmpl-dfc1ad80d0254c2aaf3e7775d1830c9d","object":"chat.completion.chunk","created":1717484084,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"content":" Turkey"},"logprobs":null,"finish_reason":null}]}

  data: {"id":"cmpl-dfc1ad80d0254c2aaf3e7775d1830c9d","object":"chat.completion.chunk","created":1717484084,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"content":" is"},"logprobs":null,"finish_reason":null}]}

  data: {"id":"cmpl-dfc1ad80d0254c2aaf3e7775d1830c9d","object":"chat.completion.chunk","created":1717484084,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"content":" Ankara"},"logprobs":null,"finish_reason":null}]}

  data: {"id":"cmpl-dfc1ad80d0254c2aaf3e7775d1830c9d","object":"chat.completion.chunk","created":1717484084,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"content":"."},"logprobs":null,"finish_reason":null}]}

  data: {"id":"cmpl-dfc1ad80d0254c2aaf3e7775d1830c9d","object":"chat.completion.chunk","created":1717484084,"model":"meta-llama/Meta-Llama-3-8B-Instruct","choices":[{"index":0,"delta":{"content":""},"finish_reason":"stop"}],"usage":{"prompt_tokens":18,"total_tokens":26,"completion_tokens":8}}

  data: [DONE]
  ```
</ResponseExample>


# Batch Messages

POST https://qstash.upstash.io/v2/batch
Send multiple messages in a single request

You can learn more about batching in the [batching section](/qstash/features/batch).

<Note>
  API playground is not available for this endpoint. You can use the cURL example below.
</Note>

<Info>You can publish to destination, URL Group or queue in the same batch request.</Info>

## Request

The endpoint is `POST https://qstash.upstash.io/v2/batch` and the body is an array of
messages. Each message has the following fields:

```
destination: string
headers: headers object
body: string
```

The headers are identical to the headers in the [create](/qstash/api/publish#request) endpoint.

```shell cURL
curl -XPOST https://qstash.upstash.io/v2/batch   -H "Authorization: Bearer XXX" \
    -H "Content-Type: application/json" \
    -d '
    [
      {
          "destination": "myUrlGroup",
          "headers":{
            "Upstash-Delay":"5s",
            "Upstash-Forward-Hello":"123456"
          },
          "body": "Hello World"
      },
      {
          "queue": "test",
          "destination": "https://example.com/destination",
          "headers":{
            "Upstash-Forward-Hello":"789"
          }
      },
      {
          "destination": "https://example.com/destination1",
          "headers":{
            "Upstash-Delay":"7s",
            "Upstash-Forward-Hello":"789"
          }
      },
      {
          "destination": "https://example.com/destination2",
          "headers":{
            "Upstash-Delay":"9s",
            "Upstash-Forward-Hello":"again"
          }
      }
    ]'
```

## Response

```json
[
  [
    {
      "messageId": "msg_...",
      "url": "https://myUrlGroup-endpoint1.com"
    },
    {
      "messageId": "msg_...",
      "url": "https://myUrlGroup-endpoint2.com"
    }
  ],
  {
    "messageId": "msg_...",
  },
  {
    "messageId": "msg_..."
  },
  {
    "messageId": "msg_..."
  }
]
```


# Bulk Cancel Messages

DELETE https://qstash.upstash.io/v2/messages
Stop delivery of multiple messages at once

Bulk cancel allows you to cancel multiple messages at once.

<Note>
  Cancelling a message will remove it from QStash and stop it from being delivered
  in the future. If a message is in flight to your API, it might be too late to
  cancel.
</Note>

<Warning>
  If you provide a set of message IDs in the body of the request, only those messages will be cancelled.
  If no body is sent, QStash will cancel all of your messages.
</Warning>

This operation scans all your messages and attempts to cancel them.
If an individual message cannot be cancelled, it will not continue and will return an error message.
Therefore, some messages may not be cancelled at the end.
In such cases, you can run the bulk cancel operation multiple times.

## Request

<ParamField body="messageIds" type="Array">
  The list of message IDs to cancel
</ParamField>

## Response

A cancelled object with the number of cancelled messages.

```JSON
{
  "cancelled": number
}
```

<RequestExample>
  ```sh curl
  curl -XDELETE https://qstash.upstash.io/v2/messages/ \
     -H "Content-Type: application/json" \
    -H "Authorization: Bearer <token>" \
    -d '{"messageIds": ["msg_id_1", "msg_id_2", "msg_id_3"]}'
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/messages', {
    method: 'DELETE',
    headers: {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json',
      body: {
          messageIds: [
              "msg_id_1",
              "msg_id_2",
              "msg_id_3",
          ],
      },
    }
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json',
  }

  data = {
    "messageIds": [
      "msg_id_1",
      "msg_id_2",
      "msg_id_3"
    ]
  }

  response = requests.delete(
    'https://qstash.upstash.io/v2/messages',
    headers=headers,
    data=data
  )
  ```

  ```go Go
  var data = strings.NewReader(`{
    "messageIds": [
      "msg_id_1",
      "msg_id_2",
      "msg_id_3"
    ]
  }`)
  req, err := http.NewRequest("DELETE", "https://qstash.upstash.io/v2/messages", data)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  req.Header.Set("Content-Type", "application/json")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 202 Accepted
  {
    "cancelled": 10
  }
  ```
</ResponseExample>


# Cancel Message

DELETE https://qstash.upstash.io/v2/messages/{messageId}
Stop delivery of an existing message

Cancelling a message will remove it from QStash and stop it from being delivered
in the future. If a message is in flight to your API, it might be too late to
cancel.

## Request

<ParamField path="messageId" type="string" required>
  The id of the message to cancel.
</ParamField>

## Response

This endpoint only returns `202 OK`

<RequestExample>
  ```sh curl
  curl -XDELETE https://qstash.upstash.io/v2/messages/msg_123 \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/messages/msg_123', {
    method: 'DELETE',
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.delete(
    'https://qstash.upstash.io/v2/messages/msg_123', 
    headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("DELETE", "https://qstash.upstash.io/v2/messages/msg_123", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```text 202 Accepted
  OK
  ```
</ResponseExample>


# Get Message

GET https://qstash.upstash.io/v2/messages/{messageId}
Retrieve a message by its id

## Request

<ParamField path="messageId" type="string" required>
  The id of the message to retrieve.
</ParamField>

<Warning>
  Messages are removed from the database shortly after they're delivered, so you
  will not be able to retrieve a message after. This endpoint is intended to be used
  for accessing messages that are in the process of being delivered/retried.
</Warning>

## Response

<Snippet file="qstash-message-type.mdx" />

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/messages/msg_123 \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch("https://qstash.upstash.io/v2/messages/msg_123", {
    headers: {
      Authorization: "Bearer <token>",
    },
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.get(
    'https://qstash.upstash.io/v2/messages/msg_123',
    headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/messages/msg_123", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "messageId": "msg_123",
    "topicName": "myTopic",
    "url":"https://example.com",
    "method": "POST",
    "header": {
      "My-Header": ["my-value"]
    },
    "body": "{\"foo\":\"bar\"}",
    "createdAt": 1620000000000
  }
  ```
</ResponseExample>


# Publish a Message

POST https://qstash.upstash.io/v2/publish/{destination}
Publish a message

## Request

<ParamField path="destination" type="string" required>
  Destination can either be a topic name or id that you configured in the
  Upstash console, a valid url where the message gets sent to, or a valid
  QStash API name like `api/llm`. If the destination is a URL, make sure
  the URL is prefixed with a valid protocol (`http://` or `https://`)
</ParamField>

<Snippet file="qstash-common-request.mdx" />

<ParamField header="Upstash-Delay" type="string">
  Delay the message delivery.

  Format for this header is a number followed by duration abbreviation, like
  `10s`. Available durations are `s` (seconds), `m` (minutes), `h` (hours), `d`
  (days).

  example: "50s" | "3m" | "10h" | "1d"
</ParamField>

<ParamField header="Upstash-Not-Before" type="int">
  Delay the message delivery until a certain time in the future.

  The format is a unix timestamp in seconds, based on the UTC timezone.

  When both `Upstash-Not-Before` and `Upstash-Delay` headers are provided,
  `Upstash-Not-Before` will be used.
</ParamField>

<ParamField header="Upstash-Deduplication-Id" type="string">
  Id to use while deduplicating messages, so that only one message with
  the given deduplication id is published.
</ParamField>

<ParamField header="Upstash-Content-Based-Deduplication" type="boolean">
  When set to true, automatically deduplicates messages based on their content,
  so that only one message with the same content is published.

  Content based deduplication creates unique deduplication ids based on the
  following message fields:

  *   Destination
  *   Body
  *   Headers
</ParamField>

## Response

<Snippet file="qstash-publish-response.mdx" />

<RequestExample>
  ```sh curl
  curl -X POST "https://qstash.upstash.io/v2/publish/https://www.example.com" \
    -H "Authorization: Bearer <token>" \
    -H "Content-Type: application/json" \
    -H "Upstash-Method: POST" \
    -H "Upstash-Delay: 10s" \
    -H "Upstash-Retries: 3" \
    -H "Upstash-Forward-Custom-Header: custom-value" \
    -d '{"message":"Hello, World!"}'
  ```

  ```js Node
  const response = await fetch(
    "https://qstash.upstash.io/v2/publish/https://www.example.com",
    {
      method: "POST",
      headers: {
        Authorization: "Bearer <token>",
        "Content-Type": "application/json",
        "Upstash-Method": "POST",
        "Upstash-Delay": "10s",
        "Upstash-Retries": "3",
        "Upstash-Forward-Custom-Header": "custom-value",
      },
      body: JSON.stringify({
        message: "Hello, World!",
      }),
    }
  );
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json',
      'Upstash-Method': 'POST',
      'Upstash-Delay': '10s',
      'Upstash-Retries': '3',
      'Upstash-Forward-Custom-Header': 'custom-value',
  }

  json_data = {
      'message': 'Hello, World!',
  }

  response = requests.post(
    'https://qstash.upstash.io/v2/publish/https://www.example.com',
     headers=headers,
     json=json_data
  )
  ```

  ```go Go
  var data = strings.NewReader(`{"message":"Hello, World!"}`)
  req, err := http.NewRequest("POST", "https://qstash.upstash.io/v2/publish/https://www.example.com", data)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  req.Header.Set("Content-Type", "application/json")
  req.Header.Set("Upstash-Method", "POST")
  req.Header.Set("Upstash-Delay", "10s")
  req.Header.Set("Upstash-Retries", "3")
  req.Header.Set("Upstash-Forward-Custom-Header", "custom-value")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json URL
  {
    "messageId": "msd_1234",
    "url": "https://www.example.com"
  }
  ```

  ```json URL Group
  [
    {
      "messageId": "msd_1234",
      "url": "https://www.example.com"
    },
    {
      "messageId": "msd_5678",
      "url": "https://www.somewhere-else.com",
      "deduplicated": true
    }
  ]
  ```
</ResponseExample>


# Get a Queue

GET https://qstash.upstash.io/v2/queues/{queueName}
Retrieves a queue

## Request

<ParamField path="queueName" type="string" required>
  The name of the queue to retrieve.
</ParamField>

## Response

<ResponseField name="createdAt" type="int" required>
  The creation time of the queue. UnixMilli
</ResponseField>

<ResponseField name="updatedAt" type="int" required>
  The update time of the queue. UnixMilli
</ResponseField>

<ResponseField name="name" type="string" required>
  The name of the queue.
</ResponseField>

<ResponseField name="parallelism" type="int" required>
  The number of parallel consumers consuming from [the queue](/qstash/features/queues).
</ResponseField>

<ResponseField name="lag" type="int" required>
  The number of unprocessed messages that exist in [the queue](/qstash/features/queues).
</ResponseField>

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/queues/my-queue \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/queue/my-queue', {
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python 
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.get(
    'https://qstash.upstash.io/v2/queue/my-queue',
     headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/queue/my-queue", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
   
    "createdAt": 1623345678001,
    "updatedAt": 1623345678001,
    "name": "my-queue",
    "parallelism" : 5, 
    "lag" : 100
  }
  ```
</ResponseExample>


# List Queues

GET https://qstash.upstash.io/v2/queues
List all your queues

## Request

No parameters

## Response

<ResponseField name="" type="Array" required>
  <Expandable defaultOpen>
    <ResponseField name="createdAt" type="int" required>
      The creation time of the queue. UnixMilli
    </ResponseField>

    <ResponseField name="updatedAt" type="int" required>
      The update time of the queue. UnixMilli
    </ResponseField>

    <ResponseField name="name" type="string" required>
      The name of the queue.
    </ResponseField>

    <ResponseField name="parallelism" type="int" required>
      The number of parallel consumers consuming from [the queue](/qstash/features/queues).
    </ResponseField>

    <ResponseField name="lag" type="int" required>
      The number of unprocessed messages that exist in [the queue](/qstash/features/queues).
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/queues \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch("https://qstash.upstash.io/v2/queues", {
    headers: {
      Authorization: "Bearer <token>",
    },
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.get(
    'https://qstash.upstash.io/v2/queues',
    headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/queues", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
    {
      "createdAt": 1623345678001,
      "updatedAt": 1623345678001,
      "name": "my-queue",
      "parallelism" : 5, 
      "lag" : 100
    },
    // ...
  ]
  ```
</ResponseExample>


# Pause Queue

POST https://qstash.upstash.io/v2/queues/{queueName}/pause
Pause an active queue

Pausing a queue stops the delivery of enqueued messages.
The queue will still accept new messages, but they will wait until the queue becomes active for delivery.
If the queue is already paused, this action has no effect.

## Request

<ParamField path="queueName" type="string" required>
  The name of the queue to pause.
</ParamField>

## Response

This endpoint simply returns 200 OK if the queue is paused successfully.

<RequestExample>
  ```sh curl
  curl -X POST https://qstash.upstash.io/v2/queues/queue_1234/pause \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  import { Client } from "@upstash/qstash";
  /**
   * Import a fetch polyfill only if you are using node prior to v18.
   * This is not necessary for nextjs, deno or cloudflare workers.
   */
  import "isomorphic-fetch";

  const c = new Client({
    token: "<QSTASH_TOKEN>",
  });

  c.queue({ queueName: "<QUEUE_NAME>" }).pause()
  ```

  ```python Python 
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")

  client.queue.pause("<QUEUE_NAME>")
  ```

  ```go Go
  package main

  import (
  	"github.com/upstash/qstash-go"
  )

  func main() {
  	client := qstash.NewClient("<QSTASH_TOKEN>")

  	// error checking is omitted for brevity
  	err := client.Queues().Pause("<QUEUE_NAME>")
  }
  ```
</RequestExample>


# Remove a Queue

DELETE https://qstash.upstash.io/v2/queues/{queueName}
Removes a queue

## Request

<ParamField path="queueName" type="string" required>
  The name of the queue to remove.
</ParamField>

## Response

This endpoint returns 200 if the queue is removed successfully,
or it doesn't exist.

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/queues/my-queue \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/queue/my-queue', {
    method: "DELETE",
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python 
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.delete(
    'https://qstash.upstash.io/v2/queue/my-queue',
     headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("DELETE", "https://qstash.upstash.io/v2/queue/my-queue", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>


# Resume Queue

POST https://qstash.upstash.io/v2/queues/{queueName}/resume
Resume a paused queue

Resuming a queue starts the delivery of enqueued messages from the earliest undelivered message.
If the queue is already active, this action has no effect.

## Request

<ParamField path="queueName" type="string" required>
  The name of the queue to resume.
</ParamField>

## Response

This endpoint simply returns 200 OK if the queue is resumed successfully.

<RequestExample>
  ```sh curl
  curl -X POST https://qstash.upstash.io/v2/queues/queue_1234/resume \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  import { Client } from "@upstash/qstash";
  /**
   * Import a fetch polyfill only if you are using node prior to v18.
   * This is not necessary for nextjs, deno or cloudflare workers.
   */
  import "isomorphic-fetch";

  const c = new Client({
    token: "<QSTASH_TOKEN>",
  });

  c.queue({ queueName: "<QUEUE_NAME>" }).resume()
  ```

  ```python Python 
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")

  client.queue.resume("<QUEUE_NAME>")
  ```

  ```go Go
  package main

  import (
  	"github.com/upstash/qstash-go"
  )

  func main() {
  	client := qstash.NewClient("<QSTASH_TOKEN>")

  	// error checking is omitted for brevity
  	err := client.Queues().Resume("<QUEUE_NAME>")
  }
  ```
</RequestExample>


# Upsert a Queue

POST https://qstash.upstash.io/v2/queues/
Updates or creates a queue

## Request

<ParamField body="queueName" type="string" required>
  The name of the queue.
</ParamField>

<ParamField body="parallelism" type="int" required>
  The number of parallel consumers consuming from [the queue](/qstash/features/queues).
</ParamField>

## Response

This endpoint returns

*   200 if the queue is added successfully.
*   412 if it fails because of the the allowed number of queues limit

<RequestExample>
  ```sh curl
  curl -XPOST https://qstash.upstash.io/v2/queues/ \
    -H "Authorization: Bearer <token>" \
    -H "Content-Type: application/json" \
    -d '{
      "queueName": "my-queue" , 
      "parallelism" : 5,
    }'
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/queues/', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      "queueName": "my-queue" , 
      "parallelism" : 5,
    })
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json',
  }

  json_data = {
      "queueName": "my-queue" , 
      "parallelism" : 5,
    }

  response = requests.post(
    'https://qstash.upstash.io/v2/queues/',
    headers=headers, 
    json=json_data
  )
  ```

  ```go Go 
  var data = strings.NewReader(`{
      "queueName": "my-queue" , 
      "parallelism" : 5,
    }`)
  req, err := http.NewRequest("POST", "https://qstash.upstash.io/v2/queues/", data)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  req.Header.Set("Content-Type", "application/json")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>


# Create Schedule

POST https://qstash.upstash.io/v2/schedules/{destination}
Create a schedule to send messages periodically

## Request

<ParamField path="destination" type="string" required>
  Destination can either be a topic name or id that you configured in the
  Upstash console or a valid url where the message gets sent to.
  If the destination is a URL, make sure
  the URL is prefixed with a valid protocol (`http://` or `https://`)
</ParamField>

<ParamField header="Upstash-Cron" type="string" required>
  Cron allows you to send this message periodically on a schedule.

  Add a Cron expression and we will requeue this message automatically whenever
  the Cron expression triggers. We offer an easy to use UI for creating Cron
  expressions in our [console](https://console.upstash.com/qstash) or you can
  check out [Crontab.guru](https://crontab.guru).

  Note: it can take up to 60 seconds until the schedule is registered on an
  available qstash node.

  Example: `*/5 * * * *`
</ParamField>

<Snippet file="qstash-common-request.mdx" />

<ParamField header="Upstash-Delay" type="string">
  Delay the message delivery.

  Delay applies to the delivery of the scheduled messages.
  For example, with the delay set to 10 minutes for a schedule
  that runs everyday at 00:00, the scheduled message will be
  created at 00:00 and it will be delivered at 00:10.

  Format for this header is a number followed by duration abbreviation, like
  `10s`. Available durations are `s` (seconds), `m` (minutes), `h` (hours), `d`
  (days).

  example: "50s" | "3m" | "10h" | "1d"
</ParamField>

<ParamField header="Upstash-Schedule-Id" type="string">
  Assign a schedule id to the created schedule.

  This header allows you to set the schedule id yourself instead of QStash assigning
  a random id.

  If a schedule with the provided id exists, the settings of the existing schedule
  will be updated with the new settings.
</ParamField>

## Response

<ResponseField name="scheduleId" type="string" required>
  The unique id of this schedule. You can use it to delete the schedule later.
</ResponseField>

<RequestExample>
  ```sh curl
  curl -XPOST https://qstash.upstash.io/v2/schedules/https://www.example.com/endpoint \
    -H "Authorization: Bearer <token>" \
    -H "Upstash-Cron: */5 * * * *"
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/schedules/https://www.example.com/endpoint', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <token>',
      'Upstash-Cron': '*/5 * * * *'
    }
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
      'Upstash-Cron': '*/5 * * * *'
  }

  response = requests.post(
    'https://qstash.upstash.io/v2/schedules/https://www.example.com/endpoint',
     headers=headers
  )
  ```

  ```go Go 
  req, err := http.NewRequest("POST", "https://qstash.upstash.io/v2/schedules/https://www.example.com/endpoint", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  req.Header.Set("Upstash-Cron", "*/5 * * * *")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "scheduleId": "scd_1234"
  }
  ```
</ResponseExample>


# Get Schedule

GET https://qstash.upstash.io/v2/schedules/{scheduleId}
Retrieves a schedule by id.

## Request

<ParamField path="scheduleId" type="string" required>
  The id of the schedule to retrieve.
</ParamField>

## Response

<ResponseField name="createdAt" type="int" required>
  The creation time of the object. UnixMilli
</ResponseField>

<ResponseField name="scheduleId" type="string" required>
  The id of the schedule.
</ResponseField>

<ResponseField name="cron" type="string" required>
  The cron expression used to schedule the message.
</ResponseField>

<ResponseField name="callerIP" type="string" required>
  IP address where this schedule created from.
</ResponseField>

<ResponseField name="destination" type="string" required>
  Url or URL Group name
</ResponseField>

<ResponseField name="method" type="string" required>
  The HTTP method to use for the message.
</ResponseField>

<ResponseField name="header" type="Record<string, string[]>" required>
  The headers of the message.
</ResponseField>

<ResponseField name="body" type="string" required>
  The body of the message.
</ResponseField>

<ResponseField name="retries" type="int">
  The number of retries that should be attempted in case of delivery failure.
</ResponseField>

<ResponseField name="delay" type="int">
  The delay in seconds before the message is delivered.
</ResponseField>

<ResponseField name="callback" type="string">
  The url where we send a callback to after the message is delivered
</ResponseField>

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/schedules/scd_1234 \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/schedules/scd_1234', {
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python 
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.get(
    'https://qstash.upstash.io/v2/schedules/scd_1234', 
    headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/schedules/scd_1234", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "scheduleId": "scd_1234",
    "createdAt": 1623345678001,
    "cron": "0 0 1 * *",
    "destination": "https://example.com",
    "method": "POST",
    "header": {
      "Content-Type": ["application/json"]
    },
    "body": "{\"message\":\"hello\"}",
    "retries": 3
  }
  ```
</ResponseExample>


# List Schedules

GET https://qstash.upstash.io/v2/schedules
List all your schedules

## Response

<ResponseField name="array" type="Object[]">
  <ResponseField name="createdAt" type="int" required>
    The creation time of the object. UnixMilli
  </ResponseField>

  <ResponseField name="id" type="string" required>
    The id of the schedule.
  </ResponseField>

  <ResponseField name="cron" type="string" required>
    The cron expression used to schedule the message.
  </ResponseField>

  <ResponseField name="destination" type="string" required>
    Url or URL Group (topic) name
  </ResponseField>

  <ResponseField name="method" type="string" required>
    The HTTP method to use for the message.
  </ResponseField>

  <ResponseField name="header" type="Record<string, string[]>" required>
    The headers of the message.
  </ResponseField>

  <ResponseField name="body" type="string" required>
    The body of the message.
  </ResponseField>

  <ResponseField name="retries" type="int">
    The number of retries that should be attempted in case of delivery failure.
  </ResponseField>

  <ResponseField name="delay" type="int">
    The delay in seconds before the message is delivered.
  </ResponseField>

  <ResponseField name="callback" type="string">
    The url where we send a callback to after the message is delivered
  </ResponseField>
</ResponseField>

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/schedules \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/schedules', {
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python 
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.get(
    'https://qstash.upstash.io/v2/schedules', 
    headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/schedules", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
    {
      "scheduleId": "scd_1234",
      "createdAt": 1623345678001,
      "cron": "0 0 1 * *",
      "destination": "https://example.com",
      "method": "POST",
      "header": {
        "Content-Type": ["application/json"]
      },
      "body": "{\"message\":\"hello\"}",
      "retries": 3
    }
  ]
  ```
</ResponseExample>


# Pause Schedule

POST https://qstash.upstash.io/v2/schedules/{scheduleId}/pause
Pause an active schedule

Pausing a schedule will not change the next delivery time, but the delivery will be ignored.
If the schedule is already paused, this action has no effect.

## Request

<ParamField path="scheduleId" type="string" required>
  The id of the schedule to pause.
</ParamField>

## Response

This endpoint simply returns 200 OK if the schedule is paused successfully.

<RequestExample>
  ```sh curl
  curl -X POST https://qstash.upstash.io/v2/schedules/scd_1234/pause \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  import { Client } from "@upstash/qstash";
  /**
   * Import a fetch polyfill only if you are using node prior to v18.
   * This is not necessary for nextjs, deno or cloudflare workers.
   */
  import "isomorphic-fetch";

  const c = new Client({
    token: "<QSTASH_TOKEN>",
  });

  c.schedules.pause({
    schedule: "<SCHEDULE_ID>"
  });

  ```

  ```python Python 
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")

  client.schedule.pause("<SCHEDULE_ID>")
  ```

  ```go Go
  package main

  import "github.com/upstash/qstash-go"

  func main() {

  	client := qstash.NewClient("<QSTASH_TOKEN>")

  	// error checking is omitted for brevity
  	err := client.Schedules().Pause("<SCHEDULE_ID>")
  }
  ```
</RequestExample>


# Remove Schedule

DELETE https://qstash.upstash.io/v2/schedules/{scheduleId}
Remove a schedule

## Request

<ParamField path="scheduleId" type="string">
  The schedule id to remove
</ParamField>

## Response

This endpoint simply returns 200 OK if the schedule is removed successfully.

<RequestExample>
  ```sh curl
  curl -XDELETE https://qstash.upstash.io/v2/schedules/scd_123 \
    -H "Authorization: Bearer <token>"
  ```

  ```javascript Node
  const response = await fetch('https://qstash.upstash.io/v2/schedules/scd_123', {
    method: 'DELETE',
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.delete(
    'https://qstash.upstash.io/v2/schedules/scd_123', 
    headers=headers
  )
  ```

  ```go Go 
  req, err := http.NewRequest("DELETE", "https://qstash.upstash.io/v2/schedules/scd_123", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>


# Resume Schedule

POST https://qstash.upstash.io/v2/schedules/{scheduleId}/resume
Resume a paused schedule

Resuming a schedule marks the schedule as active.
This means the upcoming messages will be delivered and will not be ignored.
If the schedule is already active, this action has no effect.

## Request

<ParamField path="scheduleId" type="string" required>
  The id of the schedule to resume.
</ParamField>

## Response

This endpoint simply returns 200 OK if the schedule is resumed successfully.

<RequestExample>
  ```sh curl
  curl -X POST https://qstash.upstash.io/v2/schedules/scd_1234/resume \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  import { Client } from "@upstash/qstash";
  /**
   * Import a fetch polyfill only if you are using node prior to v18.
   * This is not necessary for nextjs, deno or cloudflare workers.
   */
  import "isomorphic-fetch";

  const c = new Client({
    token: "<QSTASH_TOKEN>",
  });

  c.schedules.resume({
    schedule: "<SCHEDULE_ID>"
  });

  ```

  ```python Python 
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")

  client.schedule.resume("<SCHEDULE_ID>")
  ```

  ```go Go
  package main

  import "github.com/upstash/qstash-go"

  func main() {

  	client := qstash.NewClient("<QSTASH_TOKEN>")

  	// error checking is omitted for brevity
  	err := client.Schedules().Resume("<SCHEDULE_ID>")
  }
  ```
</RequestExample>


# Get Signing Keys

GET https://qstash.upstash.io/v2/keys
Retrieve your signing keys

## Response

<ResponseField name="current" type="string" required>
  Your current signing key.
</ResponseField>

<ResponseField name="next" type="string" required>
  The next signing key.
</ResponseField>

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/keys \
    -H "Authorization: Bearer <token>"
  ```

  ```javascript Node
  const response = await fetch('https://qstash.upstash.io/v2/keys', {
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.get(
    'https://qstash.upstash.io/v2/keys',
     headers=headers
  )
  ```

  ```go Go 
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/keys", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  { "current": "sig_123", "next": "sig_456" }
  ```
</ResponseExample>


# Rotate Signing Keys

POST https://qstash.upstash.io/v2/keys/rotate
Rotate your signing keys

## Response

<ResponseField name="current" type="string" required>
  Your current signing key.
</ResponseField>

<ResponseField name="next" type="string" required>
  The next signing key.
</ResponseField>

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/keys/rotate \
    -H "Authorization: Bearer <token>"
  ```

  ```javascript Node
  const response = await fetch('https://qstash.upstash.io/v2/keys/rotate', {
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python 
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }
  response = requests.get(
    'https://qstash.upstash.io/v2/keys/rotate', 
    headers=headers
  )
  ```

  ```go Go 
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/keys/rotate", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  { "current": "sig_123", "next": "sig_456" }
  ```
</ResponseExample>


# Upsert URL Group and Endpoint

POST https://qstash.upstash.io/v2/topics/{urlGroupName}/endpoints
Add an endpoint to a URL Group

If the URL Group does not exist, it will be created. If the endpoint does not exist, it will be created.

## Request

<ParamField path="urlGroupName" type="string" required>
  The name of your URL Group (topic). If it doesn't exist yet, it will be created.
</ParamField>

<ParamField body="endpoints" type="Array" required>
  The endpoints to add to the URL Group.

  <Expandable defaultOpen>
    <ParamField body="name" type="string">
      The name of the endpoint
    </ParamField>

    <ParamField body="url" type="string" required>
      The URL of the endpoint
    </ParamField>
  </Expandable>
</ParamField>

## Response

This endpoint returns 200 if the endpoints are added successfully.

<RequestExample>
  ```sh curl
  curl -XPOST https://qstash.upstash.io/v2/topics/:urlGroupName/endpoints \
    -H "Authorization: Bearer <token>" \
    -H "Content-Type: application/json" \
    -d '{
      "endpoints": [
        {
          "name": "endpoint1",
          "url": "https://example.com"
        },
        {
          "name": "endpoint2",
          "url": "https://somewhere-else.com"
        }
      ]
    }'
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/topics/:urlGroupName/endpoints', {
    method: 'POST',
    headers: {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      'endpoints': [
        {
          'name': 'endpoint1',
          'url': 'https://example.com'
        },
        {
          'name': 'endpoint2',
          'url': 'https://somewhere-else.com'
        }
      ]
    })
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json',
  }

  json_data = {
      'endpoints': [
          {
              'name': 'endpoint1',
              'url': 'https://example.com',
          },
          {
              'name': 'endpoint2',
              'url': 'https://somewhere-else.com',
          },
      ],
  }

  response = requests.post(
    'https://qstash.upstash.io/v2/topics/:urlGroupName/endpoints',
    headers=headers, 
    json=json_data
  )
  ```

  ```go Go 
  var data = strings.NewReader(`{
    "endpoints": [
      {
        "name": "endpoint1",
        "url": "https://example.com"
      },
      {
        "name": "endpoint2",
        "url": "https://somewhere-else.com"
      }
    ]
  }`)
  req, err := http.NewRequest("POST", "https://qstash.upstash.io/v2/topics/:urlGroupName/endpoints", data)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  req.Header.Set("Content-Type", "application/json")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>


# Get a URL Group

GET https://qstash.upstash.io/v2/topics/{urlGroupName}
Retrieves a URL Group

## Request

<ParamField path="urlGroupName" type="string" required>
  The name of the URL Group (topic) to retrieve.
</ParamField>

## Response

<ResponseField name="createdAt" type="int" required>
  The creation time of the URL Group. UnixMilli
</ResponseField>

<ResponseField name="updatedAt" type="int" required>
  The update time of the URL Group. UnixMilli
</ResponseField>

<ResponseField name="name" type="string" required>
  The name of the URL Group.
</ResponseField>

<ResponseField name="endpoints" type="Array" required>
  <Expandable openDefault>
    <ParamField body="name" type="string">
      The name of the endpoint
    </ParamField>

    <ParamField body="url" type="string" required>
      The URL of the endpoint
    </ParamField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/topics/my-url-group \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/topics/my-url-group', {
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python 
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.get(
    'https://qstash.upstash.io/v2/topics/my-url-group',
     headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/topics/my-url-group", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
   
    "createdAt": 1623345678001,
    "updatedAt": 1623345678001,
    "name": "my-url-group",
    "endpoints": [
      {
        "name": "my-endpoint",
        "url": "https://my-endpoint.com"
      }
    ]
  }
  ```
</ResponseExample>


# List URL Groups

GET https://qstash.upstash.io/v2/topics
List all your URL Groups

## Request

No parameters

## Response

<ResponseField name="" type="Array" required>
  <Expandable defaultOpen>
    <ResponseField name="createdAt" type="int" required>
      The creation time of the URL Group. UnixMilli
    </ResponseField>

    <ResponseField name="updatedAt" type="int" required>
      The update time of the URL Group. UnixMilli
    </ResponseField>

    <ResponseField name="name" type="string" required>
      The name of the URL Group.
    </ResponseField>

    <ResponseField name="endpoints" type="Array" required>
      <Expandable defaultOpen>
        <ParamField body="name" type="string" required>
          The name of the endpoint.
        </ParamField>

        <ParamField body="url" type="string" required>
          The URL of the endpoint
        </ParamField>
      </Expandable>
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl https://qstash.upstash.io/v2/topics \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch("https://qstash.upstash.io/v2/topics", {
    headers: {
      Authorization: "Bearer <token>",
    },
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.get(
    'https://qstash.upstash.io/v2/topics',
    headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("GET", "https://qstash.upstash.io/v2/topics", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
    {
    
      "createdAt": 1623345678001,
      "updatedAt": 1623345678001,
      "name": "my-url-group",
      "endpoints": [
        {
          "name": "my-endpoint",
          "url": "https://my-endpoint.com"
        }
      ]
    },
    // ...
  ]
  ```
</ResponseExample>


# Remove URL Group

DELETE https://qstash.upstash.io/v2/topics/{urlGroupName}
Remove a URL group and all its endpoints

The URL Group and all its endpoints are removed. In flight messages in the URL Group are not removed but you will not be able to send messages to the topic anymore.

## Request

<ParamField path="urlGroupName" type="string" required>
  The name of the URL Group to remove.
</ParamField>

## Response

This endpoint returns 200 if the URL Group is removed successfully.

<RequestExample>
  ```sh curl
  curl -XDELETE https://qstash.upstash.io/v2/topics/my-url-group \
    -H "Authorization: Bearer <token>"
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/topics/my-url-group', {
    method: 'DELETE',
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.delete(
    'https://qstash.upstash.io/v2/topics/my-url-group', 
    headers=headers
  )
  ```

  ```go Go 
  req, err := http.NewRequest("DELETE", "https://qstash.upstash.io/v2/topics/my-url-group", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>


# Remove Endpoints

DELETE https://qstash.upstash.io/v2/topics/{urlGroupName}/endpoints
Remove one or more endpoints

Remove one or multiple endpoints from a URL Group. If all endpoints have been removed, the URL Group will be deleted.

## Request

<ParamField path="urlGroupName" type="string" required>
  The name of your URL Group. If it doesn't exist, we return an error.
</ParamField>

<ParamField body="endpoints" type="Array" required>
  The endpoints to be removed from to the URL Group.

  <Expandable defaultOpen>
    Either `name` or `url` must be provided

    <ParamField body="name" type="string">
      The name of the endpoint
    </ParamField>

    <ParamField body="url" type="string">
      The URL of the endpoint
    </ParamField>
  </Expandable>
</ParamField>

## Response

This endpoint simply returns 200 OK if the endpoints have been removed successfully.

<RequestExample>
  ```sh curl
  curl -XDELETE https://qstash.upstash.io/v2/topics/:urlGroupName/endpoints \
    -H "Authorization: Bearer <token>" \
    -H "Content-Type: application/json" \
    -d '{
      "endpoints": [
        {
          "name": "endpoint1",
        },
        {
          "url": "https://somewhere-else.com"
        }
      ]
    }'
  ```

  ```js Node
  const response = await fetch("https://qstash.upstash.io/v2/topics/:urlGroupName/endpoints", {
    method: "DELETE",
    headers: {
      Authorization: "Bearer <token>",
      "Content-Type": "application/json",
    },
    body: {
      endpoints: [
        {
          name: "endpoint1",
        },
        {
          url: "https://somewhere-else.com",
        },
      ],
    },
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
      'Content-Type': 'application/json',
  }

  data = {
     "endpoints": [
        {
          "name": "endpoint1",
        },
        {
          "url": "https://somewhere-else.com"
        }
      ]
  }

  response = requests.delete(
    'https://qstash.upstash.io/v2/topics/:urlGroupName/endpoints',
    headers=headers,
    data=data
  )
  ```

  ```go Go
  var data = strings.NewReader(`{
    "endpoints": [
      {
        "name": "endpoint1",
      },
      {
        "url": "https://somewhere-else.com"
      }
    ]
  }`)
  req, err := http.NewRequest("DELETE", "https://qstash.upstash.io/v2/topics/:urlGroupName/endpoints", data)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  req.Header.Set("Content-Type", "application/json")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>


# Background Jobs



## When do you need background jobs

Background jobs are essential for executing tasks that are too time-consuming to run in the
main execution thread without affecting the user experience.

These tasks might include data processing, sending batch emails, performing scheduled maintenance,
or any other operations that are not immediately required to respond to user requests.

Utilizing background jobs allows your application to remain responsive and scalable, handling more requests simultaneously by offloading
heavy lifting to background processes.

<Note>
  In Serverless frameworks, your hosting provider will likely have a limit for how long each task can last. Try searching
  for the maximum execution time for your hosting provider to find out more.
</Note>

## How to use QStash for background jobs

QStash provides a simple and efficient way to run background jobs, you can understand it as a 2 step process:

1.  **Public API** Create a public API endpoint within your application. The endpoint should contain the logic for the background job.

<Warning>
  Since QStash requires a public endpoint to invoke the background job, it won't be able to access any localhost APIs.
  To get around this, you can set up a [local tunnel](/qstash/howto/local-tunnel) for local development.
</Warning>

2.  **QStash Request** Invoke QStash to start/schedule the execution of the API endpoint.

Here's what this looks like in a simple Next.js application:

<CodeGroup>
  ```tsx app/page.tsx
  "use client"

  export default function Home() {
    async function handleClick() {
      // Send a request to our server to start the background job.
      // For proper error handling, refer to the quick start.
      // Note: This can also be a server action instead of a route handler
      await fetch("/api/start-email-job", {
        method: "POST",
        body: JSON.stringify({
          users: ["a@gmail.com", "b@gmail.com", "c.gmail.com"]
        }),
      })

    }

    return (
      <main>
        <button onClick={handleClick}>Run background job</button>
      </main>
    );
  }
  ```

  ```typescript app/api/start-email-job/route.ts
  import { Client } from "@upstash/qstash";

  const qstashClient = new Client({
    token: "YOUR_TOKEN",
  });

  export async function POST(request: Request) {
    const body = await request.json();
    const users: string[] = body.users;
    // If you know the public URL of the email API, you can use it directly
    const rootDomain = request.url.split('/').slice(0, 3).join('/');
    const emailAPIURL = `${rootDomain}/api/send-email`; // ie: https://yourapp.com/api/send-email

    // Tell QStash to start the background job.
    // For proper error handling, refer to the quick start.
    await qstashClient.publishJSON({
      url: emailAPIURL,
      body: {
        users
      }
    });

    return new Response("Job started", { status: 200 });
  }

  ```

  ```typescript app/api/send-email/route.ts
  // This is a public API endpoint that will be invoked by QStash.
  // It contains the logic for the background job and may take a long time to execute.
  import { sendEmail } from "your-email-library";

  export async function POST(request: Request) {
    const body = await request.json();
    const users: string[] = body.users;

    // Send emails to the users
    for (const user of users) {
      await sendEmail(user);
    }

    return new Response("Job started", { status: 200 });
  }
  ```
</CodeGroup>

To better understand the application, let's break it down:

1.  **Client**: The client application contains a button that, when clicked, sends a request to the server to start the background job.
2.  **Next.js server**: The first endpoint, `/api/start-email-job`, is invoked by the client to start the background job.
3.  **QStash**: The QStash client is used to invoke the `/api/send-email` endpoint, which contains the logic for the background job.

Here is a visual representation of the process:

<Frame>
  <img className="block dark:hidden" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/qstash-bgjob-light.png" alt="Background job diagram" />

  <img className="hidden dark:block" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/qstash-bgjob-dark.png" alt="Background job diagram" />
</Frame>

To view a more detailed Next.js quick start guide for setting up QStash, refer to the [quick start](/qstash/quickstarts/vercel-nextjs) guide.

It's also possible to schedule a background job to run at a later time using [schedules](/qstash/features/schedules).

If you'd like to invoke another endpoint when the background job is complete, you can use [callbacks](/qstash/features/callbacks).


# Batching



[Publishing](/qstash/howto/publishing) is great for sending one message
at a time, but sometimes you want to send a batch of messages at once.

This can be useful to send messages to a single or multiple destinations.
QStash provides the `batch` endpoint to help
you with this.

If the format of the messages are valid, the response will be an array of
responses for each message in the batch. When batching URL Groups, the response
will be an array of responses for each destination in the URL Group. If one
message fails to be sent, that message will have an error response, but the
other messages will still be sent.

<Note>You can publish to destination, URL Group or queue in the same batch request.</Note>

## Batching messages with destinations

<Info>You can also send messages to the same destination!</Info>

<CodeGroup>
  ```shell cURL
  curl -XPOST https://qstash.upstash.io/v2/batch \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -d '
       [
        {
          "destination": "https://example.com/destination1"
        },
        {
          "destination": "https://example.com/destination2"
        }
       ]'
  ```

  ```typescript TypeScript
  import { Client } from "@upstash/qstash";

  // Each message is the same as the one you would send with the publish endpoint
  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.batchJSON([
    {
      url: "https://example.com/destination1",
    },
    {
      url: "https://example.com/destination2",
    },
  ]);
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH-TOKEN>")
  client.message.batch_json(
      [
          {"url": "https://example.com/destination1"},
          {"url": "https://example.com/destination2"},
      ]
  )
  ```
</CodeGroup>

## Batching messages with URL Groups

If you have a [URL Group](/qstash/howto/url-group-endpoint), you can batch send with
the URL Group as well.

<CodeGroup>
  ```shell cURL
  curl -XPOST https://qstash.upstash.io/v2/batch \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -d '
       [
        {
          "destination": "myUrlGroup"
        },
        {
          "destination": "https://example.com/destination2"
        }
       ]'
  ```

  ```typescript TypeScript
  const client = new Client({ token: "<QSTASH_TOKEN>" });

  // Each message is the same as the one you would send with the publish endpoint
  const res = await client.batchJSON([
    {
      urlGroup: "myUrlGroup",
    },
    {
      url: "https://example.com/destination2",
    },
  ]);
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH-TOKEN>")
  client.message.batch_json(
      [
          {"url_group": "my-url-group"},
          {"url": "https://example.com/destination2"},
      ]
  )
  ```
</CodeGroup>

## Batching messages with queue

If you have a [queue](/qstash/features/queues), you can batch send with
the queue. It is the same as publishing to a destination, but you need to set the queue name.

<CodeGroup>
  ```shell cURL
  curl -XPOST https://qstash.upstash.io/v2/batch \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -d '
       [
        {
          "queue": "my-queue",
          "destination": "https://example.com/destination1"
        },
        {
          "queue": "my-second-queue",
          "destination": "https://example.com/destination2"
        }
       ]'
  ```

  ```typescript TypeScript
  const client = new Client({ token: "<QSTASH_TOKEN>" });

  const res = await client.batchJSON([
    {
      queueName: "my-queue",
      url: "https://example.com/destination1",
    },
    {
      queueName: "my-second-queue",
      url: "https://example.com/destination2",
    },
  ]);
  ```

  ```python Python
  from upstash_qstash import QStash
  from upstash_qstash.message import BatchRequest

  qstash = QStash("<QSTASH_TOKEN>")

  messages = [
      BatchRequest(
          queue="my-queue",
          url="https://httpstat.us/200",
          body=f"hi 1",
          retries=0
      ),
      BatchRequest(
          queue="my-second-queue",
          url="https://httpstat.us/200",
          body=f"hi 2",
          retries=0
      ),
  ]

  qstash.message.batch(messages)
  ```
</CodeGroup>

## Batching messages with headers and body

You can provide custom headers and a body for each message in the batch.

<CodeGroup>
  ```shell cURL
  curl -XPOST https://qstash.upstash.io/v2/batch   -H "Authorization: Bearer XXX" \
      -H "Content-Type: application/json" \
      -d '
      [
        {
            "destination": "myUrlGroup",
            "headers":{
              "Upstash-Delay":"5s",
              "Upstash-Forward-Hello":"123456"
            },
            "body": "Hello World"
        },
        {
            "destination": "https://example.com/destination1",
            "headers":{
              "Upstash-Delay":"7s",
              "Upstash-Forward-Hello":"789"
            }
        },
        {
            "destination": "https://example.com/destination2",
            "headers":{
              "Upstash-Delay":"9s",
              "Upstash-Forward-Hello":"again"
            }
        }
      ]'
  ```

  ```typescript TypeScript
  const client = new Client({ token: "<QSTASH_TOKEN>" });

  // Each message is the same as the one you would send with the publish endpoint
  const msgs = [
    {
      urlGroup: "myUrlGroup",
      delay: 5,
      body: "Hello World",
      headers: {
        hello: "123456",
      },
    },
    {
      url: "https://example.com/destination1",
      delay: 7,
      headers: {
        hello: "789",
      },
    },
    {
      url: "https://example.com/destination2",
      delay: 9,
      headers: {
        hello: "again",
      },
      body: {
        Some: "Data",
      },
    },
  ];

  const res = await client.batchJSON(msgs);
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH-TOKEN>")
  client.message.batch_json(
      [
          {
              "url_group": "my-url-group",
              "delay": "5s",
              "body": {"hello": "world"},
              "headers": {"random": "header"},
          },
          {
              "url": "https://example.com/destination1",
              "delay": "1m",
          },
          {
              "url": "https://example.com/destination2",
              "body": {"hello": "again"},
          },
      ]
  )
  ```
</CodeGroup>

#### The response for this will look like

```json
[
  [
    {
      "messageId": "msg_...",
      "url": "https://myUrlGroup-endpoint1.com"
    },
    {
      "messageId": "msg_...",
      "url": "https://myUrlGroup-endpoint2.com"
    }
  ],
  {
    "messageId": "msg_..."
  },
  {
    "messageId": "msg_..."
  }
]
```


# Callbacks



All serverless function providers have a maximum execution time for each
function. Usually you can extend this time by paying more, but it's still
limited. QStash provides a way to go around this problem by using callbacks.

## What is a callback?

A callback allows you to call a long running function without having to wait for
its response. Instead of waiting for the request to finish, you can add a
callback url to your published message and when the request finishes, we will
call your callback URL with the response.

<img className="block h-32 dark:hidden" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/callbacks.png" />

<img className="hidden h-40 dark:block" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/callback_dark.png" />

1.  You publish a message to QStash using the `/v2/publish` endpoint
2.  QStash will enqueue the message and deliver it to the destination
3.  QStash waits for the response from the destination
4.  When the response is ready, QStash calls your callback URL with the response

Callbacks publish a new message with the response to the callback URL. Messages
created by callbacks are charged as any other message.

## How do I use Callbacks?

You can add a callback url in the `Upstash-Callback` header when publishing a
message. The value must be a valid URL.

<CodeGroup>
  ```bash cURL
  curl -X POST \
    https://qstash.upstash.io/v2/publish/https://my-api... \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer <QSTASH_TOKEN>' \
    -H 'Upstash-Callback: <CALLBACK_URL>' \
    -d '{ "hello": "world" }'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    url: "https://my-api...",
    body: { hello: "world" },
    callback: "https://my-callback...",
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url="https://my-api...",
      body={
          "hello": "world",
      },
      callback="https://my-callback...",
  )
  ```
</CodeGroup>

The callback body sent to you will be a JSON object with the following fields:

```json
{
  "status": 200,
  "header": { "key": ["value"] },         // Response header
  "body": "YmFzZTY0IGVuY29kZWQgcm9keQ==", // base64 encoded response body
  "retried": 2,                           // How many times we retried to deliver the original message
  "maxRetries" : 3,                       // Number of retries before the message assumed to be failed to delivered.
  "sourceMessageId": "msg_xxx",          // The ID of the message that triggered the callback
  "topicName" : "myTopic",                // The name of the URL Group (topic) if the request was part of a URL Group
  "endpointName" : "myEndpoint",          // The endpoint name if the endpoint is given a name within a topic
  "url" : "http://myurl.com" ,            // The destination url of the message that triggered the callback
  "method" : "GET",                       // The http method of the message that triggered the callback
  "sourceHeader" : { "key": "value"},     // The http header of the message that triggered the callback
  "sourceBody" : "YmFzZTY0kZWQgcm9keQ==", // The base64 encoded body of the message that triggered the callback
  "notBefore" : "1701198458025",          // The unix timestamp of the message that triggered the callback is/will be delivered in milliseconds
  "createdAt" : "1701198447054",          // The unix timestamp of the message that triggered the callback is created in milliseconds
  "scheduleId" : "scd_xxx"                // The scheduleId of the message if the message is triggered by a schedule
  "callerIP": "178.247.74.179"            // The IP address where the message that triggered the callback is published from
}
```

In Next.js you could use the following code to handle the callback:

```js
// pages/api/callback.js

import { verifySignature } from "@upstash/qstash/nextjs";

function handler(req, res) {
  // responses from qstash are base64-encoded
  const decoded = atob(req.body.body);
  console.log(decoded);

  return res.status(200).end();
}

export default verifySignature(handler);

export const config = {
  api: {
    bodyParser: false,
  },
};
```

We may truncate the response body if it exceeds your plan limits. You can check
your `Max Message Size` in the
[console](https://console.upstash.com/qstash?tab=details).

Make sure you verify the authenticity of the callback request made to your API
by
[verifying the signature](/qstash/features/security/#request-signing-optional).

# What is a Failure-Callback?

Failure callbacks are similar to callbacks but they are called only when all the retries are exhausted and still
the message can not be delivered to the given endpoint.

This is designed to be an serverless alternative to [List messages to DLQ](/qstash/api/dlq/listMessages).

You can add a failure callback URL in the `Upstash-Failure-Callback` header when publishing a
message. The value must be a valid URL.

<CodeGroup>
  ```bash cURL
  curl -X POST \
    https://qstash.upstash.io/v2/publish/<DESTINATION_URL> \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer <QSTASH_TOKEN>' \
    -H 'Upstash-Failure-Callback: <CALLBACK_URL>' \
    -d '{ "hello": "world" }'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    url: "https://my-api...",
    body: { hello: "world" },
    failureCallback: "https://my-callback...",
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url="https://my-api...",
      body={
          "hello": "world",
      },
      failure_callback="https://my-callback...",
  )
  ```
</CodeGroup>

The callback body sent to you will be a JSON object with the following fields:

```json
{
  "status": 400,
  "header": { "key": ["value"] },         // Response header
  "body": "YmFzZTY0IGVuY29kZWQgcm9keQ==", // base64 encoded response body
  "retried": 3,                           // How many times we retried to deliver the original message
  "maxRetries" : 3,                       // Number of retries before the message assumed to be failed to delivered.
  "dlqId" : "1725323658779-0"             // Dead Letter Queue id. This can be used to retrieve/remove the related message from DLQ.
  "sourceMessageId": "msg_xxx",          // The ID of the message that triggered the callback
  "topicName" : "myTopic",                // The name of the URL Group (topic) if the request was part of a topic
  "endpointName" : "myEndpoint",          // The endpoint name if the endpoint is given a name within a topic
  "url" : "http://myurl.com" ,            // The destination url of the message that triggered the callback
  "method" : "GET",                       // The http method of the message that triggered the callback
  "sourceHeader" : { "key": "value"},     // The http header of the message that triggered the callback
  "sourceBody" : "YmFzZTY0kZWQgcm9keQ==", // The base64 encoded body of the message that triggered the callback
  "notBefore" : "1701198458025",          // The unix timestamp of the message that triggered the callback is/will be delivered in milliseconds
  "createdAt" : "1701198447054",          // The unix timestamp of the message that triggered the callback is created in milliseconds
  "scheduleId" : "scd_xxx",               // The scheduleId of the message if the message is triggered by a schedule
  "callerIP": "178.247.74.179"            // The IP address where the message that triggered the callback is published from
}
```

You can also use a callback and failureCallback together!

## Configuring Callbacks

Publishes/enqueues for callbacks can also be configured with the same HTTP headers that are used to configure direct publishes/enqueues.

<Tip> You can refer to headers that are used to configure `publishes` [here](https://upstash.com/docs/qstash/api/publish) and for `enqueues`
[here](https://upstash.com/docs/qstash/api/enqueue) </Tip>

Instead of the `Upstash` prefix for headers, the `Upstash-Callback`/`Upstash-Failure-Callback` prefix can be used to configure callbacks as follows:

```
Upstash-Callback-Timeout 
Upstash-Callback-Retries
Upstash-Callback-Delay 
Upstash-Callback-Method 
Upstash-Failure-Callback-Timeout 
Upstash-Failure-Callback-Retries
Upstash-Failure-Callback-Delay
Upstash-Failure-Callback-Method
```

You can also forward headers to your callback endpoints as follows:

```
Upstash-Callback-Forward-MyCustomHeader 
Upstash-Failure-Callback-Forward-MyCustomHeader  
```


# Deduplication



Messages can be deduplicated to prevent duplicate messages from being sent. When
a duplicate message is detected, it is accepted by QStash but not enqueued. This
can be useful when the connection between your service and QStash fails, and you
never receive the acknowledgement. You can simply retry publishing and can be
sure that the message will enqueued only once.

In case a message is a duplicate, we will accept the request and return the
messageID of the existing message. The only difference will be the response
status code. We'll send HTTP `202 Accepted` code in case of a duplicate message.

## Deduplication ID

To deduplicate a message, you can send the `Upstash-Deduplication-Id` header
when publishing the message.

<CodeGroup>
  ```shell cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -H "Upstash-Deduplication-Id: abcdef" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/publish/https://my-api..."'
  ```

  ```typescript TypeScript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    url: "https://my-api...",
    body: { hello: "world" },
    deduplicationId: "abcdef",
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url="https://my-api...",
      body={
          "hello": "world",
      },
      deduplication_id="abcdef",
  )
  ```
</CodeGroup>

## Content Based Deduplication

If you want to deduplicate messages automatically, you can set the
`Upstash-Content-Based-Deduplication` header to `true`.

<CodeGroup>
  ```shell cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -H "Upstash-Content-Based-Deduplication: true" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/publish/...'
  ```

  ```typescript TypeScript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    url: "https://my-api...",
    body: { hello: "world" },
    contentBasedDeduplication: true,
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url="https://my-api...",
      body={
          "hello": "world",
      },
      content_based_deduplication=True,
  )
  ```
</CodeGroup>

Content based deduplication creates a unique deduplication ID for the message
based on the following fields:

*   **Destination**: The URL Group or endpoint you are publishing the message to.

*   **Body**: The body of the message.

*   **Header**: This includes the `Content-Type` header and all headers, that you
    forwarded with the `Upstash-Forward-` prefix. See
    [custom HTTP headers section](/qstash/howto/publishing#sending-custom-http-headers).


# Delay



When publishing a message, you can delay it for a certain amount of time before
it will be delivered to your API. See the [pricing table](https://upstash.com/pricing/qstash) for more information

<Warning>
  For free: The maximum allowed delay is  **7 days**.

  For pay-as-you-go: The maximum allowed delay is  **1 year**.

  For fixed pricing: The maximum allowed delay is  **Custom(you may delay as much as needed)**.
</Warning>

## Relative Delay

Delay a message by a certain amount of time relative to the time the message was
published.

The format for the duration is `<number><unit>`. Here are some examples:

*   `10s` = 10 seconds
*   `1m` = 1 minute
*   `30m` = half an hour
*   `2h` = 2 hours
*   `7d` = 7 days

You can send this duration inside the `Upstash-Delay` header.

<CodeGroup>
  ```shell cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -H "Upstash-Delay: 1m" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/publish/https://my-api...'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    url: "https://my-api...",
    body: { hello: "world" },
    delay: 60,
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url="https://my-api...",
      body={
          "hello": "world",
      },
      headers={
          "test-header": "test-value",
      },
      delay="60s",
  )
  ```
</CodeGroup>

<Warning>
  `Upstash-Delay` will get overridden by `Upstash-Not-Before` header when both are
  used together.
</Warning>

## Absolute Delay

Delay a message until a certain time in the future. The format is a unix
timestamp in seconds, based on the UTC timezone.

You can send the timestamp inside the `Upstash-Not-Before` header.

<CodeGroup>
  ```shell cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -H "Upstash-Not-Before: 1657104947" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/publish/https://my-api...'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    url: "https://my-api...",
    body: { hello: "world" },
    notBefore: 1657104947,
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url="https://my-api...",
      body={
          "hello": "world",
      },
      headers={
          "test-header": "test-value",
      },
      not_before=1657104947,
  )
  ```
</CodeGroup>

<Info>
  `Upstash-Not-Before` will override the `Upstash-Delay` header when both are used
  together.
</Info>

## Delays in Schedules

Adding a delay in schedules is only possible via via `Upstash-Delay`. The
delay will affect the messages that will be created by the schedule and not the
schedule itself.

For example when you create a new schedule with a delay of `30s`, the messages
will be created when the schedule triggers but only delivered after 30 seconds.


# Dead Letter Queues



At times, your API may fail to process a request. This could be due to a bug in your code, a temporary issue with a third-party service, or even network issues.
QStash automatically retries messages that fail due to a temporary issue but eventually stops and moves the message to a dead letter queue to be handled manually.

Read more about retries [here](/qstash/features/retry).

## How to Use the Dead Letter Queue

You can manually republish messages from the dead letter queue in the console.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-dlq/dlq.png" />
</Frame>

1.  **Retry** - Republish the message and remove it from the dead letter queue. Republished messages are just like any other message and will be retried automatically if they fail.
2.  **Delete** - Delete the message from the dead letter queue.

## Limitations

Dead letter queues are limited to a certain number of messages. If you exceed this limit, the oldest messages will be dropped.
Unhandled messages are evicted after some time.

See the [pricing](https://upstash.com/pricing/qstash) page for more information.


# Queues



The queue concept in QStash allows ordered delivery (FIFO) and also controlled parallelism.
Here we list common use cases for Queue and how to use them.
See the [API doc](/qstash/api/queues/upsert) for the full list of related Rest APIs.

## Ordered Delivery

With Queues, the ordered delivery is guaranteed by default.
This means:

*   Your messages will be queued without blocking the REST API and sent one by one in FIFO order. Queued means [CREATED](/qstash/howto/debug-logs) event will be logged.
*   The next message will wait for retries of the current one if it can not be delivered because your endpoint returns non-2xx code.
    In other words, the next message will be [ACTIVE](/qstash/howto/debug-logs) only after the last message is either [DELIVERED](/qstash/howto/debug-logs) or
    [FAILED](/qstash/howto/debug-logs).
*   Next message will wait for [callbacks](/qstash/features/callbacks#what-is-a-callback) or [failure callbacks](/qstash/features/callbacks#what-is-a-failure-callback) to finish.

<CodeGroup>
  ```bash cURL
  curl -XPOST -H 'Authorization: Bearer XXX' \
              -H "Content-type: application/json" \
    'https://qstash.upstash.io/v2/enqueue/my-queue/https://example.com' -d '{"message":"Hello, World!"}'
  ```

  ```typescript TypeScript
  const client = new Client({ token: "<QSTASH_TOKEN>" });

  const queue = client.queue({
    queueName: "my-queue"
  })

  await queue.enqueueJSON({
    url: "https://example.com",
    body: {
      "Hello": "World"
    }
  })
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.enqueue_json(
      queue="my-queue",
      url="https://example.com",
      body={
          "Hello": "World",
      },
  )
  ```
</CodeGroup>

## Controlled Parallelism

To ensure that your endpoint is not overwhelmed and also you want more than one-by-one delivery for better throughput,
you can achieve controlled parallelism with queues.

By default, queues have parallelism 1.
Depending on your [plan](https://upstash.com/pricing/qstash), you can configure the parallelism of your queues as follows:

<CodeGroup>
  ```bash cURL
  curl -XPOST https://qstash.upstash.io/v2/queues/ \
    -H "Authorization: Bearer <token>" \
    -H "Content-Type: application/json" \
    -d '{
      "queueName": "my-queue", 
      "parallelism": 5,
    }'
  ```

  ```typescript TypeScript
  const client = new Client({ token: "<QSTASH_TOKEN>" });

  const queue = client.queue({
    queueName: "my-queue"
  })

  await queue.upsert({
    parallelism: 1,
  })
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.queue.upsert("my-queue", parallelism=5)
  ```
</CodeGroup>

After that, you can use the `enqueue` path to send your messages.

<CodeGroup>
  ```bash cURL
  curl -XPOST -H 'Authorization: Bearer XXX' \ 
              -H "Content-type: application/json" \
    'https://qstash.upstash.io/v2/enqueue/my-queue/https://example.com' -d '{"message":"Hello, World!"}'
  ```

  ```typescript TypeScript
  const client = new Client({ token: "<QSTASH_TOKEN>" });

  const queue = QStashClient.queue({
    queueName: "my-queue"
  })

  await queue.enqueueJSON({
    url: "https://example.com",
    body: {
      "Hello": "World"
    }
  })
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.enqueue_json(
      queue="my-queue",
      url="https://example.com",
      body={
          "Hello": "World",
      },
  )
  ```
</CodeGroup>

You can check the parallelism of your queues with the following API:

<CodeGroup>
  ```bash cURL
  curl https://qstash.upstash.io/v2/queues/my-queue \
    -H "Authorization: Bearer <token>"
  ```

  ```typescript TypeScript
  const client = new Client({ token: "<QSTASH_TOKEN>" });

  const queue = client.queue({
    queueName: "my-queue"
  })

  const res = await queue.get()
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.queue.get("my-queue")
  ```
</CodeGroup>


# Retry



<Warning>
  For free: Requests will be considered failed if not processed within **15  minutes**.

  For pay-as-you-go: Requests will be considered failed if not processed within **2 hours**.

  For fixed pricing: Requests will be considered failed if not processed within  **Custom(you may timeout as much as needed)**.
</Warning>

Many things can go wrong in a serverless environment. If your API does not
respond with a success status code (2XX), we retry the request to ensure every
message will be delivered.

The maximum number of retries depends on your current plan. By default, we retry
the maximum amount of times, but you can set it lower by sending the
`Upstash-Retries` header:

<CodeGroup>
  ```shell cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -H "Upstash-Retries: 2" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/publish/https://my-api...'
  ```

  ```typescript TypeScript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    url: "https://my-api...",
    body: { hello: "world" },
    retries: 2,
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url="https://my-api...",
      body={
          "hello": "world",
      },
      retries=2,
  )
  ```
</CodeGroup>

The backoff algorithm calculates the retry delay based on the number of retries.
Each delay is capped at 1 day.

```
n = how many times this request has been retried
delay =  min(86400, e ** (2.5*n)) // in seconds
```

| n | delay  |
| - | ------ |
| 1 | 12s    |
| 2 | 2m28s  |
| 3 | 30m8ss |
| 4 | 6h7m6s |
| 5 | 24h    |
| 6 | 24h    |

## Retry-After Headers

Instead of using the default backoff algorithm, you can specify when QStash should retry your message.
To do this, include one of the following headers in your response to QStash request.

*   Retry-After
*   X-RateLimit-Reset
*   X-RateLimit-Reset-Requests
*   X-RateLimit-Reset-Tokens

These headers can be set to a value in seconds, the RFC1123 date format, or a duration format (e.g., 6m5s).
For the duration format, valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h".

Note that you can only delay retries up to the maximum value of the default backoff algorithm, which is one day.
If you specify a value beyond this limit, the backoff algorithm will be applied.

This feature is particularly useful if your application has rate limits, ensuring retries are scheduled appropriately without wasting attempts during restricted periods.

```
Retry-After: 0                             // Next retry will be scheduled immediately without any delay.
Retry-After: 10                            // Next retry will be scheduled after a 10-second delay.
Retry-After: 6m5s                          // Next retry will be scheduled after 6 minutes 5 seconds delay.
Retry-After: Sun, 27 Jun 2024 12:16:24 GMT // Next retry will be scheduled for the specified date, within the allowable limits.
```

## Upstash-Retried Header

QStash adds the `Upstash-Retried` header to requests sent to your API. This
indicates how many times the request has been retried.

```
Upstash-Retried: 0 // This is the first attempt
Upstash-Retried: 1 // This request has been sent once before and now is the second attempt
Upstash-Retried: 2 // This request has been sent twice before and now is the third attempt
```


# Schedules



In addition to sending a message once, you can create a schedule, and we will
publish the message in the given period. To create a schedule, you simply need
to add the `Upstash-Cron` header to your `publish` request.

Schedules can be configured using `cron` expressions.
[crontab.guru](https://crontab.guru/) is a great tool for understanding and
creating cron expressions.

We use `UTC` as timezone when evaluating cron expressions.

The following request would create a schedule that will automatically publish
the message every minute:

<CodeGroup>
  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  await client.schedules.create({
    destination: "https://example.com",
    cron: "* * * * *",
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.schedule.create(
      destination="https://example.com",
      cron="* * * * *",
  )
  ```

  ```shell cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -H "Upstash-Cron: * * * * *" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/schedules/https://example.com'
  ```
</CodeGroup>

All of the [other config options](/qstash/howto/publishing#optional-parameters-and-configuration)
can still be used.

<Info>
  It can take up to 60 seconds for the schedule to be loaded on an active node and
  triggered for the first time.
</Info>

You can see and manage your schedules in the
[Upstash Console](https://console.upstash.com/qstash).

### Scheduling to a URL Group

Instead of scheduling a message to a specific URL, you can also create a
schedule, that publishes to a URL Group. Simply use either the URL Group name or its id:

<CodeGroup>
  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  await client.schedules.create({
    destination: "urlGroupName",
    cron: "* * * * *",
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.schedule.create(
      destination="url-group-name",
      cron="* * * * *",
  )
  ```

  ```bash cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -H "Upstash-Cron: * * * * *" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/schedules/<URL_GROUP_ID_OR_NAME>'
  ```
</CodeGroup>

### Scheduling to a Queue

You can schedule an item to be added to a queue at a specified time.

<CodeGroup>
  ```bash typescript
  curl -XPOST \
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  await client.schedules.create({
    destination: "https://example.com",
    cron: "* * * * *",
    queueName: "yourQueueName",
  });
  ```

  ```bash cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -H "Upstash-Cron: * * * * *" \
      -H "Upstash-Queue-Name: yourQueueName" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/schedules/https://example.com'
  ```
</CodeGroup>

### Overwriting an existing schedule

You can pass scheduleId explicitly to overwrite an existing schedule or just simply create the schedule
with the given schedule id.

<CodeGroup>
  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  await client.schedules.create({
    destination: "https://example.com",
    scheduleId: "existingScheduleId",
    cron: "* * * * *",
  });
  ```

  ```shell cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -H "Upstash-Cron: * * * * *" \
      -H "Upstash-Schedule-Id: existingScheduleId" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/schedules/https://example.com'
  ```
</CodeGroup>


# Security



### Request Authorization

When interacting with the QStash API, you will need an authorization token. You
can get your token from the [Console](https://console.upstash.com/qstash).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/rest_token.png" />
</Frame>

Send this token along with every request made to `QStash` inside the
`Authorization` header like this:

```
"Authorization": "Bearer <QSTASH_TOKEN>"
```

### Request Signing (optional)

Because your endpoint needs to be publicly available, we recommend you verify
the authenticity of each incoming request.

#### The `Upstash-Signature` header

With each request we are sending a JWT inside the `Upstash-Signature` header.
You can learn more about them [here](https://jwt.io).

An example token would be:

**Header**

```json
{
  "alg": "HS256",
  "typ": "JWT"
}
```

**Payload**

```json
{
  "iss": "Upstash",
  "sub": "https://qstash-remote.requestcatcher.com/test",
  "exp": 1656580612,
  "nbf": 1656580312,
  "iat": 1656580312,
  "jti": "jwt_67kxXD6UBAk7DqU6hzuHMDdXFXfP",
  "body": "qK78N0k3pNKI8zN62Fq2Gm-_LtWkJk1z9ykio3zZvY4="
}
```

The JWT is signed using `HMAC SHA256` algorithm with your current signing key
and includes the following claims:

#### Claims

##### `iss`

The issuer field is always `Upstash`.

##### `sub`

The url of your endpoint, where this request is sent to.

For example when you are using a nextjs app on vercel, this would look something
like `https://my-app.vercel.app/api/endpoint`

##### `exp`

A unix timestamp in seconds after which you should no longer accept this
request. Our JWTs have a lifetime of 5 minutes by default.

##### `iat`

A unix timestamp in seconds when this JWT was created.

##### `nbf`

A unix timestamp in seconds before which you should not accept this request.

##### `jti`

A unique id for this token.

##### `body`

The body field is a base64 encoded sha256 hash of the request body. We use url
encoding as specified in
[RFC 4648](https://datatracker.ietf.org/doc/html/rfc4648#section-5).

#### Verifying the signature

See [how to verify the signature](/qstash/howto/signature).


# URL Groups



Sending messages to a single endpoint and not having to worry about retries is
already quite useful, but we also added the concept of URL Groups to QStash.

In short, a URL Group is just a namespace where you can publish messages to, the
same way as publishing a message to an endpoint directly.

After creating a URL Group, you can create one or multiple endpoints. An endpoint is
defined by a publicly available URL where the request will be sent to each
endpoint after it is published to the URL Group.

When you publish a message to a URL Group, it will be fanned out and sent to all the
subscribed endpoints.

## When should I use URL Groups?

URL Groups decouple your message producers from consumers by grouping one or more
endpoints into a single namespace.

Here's an example: You have a serverless function which is invoked with each
purchase in your e-commerce site. You want to send email to the customer after
the purchase. Inside the function, you submit the URL `api/sendEmail` to the
QStash. Later, if you want to send a Slack notification, you need to update the
serverless function adding another call to QStash to submit
`api/sendNotification`. In this example, you need to update and redeploy the
Serverless function at each time you change (or add) the endpoints.

If you create a URL Group `product-purchase` and produce messages to that URL Group in
the function, then you can add or remove endpoints by only updating the URL Group.
URL Groups give you freedom to modify endpoints without touching the backend
implementation.

Check [here](/qstash/howto/publishing#publish-to-url-group) to learn how to publish
to URL Groups.

## How URL Groups work

When you publish a message to a URL Group, we will enqueue a unique task for each
subscribed endpoint and guarantee successful delivery to each one of them.

[![](https://mermaid.ink/img/pako:eNp1kl1rgzAUhv9KyOWoddXNtrkYVNdf0F0U5ijRHDVMjctHoRT_-2KtaztUQeS8j28e8JxxKhhggpWmGt45zSWtnKMX13GN7PX59IUc5w19iIanBDUmKbkq-qwfXuKdSVQqeQLssK1ZI3itVQ9dekdzdO6Ja9ntKKq-DxtEoP4xYGCIr-OOGCoOG4IYlPwIcqBu0V0XQRK0PE0w9lyCvP1-iB1n1CgcNwofjcJpo_Cua8ooHDWadIrGnaJHp2jaKbrrmnKK_jl1d9s98AxXICvKmd2fy8-MsS6gghgT-5oJCUrH2NKWNA2zi7BlXAuJSUZLBTNMjRa7U51ioqWBAbpu4R9VCsrAfnTG-tR0u5pzpW1lKuqM593cyNKOC60bRVy3i-c514VJ5qmoXMVZQaUujuvADbxgRT0fgqVPX32fpclivcq8l0XGls8Lj-K2bX8Bx2nzPg)](https://mermaid.live/edit#pako:eNp1kl1rgzAUhv9KyOWoddXNtrkYVNdf0F0U5ijRHDVMjctHoRT_-2KtaztUQeS8j28e8JxxKhhggpWmGt45zSWtnKMX13GN7PX59IUc5w19iIanBDUmKbkq-qwfXuKdSVQqeQLssK1ZI3itVQ9dekdzdO6Ja9ntKKq-DxtEoP4xYGCIr-OOGCoOG4IYlPwIcqBu0V0XQRK0PE0w9lyCvP1-iB1n1CgcNwofjcJpo_Cua8ooHDWadIrGnaJHp2jaKbrrmnKK_jl1d9s98AxXICvKmd2fy8-MsS6gghgT-5oJCUrH2NKWNA2zi7BlXAuJSUZLBTNMjRa7U51ioqWBAbpu4R9VCsrAfnTG-tR0u5pzpW1lKuqM593cyNKOC60bRVy3i-c514VJ5qmoXMVZQaUujuvADbxgRT0fgqVPX32fpclivcq8l0XGls8Lj-K2bX8Bx2nzPg)

Consider this scenario: You have a URL Group and 3 endpoints that are subscribed to
it. Now when you publish a message to the URL Group, internally we will create a
task for each subscribed endpoint and handle all retry mechanism isolated from
each other.

## How to create a URL Group

Please refer to the howto [here](/qstash/howto/url-group-endpoint).


# Debug Logs



To debug the logs, first you need to understand the different states a message can
be in.

Only the last 10.000 logs are kept and older logs are removed automatically.

## Lifecycle of a Message

To understand the lifecycle of each message, we'll look at the following chart:

[comment]: # "https://mermaid.live/edit#pako:eNplkmFr2zAQhv-K0MeRNIvTpa0_DEKiQiHbmJsOtnmMqyTbAlkK0rkQQv57z1Ybt1SfTs-9uns53ZFLrzTPeURAvTFQB2inT1npGJ2_n_6x6fQrWxditRObBNer72ux_V-Inw_inmhSDHBLtyRyHjWzukLmq48vctbqGKHWTIKT2jITKbJWq5cWqd9QebXe3f0Siad4wKIofhRUqAuWBY1dcJFVYCypPmg3YkthMTR-q4-dlOSDnd8UYlf8Hp7cru62g5604cCsaQ1SDLJ59Thq3zpMtnqa8omOBvoMTTXhlybv2DjId7h0fMJbHVowin7r2NOSY6NbXfKcQqUr6CyWvHQnkkKH_v7gJM8xdHrCu70a_5fnFdh4pkIZ9OEMrQel6XrkeNj3q1GbiFRSeleZuuc0Q8IN4j7ms1mfvqgNNt3jhfTtLBrVQMDm6WY5W2bLa8gWenm1gC-LhZKP85vrKrucV-rq8zwDfjpN-B7cH-9HV3rw8y3t5bCep2crG8on"

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/debuglogs.png" />
</Frame>

Either you or a previously setup schedule will create a message.

When a message is ready for execution, it will be become `ACTIVE` and a delivery to
your API is attempted.

If you API responds with a status code between `200 - 299`, the task is
considered successful and will be marked as `DELIVERED`.

Otherwise the message is being retried if there are any retries left and moves to `RETRY`. If all retries are exhausted, the task has `FAILED` and the message will be moved to the DLQ.

During all this a message can be cancelled via [DELETE /v2/messages/:messageId](https://docs.upstash.com/qstash/api/messages/cancel). When the request is received, `CANCEL_REQUESTED` will be logged first.
If retries are not exhausted yet, in the next deliver time, the message will be marked as `CANCELLED` and will be completely removed from the system.

## Console

Head over to the [Upstash Console](https://console.upstash.com/qstash) and go to
the `Logs` tab, where you can see the latest events.


# Delete Schedules



Deleting schedules can be done using the [schedules api](/qstash/api/schedules/remove).

<CodeGroup>
  ```shell cURL
  curl -XDELETE \
      -H 'Authorization: Bearer XXX' \
      'https://qstash.upstash.io/v2/schedules/<schedule_id>'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  await client.schedules.delete("<scheduleId>");
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.schedule.delete("<scheduleId>")
  ```
</CodeGroup>

Deleting a schedule does not stop existing messages from being delivered. It
only stops the schedule from creating new messages.

## Schedule ID

If you don't know the schedule ID, you can get a list of all of your schedules
from [here](/qstash/api/schedules/list).

<CodeGroup>
  ```shell cURL
  curl \
      -H 'Authorization: Bearer XXX' \
      'https://qstash.upstash.io/v2/schedules'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const allSchedules = await client.schedules.list();
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.schedule.list()
  ```
</CodeGroup>


# Handling Failures



Sometimes, endpoints fail due to various reasons such as network issues or server issues.
In such cases, QStash offers a few options to handle these failures.

## Failure Callbacks

When publishing a message, you can provide a failure callback that will be called if the message fails to be published.
You can read more about callbacks [here](/qstash/features/callbacks).

With the failure callback, you can add custom logic such as logging the failure or sending an alert to the team.
Once you handle the failure, you can [delete it from the dead letter queue](/qstash/api/dlq/deleteMessage).

<CodeGroup>
  ```bash cURL
  curl -X POST \
    https://qstash.upstash.io/v2/publish/<DESTINATION_URL> \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer <QSTASH_TOKEN>' \
    -H 'Upstash-Failure-Callback: <CALLBACK_URL>' \
    -d '{ "hello": "world" }'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    url: "https://my-api...",
    body: { hello: "world" },
    failureCallback: "https://my-callback...",
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url="https://my-api...",
      body={
          "hello": "world",
      },
      failure_callback="https://my-callback...",
  )
  ```
</CodeGroup>

## Dead Letter Queue

If you don't want to handle the failure immediately, you can use the dead letter queue (DLQ) to store the failed messages.
You can read more about the dead letter queue [here](/qstash/features/dlq).

Failed messages are automatically moved to the dead letter queue upon failure, and can be retried from the console or
the API by [retrieving the message](/qstash/api/dlq/getMessage) then [publishing it](/qstash/api/publish).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/dlq-console.png" alt="DLQ from console" />
</Frame>


# Publish To Kafka



You can use QStash to forward a message to Kafka by using our
[Kafka REST API](/kafka/rest).

All you need is the `Webhook` url from the
[Upstash Console](https://console.upstash.com/kafka) and the Kafka topic to
publish to.

Here is a complete example: `<KAFKA_URL>`, `<USER>` and `<PASS>` will already be
filled in when you copy the url, but you need to replace `<TOPICNAME>` and
`<QSTASH_TOKEN>` with the correct values.

```
curl -XPOST 'https://qstash.upstash.io/v2/publish/<KAFKA_URL>/webhook?topic=<TOPICNAME>&user=<USER>&pass=<PASS>' \
    -H 'Authorization: Bearer <QSTASH_TOKEN>' \
    -d 'hello world'
```


# Local Tunnel



QStash requires a publicly available API to send messages to. If you want to
develop locally, you need to run a tunnel. We are planning to offer a native
solution, but for the moment please use a 3rd party tunnel.

## localtunnel.me

[localtunnel.me](https://github.com/localtunnel/localtunnel) is a free service to provide
a public endpoint for your local development.

It's as simple as running

```
npx localtunnel --port 3000
```

replacing `3000` with the port your application is running on.

This will give you a public URL like `https://good-months-leave.loca.lt` which can be used
as your QStash URL.

If you run into issues, you may need to set the `Upstash-Forward-bypass-tunnel-reminder` header to
any value to bypass the reminder message.

## ngrok

[ngrok](https://ngrok.com) is a free service, that provides you with a public
endpoint and forwards all traffic to your localhost.

### Sign up

Create a new account on
[dashboard.ngrok.com/signup](https://dashboard.ngrok.com/signup) and follow the
[instructions](https://dashboard.ngrok.com/get-started/setup) to download the
ngrok CLI and connect your account:

```bash
ngrok config add-authtoken XXX
```

### Start the tunnel

Choose the port where your application is running. Here I'm forwarding to port
3000, because Next.js is using it.

```bash
$ ngrok http 3000



Session Status                online
Account                       Andreas Thomas (Plan: Free)
Version                       3.1.0
Region                        Europe (eu)
Latency                       -
Web Interface                 http://127.0.0.1:4040
Forwarding                    https://e02f-2a02-810d-af40-5284-b139-58cc-89df-b740.eu.ngrok.io -> http://localhost:3000

Connections                   ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00
```

### Publish a message

Now copy the `Forwarding` url and use it as destination in QStash. Make sure to
add the path of your API at the end. (`/api/webhooks` in this case)

```
curl -XPOST \
    -H 'Authorization: Bearer XXX' \
    -H "Content-type: application/json" \
    -d '{ "hello": "world" }' \
    'https://qstash.upstash.io/v2/publish/https://e02f-2a02-810d-af40-5284-b139-58cc-89df-b740.eu.ngrok.io/api/webhooks'
```

### Debug

In case messages are not delivered or something else doesn't work as expected,
you can go to [http://127.0.0.1:4040](http://127.0.0.1:4040) to see what ngrok
is doing.


# Publish Messages



Publishing a message is as easy as sending a HTTP request to the `/publish`
endpoint. All you need is a valid url of your destination.

<Frame caption="Send a message via the Upstash Console">
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/publish.png" width="688" />
</Frame>

<Info>
  Destination URLs must always include the protocol (`http://` or `https://`)
</Info>

## The message

The message you want to send is passed in the request body. Upstash does not
use, parse, or validate the body, so you can send any kind of data you want. We
suggest you add a `Content-Type` header to your request to make sure your
destination API knows what kind of data you are sending.

## Sending custom HTTP headers

In addition to sending the message itself, you can also forward HTTP headers.
Simply add them prefixed with `Upstash-Forward-` and we will include them in the
message.

#### Here's an example

<CodeGroup>
  ```shell cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H 'Upstash-Forward-My-Header: my-value' \
      -H "Content-type: application/json" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/publish/https://example.com'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    url: "https://example.com",
    body: { "hello": "world" },
    headers: { "my-header": "my-value" },
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url="https://my-api...",
      body={
          "hello": "world",
      },
      headers={
          "my-header": "my-value",
      },
  )
  ```
</CodeGroup>

In this case, we would deliver a `POST` request to `https://example.com` with
the following body and headers:

```json
// body
{ "hello": "world" }

// headers
My-Header:      my-value
Content-Type:   application/json
```

#### What happens after publishing?

When you publish a message, it will be durably stored in an
[Upstash Redis database](https://upstash.com/redis). Then we try to deliver the
message to your chosen destination API. If your API is down or does not respond
with a success status code (200-299), the message will be retried and delivered
when it comes back online. You do not need to worry about retrying messages or
ensuring that they are delivered.

By default, the multiple messages published to QStash are sent to your API in parallel.

## Publish to URL Group

URL Groups allow you to publish a single message to more than one API endpoints. To
learn more about URL Groups, check [URL Groups section](/qstash/features/url-groups).

Publishing to a URL Group is very similar to publishing to a single destination. All
you need to do is replace the `URL` in the `/publish` endpoint with the URL Group
name.

```
https://qstash.upstash.io/v2/publish/https://example.com
https://qstash.upstash.io/v2/publish/my-url-group
```

<CodeGroup>
  ```shell cURL
  curl -XPOST \
      -H 'Authorization: Bearer XXX' \
      -H "Content-type: application/json" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/publish/my-url-group'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const res = await client.publishJSON({
    urlGroup: "my-url-group",
    body: { "hello": "world" },
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.message.publish_json(
      url_group="my-url-group",
      body={
          "hello": "world",
      },
  )
  ```
</CodeGroup>

## Optional parameters and configuration

QStash supports a number of optional parameters and configuration that you can
use to customize the delivery of your message. All configuration is done using
HTTP headers.


# Receiving Messages

What do we send to your API?

When you publish a message, QStash will deliver it to your chosen destination. This is a brief overview of how a request to your API looks like.

## Headers

We are forwarding all headers that have been prefixed with `Upstash-Forward-` to your API. [Learn more](/qstash/howto/publishing#sending-custom-http-headers)

In addition to your custom headers, we're sending these headers as well:

| Header                | Description                                                          |
| --------------------- | -------------------------------------------------------------------- |
| `User-Agent`          | Will be set to `Upstash-QStash`                                      |
| `Content-Type`        | The original `Content-Type` header                                   |
| `Upstash-Topic-Name`  | The URL Group (topic) name if sent to a URL Group                    |
| `Upstash-Signature`   | The signature you need to verify [See here](/qstash/howto/signature) |
| `Upstash-Retried`     | How often the message has been retried so far. Starts with 0.        |
| `Upstash-Message-Id`  | The message id of the message.                                       |
| `Upstash-Schedule-Id` | The schedule id of the message if it is related to a schedule.       |
| `Upstash-Caller-Ip`   | The IP address of the publisher of this message.                     |

## Body

The body is passed as is, we do not modify it at all. If you send a JSON body, you will receive a JSON body. If you send a string, you will receive a string.

## Verifying the signature

[See here](/qstash/howto/signature)


# Reset Token



Your token is used to interact with the QStash API. You need it to publish
messages as well as create, read, update or delete other resources, such as
URL Groups and endpoints.

Resetting your token will invalidate your current token and all future requests
with the old token will be rejected.

To reset your token, simply click on the "Reset token" button at the bottom in
the [QStash UI](https://console.upstash.com/qstash) and confirm the dialog.

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/reset_token.png)

Afterwards you should immediately update your token in all your applications.


# Roll Your Signing Keys



Because your API needs to be publicly accessible from the internet, you should
make sure to verify the authenticity of each request.

Upstash provides a JWT with each request. This JWT is signed by your individual
secret signing keys. [Read more](/qstash/howto/signature).

We are using 2 signing keys:

*   current: This is the key used to sign the JWT.
*   next: This key will be used to sign after you have rolled your keys.

If we were using only a single key, there would be some time between when you
rolled your keys and when you can edit the key in your applications. In order to
minimize downtime, we use 2 keys and you should always try to verify with both
keys.

## What happens when I roll my keys?

When you roll your keys, the current key will be replaced with the next key and
a new next key will be generated.

```
currentKey = nextKey
nextKey = generateNewKey()
```

<Warning>
  Rolling your keys twice without updating your applications will cause your apps
  to reject all requests, because both the current and next keys will have been
  replaced.
</Warning>

## How to roll your keys

Rolling your keys can be done by going to the
[QStash UI](https://console.upstash.com/qstash) and clicking on the "Roll keys"
button.

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/roll_keys.png)


# Verify Signatures



We send a JWT with each request. This JWT is signed by your individual secret
signing keys and sent in the `Upstash-Signature` HTTP header.

To verify the jwt you can choose from a
[wide array of open source libraries](https://jwt.io/libraries), or roll your
own solution.

## Claims

All claims in the JWT are listed [here](/qstash/features/security#claims)

## Verifying

The exact implementation depends on the language of your choice and the library
if you use one.

Instead here are the steps you need to follow:

1.  Split the JWT into its header, payload and signature
2.  Verify the signature
3.  Decode the payload and verify the claims
    *   `iss`: The issuer must be`Upstash`.
    *   `sub`: The subject must the url of your API.
    *   `exp`: Verify the token has not expired yet.
    *   `nbf`: Verify the token is already valid.
    *   `body`: Hash the raw request body using `SHA-256` and compare it with the
        `body` claim.

You can also reference the implementation in our
[typescript sdk](https://github.com/upstash/sdk-qstash-ts/blob/main/src/receiver.ts#L82) or use
the methods provided in the SDKs:

<CodeGroup>
  ```typescript Typescript
  import { Receiver } from "@upstash/qstash";

  const receiver = new Receiver({
    currentSigningKey: "YOUR_CURRENT_SIGNING_KEY",
    nextSigningKey: "YOUR_NEXT_SIGNING_KEY",
  });

  // ... in your request handler

  const signature = req.headers["Upstash-Signature"];
  const body = req.body;

  const isValid = receiver.verify({
    body,
    signature,
    url: "YOUR-SITE-URL",
  });
  ```

  ```python Python
  from qstash import Receiver

  receiver = Receiver(
      current_signing_key="YOUR_CURRENT_SIGNING_KEY",
      next_signing_key="YOUR_NEXT_SIGNING_KEY",
  )

  # ... in your request handler

  signature, body = req.headers["Upstash-Signature"], req.body

  receiver.verify(
      body=body,
      signature=signature,
      url="YOUR-SITE-URL",
  )
  ```
</CodeGroup>

<Warning>In JavaScript, the body could be presented by an object via the HTTP handler if it is a JSON. Don't forget to convert to string first via JSON.stringify(body)</Warning>

After you have verified the signature and the claims, you can be sure the
request came from Upstash and process it accordingly.


# Create URL Groups and Endpoints



QStash allows you to group multiple APIs together into a single namespace,
called a `URL Group` (Previously, it was called `Topics`).
Read more about URL Groups [here](/qstash/features/url-groups).

There are two ways to create endpoints and URL Groups: The UI and the REST API.

## UI

Go to [console.upstash.com/qstash](https://console.upstash.com/qstash) and click
on the `URL Groups` tab. Afterwards you can create a new URL Group by giving it a name.
Keep in mind that URL Group names are restricted to alphanumeric, underscore, hyphen
and dot characters.

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/create_topic.png)

After creating the URL Group, you can add endpoints to it:

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/create_endpoint.png)

## API

You can create a URL Group and endpoint using the [console](https://console.upstash.com/qstash) or [REST API](/qstash/api/url-groups/add-endpoint).

<CodeGroup>
  ```bash cURL
  curl -XPOST https://qstash.upstash.io/v2/topics/:urlGroupName/endpoints \
    -H "Authorization: Bearer <token>" \
    -H "Content-Type: application/json" \
    -d '{
      "endpoints": [
        {
          "name": "endpoint1",
          "url": "https://example.com"
        },
        {
          "name": "endpoint2",
          "url": "https://somewhere-else.com"
        }
      ]
    }'
  ```

  ```typescript Typescript
  import { Client } from "@upstash/qstash";

  const client = new Client({ token: "<QSTASH_TOKEN>" });
  const urlGroups = client.urlGroups;
  await urlGroups.addEndpoints({
    name: "urlGroupName",
    endpoints: [
      { name: "endpoint1", url: "https://example.com" },
      { name: "endpoint2", url: "https://somewhere-else.com" },
    ],
  });
  ```

  ```python Python
  from qstash import QStash

  client = QStash("<QSTASH_TOKEN>")
  client.url_group.upsert_endpoints(
      url_group="url-group-name",
      endpoints=[
          {"name": "endpoint1", "url": "https://example.com"},
          {"name": "endpoint2", "url": "https://somewhere-else.com"},
      ],
  )
  ```
</CodeGroup>


# LLM with Anthropic



QStash integrates smoothly with Anthropic's API, allowing you to send LLM requests and leverage QStash features like retries, callbacks, and batching. This is especially useful when working in serverless environments where LLM response times vary and traditional timeouts may be limiting. QStash provides an HTTP timeout of up to 2 hours, which is ideal for most LLM cases.

### Example: Publishing and Enqueueing Requests

Specify the `api` as `llm` with the provider set to `anthropic()` when publishing requests. Use the `Upstash-Callback` header to handle responses asynchronously, as streaming completions aren’t supported for this integration.

#### Publishing a Request

```typescript
import { anthropic, Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
await client.publishJSON({
  api: { name: "llm", provider: anthropic({ token: "<ANTHROPIC_TOKEN>" }) },
  body: {
    model: "claude-3-5-sonnet-20241022",
    messages: [{ role: "user", content: "Summarize recent tech trends." }],
  },
  callback: "https://example.com/callback",
});
```

### Enqueueing a Chat Completion Request

Use `enqueueJSON` with Anthropic as the provider to enqueue requests for asynchronous processing.

```typescript
import { anthropic, Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });

const result = await client.queue({ queueName: "your-queue-name" }).enqueueJSON({
  api: { name: "llm", provider: anthropic({ token: "<ANTHROPIC_TOKEN>" }) },
  body: {
    model: "claude-3-5-sonnet-20241022",
    messages: [
      {
        role: "user",
        content: "Generate ideas for a marketing campaign.",
      },
    ],
  },
  callback: "https://example.com/callback",
});

console.log(result);
```

### Sending Chat Completion Requests in Batches

Use `batchJSON` to send multiple requests at once. Each request in the batch specifies the same Anthropic provider and includes a callback URL.

```typescript
import { anthropic, Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });

const result = await client.batchJSON([
  {
    api: { name: "llm", provider: anthropic({ token: "<ANTHROPIC_TOKEN>" }) },
    body: {
      model: "claude-3-5-sonnet-20241022",
      messages: [
        {
          role: "user",
          content: "Describe the latest in AI research.",
        },
      ],
    },
    callback: "https://example.com/callback1",
  },
  {
    api: { name: "llm", provider: anthropic({ token: "<ANTHROPIC_TOKEN>" }) },
    body: {
      model: "claude-3-5-sonnet-20241022",
      messages: [
        {
          role: "user",
          content: "Outline the future of remote work.",
        },
      ],
    },
    callback: "https://example.com/callback2",
  },
  // Add more requests as needed
]);

console.log(result);
```

#### Analytics with Helicone

To monitor usage, include Helicone analytics by passing your Helicone API key under `analytics`:

```typescript
await client.publishJSON({
  api: {
    name: "llm",
    provider: anthropic({ token: "<ANTHROPIC_TOKEN>" }),
    analytics: { name: "helicone", token: process.env.HELICONE_API_KEY! },
  },
  body: { model: "claude-3-5-sonnet-20241022", messages: [{ role: "user", content: "Hello!" }] },
  callback: "https://example.com/callback",
});
```

With this setup, Anthropic can be used seamlessly in any LLM workflows in QStash.


# LLM - OpenAI



QStash has built-in support for calling LLM APIs. This allows you to take advantage of QStash features such as retries, callbacks, and batching while using LLM APIs.

QStash is especially useful for LLM processing because LLM response times are often highly variable. When accessing LLM APIs from serverless runtimes, invocation timeouts are a common issue. QStash offers an HTTP timeout of 2 hours, which is sufficient for most LLM use cases. By using callbacks and the workflows, you can easily manage the asynchronous nature of LLM APIs.

## QStash LLM API

You can publish (or enqueue) single LLM request or batch LLM requests using all existing QStash features natively. To do this, specify the destination `api` as `llm` with a valid provider. The body of the published or enqueued message should contain a valid chat completion request. For these integrations, you must specify the `Upstash-Callback` header so that you can process the response asynchronously. Note that streaming chat completions cannot be used with them. Use [the chat API](#chat-api) for streaming completions.

All the examples below can be used with **OpenAI-compatible LLM providers**.

### Publishing a Chat Completion Request

<CodeGroup>
  ```js JavaScript
  import { Client, upstash } from "@upstash/qstash";

  const client = new Client({
      token: "<QSTASH_TOKEN>",
  });

  const result = await client.publishJSON({
      api: { name: "llm", provider: openai({ token: "_OPEN_AI_TOKEN_"}) },
      body: {
          model: "gpt-3.5-turbo",
          messages: [
              {
              role: "user",
              content: "Write a hello world program in Rust.",
              },
          ],
      },
      callback: "https://abc.requestcatcher.com/",
  });

  console.log(result);
  ```

  ```python Python
  from qstash import QStash
  from qstash.chat import upstash

  q = QStash("<QSTASH_TOKEN>")

  result = q.message.publish_json(
      api={"name": "llm", "provider": openai("<OPENAI_API_KEY>")},
      body={
          "model": "gpt-3.5-turbo",
          "messages": [
              {
                  "role": "user",
                  "content": "Write a hello world program in Rust.",
              }
          ],
      },
      callback="https://abc.requestcatcher.com/",
  )

  print(result)
  ```
</CodeGroup>

### Enqueueing a Chat Completion Request

<CodeGroup>
  ```js JavaScript
  import { Client, upstash } from "@upstash/qstash";

  const client = new Client({
      token: "<QSTASH_TOKEN>",
  });

  const result = await client.queue({ queueName: "queue-name" }).enqueueJSON({
      api: { name: "llm", provider: openai({ token: "_OPEN_AI_TOKEN_"}) },
      body: {
          "model": "gpt-3.5-turbo",
          messages: [
              {
                  role: "user",
                  content: "Write a hello world program in Rust.",
              },
          ],
      },
      callback: "https://abc.requestcatcher.com",
  });

  console.log(result);
  ```

  ```python Python
  from qstash import QStash
  from qstash.chat import upstash

  q = QStash("<QSTASH_TOKEN>")

  result = q.message.enqueue_json(
      queue="queue-name",
      api={"name": "llm", "provider": openai("<OPENAI_API_KEY>")},
      body={
          "model": "gpt-3.5-turbo",
          "messages": [
              {
                  "role": "user",
                  "content": "Write a hello world program in Rust.",
              }
          ],
      },
      callback="https://abc.requestcatcher.com",
  )

  print(result)
  ```
</CodeGroup>

### Sending Chat Completion Requests in Batches

<CodeGroup>
  ```js JavaScript
  import { Client, upstash } from "@upstash/qstash";

  const client = new Client({
      token: "<QSTASH_TOKEN>",
  });

  const result = await client.batchJSON([
      {
          api: { name: "llm", provider: openai({ token: "_OPEN_AI_TOKEN_" }) },
          body: { ... },
          callback: "https://abc.requestcatcher.com",
      },
      ...
  ]);

  console.log(result);
  ```

  ```python Python
  from qstash import QStash
  from qstash.chat import upstash

  q = QStash("<QSTASH_TOKEN>")

  result = q.message.batch_json(
      [
          {
              "api":{"name": "llm", "provider": openai("<OPENAI_API_KEY>")},
              "body": {...},
              "callback": "https://abc.requestcatcher.com",
          },
          ...
      ]
  )

  print(result)
  ```

  ```shell curl
  curl "https://qstash.upstash.io/v2/batch" \
      -X POST \
      -H "Authorization: Bearer QSTASH_TOKEN" \
      -H "Content-Type: application/json" \
      -d '[
          {
              "destination": "api/llm",
              "body": {...},
              "callback": "https://abc.requestcatcher.com"
          },
          ...
      ]'
  ```
</CodeGroup>

### Retrying After Rate Limit Resets

When the rate limits are exceeded, QStash automatically schedules the retry of
publish or enqueue of chat completion tasks depending on the reset time
of the rate limits. That helps with not doing retries prematurely
when it is definitely going to fail due to exceeding rate limits.

## Analytics via Helicone

Helicone is a powerful observability platform that provides valuable insights into your LLM usage. Integrating Helicone with QStash is straightforward.

To enable Helicone observability in QStash, you simply need to pass your Helicone API key when initializing your model. Here's how to do it for both custom models and OpenAI:

```ts
import { Client, custom } from "@upstash/qstash";

const client = new Client({
  token: "<QSTASH_TOKEN>",
});

await client.publishJSON({
  api: {
    name: "llm",
    provider: custom({
      token: "XXX",
      baseUrl: "https://api.together.xyz",
    }),
    analytics: { name: "helicone", token: process.env.HELICONE_API_KEY! },
  },
  body: {
    model: "meta-llama/Llama-3-8b-chat-hf",
    messages: [
      {
        role: "user",
        content: "hello",
      },
    ],
  },
  callback: "https://oz.requestcatcher.com/",
});
```


# Pipedream

Build and run workflows with 1000s of open source triggers and actions across 900+ apps.

[Pipedream](https://pipedream.com) allows you to build and run workflows with
1000s of open source triggers and actions across 900+ apps.

Check out the [official integration](https://pipedream.com/apps/qstash).

## Trigger a Pipedream workflow from a QStash topic message

This is a step by step guide on how to trigger a Pipedream workflow from a
QStash topic message.

Alternatively [click here](https://pipedream.com/new?h=tch_3egfAX) to create a
new workflow with this QStash topic trigger added.

### 1. Create a Topic in QStash

If you haven't yet already, create a **Topic** in the
[QStash dashboard](https://console.upstash.com/qstash?tab=topics).

### 2. Create a new Pipedream workflow

Sign into [Pipedream](https://pipedream.com) and create a new workflow.

### 3. Add QStash Topic Message as a trigger

In the workflow **Trigger** search for QStash and select the **Create Topic
Endpoint** trigger.

![Select the QStash Create Topic Endpoint trigger](https://res.cloudinary.com/pipedreamin/image/upload/v1664298855/docs/components/CleanShot_2022-09-27_at_13.13.56_x6gzgk.gif)

Then, connect your QStash account by clicking the QStash prop and retrieving
your token from the
[QStash dashboard](https://console.upstash.com/qstash?tab=details).

After connecting your QStash account, click the **Topic** prop, a dropdown will
appear containing the QStash topics on your account.

Then *click* on a specific topic to listen for new messages on.

![Selecting a QStash topic to subscribe to](https://res.cloudinary.com/pipedreamin/image/upload/v1664299016/docs/components/CleanShot_2022-09-27_at_13.16.35_rewzbo.gif)

Finally, *click* **Continue**. Pipedream will create a unique HTTP endpoint and
add it to your QStash topic.

### 4. Test with a sample message

Use the *Request Builder* in the
[QStash dashboard](https://console.upstash.com/qstash?tab=details) to publish a
test message to your topic.

Alternatively, you can use the **Create topic message** action in a Pipedream
workflow to send a message to your topic.

*Don't forget* to use this action in a separate workflow, otherwise you might
cause an infinite loop of messages between QStash and Pipedream.

### 5. Add additional steps

Add additional steps to the workflow by clicking the plus icon beneath the
Trigger step.

Build a workflow with the 1,000+ pre-built components available in Pipedream,
including [Airtable](https://pipedream.com/apps/airtable),
[Google Sheets](https://pipedream.com/apps/google-sheets),
[Slack](https://pipedream.com/apps/slack) and many more.

Alternatively, use [Node.js](https://pipedream.com/docs/code/nodejs) or
[Python](https://pipedream.com/docs/code/python) code steps to retrieve,
transform, or send data to other services.

### 6. Deploy your Pipedream workflow

After you're satisfied with your changes, click the **Deploy** button in the
top right of your Pipedream workflow. Your deployed workflow will not
automatically process new messages to your QStash topic. Collapse
quickstart-trigger-pipedream-workflow-from-topic.md 3 KB

### Video tutorial

If you prefer video, you can check out this tutorial by
[pipedream](https://pipedream.com).

[![Video](https://img.youtube.com/vi/-oXlWuxNG5A/0.jpg)](https://www.youtube.com/watch?v=-oXlWuxNG5A)

## Trigger a Pipedream workflow from a QStash topic message

This is a step by step guide on how to trigger a Pipedream workflow from a
QStash endpoint message.

Alternatively [click here](https://pipedream.com/new?h=tch_m5ofX6) to create a
pre-configured workflow with the HTTP trigger and QStash webhook verification
step already added.

### 1. Create a new Pipedream workflow

Sign into [Pipedream](https://pipedream.com) and create a new workflow.

### 2. Configure the workflow with an HTTP trigger

In the workflow **Trigger** select the **New HTTP / Webhook Requests** option.

![Create new HTTP Webhook trigger](https://res.cloudinary.com/pipedreamin/image/upload/v1664296111/docs/components/CleanShot_2022-09-27_at_12.27.42_cqzolg.png)

Pipedream will create a unique HTTP endpoint for your workflow.

Then configure the HTTP trigger to *return a custom response*. By default
Pipedream will always return a 200 response, which allows us to return a non-200
response to QStash to retry the workflow again if there's an error during the
execution of the QStash message.

![Configure the webhook to return a custom response](https://res.cloudinary.com/pipedreamin/image/upload/v1664296210/docs/components/CleanShot_2022-09-27_at_12.29.45_jbwtcm.png)

Lastly, set the **Event Body** to be a **Raw request**. This will make sure the
QStash verify webhook action receives the data in the correct format.

![Set the event body to a raw body](https://res.cloudinary.com/pipedreamin/image/upload/v1664302540/docs/components/CleanShot_2022-09-27_at_14.15.15_o4xinz.png)

### 3. Test with a sample message

Use the *Request Builder* in the
[QStash dashboard](https://console.upstash.com/qstash?tab=details) to publish a
test message to your topic.

Alternatively, you can use the **Create topic message** action in a Pipedream
workflow to send a message to your topic.

*Don't forget* to use this action in a separate workflow, otherwise you might
cause an infinite loop of messages between QStash and Pipedream.

### 4. Verify the QStash webhook

Pipedream has a pre-built QStash action that will verify the content of incoming
webhooks from QStash.

First, search for **QStash** in the step search bar, then select the QStash app.

Of the available actions, select the **Verify Webhook** action.

Then connect your QStash account and select the **HTTP request** prop. In the
dropdown, click **Enter custom expression** and then paste in
`{{ steps.trigger.event }}`.

This step will automatically verify the incoming HTTP requests and exit the
workflow early if requests are not from QStash.

### 5. Add additional steps

Add additional steps to the workflow by clicking the plus icon beneath the
Trigger step.

Build a workflow with the 1,000+ pre-built components available in Pipedream,
including [Airtable](https://pipedream.com/apps/airtable),
[Google Sheets](https://pipedream.com/apps/google-sheets),
[Slack](https://pipedream.com/apps/slack) and many more.

Alternatively, use [Node.js](https://pipedream.com/docs/code/nodejs) or
[Python](https://pipedream.com/docs/code/python) code steps to retrieve,
transform, or send data to other services.

### 6. Return a 200 response

In the final step of your workflow, return a 200 response by adding a new step
and selecting **Return an HTTP Response**.

![Returning an HTTP response](https://res.cloudinary.com/pipedreamin/image/upload/v1664296812/docs/components/CleanShot_2022-09-27_at_12.39.25_apkngf.png)

This will generate Node.js code to return an HTTP response to QStash using the
`$.respond` helper in Pipedream.

### 7. Deploy your Pipedream workflow

After you're satisfied with your changes, click the **Deploy** button in the
top right of your Pipedream workflow. Your deployed workflow will not
automatically process new messages to your QStash topic.

### Video tutorial

If you prefer video, you can check out this tutorial by
[pipedream](https://pipedream.com).

[![Video](https://img.youtube.com/vi/uG8eO7BNok4/0.jpg)](https://youtu.be/uG8eO7BNok4)


# Email - Resend



The `qstash-js` SDK offers an integration to easily send emails using [Resend](https://resend.com/), streamlining email delivery in your applications.

## Basic Email Sending

To send a single email, use the `publishJSON` method with the `resend` provider. Ensure your `QSTASH_TOKEN` and `RESEND_TOKEN` are set for authentication.

```typescript
import { Client, resend } from "@upstash/qstash";
const client = new Client({ token: "<QSTASH_TOKEN>" });

await client.publishJSON({
  api: {
    name: "email",
    provider: resend({ token: "<RESEND_TOKEN>" }),
  },
  body: {
    from: "Acme <onboarding@resend.dev>",
    to: ["delivered@resend.dev"],
    subject: "Hello World",
    html: "<p>It works!</p>",
  },
});
```

In the `body` field, specify any parameters supported by [the Resend Send Email API](https://resend.com/docs/api-reference/emails/send-email), such as `from`, `to`, `subject`, and `html`.

## Sending Batch Emails

To send multiple emails at once, use Resend’s [Batch Email API](https://resend.com/docs/api-reference/emails/send-batch-emails). Set the `batch` option to `true` to enable batch sending. Each email configuration is defined as an object within the `body` array.

```typescript
await client.publishJSON({
  api: {
    name: "email",
    provider: resend({ token: "<RESEND_TOKEN>", batch: true }),
  },
  body: [
    {
      from: "Acme <onboarding@resend.dev>",
      to: ["foo@gmail.com"],
      subject: "Hello World",
      html: "<h1>It works!</h1>",
    },
    {
      from: "Acme <onboarding@resend.dev>",
      to: ["bar@outlook.com"],
      subject: "World Hello",
      html: "<p>It works!</p>",
    },
  ],
});
```

Each entry in the `body` array represents an individual email, allowing customization of `from`, `to`, `subject`, `html`, and any other Resend-supported fields.


# API Examples



### Use QStash via:

*   cURL
*   [Typescript SDK](https://github.com/upstash/sdk-qstash-ts)
*   [Python SDK](https://github.com/upstash/qstash-python)

Below are some examples to get you started. You can also check the [how to](/qstash/howto/publishing) section for
more technical details or the [API reference](/qstash/api/messages) to test the API.

### Publish a message to an endpoint

Simple example to [publish](/qstash/howto/publishing) a message to an endpoint.

<Tabs>
  <Tab title="cURL">
    ```shell
    curl -XPOST \
        -H 'Authorization: Bearer XXX' \
        -H "Content-type: application/json" \
        -d '{ "hello": "world" }' \
        'https://qstash.upstash.io/v2/publish/https://example.com'
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });
    await client.publishJSON({
      url: "https://example.com",
      body: {
        hello: "world",
      },
    });
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.message.publish_json(
        url="https://example.com",
        body={
            "hello": "world",
        },
    )
    # Async version is also available
    ```
  </Tab>
</Tabs>

### Publish a message to a URL Group

The [URL Group](/qstash/features/url-groups) is a way to publish a message to multiple endpoints in a
fan out pattern.

<Tabs>
  <Tab title="cURL">
    ```shell
    curl -XPOST \
        -H 'Authorization: Bearer XXX' \
        -H "Content-type: application/json" \
        -d '{ "hello": "world" }' \
        'https://qstash.upstash.io/v2/publish/myUrlGroup'
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });
    await client.publishJSON({
      urlGroup: "myUrlGroup",
      body: {
        hello: "world",
      },
    });
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.message.publish_json(
        url_group="my-url-group",
        body={
            "hello": "world",
        },
    )
    # Async version is also available
    ```
  </Tab>
</Tabs>

### Publish a message with 5 minutes delay

Add a delay to the message to be published. After QStash receives the message,
it will wait for the specified time (5 minutes in this example) before sending the message to the endpoint.

<Tabs>
  <Tab title="cURL">
    ```shell
    curl -XPOST \
        -H 'Authorization: Bearer XXX' \
        -H "Content-type: application/json" \
        -H "Upstash-Delay: 5m" \
        -d '{ "hello": "world" }' \
        'https://qstash.upstash.io/v2/publish/https://example.com'
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });
    await client.publishJSON({
      url: "https://example.com",
      body: {
        hello: "world",
      },
      delay: 300,
    });
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.message.publish_json(
        url="https://example.com",
        body={
            "hello": "world",
        },
        delay="5m",
    )
    # Async version is also available
    ```
  </Tab>
</Tabs>

### Send a custom header

Add a custom header to the message to be published.

<Tabs>
  <Tab title="cURL">
    ```shell
    curl -XPOST \
        -H 'Authorization: Bearer XXX' \
        -H 'Upstash-Forward-My-Header: my-value' \
        -H "Content-type: application/json" \
        -d '{ "hello": "world" }' \
        'https://qstash.upstash.io/v2/publish/https://example.com'
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });
    await client.publishJSON({
      url: "https://example.com",
      body: {
        hello: "world",
      },
      headers: {
        "My-Header": "my-value",
      },
    });
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.message.publish_json(
        url="https://example.com",
        body={
            "hello": "world",
        },
        headers={
            "My-Header": "my-value",
        },
    )
    # Async version is also available
    ```
  </Tab>
</Tabs>

### Schedule to run once a day

<Tabs>
  <Tab title="cURL">
    ```shell
    curl -XPOST \
        -H 'Authorization: Bearer XXX' \
        -H "Upstash-Cron: 0 0 * * *" \
        -H "Content-type: application/json" \
        -d '{ "hello": "world" }' \
        'https://qstash.upstash.io/v2/schedules/https://example.com'
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });
    await client.schedules.create({
      destination: "https://example.com",
      cron: "0 0 * * *",
    });
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.schedule.create(
        destination="https://example.com",
        cron="0 0 * * *",
    )
    # Async version is also available
    ```
  </Tab>
</Tabs>

### Publish messages to a FIFO queue

By default, messges are published concurrently. With a [queue](/qstash/features/queues), you can enqueue messages in FIFO order.

<Tabs>
  <Tab title="cURL">
    ```shell
    curl -XPOST -H 'Authorization: Bearer XXX' \
                -H "Content-type: application/json" \
                'https://qstash.upstash.io/v2/enqueue/my-queue/https://example.com' 
                -d '{"message":"Hello, World!"}'
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });

    const queue = client.queue({
      queueName: "my-queue"
    })

    await queue.enqueueJSON({
      url: "https://example.com",
      body: {
        "Hello": "World"
      }
    })
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.message.enqueue_json(
        queue="my-queue",
        url="https://example.com",
        body={
            "Hello": "World",
        },
    )
    # Async version is also available
    ```
  </Tab>
</Tabs>

### Publish messages in a [batch](/qstash/features/batch)

Publish multiple messages in a single request.

<Tabs>
  <Tab title="cURL">
    ```shell
    curl -XPOST https://qstash.upstash.io/v2/batch \
        -H 'Authorization: Bearer XXX' \
        -H "Content-type: application/json" \
        -d '
         [
          {
            "destination": "https://example.com/destination1"
          },
          {
            "destination": "https://example.com/destination2"
          }
         ]'
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    import { Client } from "@upstash/qstash";

    const client = new Client({ token: "<QSTASH_TOKEN>" });
    const res = await client.batchJSON([
      {
        url: "https://example.com/destination1",
      },
      {
        url: "https://example.com/destination2",
      },
    ]);
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.message.batch_json(
        [
            {
                "url": "https://example.com/destination1",
            },
            {
                "url": "https://example.com/destination2",
            },
        ]
    )
    # Async version is also available
    ```
  </Tab>
</Tabs>

### Set max retry count to 3

Configure how many times QStash should retry to send the message to the endpoint before
sending it to the [dead letter queue](/qstash/features/dlq).

<Tabs>
  <Tab title="cURL">
    ```shell
    curl -XPOST \
        -H 'Authorization: Bearer XXX' \
        -H "Upstash-Retries: 3" \
        -H "Content-type: application/json" \
        -d '{ "hello": "world" }' \
        'https://qstash.upstash.io/v2/publish/https://example.com'
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });
    await client.publishJSON({
      url: "https://example.com",
      body: {
        hello: "world",
      },
      retries: 3,
    });
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.message.publish_json(
        url="https://example.com",
        body={
            "hello": "world",
        },
        retries=3,
    )
    # Async version is also available
    ```
  </Tab>
</Tabs>

### Set callback url

Receive a response from the endpoint and send it to the specified callback URL.
If the endpoint returns a response, QStash will send it to the failure callback URL.

<Tabs>
  <Tab title="cURL">
    ```shell
    curl -XPOST \
        -H 'Authorization: Bearer XXX' \
        -H "Content-type: application/json" \
        -H "Upstash-Callback: https://example.com/callback" \
        -H "Upstash-Failure-Callback: https://example.com/failure" \
        -d '{ "hello": "world" }' \
        'https://qstash.upstash.io/v2/publish/https://example.com'
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });
    await client.publishJSON({
      url: "https://example.com",
      body: {
        hello: "world",
      },
      callback: "https://example.com/callback",
      failureCallback: "https://example.com/failure",
    });
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.message.publish_json(
        url="https://example.com",
        body={
            "hello": "world",
        },
        callback="https://example.com/callback",
        failure_callback="https://example.com/failure",
    )
    # Async version is also available
    ```
  </Tab>
</Tabs>

### List all events

Retrieve a list of all [events](https://upstash.com/docs/qstash/api/events/list) that have
been published (filtering is also available).

<Tabs>
  <Tab title="cURL">
    ```shell
    curl https://qstash.upstash.io/v2/events \
        -H "Authorization: Bearer XXX"
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });
    const events = await client.events()
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.event.list()
    # Async version is also available
    ```
  </Tab>
</Tabs>

### List all schedules

<Tabs>
  <Tab title="cURL">
    ```shell
    curl https://qstash.upstash.io/v2/schedules \
        -H "Authorization: Bearer XXX"
    ```
  </Tab>

  <Tab title="Typescript SDK">
    ```typescript
    const client = new Client({ token: "<QSTASH_TOKEN>" });
    const scheds = await client.schedules.list();
    ```
  </Tab>

  <Tab title="Python SDK">
    ```python
    from qstash import QStash

    client = QStash("<QSTASH_TOKEN>")
    client.schedule.list()
    # Async version is also available
    ```
  </Tab>
</Tabs>


# Changelog



<Warning>Workflow changelogs are [here](/workflow/changelog)</Warning>

<Update label="October 2024">
  *   Added resend API. See [here](/qstash/integrations/resend). Released with qstash-js v2.7.14.
  *   Fixed a bug on qstash-js where we skipped using the next signing key when the current signing key fails to verify the `upstash-signature`. Released with qstash-js v2.7.14.
  *   Added `schedule to queues` feature to the qstash-js. See [here](/qstash/features/schedules#scheduling-to-a-queue). Released with qstash-js v2.7.14.
  *   Optimized the console by trimming event bodies, reducing resource usage and enabling efficient querying of events with large payloads.
  *   Began development on a new architecture to deliver faster event processing on the server.
  *   Added more fields to events in the [REST API](/qstash/api/events/list), including `Timeout`, `Method`, `Callback`, `CallbackHeaders`, `FailureCallback`, `FailureCallbackHeaders`, and `MaxRetries`.
  *   Enhanced retry backoff logic by supporting additional headers for retry timing. Along with `Retry-After`, Qstash now recognizes `X-RateLimit-Reset`, `X-RateLimit-Reset-Requests`, and `X-RateLimit-Reset-Tokens` as backoff time indicators. See [here](/qstash/features/retry#retry-after-headers) for more details.
</Update>

<Update label="September 2024">
  *   Improved performance, resulting in reduced latency for average publish times.
  *   Set the `nbf` (not before) claim on Signing Keys to 0. This claim specifies the time before which the JWT must not be processed. Previously, this was incorrectly used, causing validation issues when there were minor clock discrepancies between systems.
  *   Fixed queue name validation. Queue names must now be alphanumeric or include hyphens, underscores, or periods, consistent with other API resources.
  *   Resolved bugs related to  [overwriting a schedule](https://upstash.com/docs/qstash/features/schedules#overwriting-an-existing-schedule).
</Update>

<Update label="August 2024">
  *   Released [Upstash Workflow](https://upstash.com/docs/qstash/workflow).
  *   Fixed a bug where paused schedules were mistakenly resumed after a process restart (typically occurring during new version releases).
</Update>

<Update label="July 2024">
  *   Big update on the UI, where all the Rest functinality exposed in the Console.
  *   Addded order query parameter to [/v2/events](https://upstash.com/docs/qstash/api/events/list) and [/v2/dlq](https://upstash.com/docs/qstash/api/dlq/listMessages) endpoints.
  *   Added [ability to configure](https://upstash.com/docs/qstash/features/callbacks#configuring-callbacks) callbacks(/failure\_callbacks)
  *   A critical fix for schedule pause and resume Rest APIs where the endpoints were not working at all before the fix.
</Update>

<Update label="June 2024">
  *   Pause and resume for scheduled messages
  *   Pause and resume for queues
  *   [Bulk cancel](https://upstash.com/docs/qstash/api/messages/bulk-cancel) messages
  *   Body and headers on [events](https://upstash.com/docs/qstash/api/events/list)
  *   Fixed inaccurate queue lag
</Update>

<Update label="May 2024">
  *   [Retry-After](https://upstash.com/docs/qstash/features/retry#retry-after-header) support for rate-limited endpoints
  *   [Upstash-Timeout](https://upstash.com/docs/qstash/api/publish) header
</Update>

<Update label="April 2024">
  *   [Queues and parallelism](https://upstash.com/docs/qstash/features/queues)
  *   [Event filtering](https://upstash.com/docs/qstash/api/events/list)
</Update>

<Update label="March 2024">
  *   [Batch publish messages](https://upstash.com/docs/qstash/api/messages/batch)
  *   [Bulk delete](https://upstash.com/docs/qstash/api/dlq/deleteMessages) for DLQ
</Update>

<Update label="February 2024">
  *   Added [failure callback support](https://upstash.com/docs/qstash/api/schedules/create) to scheduled messages
  *   Added Upstash-Caller-IP header to outgoing messages. See \[[https://upstash.com/docs/qstash/howto/receiving](https://upstash.com/docs/qstash/howto/receiving)] for all headers
  *   Added Schedule ID to [events](https://upstash.com/docs/qstash/api/events/list) and [messages](https://upstash.com/docs/qstash/api/messages/get)
</Update>

<Update label="November 2023">
  *   Put last response in DLQ
  *   DLQ [get message](https://upstash.com/docs/qstash/api/dlq/getMessage)
  *   Pass schedule ID to the header when calling the user's endpoint
  *   Added more information to [callbacks](https://upstash.com/docs/qstash/features/callbacks)
</Update>

<Update label="October 2023">
  *   Added [Upstash-Failure-Callback](https://upstash.com/docs/qstash/features/callbacks#what-is-a-failure-callback)
</Update>


# Compare



In this section, we will compare QStash with alternative solutions.

### BullMQ

BullMQ is a message queue for NodeJS based on Redis. BullMQ is open source
project, you can run BullMQ yourself.

*   Using BullMQ in serverless environments is problematic due to stateless nature
    of serverless. QStash is designed for serverless environments.

*   With BullMQ, you need to run a stateful application to consume messages.
    QStash calls the API endpoints, so you do not need your application to consume
    messages continuously.

*   You need to run and maintain BullMQ and Redis yourself. QStash is completely
    serverless, you maintain nothing and pay for just what you use.

### Zeplo

Zeplo is a message queue targeting serverless. Just like QStash it allows users
to queue and schedule HTTP requests.

While Zeplo targets serverless, it has a fixed monthly price in paid plans which
is \$39/month. In QStash, price scales to zero, you do not pay if you are not
using it.

With Zeplo, you can send messages to a single endpoint. With QStash, in addition
to endpoint, you can submit messages to a URL Group which groups one or more
endpoints into a single namespace. Zeplo does not have URL Group functionality.

### Quirrel

Quirrel is a job queueing service for serverless. It has a similar functionality
with QStash.

Quirrel is acquired by Netlify, some of its functionality is available as
Netlify scheduled functions. QStash is platform independent, you can use it
anywhere.


# Getting Started



QStash is a **serverless messaging and scheduling solution**. It fits easily into your existing workflow and allows you to build reliable systems without managing infrastructure.

Instead of calling an endpoint directly, QStash acts as a middleman between you and an API to guarantee delivery, perform automatic retries on failure, and more.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/qstash-benefits.png" />
</Frame>

<Tip href="/workflow/getstarted">
  We have a new SDK called [Upstash Workflow](/workflow/getstarted).

  **Upstash Workflow SDK** is **QStash** simplified for your complex applications

  *   Skip the details of preparing a complex dependent endpoints.
  *   Focus on the essential parts.
  *   Enjoy automatic retries and delivery guarantees.
  *   Avoid platform-specific timeouts.

  Check out [Upstash Workflow Getting Started](/workflow/getstarted) for more.
</Tip>

## Quick Start

Check out these Quick Start guides to get started with QStash in your application.

<CardGroup cols={2}>
  <Card title="Next.js" icon="node-js" href="/qstash/quickstarts/vercel-nextjs">
    Build a Next application that uses QStash to start a long-running job on your platform
  </Card>

  <Card title="Python" icon="python" href="/qstash/quickstarts/python-vercel">
    Build a Python application that uses QStash to schedule a daily job that clean up a database
  </Card>
</CardGroup>

Or continue reading to learn how to send your first message!

## Send your first message

<Check>
  **Prerequisite**

  You need an Upstash account before publishing messages, create one
  [here](https://console.upstash.com).
</Check>

### Public API

Make sure you have a publicly available HTTP API that you want to send your
messages to. If you don't, you can use something like
[requestcatcher.com](https://requestcatcher.com/), [webhook.site](https://webhook.site/) or
[webhook-test.com](https://webhook-test.com/) to try it out.

For example, you can use this URL to test your messages: [https://firstqstashmessage.requestcatcher.com](https://firstqstashmessage.requestcatcher.com)

### Get your token

Go to the [Upstash Console](https://console.upstash.com/qstash) and copy the
`QSTASH_TOKEN`.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/rest_token.png" />
</Frame>

### Publish a message

A message can be any shape or form: json, xml, binary, anything, that can be
transmitted in the http request body. We do not impose any restrictions other
than a size limit of 1 MB (which can be customized at your request).

In addition to the request body itself, you can also send HTTP headers. Learn
more about this in the [message publishing section](/qstash/howto/publishing).

<CodeGroup>
  ```bash cURL
  curl -XPOST \
      -H 'Authorization: Bearer <QSTASH_TOKEN>' \
      -H "Content-type: application/json" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/publish/https://<your-api-url>'
  ```

  ```bash cURL RequestCatcher
  curl -XPOST \
      -H 'Authorization: Bearer <QSTASH_TOKEN>' \
      -H "Content-type: application/json" \
      -d '{ "hello": "world" }' \
      'https://qstash.upstash.io/v2/publish/https://firstqstashmessage.requestcatcher.com/test'
  ```
</CodeGroup>

Don't worry, we have SDKs for different languages so you don't
have to make these requests manually.

### Check Response

You should receive a response with a unique message ID.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/reqcatcher.png" />
</Frame>

### Check Message Status

Head over to [Upstash Console](https://console.upstash.com/qstash) and go to the
`Events` tab where you can see your message activities.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/log.png" />
</Frame>

Learn more about different states [here](/qstash/howto/debug-logs).

## Features and Use Cases

<CardGroup cols={2}>
  <Card title="Background Jobs" icon="share-all" href="/qstash/features/background-jobs">
    Run long-running tasks in the background, without blocking your application
  </Card>

  <Card title="Schedules" icon="calendar-days" href="/qstash/features/schedules">
    Schedule messages to be delivered at a time in the future
  </Card>

  <Card title="Fan out" icon="arrows-maximize" href="/qstash/features/url-groups">
    Publish messages to multiple endpoints, in parallel, using URL Groups
  </Card>

  <Card title="FIFO" icon="right-left" href="/qstash/features/queues#ordered-delivery">
    Enqueue messages to be delivered one by one in the order they have enqueued.
  </Card>

  <Card title="Parallelism" icon="arrows-up-to-line" href="/qstash/features/queues#controlled-parallelism">
    Controlled parallelism to avoid overflowing your endpoint without compromising throughput.
  </Card>

  <Card title="Callbacks" icon="phone" href="/qstash/features/callbacks">
    Get a response delivered to your API when a message is delivered
  </Card>

  <Card title="Retry Failed Jobs" icon="repeat" href="/qstash/features/dlq">
    Use a Dead Letter Queue to have full control over failed messages
  </Card>

  <Card title="Deduplication" icon="copy" href="/qstash/features/deduplication">
    Prevent duplicate messages from being delivered
  </Card>

  <Card title="LLM" icon="message" href="/qstash/features/llm">
    Generate responses for one or more chat messages using large language models
  </Card>

  <Card title="LLM Integrations" icon="shapes" href="/qstash/features/llm#integrations-with-qstash-apis">
    Publish, enqueue, or batch chat completion requests using large language models with QStash
    features.
  </Card>
</CardGroup>


# Pricing & Limits



Please check our [pricing page](https://upstash.com/pricing/qstash) for the most up-to-date information on pricing and limits.


# Use Cases



TODO: andreas: rework and reenable this page after we have 2 use cases ready
[https://linear.app/upstash/issue/QSTH-84/use-cases-summaryhighlights-of-recipes](https://linear.app/upstash/issue/QSTH-84/use-cases-summaryhighlights-of-recipes)

This section is still a work in progress.

We will be adding detailed tutorials for each use case soon.

Tell us on [Discord](https://discord.gg/w9SenAtbme) or
[X](https://x.com/upstash) what you would like to see here.

### Triggering Nextjs Functions on a schedule

Create a schedule in QStash that runs every hour and calls a Next.js serverless
function hosted on Vercel.

### Reset Billing Cycle in your Database

Once a month, reset database entries to start a new billing cycle.

### Fanning out alerts to Slack, email, Opsgenie, etc.

Createa QStash URL Group that receives alerts from a single source and delivers them
to multiple destinations.

### Send delayed message when a new user signs up

Publish delayed messages whenever a new user signs up in your app. After a
certain delay (e.g. 10 minutes), QStash will send a request to your API,
allowing you to email the user a welcome message.


# AWS Lambda (Node)



## Setting up a Lambda

The [AWS CDK](https://aws.amazon.com/cdk/) is the most convenient way to create a new project on AWS Lambda. For example, it lets you directly define integrations such as APIGateway, a tool to make our lambda publicly available as an API, in your code.

```bash Terminal
mkdir my-app
cd my-app
cdk init app -l typescript
npm i esbuild @upstash/qstash
mkdir lambda
touch lambda/index.ts
```

## Webhook verification

### Using the SDK (recommended)

Edit `lambda/index.ts`, the file containing our core lambda logic:

```ts lambda/index.ts
import { Receiver } from "@upstash/qstash"
import type { APIGatewayProxyEvent, APIGatewayProxyResult } from "aws-lambda"

const receiver = new Receiver({
  currentSigningKey: process.env.QSTASH_CURRENT_SIGNING_KEY ?? "",
  nextSigningKey: process.env.QSTASH_NEXT_SIGNING_KEY ?? "",
})

export const handler = async (
  event: APIGatewayProxyEvent
): Promise<APIGatewayProxyResult> => {
  const signature = event.headers["upstash-signature"]
  const lambdaFunctionUrl = `https://${event.requestContext.domainName}`

  if (!signature) {
    return {
      statusCode: 401,
      body: JSON.stringify({ message: "Missing signature" }),
    }
  }

  try {
    await receiver.verify({
      signature: signature,
      body: event.body ?? "",
      url: lambdaFunctionUrl,
    })
  } catch (err) {
    return {
      statusCode: 401,
      body: JSON.stringify({ message: "Invalid signature" }),
    }
  }

  // Request is valid, perform business logic

  return {
    statusCode: 200,
    body: JSON.stringify({ message: "Request processed successfully" }),
  }
}
```

We'll set the `QSTASH_CURRENT_SIGNING_KEY` and `QSTASH_NEXT_SIGNING_KEY` environment variables together when deploying our Lambda.

### Manual Verification

In this section, we'll manually verify our incoming QStash requests without additional packages. Also see our [manual verification example](https://github.com/upstash/qstash-examples/tree/main/aws-lambda).

1.  Implement the handler function

```ts lambda/index.ts
import type { APIGatewayEvent, APIGatewayProxyResult } from "aws-lambda"
import { createHash, createHmac } from "node:crypto"

export const handler = async (
  event: APIGatewayEvent,
): Promise<APIGatewayProxyResult> => {
  const signature = event.headers["upstash-signature"] ?? ""
  const currentSigningKey = process.env.QSTASH_CURRENT_SIGNING_KEY ?? ""
  const nextSigningKey = process.env.QSTASH_NEXT_SIGNING_KEY ?? ""

  const url = `https://${event.requestContext.domainName}`

  try {
    // Try to verify the signature with the current signing key and if that fails, try the next signing key
    // This allows you to roll your signing keys once without downtime
    await verify(signature, currentSigningKey, event.body, url).catch((err) => {
      console.error(
        `Failed to verify signature with current signing key: ${err}`
      )

      return verify(signature, nextSigningKey, event.body, url)
    })
  } catch (err) {
    const message = err instanceof Error ? err.toString() : err

    return {
      statusCode: 400,
      body: JSON.stringify({ error: message }),
    }
  }

  // Add your business logic here

  return {
    statusCode: 200,
    body: JSON.stringify({ message: "Request processed successfully" }),
  }
}
```

2.  Implement the `verify` function:

```ts lambda/index.ts
/**
 * @param jwt - The content of the `upstash-signature` header (JWT)
 * @param signingKey - The signing key to use to verify the signature (Get it from Upstash Console)
 * @param body - The raw body of the request
 * @param url - The public URL of the lambda function
 */
async function verify(
  jwt: string,
  signingKey: string,
  body: string | null,
  url: string
): Promise<void> {
  const split = jwt.split(".")
  if (split.length != 3) {
    throw new Error("Invalid JWT")
  }
  const [header, payload, signature] = split

  if (
    signature !=
    createHmac("sha256", signingKey)
      .update(`${header}.${payload}`)
      .digest("base64url")
  ) {
    throw new Error("Invalid JWT signature")
  }

  // JWT is verified, start looking at payload claims
  const p: {
    sub: string
    iss: string
    exp: number
    nbf: number
    body: string
  } = JSON.parse(Buffer.from(payload, "base64url").toString())

  if (p.iss !== "Upstash") {
    throw new Error(`invalid issuer: ${p.iss}, expected "Upstash"`)
  }
  if (p.sub !== url) {
    throw new Error(`invalid subject: ${p.sub}, expected "${url}"`)
  }

  const now = Math.floor(Date.now() / 1000)
  if (now > p.exp) {
    throw new Error("token has expired")
  }
  if (now < p.nbf) {
    throw new Error("token is not yet valid")
  }

  if (body != null) {
    if (
      p.body.replace(/=+$/, "") !=
      createHash("sha256").update(body).digest("base64url")
    ) {
      throw new Error("body hash does not match")
    }
  }
}
```

You can find the complete example
[here](https://github.com/upstash/qstash-examples/blob/main/aws-lambda/typescript-example/index.ts).

## Deploying a Lambda

### Using the AWS CDK (recommended)

Because we used the AWS CDK to initialize our project, deployment is straightforward. Edit the `lib/<your-stack-name>.ts` file the CDK created when bootstrapping the project. For example, if our lambda webhook does video processing, it could look like this:

```ts lib/<your-stack-name>.ts
import * as cdk from "aws-cdk-lib";
import * as lambda from "aws-cdk-lib/aws-lambda";
import { NodejsFunction } from "aws-cdk-lib/aws-lambda-nodejs";
import { Construct } from "constructs";
import path from "path";
import * as apigateway from 'aws-cdk-lib/aws-apigateway';

export class VideoProcessingStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props)

    // Create the Lambda function
    const videoProcessingLambda = new NodejsFunction(this, 'VideoProcessingLambda', {
      runtime: lambda.Runtime.NODEJS_20_X,
      handler: 'handler',
      entry: path.join(__dirname, '../lambda/index.ts'),
    });

    // Create the API Gateway
    const api = new apigateway.RestApi(this, 'VideoProcessingApi', {
      restApiName: 'Video Processing Service',
      description: 'This service handles video processing.',
      defaultMethodOptions: {
        authorizationType: apigateway.AuthorizationType.NONE,
      },
    });

    api.root.addMethod('POST', new apigateway.LambdaIntegration(videoProcessingLambda));
  }
}
```

Every time we now run the following deployment command in our terminal, our changes are going to be deployed right to a publicly available API, authorized by our QStash webhook logic from before.

```bash Terminal
cdk deploy
```

You may be prompted to confirm the necessary AWS permissions during this process, for example allowing APIGateway to invoke your lambda function.

Once your code has been deployed to Lambda, you'll receive a live URL to your endpoint via the CLI and can see the new APIGateway connection in your AWS dashboard:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/api-gateway.png" />
</Frame>

The URL you use to invoke your function typically follows this format, especially if you follow the same stack configuration as shown above:

`https://<API-GATEWAY-ID>.execute-api.<API-REGION>.amazonaws.com/prod/`

To provide our `QSTASH_CURRENT_SIGNING_KEY` and `QSTASH_NEXT_SIGNING_KEY` environment variables, navigate to your QStash dashboard:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_signing_keys.png" />
</Frame>

and make these two variables available to your Lambda in your function configuration:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/environment.png" />
</Frame>

Tada, we just deployed a live Lambda with the AWS CDK! 🎉

### Manual Deployment

1.  Create a new Lambda function by going to the [AWS dashboard](https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#/create/function) for your desired lambda region. Give your new function a name and select `Node.js 20.x` as runtime, then create the function.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/create_lambda.png" />
</Frame>

2.  To make this Lambda available under a public URL, navigate to the `Configuration` tab and click `Function URL`:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/create_url.png" />
</Frame>

3.  In the following dialog, you'll be asked to select one of two authentication types. Select `NONE`, because we are handling authentication ourselves. Then, click `Save`.

    You'll see the function URL on the right side of your function overview:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/overview.png" />
</Frame>

4.  Get your current and next signing key from the
    [Upstash Console](https://console.upstash.com/qstash).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_signing_keys.png" />
</Frame>

5.  Still under the `Configuration` tab, set the `QSTASH_CURRENT_SIGNING_KEY` and `QSTASH_NEXT_SIGNING_KEY`
    environment variables:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/environment.png" />
</Frame>

6.  Add the following script to your `package.json` file to build and zip your code:

```json package.json
{
  "scripts": {
    "build": "rm -rf ./dist; esbuild index.ts --bundle --minify --sourcemap --platform=node --target=es2020 --outfile=dist/index.js && cd dist && zip -r index.zip index.js*"
  }
}
```

7.  Click the `Upload from` button for your Lambda and
    deploy the code to AWS. Select `./dist/index.zip` as the upload file.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/upload.png" />
</Frame>

Tada, you've manually deployed a zip file to AWS Lambda! 🎉

## Testing the Integration

To make sure everything works as expected, navigate to your QStash request builder and send a request to your freshly deployed Lambda function:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/verify-integration.png" />
</Frame>

Alternatively, you can also send a request via CURL:

```bash Terminal
curl --request POST "https://qstash.upstash.io/v2/publish/<YOUR-LAMBDA-URL>" \
-H "Authorization: Bearer <QSTASH_TOKEN>" \
-H "Content-Type: application/json" \
-d "{ \"hello\": \"world\"}"
```


# AWS Lambda (Python)



[Source Code](https://github.com/upstash/qstash-examples/tree/main/aws-lambda/python-example)

This is a step by step guide on how to receive webhooks from QStash in your
Lambda function on AWS.

### 1. Create a new project

Let's create a new folder called `aws-lambda` and initialize a new project by
creating `lambda_function.py` This example uses Makefile, but the scripts can
also be written for `Pipenv`.

```bash
mkdir aws-lambda
cd aws-lambda
touch lambda_function.py
```

### 2. Dependencies

We are using `PyJwt` for decoding the JWT token in our code. We will install the
package in the zipping stage.

### 3. Creating the handler function

In this example we will show how to receive a webhook from QStash and verify the
signature.

First, let's import everything we need:

```python
import json
import os
import hmac
import hashlib
import base64
import time
import jwt
```

Now, we create the handler function. In the handler we will prepare all
necessary variables that we need for verification. This includes the signature,
the signing keys and the url of the lambda function. Then we try to verify the
request using the current signing key and if that fails we will try the next
one. If the signature could be verified, we can start processing the request.

```python
def lambda_handler(event, context):

    # parse the inputs
    current_signing_key = os.environ['QSTASH_CURRENT_SIGNING_KEY']
    next_signing_key = os.environ['QSTASH_NEXT_SIGNING_KEY']

    headers = event['headers']
    signature = headers['upstash-signature']
    url = 'https://{}'.format(event['requestContext']['domainName'])
    body = None
    if 'body' in event:
        body = event['body']


    # check verification now
    try:
        verify(signature, current_signing_key, body, url)
    except Exception as e:
        print("Failed to verify signature with current signing key:", e)
        try:
            verify(signature, next_signing_key, body, url)
        except Exception as e2:
            return {
                "statusCode": 400,
                "body": json.dumps({
                    "error": str(e2),
                }),
            }


    # Your logic here...

    return {
        "statusCode": 200,
        "body": json.dumps({
            "message": "ok",
        }),
    }
```

The `verify` function will handle the actual verification of the signature. The
signature itself is actually a [JWT](https://jwt.io) and includes claims about
the request. See [here](/qstash/features/security#claims).

```python
# @param jwt_token - The content of the `upstash-signature` header
# @param signing_key - The signing key to use to verify the signature (Get it from Upstash Console)
# @param body - The raw body of the request
# @param url - The public URL of the lambda function
def verify(jwt_token, signing_key, body, url):
    split = jwt_token.split(".")
    if len(split) != 3:
        raise Exception("Invalid JWT.")

    header, payload, signature = split

    message = header + '.' + payload
    generated_signature = base64.urlsafe_b64encode(hmac.new(bytes(signing_key, 'utf-8'), bytes(message, 'utf-8'), digestmod=hashlib.sha256).digest()).decode()

    if generated_signature != signature and signature + "=" != generated_signature :
        raise Exception("Invalid JWT signature.")

    decoded = jwt.decode(jwt_token, options={"verify_signature": False})
    sub = decoded['sub']
    iss = decoded['iss']
    exp = decoded['exp']
    nbf = decoded['nbf']
    decoded_body = decoded['body']

    if iss != "Upstash":
        raise Exception("Invalid issuer: {}".format(iss))

    if sub != url:
        raise Exception("Invalid subject: {}".format(sub))

    now = time.time()
    if now > exp:
        raise Exception("Token has expired.")

    if now < nbf:
        raise Exception("Token is not yet valid.")


    if body != None:
        while decoded_body[-1] == "=":
            decoded_body = decoded_body[:-1]

        m = hashlib.sha256()
        m.update(bytes(body, 'utf-8'))
        m = m.digest()
        generated_hash = base64.urlsafe_b64encode(m).decode()

        if generated_hash != decoded_body and generated_hash != decoded_body + "=" :
                raise Exception("Body hash doesn't match.")
```

You can find the complete file
[here](https://github.com/upstash/qstash-examples/tree/main/aws-lambda/python-example/lambda_function.py).

That's it, now we can create the function on AWS and test it.

### 4. Create a Lambda function on AWS

Create a new Lambda function from scratch by going to the
[AWS console](https://us-east-1.console.aws.amazon.com/lambda/home?region=us-east-1#/create/function).
(Make sure you select your desired region)

Give it a name and select `Python 3.8` as runtime, then create the function.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/python/create_lambda.png" />
</Frame>

Afterwards we will add a public URL to this lambda by going to the
`Configuration` tab:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/create_url.png" />
</Frame>

Select `Auth Type = NONE` because we are handling authentication ourselves.

After creating the url, you should see it on the right side of the overview of
your function:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/python/overview.png" />
</Frame>

### 5. Set Environment Variables

Get your current and next signing key from the
[Upstash Console](https://console.upstash.com/qstash)

On the same `Configuration` tab from earlier, we will now set the required
environment variables:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/environment.png" />
</Frame>

### 6. Deploy your Lambda function

We need to bundle our code and zip it to deploy it to AWS.

Add the following script to your `Makefile` file (or corresponding pipenv
script):

```yaml
zip:
    rm -rf dist
	pip3 install --target ./dist pyjwt
	cp lambda_function.py ./dist/lambda_function.py
	cd dist && zip -r lambda.zip .
	mv ./dist/lambda.zip ./
```

When calling `make zip` this will install PyJwt and zip the code.

Afterwards we can click the `Upload from` button in the lower right corner and
deploy the code to AWS. Select `lambda.zip` as upload file.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/aws/upload.png" />
</Frame>

### 7. Publish a message

Open a different terminal and publish a message to QStash. Note the destination
url is the URL from step 4.

```bash
curl --request POST "https://qstash.upstash.io/v2/publish/https://urzdbfn4et56vzeasu3fpcynym0zerme.lambda-url.eu-west-1.on.aws" \
     -H "Authorization: Bearer <QSTASH_TOKEN>" \
     -H "Content-Type: application/json" \
     -d "{ \"hello\": \"world\"}"
```

## Next Steps

That's it, you have successfully created a secure AWS lambda function, that
receives and verifies incoming webhooks from qstash.

Learn more about publishing a message to qstash [here](/qstash/howto/publishing)


# Cloudflare Workers



This is a step by step guide on how to receive webhooks from QStash in your
Cloudflare Worker.

### Project Setup

We will use **C3 (create-cloudflare-cli)** command-line tool to create our functions. You can open a new terminal window and run C3 using the prompt below.

<CodeGroup>
  ```shell npm
  npm create cloudflare@latest
  ```

  ```shell yarn
  yarn create cloudflare@latest
  ```
</CodeGroup>

This will install the `create-cloudflare` package, and lead you through setup. C3 will also install Wrangler in projects by default, which helps us testing and deploying the projects.

```text
➜  npm create cloudflare@latest
Need to install the following packages:
  create-cloudflare@2.1.0
Ok to proceed? (y) y

using create-cloudflare version 2.1.0

╭ Create an application with Cloudflare Step 1 of 3
│
├ In which directory do you want to create your application?
│ dir ./cloudflare_starter
│
├ What type of application do you want to create?
│ type "Hello World" Worker
│
├ Do you want to use TypeScript?
│ yes typescript
│
├ Copying files from "hello-world" template
│
├ Do you want to use TypeScript?
│ yes typescript
│
├ Retrieving current workerd compatibility date
│ compatibility date 2023-08-07
│
├ Do you want to use git for version control?
│ yes git
│
╰ Application created
```

We will also install the **Upstash QStash library**.

```bash
npm install @upstash/qstash
```

### 3. Use QStash in your handler

First we import the library:

```ts src/index.ts
import { Receiver } from "@upstash/qstash";
```

Then we adjust the `Env` interface to include the `QSTASH_CURRENT_SIGNING_KEY`
and `QSTASH_NEXT_SIGNING_KEY` environment variables.

```ts src/index.ts
export interface Env {
  QSTASH_CURRENT_SIGNING_KEY: string;
  QSTASH_NEXT_SIGNING_KEY: string;
}
```

And then we validate the signature in the `handler` function.

First we create a new receiver and provide it with the signing keys.

```ts src/index.ts
const receiver = new Receiver({
  currentSigningKey: env.QSTASH_CURRENT_SIGNING_KEY,
  nextSigningKey: env.QSTASH_NEXT_SIGNING_KEY,
});
```

Then we verify the signature.

```ts src/index.ts
const body = await request.text();

const isValid = receiver.verify({
  signature: request.headers.get("Upstash-Signature")!,
  body,
});
```

The entire file looks like this now:

```ts src/index.ts
import { Receiver } from "@upstash/qstash";
export interface Env {
  QSTASH_CURRENT_SIGNING_KEY: string;
  QSTASH_NEXT_SIGNING_KEY: string;
}

export default {
  async fetch(
    request: Request,
    env: Env,
    ctx: ExecutionContext
  ): Promise<Response> {
    const c = new Receiver({
      currentSigningKey: env.QSTASH_CURRENT_SIGNING_KEY,
      nextSigningKey: env.QSTASH_NEXT_SIGNING_KEY,
    });

    const body = await request.text();

    const isValid = await c
      .verify({
        signature: request.headers.get("Upstash-Signature")!,
        body,
      })
      .catch((err) => {
        console.error(err);
        return false;
      });
    if (!isValid) {
      return new Response("Invalid signature", { status: 401 });
    }
    console.log("The signature was valid");

    // do work here

    return new Response("Hello World!");
  },
};
```

### Configure Credentials

There are two methods for setting up the credentials for QStash. The recommended way is to use Cloudflare Upstash Integration. Alternatively, you can add the credentials manually.

#### Using the Cloudflare Integration

Access to the [Cloudflare Dashboard](https://dash.cloudflare.com) and login with the same account that you've used while setting up the Worker application. Then, navigate to **Workers & Pages > Overview** section on the sidebar. Here, you'll find your application listed.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/cloudflare-integration/overview.png" />
</Frame>

Clicking on the application will direct you to the application details page, where you can perform the integration process. Switch to the **Settings** tab in the application details, and proceed to **Integrations** section. You will see various Worker integrations listed. To proceed, click the **Add Integration** button associated with the QStash.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/cloudflare-integration/qstash-add-integration.png" />
</Frame>

On the Integration page, connect to your Upstash account. Then, select the related database from the dropdown menu. Finalize the process by pressing Save button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/cloudflare-integration/qstash-credentials.png" />
</Frame>

#### Setting up Manually

Navigate to [Upstash Console](https://console.upstash.com) and copy/paste your QStash credentials to `wrangler.toml` as below.

```yaml
[vars]
QSTASH_URL="REPLACE_HERE"
QSTASH_TOKEN="REPLACE_HERE"
QSTASH_CURRENT_SIGNING_KEY="REPLACE_HERE"
QSTASH_NEXT_SIGNING_KEY="REPLACE_HERE"
```

### Test and Deploy

You can test the function locally with `npx wrangler dev`

Deploy your function to Cloudflare with `npx wrangler deploy`

The endpoint of the function will be provided to you, once the deployment is done.

### Publish a message

Open a different terminal and publish a message to QStash. Note the destination
url is the same that was printed in the previous deploy step.

```bash
curl --request POST "https://qstash.upstash.io/v2/publish/https://cloudflare-workers.upstash.workers.dev" \
     -H "Authorization: Bearer <QSTASH_TOKEN>" \
     -H "Content-Type: application/json" \
     -d "{ \"hello\": \"world\"}"
```

In the logs you should see something like this:

```bash
$ npx wrangler tail

⛅️ wrangler 2.0.17
--------------------
Retrieving cached values for account from node_modules/.cache/wrangler
Successfully created tail, expires at 2022-07-11T21:11:36Z
Connected to cloudflare-workers, waiting for logs...
POST https://cloudflare-workers.upstash.workers.dev/ - Ok @ 7/11/2022, 5:13:19 PM
  (log) The signature was valid
```

## Next Steps

That's it, you have successfully created a secure Cloudflare Worker, that
receives and verifies incoming webhooks from qstash.

Learn more about publishing a message to qstash [here](/qstash/howto/publishing).

You can find the source code [here](https://github.com/upstash/qstash-examples/tree/main/cloudflare-workers).


# Deno Deploy



[Source Code](https://github.com/upstash/qstash-examples/tree/main/deno-deploy)

This is a step by step guide on how to receive webhooks from QStash in your Deno
deploy project.

### 1. Create a new project

Go to [https://dash.deno.com/projects](https://dash.deno.com/projects) and
create a new playground project.

### 2. Edit the handler function

Then paste the following code into the browser editor:

```ts
import { serve } from "https://deno.land/std@0.142.0/http/server.ts";
import { Receiver } from "https://deno.land/x/upstash_qstash@v0.1.4/mod.ts";

serve(async (req: Request) => {
  const r = new Receiver({
    currentSigningKey: Deno.env.get("QSTASH_CURRENT_SIGNING_KEY")!,
    nextSigningKey: Deno.env.get("QSTASH_NEXT_SIGNING_KEY")!,
  });

  const isValid = await r
    .verify({
      signature: req.headers.get("Upstash-Signature")!,
      body: await req.text(),
    })
    .catch((err: Error) => {
      console.error(err);
      return false;
    });

  if (!isValid) {
    return new Response("Invalid signature", { status: 401 });
  }

  console.log("The signature was valid");

  // do work

  return new Response("OK", { status: 200 });
});
```

### 3. Add your signing keys

Click on the `settings` button at the top of the screen and then click
`+ Add Variable`

Get your current and next signing key from
[Upstash](https://console.upstash.com/qstash) and then set them in deno deploy.

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/deno_deploy_env.png)

### 4. Deploy

Simply click on `Save & Deploy` at the top of the screen.

### 5. Publish a message

Make note of the url displayed in the top right. This is the public url of your
project.

```bash
curl --request POST "https://qstash.upstash.io/v2/publish/https://early-frog-33.deno.dev" \
     -H "Authorization: Bearer <QSTASH_TOKEN>" \
     -H "Content-Type: application/json" \
     -d "{ \"hello\": \"world\"}"
```

In the logs you should see something like this:

```basheurope-west3isolate start time: 2.21 ms
Listening on http://localhost:8000/
The signature was valid
```

## Next Steps

That's it, you have successfully created a secure deno API, that receives and
verifies incoming webhooks from qstash.

Learn more about publishing a message to qstash [here](/qstash/howto/publishing)


# Golang



[Source Code](https://github.com/upstash/qstash-examples/tree/main/fly.io/go)

This is a step by step guide on how to receive webhooks from QStash in your
Golang application running on [fly.io](https://fly.io).

## 0. Prerequisites

*   [flyctl](https://fly.io/docs/getting-started/installing-flyctl/) - The fly.io
    CLI

## 1. Create a new project

Let's create a new folder called `flyio-go` and initialize a new project.

```bash
mkdir flyio-go
cd flyio-go
go mod init flyio-go
```

## 2. Creating the main function

In this example we will show how to receive a webhook from QStash and verify the
signature using the popular [golang-jwt/jwt](https://github.com/golang-jwt/jwt)
library.

First, let's import everything we need:

```go
package main

import (
	"crypto/sha256"
	"encoding/base64"
	"fmt"
	"github.com/golang-jwt/jwt/v4"
	"io"
	"net/http"
	"os"
	"time"
)
```

Next we create `main.go`. Ignore the `verify` function for now. We will add that
next. In the handler we will prepare all necessary variables that we need for
verification. This includes the signature and the signing keys. Then we try to
verify the request using the current signing key and if that fails we will try
the next one. If the signature could be verified, we can start processing the
request.

```go
func main() {
	port := os.Getenv("PORT")
	if port == "" {
		port = "8080"
	}

	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		defer r.Body.Close()

		currentSigningKey := os.Getenv("QSTASH_CURRENT_SIGNING_KEY")
		nextSigningKey := os.Getenv("QSTASH_NEXT_SIGNING_KEY")
		tokenString := r.Header.Get("Upstash-Signature")

		body, err := io.ReadAll(r.Body)
		if err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}

		err = verify(body, tokenString, currentSigningKey)
		if err != nil {
			fmt.Printf("Unable to verify signature with current signing key: %v", err)
			err = verify(body, tokenString, nextSigningKey)
		}

		if err != nil {
			http.Error(w, err.Error(), http.StatusUnauthorized)
			return
		}

		// handle your business logic here

		w.WriteHeader(http.StatusOK)

	})

	fmt.Println("listening on", port)
	err := http.ListenAndServe(":"+port, nil)
	if err != nil {
		panic(err)
	}
}
```

The `verify` function will handle verification of the [JWT](https://jwt.io),
that includes claims about the request. See
[here](/qstash/features/security#claims).

```go
func verify(body []byte, tokenString, signingKey string) error {
	token, err := jwt.Parse(
		tokenString,
		func(token *jwt.Token) (interface{}, error) {
			if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
				return nil, fmt.Errorf("Unexpected signing method: %v", token.Header["alg"])
			}
			return []byte(signingKey), nil
		})

	if err != nil {
		return err
	}

	claims, ok := token.Claims.(jwt.MapClaims)
	if !ok || !token.Valid {
		return fmt.Errorf("Invalid token")
	}

	if !claims.VerifyIssuer("Upstash", true) {
		return fmt.Errorf("invalid issuer")
	}
	if !claims.VerifyExpiresAt(time.Now().Unix(), true) {
		return fmt.Errorf("token has expired")
	}
	if !claims.VerifyNotBefore(time.Now().Unix(), true) {
		return fmt.Errorf("token is not valid yet")
	}

	bodyHash := sha256.Sum256(body)
	if claims["body"] != base64.URLEncoding.EncodeToString(bodyHash[:]) {
		return fmt.Errorf("body hash does not match")
	}

	return nil
}
```

You can find the complete file
[here](https://github.com/upstash/qstash-examples/blob/main/fly.io/go/main.go).

That's it, now we can deploy our API and test it.

## 3. Create app on fly.io

[Login](https://fly.io/docs/getting-started/log-in-to-fly/) with `flyctl` and
then `flyctl launch` the new app. This will create the necessary `fly.toml` for
us. It will ask you a bunch of questions. I chose all defaults here except for
the last question. We do not want to deploy just yet.

```bash
$ flyctl launch
Creating app in /Users/andreasthomas/github/upstash/qstash-examples/fly.io/go
Scanning source code
Detected a Go app
Using the following build configuration:
        Builder: paketobuildpacks/builder:base
        Buildpacks: gcr.io/paketo-buildpacks/go
? App Name (leave blank to use an auto-generated name):
Automatically selected personal organization: Andreas Thomas
? Select region: fra (Frankfurt, Germany)
Created app winer-cherry-9545 in organization personal
Wrote config file fly.toml
? Would you like to setup a Postgresql database now? No
? Would you like to deploy now? No
Your app is ready. Deploy with `flyctl deploy`
```

## 4. Set Environment Variables

Get your current and next signing key from the
[Upstash Console](https://console.upstash.com/qstash)

Then set them using `flyctl secrets set ...`

```bash
flyctl secrets set QSTASH_CURRENT_SIGNING_KEY=...
flyctl secrets set QSTASH_NEXT_SIGNING_KEY=...
```

## 5. Deploy the app

Fly.io made this step really simple. Just `flyctl deploy` and enjoy.

```bash
flyctl deploy
```

## 6. Publish a message

Now you can publish a message to QStash. Note the destination url is basically
your app name, if you are not sure what it is, you can go to
[fly.io/dashboard](https://fly.io/dashboard) and find out. In my case the app is
named "winter-cherry-9545" and the public url is
"[https://winter-cherry-9545.fly.dev](https://winter-cherry-9545.fly.dev)".

```bash
curl --request POST "https://qstash.upstash.io/v2/publish/https://winter-cherry-9545.fly.dev" \
     -H "Authorization: Bearer <QSTASH_TOKEN>" \
     -H "Content-Type: application/json" \
     -d "{ \"hello\": \"world\"}"
```

## Next Steps

That's it, you have successfully created a Go API hosted on fly.io, that
receives and verifies incoming webhooks from qstash.

Learn more about publishing a message to qstash [here](/qstash/howto/publishing)


# Python on Vercel



## Introduction

This quickstart will guide you through setting up QStash to run a daily script
to clean up your database. This is useful for testing and development environments
where you want to reset the database every day.

## Prerequisites

*   Create an Upstash account and get your [QStash token](https://console.upstash.com/qstash)

<Steps>
  <Step titleSize="h3" title="Create Python app">
    First, we'll create a new directory for our Python app. We'll call it `clean-db-cron`.

    The database we'll be using is Redis, so we'll need to install the `upstash_redis` package.

    ```bash
    mkdir clean-db-cron
    ```

    ```bash
    cd clean-db-cron
    ```

    ```bash
    pip install upstash-redis
    ```
  </Step>

  <Step titleSize="h3" title="Cleanup logic">
    Let's write the Python code to clean up the database. We'll use the `upstash_redis`
    package to connect to the database and delete all keys.

    ```python index.py
    from upstash_redis import Redis

    redis = Redis(url="https://YOUR_REDIS_URL", token="YOUR_TOKEN")

    def delete_all_entries():
      keys = redis.keys("*") # Match all keys
      redis.delete(*keys)


    delete_all_entries()
    ```

    Try running the code to see if it works. Your database keys should be deleted!
  </Step>

  <Step titleSize="h3" title="Make the Python code into a public endpoint">
    In order to use QStash, we need to make the Python code into a public endpoint. There
    are many ways to do this such as using Flask, FastAPI, or Django. In this example, we'll
    use the Python `http.server` module to create a simple HTTP server.

    ```python api/index.py
    from http.server import BaseHTTPRequestHandler
    from upstash_redis import Redis

    redis = Redis(url="https://YOUR_REDIS_URL", token="YOUR_TOKEN")

    def delete_all_entries():
      keys = redis.keys("*") # Match all keys
      redis.delete(*keys)


    class handler(BaseHTTPRequestHandler):
      def do_POST(self):
        delete_all_entries()
        self.send_response(200)
        self.end_headers()
    ```

    For the purpose of this tutorial, I'll deploy the application to Vercel using the
    [Python Runtime](https://vercel.com/docs/functions/runtimes/python), but feel free to
    use any other hosting provider.

    <Accordion title="Deploying to Vercel">
      There are many ways to [deploy to Vercel](https://vercel.com/docs/deployments/overview), but
      I'm going to use the Vercel CLI.

      ```bash
      npm install -g vercel
      ```

      ```bash
      vercel
      ```
    </Accordion>

    Once deployed, you can find the public URL in the dashboard.
  </Step>

  <Step titleSize="h3" title="Have QStash invoke the endpoint">
    There are two ways we can go about configuring QStash. We can either use the QStash dashboard
    or the QStash API. In this example, it makes more sense to utilize the dashboard since we
    only need to set up a singular cronjob.

    However, you can imagine a scenario where you have a large number of cronjobs and you'd
    want to automate the process. In that case, you'd want to use the QStash Python SDK.

    To create the schedule, go to the [QStash dashboard](https://console.upstash.com/qstash) and enter
    the URL of the public endpoint you created. Then, set the type to schedule and change the
    `Upstash-Cron` header to run daily at a time of your choosing.

    ```
    URL: https://your-vercel-app.vercel.app/api
    Type: Schedule
    Every: every day at midnight (feel free to customize)
    ```

    <Frame>
      <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/python-quickstart-schedule.png" alt="QStash console scheduling" />
    </Frame>

    Once you start the schedule, QStash will invoke the endpoint at the specified time. You can
    scroll down and verify the job has been created!

    <Accordion title="Using the SDK">
      If you have a use case where you need to automate the creation of jobs, you can use the SDK instead.

      ```python
      from qstash import QStash

      client = QStash("<QSTASH_TOKEN>")
      client.schedule.create(
          destination="https://YOUR_URL.vercel.app/api",
          cron="0 12 * * *",
      )
      ```
    </Accordion>
  </Step>
</Steps>

Now, go ahead and try it out for yourself! Try using some of the other features of QStash, such as
[callbacks](/qstash/features/callbacks) and [URL Groups](/qstash/features/url-groups).


# Next.js



QStash is a robust message queue and task-scheduling service that integrates perfectly with Next.js. This guide will show you how to use QStash in your Next.js projects, including a quickstart and a complete example.

## Quickstart

At its core, each QStash message contains two pieces of information:

*   URL (which endpoint to call)
*   Request body (e.g. IDs of items you want to process)

The following endpoint could be used to upload an image and then asynchronously queue a processing task to optimize the image in the background.

```tsx upload-image/route.ts
import { Client } from "@upstash/qstash"
import { NextResponse } from "next/server"

const client = new Client({ token: process.env.QSTASH_TOKEN! })

export const POST = async (req: Request) => {
  // Image uploading logic

  // 👇 Once uploading is done, queue an image processing task
  const result = await client.publishJSON({
    url: "https://your-api-endpoint.com/process-image",
    body: { imageId: "123" },
  })

  return NextResponse.json({
    message: "Image queued for processing!",
    qstashMessageId: result.messageId,
  })
}
```

Note that the URL needs to be publically available for QStash to call, either as a deployed project or by [developing with QStash locally](/qstash/howto/local-tunnel).

Because QStash calls our image processing task, we get automatic retries whenever the API throws an error. These retries make our function very reliable. We also let the user know immediately that their image has been successfully queued.

Now, let's **receive the QStash message** in our image processing endpoint:

```tsx process-image/route.ts
import { verifySignatureAppRouter } from "@upstash/qstash/nextjs"

// 👇 Verify that this messages comes from QStash
export const POST = verifySignatureAppRouter(async (req: Request) => {
  const body = await req.json()
  const { imageId } = body as { imageId: string }

  // Image processing logic, i.e. using sharp

  return new Response(`Image with id "${imageId}" processed successfully.`)
})
```

```bash .env
# Copy all three from your QStash dashboard
QSTASH_TOKEN=
QSTASH_CURRENT_SIGNING_KEY=
QSTASH_NEXT_SIGNING_KEY=
```

Just like that, we set up a reliable and asynchronous image processing system in Next.js. The same logic works for email queues, reliable webhook processing, long-running report generations and many more.

## Example project

*   Create an Upstash account and get your [QStash token](https://console.upstash.com/qstash)
*   Node.js installed

<Steps>
  <Step titleSize="h3" title="Create Next.js app and install QStash">
    ```bash
    npx create-next-app@latest qstash-bg-job
    ```

    ```bash
    cd qstash-bg-job
    ```

    ```bash
    npm install @upstash/qstash
    ```

    ```bash
    npm run dev
    ```
  </Step>

  <Step titleSize="h3" title="Create UI">
    After removing the default content in `src/app/page.tsx`, let's create a simple UI to trigger the background job
    using a button.

    ```tsx src/app/page.tsx
    "use client"

    export default function Home() {
      return (
        <main className="flex h-lvh items-center justify-center">
          <button
            onClick={handleClick}
            className="btn btn-primary w-1/2 h-56 bg-green-500 text-xl sm:text-3xl rounded-lg hover:bg-green-600"
          >
            Start Background Job
          </button>
        </main>
      )
    }
    ```

    <Accordion title="Beautiful">
      <Frame>
        <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/quickstart-qstash-ui.png" alt="Quickstart UI" />
      </Frame>
    </Accordion>
  </Step>

  <Step titleSize="h3" title="Start Background Job">
    We can use QStash to start a background job by calling the `publishJSON` method.
    In this example, we're using Next.js server actions, but you can also use route handlers.

    Since we don't have our public API endpoint yet, we can use [Request Catcher](https://requestcatcher.com/) to test the background job.
    This will eventually be replaced with our own API endpoint.

    ```ts src/app/actions.ts
    "use server"
    import { Client } from "@upstash/qstash"

    const qstashClient = new Client({
      // Add your token to a .env file
      token: process.env.QSTASH_TOKEN!,
    })

    export async function startBackgroundJob() {
      await qstashClient.publishJSON({
        url: "https://firstqstashmessage.requestcatcher.com/test",
        body: {
          hello: "world",
        },
      })
    }
    ```

    Now let's invoke the `startBackgroundJob` function when the button is clicked.

    ```tsx src/app/page.tsx
    "use client"
    import { startBackgroundJob } from "@/app/actions"

    export default function Home() {
      async function handleClick() {
        await startBackgroundJob()
      }

      return (
        <main className="flex h-lvh items-center justify-center">
          <button
            onClick={handleClick}
            className="btn btn-primary w-1/2 h-56 bg-green-500 text-xl sm:text-3xl rounded-lg hover:bg-green-600"
          >
            Start Background Job
          </button>
        </main>
      )
    }
    ```

    To test the background job, click the button and check the Request Catcher for the incoming request.

    <Accordion title="Verification screenshots">
      <Frame>
        <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/reqcatcher.png" />
      </Frame>

      You can also head over to [Upstash Console](https://console.upstash.com/qstash) and go to the
      `Logs` tab where you can see your message activities.

      <Frame>
        <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/log.png" />
      </Frame>
    </Accordion>
  </Step>

  <Step titleSize="h3" title="Create your own endpoint">
    Now that we know QStash is working, let's create our own endpoint to handle a background job. This
    is the endpoint that will be invoked by QStash.

    This job will be responsible for sending 10 requests, each with a 500ms delay. Since we're deploying
    to Vercel, we have to be cautious of the [time limit for serverless functions](https://vercel.com/docs/functions/runtimes#max-duration).

    ```ts src/app/api/long-task/route.ts
    export async function POST(request: Request) {
      const data = await request.json()

      for (let i = 0; i < 10; i++) {
        await fetch("https://firstqstashmessage.requestcatcher.com/test", {
          method: "POST",
          body: JSON.stringify(data),
          headers: { "Content-Type": "application/json" },
        })
        await new Promise((resolve) => setTimeout(resolve, 500))
      }

      return Response.json({ success: true })
    }
    ```

    Now let's update our `startBackgroundJob` function to use our new endpoint.

    There's 1 problem: our endpoint is not public. We need to make it public so that QStash can call it.
    We have 2 options:

    1.  Deploy our application to a platform like Vercel and use the public URL.
    2.  Create a [local tunnel](/qstash/howto/local-tunnel) to test the endpoint locally.

    For the purpose, of this tutorial, I'll deploy the application to Vercel, but
    feel free to use a local tunnel if you prefer.

    <Accordion title="Deploying to Vercel">
      There are many ways to [deploy to Vercel](https://vercel.com/docs/deployments/overview), but
      I'm going to use the Vercel CLI.

      ```bash
      npm install -g vercel
      ```

      ```bash
      vercel
      ```

      Once deployed, you can find the public URL in the Vercel dashboard.
    </Accordion>

    Now that we have a public URL, we can update the URL.

    ```ts src/app/actions.ts
    "use server"
    import { Client } from "@upstash/qstash"

    const qstashClient = new Client({
      token: process.env.QSTASH_TOKEN!,
    })

    export async function startBackgroundJob() {
      await qstashClient.publishJSON({
        // Replace with your public URL
        url: "https://qstash-bg-job.vercel.app/api/long-task",
        body: {
          hello: "world",
        },
      })
    }
    ```

    And voila! You've created a Next.js app that calls a long-running background job using QStash.
  </Step>

  <Step title="Error catching and security">
    QStash is a great way to handle background jobs, but it's important to remember that it's a public
    API. This means that anyone can call your endpoint. Make sure to add security measures to your endpoint
    to ensure that QStash is the sender of the request.

    Luckily, our SDK provides a way to verify the sender of the request. Make sure to get your signing keys
    from the QStash console and add them to your environment variables. The `verifySignatureAppRouter` will try to
    load `QSTASH_CURRENT_SIGNING_KEY` and `QSTASH_NEXT_SIGNING_KEY` from the environment. If one of them is missing,
    an error is thrown.

    ```ts src/app/api/long-task/route.ts
    import { verifySignatureAppRouter } from "@upstash/qstash/nextjs"

    async function handler(request: Request) {
      const data = await request.json()

      for (let i = 0; i < 10; i++) {
        await fetch("https://firstqstashmessage.requestcatcher.com/test", {
          method: "POST",
          body: JSON.stringify(data),
          headers: { "Content-Type": "application/json" },
        })
        await new Promise((resolve) => setTimeout(resolve, 500))
      }

      return Response.json({ success: true })
    }

    export const POST = verifySignatureAppRouter(handler)
    ```

    Let's also add error catching to our action and a loading state to our UI.

    <CodeGroup>
      ```ts src/app/actions.ts
      "use server"
      import { Client } from "@upstash/qstash";

      const qstashClient = new Client({
        token: process.env.QSTASH_TOKEN!,
      });

      export async function startBackgroundJob() {
        try {
          const response = await qstashClient.publishJSON({
            "url": "https://qstash-bg-job.vercel.app/api/long-task",
            body: {
              "hello": "world"
            }
          });
          return response.messageId;
        } catch (error) {
          console.error(error);
          return null;
        }
      }
      ```

      ```tsx src/app/page.tsx
      "use client"
      import { startBackgroundJob } from "@/app/actions";
      import { useState } from "react";

      export default function Home() {
        const [loading, setLoading] = useState(false);
        const [msg, setMsg] = useState("");

        async function handleClick() {
          setLoading(true);
          const messageId = await startBackgroundJob();
          if (messageId) {
            setMsg(`Started job with ID ${messageId}`);
          } else {
            setMsg("Failed to start background job");
          }
          setLoading(false);
        }

        return (
          <main className="flex flex-col h-lvh items-center justify-center">
            <button onClick={handleClick} disabled={loading} className="btn btn-primary w-1/2 h-56 bg-green-500 text-xl sm:text-3xl rounded-lg hover:bg-green-600 disabled:bg-gray-500">
              Start Background Job
            </button>

            {loading && <div className="text-2xl mt-8">Loading...</div>}
            {msg && <p className="text-center text-lg">{msg}</p>}
          </main>
        );
      }
      ```
    </CodeGroup>
  </Step>
</Steps>

## Result

We have now created a Next.js app that calls a long-running background job using QStash!
Here's the app in action:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/nextjs-quickstart-result-gif.gif" alt="Quickstart Result Gif" />
</Frame>

We can also view the logs on Vercel and QStash

<Accordion title="Logs">
  Vercel

  <Frame>
    <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/vercel-log-quickstart.png" alt="Vercel Logs" />
  </Frame>

  QStash

  <Frame>
    <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/quickstart-qstash-result.png" alt="Vercel Logs" />
  </Frame>
</Accordion>

And the code for the 3 files we created:

<CodeGroup>
  ```tsx src/app/page.tsx
  "use client"
  import { startBackgroundJob } from "@/app/actions";
  import { useState } from "react";

  export default function Home() {
  const [loading, setLoading] = useState(false);
  const [msg, setMsg] = useState("");

    async function handleClick() {
      setLoading(true);
      const messageId = await startBackgroundJob();
      if (messageId) {
        setMsg(`Started job with ID ${messageId}`);
      } else {
        setMsg("Failed to start background job");
      }
      setLoading(false);
    }

    return (
      <main className="flex flex-col h-lvh items-center justify-center">
        <button onClick={handleClick} disabled={loading} className="btn btn-primary w-1/2 h-56 bg-green-500 text-xl sm:text-3xl rounded-lg hover:bg-green-600 disabled:bg-gray-500">
          Start Background Job
        </button>

        {loading && <div className="text-2xl mt-8">Loading...</div>}
        {msg && <p className="text-center text-lg">{msg}</p>}
      </main>
    );

  }

  ```

  ```ts src/app/actions.ts
  "use server"
  import { Client } from "@upstash/qstash";

  const qstashClient = new Client({
    token: process.env.QSTASH_TOKEN!,
  });

  export async function startBackgroundJob() {
    try {
      const response = await qstashClient.publishJSON({
        "url": "https://qstash-bg-job.vercel.app/api/long-task",
        body: {
          "hello": "world"
        }
      });
      return response.messageId;
    } catch (error) {
      console.error(error);
      return null;
    }
  }
  ```

  ```ts src/app/api/long-task/route.ts
  import { verifySignatureAppRouter } from "@upstash/qstash/nextjs"

  async function handler(request: Request) {
    const data = await request.json()

    for (let i = 0; i < 10; i++) {
      await fetch("https://firstqstashmessage.requestcatcher.com/test", {
        method: "POST",
        body: JSON.stringify(data),
        headers: { "Content-Type": "application/json" },
      })
      await new Promise((resolve) => setTimeout(resolve, 500))
    }

    return Response.json({ success: true })
  }

  export const POST = verifySignatureAppRouter(handler)
  ```
</CodeGroup>

Now, go ahead and try it out for yourself! Try using some of the other features of QStash, like
[schedules](/qstash/features/schedules), [callbacks](/qstash/features/callbacks), and [URL Groups](/qstash/features/url-groups).


# Periodic Data Updates



<Note>
  *   Code:
      [Repository](https://github.com/upstash/qstash-examples/tree/main/periodic-data-updates)
  *   App:
      [qstash-examples-periodic-data-updates.vercel.app](https://qstash-examples-periodic-data-updates.vercel.app)
</Note>

This recipe shows how to use QStash as a trigger for a Next.js api route, that
fetches data from somewhere and stores it in your database.

For the database we will use Redis because it's very simple to setup and is not
really the main focus of this recipe.

## What will be build?

Let's assume there is a 3rd party API that provides some data. One approach
would be to just query the API whenever you or your users need it, however that
might not work well if the API is slow, unavailable or rate limited.

A better approach would be to continuously fetch fresh data from the API and
store it in your database.

Traditionally this would require a long running process, that would continuously
call the API. With QStash you can do this inside your Next.js app and you don't
need to worry about maintaining anything.

For the purpose of this recipe we will build a simple app, that scrapes the
current Bitcoin price from a public API, stores it in redis and then displays a
chart in the browser.

## Setup

If you don't have one already, create a new Next.js project with
`npx create-next-app@latest --ts`.

Then install the required packages

```bash
npm install @upstash/qstash @upstash/redis
```

You can replace `@upstash/redis` with any kind of database client you want.

## Scraping the API

Create a new serverless function in `/pages/api/cron.ts`

````ts
import { NextApiRequest, NextApiResponse } from "next";
import { Redis } from "@upstash/redis";

import { verifySignature } from "@upstash/qstash/nextjs";

/**
 * You can use any database you want, in this case we use Redis
 */
const redis = Redis.fromEnv();

/**
 * Load the current bitcoin price in USD and store it in our database at the
 * current timestamp
 */
async function handler(_req: NextApiRequest, res: NextApiResponse) {
  try {
    /**
     * The API returns something like this:
     * ```json
     * {
     *   "USD": {
     *     "last": 123
     *   },
     *   ...
     * }
     * ```
     */
    const raw = await fetch("https://blockchain.info/ticker");
    const prices = await raw.json();
    const bitcoinPrice = prices["USD"]["last"] as number;

    /**
     * After we have loaded the current bitcoin price, we can store it in the
     * database together with the current time
     */
    await redis.zadd("bitcoin-prices", {
      score: Date.now(),
      member: bitcoinPrice,
    });

    res.send("OK");
  } catch (err) {
    res.status(500).send(err);
  } finally {
    res.end();
  }
}

/**
 * Wrap your handler with `verifySignature` to automatically reject all
 * requests that are not coming from Upstash.
 */
export default verifySignature(handler);

/**
 * To verify the authenticity of the incoming request in the `verifySignature`
 * function, we need access to the raw request body.
 */
export const config = {
  api: {
    bodyParser: false,
  },
};
````

## Deploy to Vercel

That's all we need to fetch fresh data. Let's deploy our app to Vercel.

You can either push your code to a git repository and deploy it to Vercel, or
you can deploy it directly from your local machine using the
[vercel cli](https://vercel.com/docs/cli).

For a more indepth tutorial on how to deploy to Vercel, check out this
[quickstart](/qstash/quickstarts/vercel-nextjs#4-deploy-to-vercel).

After you have deployed your app, it is time to add your secrets to your
environment variables.

## Secrets

Head over to [QStash](https://console.upstash.com/qstash) and copy the
`QSTASH_CURRENT_SIGNING_KEY` and `QSTASH_NEXT_SIGNING_KEY` to vercel's
environment variables. ![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/vercel_env.png)

If you are not using a custom database, you can quickly create a new
[Redis database](https://console.upstash.com/redis). Afterwards copy the
`UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to vercel.

In the near future we will update our
[Vercel integration](https://vercel.com/integrations/upstash) to do this for
you.

## Redeploy

To use the environment variables, you need to redeploy your app. Either with
`npx vercel --prod` or in the UI.

## Create cron trigger in QStash

The last part is to add the trigger in QStash. Go to
[QStash](https://console.upstash.com/qstash) and create a new schedule.

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/schedule.png)

Now we will call your api function whenever you schedule is triggered.

## Adding frontend UI

This part is probably the least interesting and would require more dependencies
for styling etc. Check out the
[index.tsx](https://github.com/upstash/qstash-examples/blob/main/periodic-data-updates/pages/index.tsx)
file, where we load the data from the database and display it in a chart.

## Hosted example

You can find a running example of this recipe
[here](https://qstash-examples-periodic-data-updates.vercel.app/).


# DLQ



<Info>
  You can run the async code by importing `AsyncQStash` from `qstash`
  and awaiting the methods.
</Info>

#### Get all messages with pagination using cursor

Since the DLQ can have a large number of messages, they are paginated.
You can go through the results using the `cursor`.

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")

all_messages = []
cursor = None
while True:
    res = client.dlq.list(cursor=cursor)
    all_messages.extend(res.messages)
    cursor = res.cursor
    if cursor is None:
        break
```

#### Get a message from the DLQ

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
msg = client.dlq.get("<dlq-id>")
```

#### Delete a message from the DLQ

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.dlq.delete("<dlq-id>")
```


# Events



<Info>
  You can run the async code by importing `AsyncQStash` from `qstash`
  and awaiting the methods.
</Info>

#### Get all events with pagination using cursor

Since there can be a large number of events, they are paginated.
You can go through the results using the `cursor`.

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")

all_events = []
cursor = None
while True:
    res = client.event.list(cursor=cursor)
    all_events.extend(res.events)
    cursor = res.cursor
    if cursor is None:
        break
```


# Keys



<Info>
  You can run the async code by importing `AsyncQStash` from `qstash`
  and awaiting the methods.
</Info>

#### Retrieve your signing Keys

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
signing_key = client.signing_key.get()

print(signing_key.current, signing_key.next)
```

#### Rotate your signing Keys

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
new_signing_key = client.signing_key.rotate()

print(new_signing_key.current, new_signing_key.next)
```


# Messages



<Info>
  You can run the async code by importing `AsyncQStash` from `qstash`
  and awaiting the methods.
</Info>

Messages are removed from the database shortly after they're delivered, so you
will not be able to retrieve a message after. This endpoint is intended to be used
for accessing messages that are in the process of being delivered/retried.

#### Retrieve a message

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
msg = client.message.get("<msg-id>")
```

#### Cancel/delete a message

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.message.cancel("<msg-id>")
```

#### Cancel messages in bulk

Cancel many messages at once or cancel all messages

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")

# cancel more than one message
client.message.cancel_many(["<msg-id-0>", "<msg-id-1>"])

# cancel all messages
client.message.cancel_all()
```


# Overview



These are example usages of each method in the QStash SDK. You can also reference the
[examples repo](https://github.com/upstash/qstash-py/tree/main/examples) and [API examples](/qstash/overall/apiexamples) for more.


# Publish



<Info>
  You can run the async code by importing `AsyncQStash` from `qstash`
  and awaiting the methods.
</Info>

#### Publish to a URL with a 3 second delay and headers/body

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
res = client.message.publish_json(
    url="https://my-api...",
    body={
        "hello": "world",
    },
    headers={
        "test-header": "test-value",
    },
    delay="3s",
)

print(res.message_id)
```

#### Publish to a URL group with a 3 second delay and headers/body

You can make a URL group on the QStash console or using the [URL group API](/qstash/sdks/py/examples/url-groups)

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
res = client.message.publish_json(
    url_group="my-url-group",
    body={
        "hello": "world",
    },
    headers={
        "test-header": "test-value",
    },
    delay="3s",
)

# When publishing to a URL group, the response is an array of messages for each URL in the group
print(res[0].message_id)
```

#### Publish a method with a callback URL

[Callbacks](/qstash/features/callbacks) are useful for long running functions. Here, QStash will return the response
of the publish request to the callback URL.

We also change the `method` to `GET` in this use case so QStash will make a `GET` request to the `url`. The default
is `POST`.

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.message.publish_json(
    url="https://my-api...",
    body={
        "hello": "world",
    },
    callback="https://my-callback...",
    failure_callback="https://my-failure-callback...",
    method="GET",
)
```

#### Configure the number of retries

The max number of retries is based on your [QStash plan](https://upstash.com/pricing/qstash)

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.message.publish_json(
    url="https://my-api...",
    body={
        "hello": "world",
    },
    retries=1,
)
```

#### Publish HTML content instead of JSON

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.message.publish(
    url="https://my-api...",
    body="<html><body><h1>Hello World</h1></body></html>",
    content_type="text/html",
)
```

#### Publish a message with [content-based-deduplication](/qstash/features/deduplication)

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.message.publish_json(
    url="https://my-api...",
    body={
        "hello": "world",
    },
    content_based_deduplication=True,
)
```

#### Publish a message with timeout

Timeout value to use when calling a url ([See `Upstash-Timeout` in Publish Message page](/qstash/api/publish#request))

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.message.publish_json(
    url="https://my-api...",
    body={
        "hello": "world",
    },
    timeout="30s",
)
```


# Queues



#### Create a queue with parallelism

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")

queue_name = "upstash-queue"
client.queue.upsert(queue_name, parallelism=2)

print(client.queue.get(queue_name))
```

#### Delete a queue

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")

queue_name = "upstash-queue"
client.queue.delete(queue_name)
```

#### Pause/Resume a queue

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")

queue_name = "upstash-queue"
client.queue.upsert(queue_name, parallelism=1)

client.queue.pause(queue_name)

queue = client.queue.get(queue_name)
print(queue.paused) # prints True

client.queue.resume(queue_name)
```


# Receiver



When receiving a message from QStash, you should [verify the signature](/qstash/howto/signature).
The QStash Python SDK provides a helper function for this.

```python
from qstash import Receiver

receiver = Receiver(
    current_signing_key="YOUR_CURRENT_SIGNING_KEY",
    next_signing_key="YOUR_NEXT_SIGNING_KEY",
)

# ... in your request handler

signature, body = req.headers["Upstash-Signature"], req.body

receiver.verify(
    body=body,
    signature=signature,
    url="YOUR-SITE-URL",
)
```


# Schedules



<Info>
  You can run the async code by importing `AsyncQStash` from `qstash`
  and awaiting the methods.
</Info>

#### Create a schedule that runs every 5 minutes

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
schedule_id = client.schedule.create(
    destination="https://my-api...",
    cron="*/5 * * * *",
)

print(schedule_id)
```

#### Create a schedule that runs every hour and sends the result to a [callback URL](/qstash/features/callbacks)

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.schedule.create(
    destination="https://my-api...",
    cron="0 * * * *",
    callback="https://my-callback...",
    failure_callback="https://my-failure-callback...",
)
```

#### Create a schedule to a URL group that runs every minute

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.schedule.create(
    destination="my-url-group",
    cron="0 * * * *",
)
```

#### Get a schedule by schedule id

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
schedule = client.schedule.get("<schedule-id>")

print(schedule.cron)
```

#### List all schedules

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
all_schedules = client.schedule.list()

print(all_schedules)
```

#### Delete a schedule

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.schedule.delete("<schedule-id>")
```

#### Create a schedule with timeout

Timeout value to use when calling a schedule URL ([See `Upstash-Timeout` in Create Schedule page](/qstash/api/schedules/create)).

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
schedule_id = client.schedule.create(
    destination="https://my-api...",
    cron="*/5 * * * *",
    timeout="30s",
)

print(schedule_id)
```

#### Pause/Resume a schedule

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
schedule_id = "scd_1234"

client.schedule.pause(schedule_id)

schedule = client.schedule.get(schedule_id)
print(schedule.paused) # prints True

client.schedule.resume(schedule_id)
```


# URL Groups



<Info>
  You can run the async code by importing `AsyncQStash` from `qstash`
  and awaiting the methods.
</Info>

#### Create a URL group and add 2 endpoints

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.url_group.upsert_endpoints(
    url_group="my-url-group",
    endpoints=[
        {"url": "https://my-endpoint-1"},
        {"url": "https://my-endpoint-2"},
    ],
)
```

#### Get URL group by name

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
url_group = client.url_group.get("my-url-group")

print(url_group.name, url_group.endpoints)
```

#### List URL groups

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
all_url_groups = client.url_group.list()

for url_group in all_url_groups:
    print(url_group.name, url_group.endpoints)
```

#### Remove an endpoint from a URL group

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.url_group.remove_endpoints(
    url_group="my-url-group",
    endpoints=[
        {"url": "https://my-endpoint-1"},
    ],
)
```

#### Delete a URL group

```python
from qstash import QStash

client = QStash("<QSTASH-TOKEN>")
client.url_group.delete("my-url-group")
```


# Getting Started



## Install

### PyPI

```bash
pip install qstash
```

## Get QStash token

Follow the instructions [here](/qstash/overall/getstarted) to get your QStash token and signing keys.

## Usage

#### Synchronous Client

```python
from qstash import QStash

client = QStash("<QSTASH_TOKEN>")
client.message.publish_json(...)
```

#### Asynchronous Client

```python
import asyncio

from qstash import AsyncQStash


async def main():
    client = AsyncQStash("<QSTASH_TOKEN>")
    await client.message.publish_json(...)


asyncio.run(main())
```

#### RetryConfig

You can configure the retry policy of the client by passing the configuration to the client constructor.

Note: This isn for sending the request to QStash, not for the retry policy of QStash.

The default number of retries is **5** and the default backoff function is `lambda retry_count: math.exp(retry_count) * 50`.

You can also pass in `False` to disable retrying.

```python
from qstash import QStash

client = QStash(
    "<QSTASH_TOKEN>",
    retry={
        "retries": 3,
        "backoff": lambda retry_count: (2**retry_count) * 20,
    },
)
```


# Overview



`qstash` is an Python SDK for QStash, allowing for easy access to the QStash API.

Using `qstash` you can:

*   Publish a message to a URL/URL group/API
*   Publish a message with a delay
*   Schedule a message to be published
*   Access all events
*   Create, read, update, or delete URL groups.
*   Read or remove messages from the [DLQ](/qstash/features/dlq)
*   Read or cancel messages
*   Verify the signature of a message

You can find the Github Repository [here](https://github.com/upstash/qstash-py).


# DLQ



#### Get all messages with pagination using cursor

Since the DLQ can have a large number of messages, they are paginated.
You can go through the results using the `cursor`.

```typescript
import { Client } from "@upstash/qstash";

const client = new Client("<QSTASH_TOKEN>");
const dlq = client.dlq;
const all_messages = [];
let cursor = null;
while (true) {
  const res = await dlq.listMessages({ cursor });
  all_messages.push(...res.messages);
  cursor = res.cursor;
  if (!cursor) {
    break;
  }
}
```

#### Delete a message from the DLQ

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const dlq = client.dlq;
await dlq.delete("dlqId");
```


# Events



#### Get all events with pagination using cursor

Since there can be a large number of events, they are paginated.
You can go through the results using the `cursor`.

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const allEvents = [];
let cursor = null;
while (true) {
  const res = await client.events({ cursor });
  allEvents.push(...res.events);
  cursor = res.cursor;
  if (!cursor) {
    break;
  }
}
```

#### Filter events by state and only return the first 50.

<Info>
  More filters can be found in the [API Reference](/qstash/api/events/list).
</Info>

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const res = await client.events({ 
  filter: {
    state: "DELIVERED",
    count: 50
  }
});
```


# Messages



Messages are removed from the database shortly after they're delivered, so you
will not be able to retrieve a message after. This endpoint is intended to be used
for accessing messages that are in the process of being delivered/retried.

#### Retrieve a message

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const messages = client.messages
const msg = await messages.get("msgId");
```

#### Cancel/delete a message

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const messages = client.messages
const msg = await messages.delete("msgId");
```

#### Cancel messages in bulk

Cancel many messages at once or cancel all messages

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });

// deleting two messages at once
await client.messages.deleteMany([
  "message-id-1",
  "message-id-2",
])


// deleting all messages
await client.messages.deleteAll()
```


# Overview



These are example usages of each method in the QStash SDK. You can also reference the
[examples repo](https://github.com/upstash/sdk-qstash-ts/tree/main/examples) and [API examples](/qstash/overall/apiexamples) for more.


# Publish



#### Publish to a URL with a 3 second delay and headers/body

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const res = await client.publishJSON({
  url: "https://my-api...",
  body: { hello: "world" },
  headers: { "test-header": "test-value" },
  delay: "3s",
});
```

#### Publish to a URL group with a 3 second delay and headers/body

You create URL group on the QStash console or using the [URL Group API](/qstash/sdks/ts/examples/url-groups#create-a-url-group-and-add-2-endpoints)

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const res = await client.publishJSON({
  urlGroup: "my-url-group",
  body: { hello: "world" },
  headers: { "test-header": "test-value" },
  delay: "3s",
});

// When publishing to a URL Group, the response is an array of messages for each URL in the URL Group
console.log(res[0].messageId);
```

#### Publish a method with a callback URL

[Callbacks](/qstash/features/callbacks) are useful for long running functions. Here, QStash will return the response
of the publish request to the callback URL.

We also change the `method` to `GET` in this use case so QStash will make a `GET` request to the `url`. The default
is `POST`.

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const res = await client.publishJSON({
  url: "https://my-api...",
  body: { hello: "world" },
  callback: "https://my-callback...",
  failureCallback: "https://my-failure-callback...",
  method: "GET",
});
```

#### Configure the number of retries

The max number of retries is based on your [QStash plan](https://upstash.com/pricing/qstash)

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const res = await client.publishJSON({
  url: "https://my-api...",
  body: { hello: "world" },
  retries: 1,
});
```

#### Publish HTML content instead of JSON

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const res = await client.publish({
  url: "https://my-api...",
  body: "<html><body><h1>Hello World</h1></body></html>",
  headers: {
    "Content-Type": "text/html",
  },
});
```

#### Publish a message with [content-based-deduplication](/qstash/features/deduplication)

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const res = await client.publishJSON({
  url: "https://my-api...",
  body: { hello: "world" },
  contentBasedDeduplication: true,
});
```

#### Publish a message with timeout

Timeout value in seconds to use when calling a url ([See `Upstash-Timeout` in Publish Message page](/qstash/api/publish#request))

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const res = await client.publishJSON({
  url: "https://my-api...",
  body: { hello: "world" },
  timeout: "30s" // 30 seconds timeout
});
```


# Queues



#### Create a queue with parallelism 2

```typescript
import { Client } from "@upstash/qstash";
const client = new Client({ token: "<QSTASH_TOKEN>" });

const queueName = "upstash-queue";
await client.queue({ queueName }).upsert({ parallelism: 2 });

const queueDetails = await client.queue({ queueName }).get();
```

#### Delete Queue

```typescript
import { Client } from "@upstash/qstash";
const client = new Client({ token: "<QSTASH_TOKEN>" });

const queueName = "upstash-queue";
await client.queue({ queueName: queueName }).delete();
```

#### Pause/Resume a queue

```typescript
import { Client } from "@upstash/qstash";
const client = new Client({ token: "<QSTASH_TOKEN>" });

const name = "upstash-pause-resume-queue";
const queue = client.queue({ queueName: name });
await queue.upsert({ parallelism: 1 });

// pause queue
await queue.pause();

const queueInfo = await queue.get();
console.log(queueInfo.paused); // prints true

// resume queue
await queue.resume();
```


# Receiver



When receiving a message from QStash, you should [verify the signature](/qstash/howto/signature).
The QStash Typescript SDK provides a helper function for this.

```typescript
import { Receiver } from "@upstash/qstash";

const receiver = new Receiver({
  currentSigningKey: "YOUR_CURRENT_SIGNING_KEY",
  nextSigningKey: "YOUR_NEXT_SIGNING_KEY",
});

// ... in your request handler

const signature = req.headers["Upstash-Signature"];
const body = req.body;

const isValid = receiver.verify({
  body,
  signature,
  url: "YOUR-SITE-URL",
});
```


# Schedules



#### Create a schedule that runs every 5 minutes

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
await client.schedules.create({
  destination: "https://my-api...",
  cron: "*/5 * * * *",
});
```

#### Create a schedule that runs every hour and sends the result to a [callback URL](/qstash/features/callbacks)

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
await client.schedules.create({
  destination: "https://my-api...",
  cron: "0 * * * *",
  callback: "https://my-callback...",
  failureCallback: "https://my-failure-callback...",
});
```

#### Create a schedule to a URL Group that runs every minute

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
await client.schedules.create({
  destination: "my-url-group",
  cron: "* * * * *",
});
```

#### Get a schedule by schedule id

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });

const res = await client.schedules.get("scheduleId");
console.log(res.cron);
```

#### List all schedules

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const allSchedules = await client.schedules.list();
console.log(allSchedules);
```

#### Create/overwrite a schedule with a user chosen schedule id

Note that if a schedule exists with the same id, the old one will be discarded
and new schedule will be used.

```typescript Typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
await client.schedules.create({
  destination: "https://example.com",
  scheduleId: "USER_PROVIDED_SCHEDULE_ID",
  cron: "* * * * *",
});
```

#### Delete a schedule

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
await client.schedules.delete("scheduleId");
```

#### Create a schedule with timeout

Timeout value in seconds to use when calling a schedule URL ([See `Upstash-Timeout` in Create Schedule page](/qstash/api/schedules/create)).

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
await client.schedules.create({
  url: "https://my-api...",
  cron: "* * * * *",
  timeout: "30" // 30 seconds timeout
});
```

#### Pause/Resume a schedule

```typescript
import { Client } from "@upstash/qstash";
const client = new Client({ token: "<QSTASH_TOKEN>" });
const scheduleId = "my-schedule"

// pause schedule
await client.schedules.pause({ schedule: scheduleId });

// check if paused
const result = await client.schedules.get(scheduleId);
console.log(getResult.isPaused) // prints true

// resume schedule
await client.schedules.resume({ schedule: scheduleId });
```


# URL Groups



#### Create a URL Group and add 2 endpoints

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const urlGroups = client.urlGroups;
await urlGroups.addEndpoints({
  name: "url_group_name",
  endpoints: [
    { url: "https://my-endpoint-1" },
    { url: "https://my-endpoint-2" },
  ],
});
```

#### Get URL Group by name

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const urlGroups = client.urlGroups;
const urlGroup = await urlGroups.get("urlGroupName");
console.log(urlGroup.name, urlGroup.endpoints);
```

#### List URL Groups

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const allUrlGroups = await client.urlGroups.list();
for (const urlGroup of allUrlGroups) {
  console.log(urlGroup.name, urlGroup.endpoints);
}
```

#### Remove an endpoint from a URL Group

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const urlGroups = client.urlGroups;
await urlGroups.removeEndpoints({
  name: "urlGroupName",
  endpoints: [{ url: "https://my-endpoint-1" }],
});
```

#### Delete a URL Group

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const urlGroups = client.urlGroups;
await urlGroups.delete("urlGroupName");
```


# Getting Started



## Install

### NPM

```bash
npm install @upstash/qstash
```

## Get QStash token

Follow the instructions [here](/qstash/overall/getstarted) to get your QStash token and signing keys.

## Usage

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({
  token: "<QSTASH_TOKEN>",
});
```

#### RetryConfig

You can configure the retry policy of the client by passing the configuration to the client constructor.

Note: This is for sending the request to QStash, not for the retry policy of QStash.

The default number of attempts is **6** and the default backoff function is `(retry_count) => (Math.exp(retry_count) * 50)`.

You can also pass in `false` to disable retrying.

```typescript
import { Client } from "@upstash/qstash";

const client = new Client({
  token: "<QSTASH_TOKEN>",
  retry: {
    retries: 3,
    backoff: retry_count => 2 ** retry_count * 20,
  },
});
```


# Overview



`@upstash/qstash` is a Typescript SDK for QStash, allowing for easy access to the QStash API.

Using `@upstash/qstash` you can:

*   Publish a message to a URL/URL Group
*   Publish a message with a delay
*   Schedule a message to be published
*   Access all events
*   Create, read, update, or delete URL groups.
*   Read or remove messages from the [DLQ](/qstash/features/dlq)
*   Read or cancel messages
*   Verify the signature of a message

You can find the Github Repository [here](https://github.com/upstash/sdk-qstash-ts).


# Examples Index

List of all Upstash Examples

TODO: fahreddin
import TagFilters from "../../src/components/Filter.js"

<TagFilters>
  <TagFilters.Item externalLink type="Article" tags={["Svelte"]} url="https://blog.upstash.com/sveltekit-todo-redis">
    SvelteKit TODO App with Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Strapi"]} url="https://blog.upstash.com/redis-strapi">
    Serverless Redis Caching for Strapi
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Blitz.js"]} url="https://blog.upstash.com/blitzjs-todo-redis">
    To-Do List with Blitz.js & Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Chalice", "Slackbot"]} url="https://blog.upstash.com/chalice-event-reminder-slackbot">
    Slackbot with AWS Chalice and Upstash Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Render"]} url="https://blog.upstash.com/render-serverless-redis">
    Using Render with Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Slackbot", "Vercel"]} url="https://blog.upstash.com/vercel-note-taker-slackbot">
    Slackbot with Vercel and Upstash Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Slackbot", "Vercel"]} url="https://blog.upstash.com/vercel-note-taker-slackbot">
    Slackbot with Vercel and Upstash Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Remix", "Cloudflare Workers"]} url="https://blog.upstash.com/fast_websites_with_Remix_on_Cloudflare_and_Upstash_Redis">
    Remix on Cloudflare with Upstash Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Remix"]} url="https://blog.upstash.com/remix-todo-redis">
    Remix TODO App with Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["GraphQL", "Netlify"]} url="https://blog.upstash.com/netlify-graph-upstash">
    Global Cache for Netlify Graph with Upstash Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Next.js", "NextAuth"]} url="https://blog.upstash.com/next-auth-serverless-redis">
    Next.js Authentication with NextAuth and Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Next.js"]} url="https://blog.upstash.com/survey-serverless-redis">
    Building a Survey App with Upstash Redis and Next.js
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["React Native", "Mobile"]} url="https://blog.upstash.com/serverless-react-native">
    Building React Native Apps Backed by AWS Lambda and Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Remix", "Node.js"]} url="https://blog.upstash.com/redis-with-remix">
    Using Upstash Redis with Remix
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Remix", "Node.js"]} url="https://blog.upstash.com/redis-session-remix">
    Using Upstash Redis as a Session Store for Remix
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["REST-API", "Backendless"]} url="https://docs.upstash.com/redis/tutorials/notification">
    Building a Serverless Notification API for Your Web Application with Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS App Runner", "Node.js"]} url="https://docs.upstash.com/redis/tutorials/aws_app_runner_with_redis">
    Build Stateful Applications with AWS App Runner and Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Google Cloud"]} url="https://docs.upstash.com/redis/tutorials/cloud_run_sessions">
    Session Management on Google Cloud Run with Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Cloudflare Workers", "upstash-redis", "REST-API"]} url="https://docs.upstash.com/redis/tutorials/cloudflare_workers_with_redis">
    Use Redis in Cloudflare Workers
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Fastly Compute", "upstash-redis", "REST-API"]} url="https://blog.upstash.com/fastly-compute-edge-with-redis">
    Use Redis in Fastly Compute
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Cloudflare Workers"]} url="https://docs.upstash.com/redis/tutorials/edge_leaderboard">
    Build a Leaderboard API at Edge Using Cloudflare Workers and Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Lambda", "Node.js"]} url="https://docs.upstash.com/redis/tutorials/job_processing">
    Job Processing and Event Queue with Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Lambda", "Node.js"]} url="https://docs.upstash.com/redis/tutorials/rate-limiting">
    AWS Lambda Rate Limiting with Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Lambda", "Node.js"]} url="https://docs.upstash.com/redis/tutorials/histogram">
    Build a Serverless Histogram API with Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Lambda", "Node.js"]} url="https://docs.upstash.com/redis/tutorials/auto_complete_with_serverless_redis">
    Autocomplete API with Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Next.js", "Vercel"]} url="https://docs.upstash.com/redis/tutorials/roadmapvotingapp">
    Roadmap Voting App with Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Next.js"]} url="https://blog.upstash.com/survey-serverless-redis">
    Building a Survey App with Upstash Redis only
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Google Cloud"]} url="https://docs.upstash.com/redis/tutorials/using_google_cloud_functions">
    Serverless Redis on Google Cloud Functions
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Lambda"]} url="https://docs.upstash.com/redis/tutorials/using_serverless_framework">
    Using Serverless Framework
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Lambda"]} url="https://docs.upstash.com/redis/tutorials/using_aws_sam">
    Using AWS SAM
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Lambda"]} url="https://docs.upstash.com/redis/tutorials/api_with_cdk">
    Deploy a Serverless API with AWS CDK and AWS Lambda
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Node.js"]} url="https://docs.upstash.com/redis/tutorials/express_session">
    Express Session with Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Next.js"]} url="https://docs.upstash.com/redis/tutorials/nextjs_with_redis">
    Next.js with Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Nuxt.js"]} url="https://docs.upstash.com/redis/tutorials/nuxtjs_with_redis">
    Nuxt.js with Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Java", "AWS Lambda"]} url="https://docs.upstash.com/redis/tutorials/serverless_java_redis">
    Serverless API with Java and Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Go", "AWS Lambda"]} url="https://docs.upstash.com/redis/tutorials/goapi">
    Serverless Golang API with Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Python", "AWS Lambda"]} url="https://docs.upstash.com/redis/tutorials/pythonapi">
    Serverless Python API with Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Java"]} url="https://docs.upstash.com/redis/tutorials/redisson">
    Serverless Redisson
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Svelte", "Netlify"]} url="https://blog.upstash.com/svelte-with-serverless-redis">
    Building SvelteKit Applications with Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Cloudflare Workers", "upstash-redis", "REST-API"]} url="https://blog.upstash.com/cloudflare-workers-waiting-room">
    Build Your Own Waiting Room for Your Website with Cloudflare Workers and
    Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Lambda", "Mobile", "Flutter"]} url="https://blog.upstash.com/serverless-with-flutter">
    Fullstack Serverless App with Flutter, Serverless Framework and
    Upstash(REDIS) - PART 1
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Next.js Edge", "Next.js", "Vercel"]} url="https://blog.upstash.com/getstarted-nextjs-edge-with-redis">
    Getting Started with Next.js Edge Functions
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Next.js Edge", "Next.js", "Vercel"]} url="https://blog.upstash.com/nextjs-edge-waiting-room">
    Waiting Room for Your Next.js App Using Edge Functions
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Benchmark"]} url="https://blog.upstash.com/serverless-database-benchmark">
    Serverless Battleground - DynamoDB vs Firestore vs MongoDB vs Cassandra vs
    Redis vs FaunaDB
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["AWS Lambda", "REST-API"]} url="https://blog.upstash.com/aws-lambda-redis-rest">
    Stateful AWS Lambda with Redis REST
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["REST-API"]} url="https://blog.upstash.com/pipeline">
    Pipeline REST API on Serverless Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Next.js", "Vercel", "REST-API"]} url="https://blog.upstash.com/nextjs-todo">
    The Most Minimalist Next.js TODO App
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Cloudflare Workers", "REST-API"]} url="https://blog.upstash.com/edge-guard">
    Implement IP Allow/Deny List at Edge with Cloudflare Workers and Upstash
    Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Cloudflare Workers"]} url="https://blog.upstash.com/redis-cloudflare-workers">
    Redis @ Edge with Cloudflare Workers
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Video" tags={["Next.js", "Vercel"]} url="https://www.youtube.com/watch?v=FytxaSVQROc">
    Using Serverless Redis with Next.js
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Video" tags={["Next.js"]} url="https://www.youtube.com/watch?v=sBiUqozxY4o">
    Building a Cache with Upstash Redis in Next.js
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Video" tags={["Next.js Edge", "Next.js", "Vercel"]} url="https://www.youtube.com/watch?v=hu2SrILiEgE">
    Vercel Edge Function URL Shortener with Upstash Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Video" tags={["Next.js"]} url="https://www.youtube.com/watch?v=S1oOlKQo8CY">
    Adding Feature Flags to Next.js (Upstash Redis, SWR, Hooks)
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Video" tags={["Next.js"]} url="https://www.youtube.com/watch?v=_opoQpUMqF4">
    Rate Limiting Your Serverless Functions with Upstash Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Video" tags={["Next.js"]} url="https://www.youtube.com/watch?v=1Dotv9T7nIs">
    Create a React Scoreboard with Upstash Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Video" tags={["Go", "AWS Lambda"]} url="https://www.youtube.com/watch?v=EJ6CJ0GC9lk">
    Upstash on AWS Lambda Using Golang
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Video" tags={["Cloudflare Workers"]} url="https://www.youtube.com/watch?v=g6hGJcuscoM">
    IP Address Allow/Deny with Cloudflare Workers and Upstash Redis
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Video" tags={["Vercel", "Next.js Edge"]} url="https://www.youtube.com/watch?v=yuxd2kurpzk&t=968s">
    Edge Functions Explained with Kelsey Hightower and Lee Robinson - (Next.js
    Conf 2021)
  </TagFilters.Item>

  <TagFilters.Item externalLink type="Article" tags={["Elixir"]} url="https://docs.upstash.com/redis/tutorials/elixir_with_redis">
    Elixir with Redis
  </TagFilters.Item>
</TagFilters>


# Backup/Restore



You can create a manual backup of your database and restore that backup to any of your databases. Both backup and restore operations require that your database is in one of the AWS regions.

Additionally, you can utilize the daily backup feature to automatically create backups of your database on a daily basis.

### Create A Manual Backup

To create a manual backup of your database:

*   Go to the database details page and navigate to the `Backups` tab

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/backuptab.png" width="800" />
</Frame>

*   Click on the `Backup` button and fill in a name for your backup. **Your backup name must be unique.**
*   Then click on the `Create` button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/backupmodal.png" width="600" />
</Frame>

During the process of creating a backup for your database, it is important to note that your database will be temporarily locked, which means these operations will be unavailable during this time:

*   Create Database Backup
*   Enable TLS
*   Move Database to Team
*   Restore Database Backup
*   Update Eviction
*   Update Password
*   Delete Database

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/backupinprogressscreen.png" width="800" />
</Frame>

### Restore A Backup

To restore a backup that was created from your current database, follow the steps below:

*   Go to the database details page and navigate to the `Backups` tab
*   Click on the `Restore` button next to the backup record listed.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/backuptabdailyenabled.png" width="800" />
</Frame>

*   Click on `Restore`. **Be aware of the fact that your target database will be flushed with this operation.**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/restorebackupfromcurrentdatabase.png" width="700" />
</Frame>

### Restore A Backup From Another Database

To restore a backup that was created from one of your databases other than the current one, follow the steps below:

*   Go to the database details page and navigate to the `Backups` tab
*   Click on the `Restore...` button

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/backuptabdailyenabled.png" width="800" />
</Frame>

*   Select the source database, referring to the database from which the backup was generated.
*   Select the backup record that you want to restore to the current database.
*   Click on `Start Restore`. **Be aware of the fact that your target database will be flushed with this operation.**

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/restorebackupmodal.png" width="700" />
</Frame>

### Enable Daily Automated Backup

To enable daily automated backup for your database:

*   Go to the database details page and navigate to the `Backups` tab

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/backuptabwide.png" width="800" />
</Frame>

*   Enable the switch next to the `Daily Backup`
*   Click on `Enable`

### Disable Daily Automated Backup

To disable the daily automated backup for your database, please follow the steps below:

*   Go to the database details page and navigate to the `Backups` tab

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/backuptabdailyenabled.png" width="800" />
</Frame>

*   Disable the switch next to the `Daily Backup`
*   Click on `Disable`

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/backuprestore/dailybackupdisablemodal.png" width="700" />
</Frame>


# Consistency



Upstash utilizes a leader-based replication mechanism. Under this mechanism,
each key is assigned to a leader replica, which is responsible for handling
write operations on that key. The remaining replicas serve as backups to the
leader. When a write operation is performed on a key, it is initially processed
by the leader replica and then asynchronously propagated to the backup replicas.
This ensures that data consistency is maintained across the replicas. Reads can
be performed from any replica.

Each replica employs a failure detector to track liveness of the leader replica.
When the leader replica fails for a reason, remaining replicas start a new
leader election round and elect a new leader. This is the only unavailability
window for the cluster where *write* your requests can be blocked for a short
period of time. Also in case of cluster wide failures like network partitioning
(split brain); periodically running anti entropy jobs resolve the conflicts
using `Last-Writer-Wins` algorithm and converge the replicas to the same state.

This model gives a better write consistency and read scalability but can provide
only **Eventual Consistency**. Additionally you can achieve **Causal
Consistency** (`Read-Your-Writes`, `Monotonic-Reads`, `Monotonic-Writes` and
`Writes-Follow-Reads` guarantees) for a single Redis connection. (A TCP
connection forms a session between client and server).

<Note>
  Previously, Upstash supported `Strong Consistency` mode for the single region
  databases. We decided to deprecate this feature because its effect on latency
  started to conflict with the performance expectations of Redis use cases. Also
  we are gradually moving to **CRDT** based Redis data structures, which will
  provide `Strong Eventual Consistency`.
</Note>


# Durable Storage

This article explains the persistence provided by Upstash databases.

In Upstash, persistence is always enabled, setting it apart from other Redis
offerings. Every write operation is consistently stored in both memory and the
block storage provided by cloud providers, such as AWS's EBS. This dual storage
approach ensures data durability. Read operations are optimized to first check
if the data exists in memory, facilitating faster access. If the data is not in
memory, it is retrieved from disk. This combination of memory and disk storage
in Upstash guarantees reliable data access and maintains data integrity, even
during system restarts or failures.

### Multi Tier Storage

Upstash keeps your data both in memory and disk. This design provides:

*   Data safety with persistent storage
*   Low latency with in memory access
*   Price flexibility by using memory only for active data

In Upstash, an entry in memory is evicted if it remains idle, meaning it has not
been accessed for an extended period. It's important to note that eviction does
not result in data loss since the entry is still stored in the block storage.
When a read operation occurs for an evicted entry, it is efficiently reloaded
from the block storage back into memory, ensuring fast access to the data. This
eviction mechanism in Upstash optimizes memory usage by prioritizing frequently
accessed data while maintaining the ability to retrieve less frequently accessed
data when needed.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/durability/storage.png" width="600" />
</Frame>

<Card title="Can I use Upstash as a database?" icon="lightbulb" iconType="duotone" color="#34D399">
  Definitely, yes. Some users are worried that Redis data will be lost when a
  server crashes. This is not the case for Upstash thanks to Durable Storage.
  Data is reloaded to memory from block storage in case of a server crash.
  Moreover, except for the free tier, all paid tier databases provide extra redundancy by replicating data to multiple instances.
</Card>


# Eviction



By default, Upstash will reject write operations once the maximum data size
limit has been reached. However, if you are utilizing Upstash as a cache, you
have the option to enable eviction. Enabling eviction allows older data to be
automatically removed from the cache (including Durable Storage) when the maximum size limit is reached.
This ensures that the cache remains within the allocated size and can make room
for new data to be stored. Enabling eviction is particularly useful when the
cache is intended to store frequently changing or temporary data, allowing the
cache to adapt to evolving data needs while maintaining optimal performance.

*   You can enable eviction by checking **Eviction** checkbox while creating a new
    database:

    <Frame>
      <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/eviction/create-database.png" />
    </Frame>

*   Or for an existing database by clicking **Enable** in Configuration/Eviction
    box in the database details page:
    <Frame>
      <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/eviction/configuration.png" />
    </Frame>

Upstash currently uses a single eviction algorithm, called
**optimistic-volatile**, which is a combination of *volatile-random* and
*allkeys-random* eviction policies available in
[the original Redis](https://redis.io/docs/manual/eviction/#eviction-policies).

Initially, Upstash employs random sampling to select keys for eviction, giving
priority to keys marked with a TTL (expire field). If there is a shortage of
volatile keys or they are insufficient to create space, additional non-volatile
keys are randomly chosen for eviction. In future releases, Upstash plans to
introduce more eviction policies, offering users a wider range of options to
customize the eviction behavior according to their specific needs.


# Global Database



In the global database, the replicas are distributed across multiple regions
around the world. The clients are routed to the nearest region. This helps with
minimizing latency for use cases where users can be anywhere in the world.

### Primary Region and Read Regions

The Upstash Global database is structured with a Primary Region and multiple
Read Regions. When a write command is issued, it is initially sent and processed
at the Primary Region. The write operation is then replicated to all the Read
Regions, ensuring data consistency across the database.

On the other hand, when a read command is executed, it is directed to the
nearest Read Region to optimize response time. By leveraging the Global
database's distributed architecture, read operations can be performed with
reduced latency, as data retrieval occurs from the closest available Read
Region.

The Global database's design thus aids in minimizing read operation latency by
efficiently distributing data across multiple regions and enabling requests to
be processed from the nearest Read Region.

User selects a single primary region and multiple read regions. For the best
performance, you should select the primary region in the same location where
your writes happen. Select the read regions where your clients that read the
Redis located. You may have your database with a single primary region but no
read regions which would be practically same with a single region (regional)
database. You can add or remove regions on a running Redis database.

Here the list of regions currently supported:

*   AWS US-East-1 North Virginia
*   AWS US-West-1 North California
*   AWS US-West-2 Oregon
*   AWS EU-West-1 Ireland
*   AWS EU-Central-1 Frankfurt
*   AWS AP-SouthEast-1 Singapore
*   AWS AP-SouthEast-2 Sydney
*   AWS AP-NorthEast-1 Japan
*   AWS SA-East-1 São Paulo

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/globaldb/regionselect.png" width="520" />
</Frame>

In our internal tests, we see the following latencies (99th percentile):

*   Read latency from the same region \<1ms
*   Write latency from the same region \<5ms
*   Read/write latency from the same continent \<50ms

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/globaldb/map2.png" width="1000" />
</Frame>

### Architecture

In the multi region architecture, each key is owned by a primary replica which
is located at the region that you choose as primary region. Read replicas become
the backups of the primary for the related keys. The primary replica processes
the writes, then propagates them to the read replicas. Read requests are
processed by all replicas, this means you can read a value from any of the
replicas. This model gives a better write consistency and read scalability.

Each replica employs a failure detector to track the liveness of the primary
replica. When the primary replica fails for a reason, read replicas start a new
leader election round and elect a new leader (primary). This is the only
unavailability window for the cluster where your requests can be blocked for a
short period of time.

<Note>
  Global Database is designed to optimize the latency of READ operations. It may
  not be a good choice if your use case is WRITE heavy.
</Note>

### Use Cases

*   **Edge functions:** Edge computing (Cloudflare workers, Fastly Compute) is
    becoming a popular way of building globally fast applications. But there are
    limited data solutions accessible from edge functions. Upstash Global Database
    is accessible from Edge functions with the REST API. Low latency from all edge
    locations makes it a perfect solution for Edge functions

*   Multi region serverless architectures: You can run your AWS Lambda function in
    multiple regions to lower global latency. Vercel/Netlify functions can be run
    in different regions. Upstash Global database provides low latency data
    wherever your serverless functions are.

*   Web/mobile use cases where you need low latency globally. Thanks to the read
    only REST API, you can access Redis from your web/mobile application directly.
    In such a case, Global Database will help to lower the latency as you can
    expect the clients from anywhere.

### High Availability and Disaster Recovery

Although the main motivation behind the Global Database is to provide low
latency; it also makes your database resilient to region wide failures. When a
region is not available, your requests are routed to another region; so your
database remains available.

### Consistency

Global Database is an eventually consistent database. The write request returns
after the primary replica processes the operation. Write operation is replicated
to read replicas asynchronously. Read requests can be served by any replica,
which gives better horizontal scalability but also means a read request may
return a stale value while a write operation for the same key is being
propagated to read replicas.

In case of cluster wide failures like network partitioning (split brain);
periodically running anti entropy jobs resolve the conflicts using LWW
algorithms and converge the replicas to the same state.

### Upgrade from Regional to Global

Currently, we do not support auto-upgrade from regional to global database. You
can export data from your old database and import into the global database.

### Pricing

Global Database charges \$0.2 per 100K commands. The write commands are replicated to all read regions in addition to primary region so the replications are counted as commands. For example, if you have 1 primary 1 read region, 100K writes will cost \$0.4 (\$0.2 x 2). You can use Global Database in the free tier too. Free usage is limited with max one read region.


# Replication



Replication is enabled for all paid Upstash databases. The data is replicated to
multiple instances. Replication provides you high availability and better
scalability.

### High Availability

Replication makes your database resilient to failures because even one of the
replicas is not available, your database continues to work.

There are two types of replicas in Upstash Redis: primary replicas and read replicas. Primary replicas handle both reads and writes, while read replicas are used only for reads. In all database types, multiple primary replicas ensure that if one fails, your database continues to function.

In free and pay-as-you-go databases, there is one read replica per region. If it fails, your database remains operational, and you can still read from the primary replicas, though with higher latency. In Pro and Enterprise databases, multiple read replicas ensure that if one fails, your database remains operational, and you can read from the other read replicas without any additional latency.

### Better Scalability

In a replicated database, your requests are evenly distributed among the
replicas using a round-robin approach. As your throughput requirements grow,
additional replicas can be added to the cluster to handle the increased workload
and maintain high performance. This scalability feature ensures that your
database can effectively meet the demands of high throughput scenarios.

### Architecture

We use the single leader replication model. Each key is owned by a leader
replica and other replicas become the backups of the leader. Writes on a key are
processed by the leader replica first then propagated to backup replicas. Reads
can be performed from any replica. This model gives a better write consistency
and read scalability.

### Consistency

Each replica in the cluster utilizes a failure detector to monitor the status of
the leader replica. In the event that the leader replica fails, the remaining
replicas initiate a new leader election process to select a new leader. During
this leader election round, which is the only unavailability window for the
cluster, there may be a short period of time where your requests can be
temporarily blocked.

However, once a new leader is elected, normal operations resume, ensuring the
continued availability of the cluster. This mechanism ensures that any potential
unavailability caused by leader failure is minimized, and the cluster can
quickly recover and resume processing requests.


# REST API



REST API enables you to access your Upstash database using REST.

## Get Started

If you do not have a database already, follow
[these steps](../overall/getstarted) to create one.

In the database details section of the
[Upstash Console](https://console.upstash.com), click the `REST API` button.
Copy the REST URL and the authentication token. Send an HTTP GET request to the
provided URL by adding an `Authorization: Bearer $TOKEN` header.

```shell
curl https://us1-merry-cat-32748.upstash.io/set/foo/bar \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934"
```

The above script executes a `SET foo bar` command. It will return a JSON
response:

```json
{ "result": "OK" }
```

You can also set the token as `_token` request parameter as below:

```shell
curl https://us1-merry-cat-32748.upstash.io/set/foo/bar?_token=2553feg6a2d9842h2a0gcdb5f8efe9934
```

## API Semantics

Upstash REST API follows the same convention with
[Redis Protocol](https://redis.io/commands). Give the command name and
parameters in the same order as Redis protocol by separating them with a `/`.

```shell
curl REST_URL/COMMAND/arg1/arg2/../argN
```

Here are some examples:

*   `SET foo bar` -> `REST_URL/set/foo/bar`

*   `SET foo bar EX 100` -> `REST_URL/set/foo/bar/EX/100`

*   `GET foo` -> `REST_URL/get/foo`

*   `MGET foo1 foo2 foo3` -> `REST_URL/mget/foo1/foo2/foo3`

*   `HGET employee:23381 salary` -> `REST_URL/hget/employee:23381/salary`

*   `ZADD teams 100 team-x 90 team-y` ->
    `REST_URL/zadd/teams/100/team-x/90/team-y`

#### JSON or Binary Value

To post a JSON or a binary value, you can use an HTTP POST request and set value
as the request body:

```shell
curl -X POST -d '$VALUE' https://us1-merry-cat-32748.upstash.io/set/foo \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934"
```

In the example above, `$VALUE` sent in request body is appended to the command
as `REST_URL/set/foo/$VALUE`.

Please note that when making a POST request to the Upstash REST API, the request
body is appended as the last parameter of the Redis command. If there are
additional parameters in the Redis command after the value, you should include
them as query parameters in the request:

```shell
curl -X POST -d '$VALUE' https://us1-merry-cat-32748.upstash.io/set/foo?EX=100 \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934"
```

Above command is equivalent to `REST_URL/set/foo/$VALUE/EX/100`.

#### POST Command in Body

Alternatively, you can send the whole command in the request body as a single
JSON array. Array's first element must be the command name and command
parameters should be appended next to each other in the same order as Redis
protocol.

```shell
curl -X POST -d '[COMMAND, ARG1, ARG2,.., ARGN]' REST_URL
```

For example, Redis command `SET foo bar EX 100` can be sent inside the request
body as:

```shell
curl -X POST -d '["SET", "foo", "bar", "EX", 100]' https://us1-merry-cat-32748.upstash.io \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934"
```

## HTTP Codes

*   `200 OK`: When request is accepted and successfully executed.

*   `400 Bad Request`: When there's a syntax error, an invalid/unsupported command
    is sent or command execution fails.

*   `401 Unauthorized`: When authentication fails; auth token is missing or
    invalid.

*   `405 Method Not Allowed`: When an unsupported HTTP method is used. Only
    `HEAD`, `GET`, `POST` and `PUT` methods are allowed.

## Response

REST API returns a JSON response by default. When command execution is successful, response
JSON will have a single `result` field and its value will contain the Redis
response. It can be either;

*   a `null` value

```json
{ "result": null }
```

*   an integer

```json
{ "result": 137 }
```

*   a string

```json
{ "result": "value" }
```

*   an array value:

```json
{ "result": ["value1", null, "value2"] }
```

If command is rejected or fails, response JSON will have a single `error` field
with a string value explaining the failure:

```json
{"error":"WRONGPASS invalid password"}

{"error":"ERR wrong number of arguments for 'get' command"}
```

### Base64 Encoded Responses

If the response contains an invalid utf-8 character, it will be replaced with
a � (Replacement character U+FFFD). This can happen when you are using binary
operations like `BITOP NOT` etc.

If you prefer the raw response in base64 format, you can achieve this by setting
the `Upstash-Encoding` header to `base64`. In this case, all strings in the response
will be base64 encoded, except for the "OK" response.

```shell
curl https://us1-merry-cat-32748.upstash.io/SET/foo/bar \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934" \
 -H "Upstash-Encoding: base64"

# {"result":"OK"}

curl https://us1-merry-cat-32748.upstash.io/GET/foo \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934" \
 -H "Upstash-Encoding: base64"

# {"result":"YmFy"}
```

### RESP2 Format Responses

REST API returns a JSON response by default and the response content type is set to `application/json`.

If you prefer the binary response in RESP2 format, you can achieve this by setting
the `Upstash-Response-Format` header to `resp2`. In this case, the response content type
is set to `application/octet-stream` and the raw response is returned as binary similar to a TCP-based Redis client.

The default value for this option is `json`.
Any format other than `json` and `resp2` is not allowed and will result in a HTTP 400 Bad Request.

This option is not applicable to `/multi-exec` transactions endpoint, as it only returns response in JSON format.
Additionally, setting the `Upstash-Encoding` header to `base64` is not permitted when the `Upstash-Response-Format` is set to `resp2`
and will result in a HTTP 400 Bad Request.

```shell
curl https://us1-merry-cat-32748.upstash.io/SET/foo/bar \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934" \
 -H "Upstash-Reponse-Format: resp2"

# +OK\r\n

curl https://us1-merry-cat-32748.upstash.io/GET/foo \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934" \
 -H "Upstash-Reponse-Format: resp2"

# $3\r\nbar\r\n
```

## Pipelining

Upstash REST API provides support for command pipelining, allowing you to send
multiple commands as a batch instead of sending them individually and waiting
for responses. With the pipeline API, you can include several commands in a
single HTTP request, and the response will be a JSON array. Each item in the
response array corresponds to the result of a command in the same order as they
were included in the pipeline.

API endpoint for command pipelining is `/pipeline`. Pipelined commands should be
send as a two dimensional JSON array in the request body, each row containing
name of the command and its arguments.

**Request syntax**:

```shell
curl -X POST https://us1-merry-cat-32748.upstash.io/pipeline \
 -H "Authorization: Bearer $TOKEN" \
 -d '
    [
      ["CMD_A", "arg0", "arg1", ..., "argN"],
      ["CMD_B", "arg0", "arg1", ..., "argM"],
      ...
    ]
    '
```

**Response syntax**:

```json
[{"result":"RESPONSE_A"},{"result":"RESPONSE_B"},{"error":"ERR ..."}, ...]
```

<Note>
  Execution of the pipeline is *not atomic*. Even though each command in the
  pipeline will be executed in order, commands sent by other clients can
  interleave with the pipeline. Use [transactions](#transactions) API instead if
  you need atomicity.
</Note>

For example you can write the `curl` command below to send following Redis
commands using pipeline:

```redis
SET key1 valuex
SETEX key2 13 valuez
INCR key1
ZADD myset 11 item1 22 item2
```

```shell
curl -X POST https://us1-merry-cat-32748.upstash.io/pipeline \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934" \
 -d '
    [
      ["SET", "key1", "valuex"],
      ["SETEX", "key2", 13, "valuez"],
      ["INCR", "key1"],
      ["ZADD", "myset", 11, "item1", 22, "item2"]
    ]
    '
```

And pipeline response will be:

```json
[
  { "result": "OK" },
  { "result": "OK" },
  { "error": "ERR value is not an int or out of range" },
  { "result": 2 }
]
```

You can use pipelining when;

*   You need more throughput, since pipelining saves from multiple round-trip
    times. (*But beware that latency of each command in the pipeline will be equal
    to the total latency of the whole pipeline.*)
*   Your commands are independent of each other, response of a former command is
    not needed to submit a subsequent command.

## Transactions

Upstash REST API supports transactions to execute multiple commands atomically.
With transactions API, several commands are sent using a single HTTP request,
and a single JSON array response is returned. Each item in the response array
corresponds to the command in the same order within the transaction.

API endpoint for transaction is `/multi-exec`. Transaction commands should be
send as a two dimensional JSON array in the request body, each row containing
name of the command and its arguments.

**Request syntax**:

```shell
curl -X POST https://us1-merry-cat-32748.upstash.io/multi-exec \
 -H "Authorization: Bearer $TOKEN" \
 -d '
    [
      ["CMD_A", "arg0", "arg1", ..., "argN"],
      ["CMD_B", "arg0", "arg1", ..., "argM"],
      ...
    ]
    '
```

**Response syntax**:

In case when transaction is succesful, multiple responses corresponding to each
command is returned in json as follows:

```json
[{"result":"RESPONSE_A"},{"result":"RESPONSE_B"},{"error":"ERR ..."}, ...]
```

If transaction is discarded as a whole, a single error is returned in json as
follows:

```json
{ "error": "ERR ..." }
```

A transaction might be discarded in following cases:

*   There is a syntax error on the transaction request.
*   At least one of the commands is unsupported.
*   At least one of the commands exceeds the
    [max request size](../troubleshooting/max_request_size_exceeded).
*   At least one of the commands exceeds the
    [daily request limit](../troubleshooting/max_daily_request_limit).

Note that a command may still fail even if it is a supported and valid command.
In that case, all commands will be executed. Upstash Redis will not stop the
processing of commands. This is to provide same semantics with Redis when there
are
[errors inside a transaction](https://redis.io/docs/manual/transactions/#errors-inside-a-transaction).

**Example**:

You can write the `curl` command below to send following Redis commands using
REST transaction API:

```
MULTI
SET key1 valuex
SETEX key2 13 valuez
INCR key1
ZADD myset 11 item1 22 item2
EXEC
```

```shell
curl -X POST https://us1-merry-cat-32748.upstash.io/multi-exec \
 -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934" \
 -d '
    [
      ["SET", "key1", "valuex"],
      ["SETEX", "key2", 13, "valuez"],
      ["INCR", "key1"],
      ["ZADD", "myset", 11, "item1", 22, "item2"]
    ]
    '
```

And transaction response will be:

```json
[
  { "result": "OK" },
  { "result": "OK" },
  { "error": "ERR value is not an int or out of range" },
  { "result": 2 }
]
```

## Monitor Command

Upstash REST API provides Redis [`MONITOR`](https://redis.io/docs/latest/commands/monitor/) command using
[Server Send Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events) mechanism. API endpoint is `/monitor`.

```shell
curl -X POST https://us1-merry-cat-32748.upstash.io/monitor \
  -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934" \
  -H "Accept:text/event-stream"
```

This request will listen for Redis monitor events and incoming data will be received as:

```
data: "OK"
data: 1721284005.242090 [0 0.0.0.0:0] "GET" "k"
data: 1721284008.663811 [0 0.0.0.0:0] "SET" "k" "v"
data: 1721284025.561585 [0 0.0.0.0:0] "DBSIZE"
data: 1721284030.601034 [0 0.0.0.0:0] "KEYS" "*"
```

## Subscribe & Publish Commands

Simiar to `MONITOR` command, Upstash REST API provides Redis [`SUBSCRIBE`](https://redis.io/docs/latest/commands/subscribe/) and
[`PUBLISH`](https://redis.io/docs/latest/commands/publish/) commands. The `SUBSCRIBE` endpoint works using\
[Server Send Events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events) mechanism.
API endpoints are `/subscribe` and `/publish`

Following request will subscribe to a channel named `chat`:

```shell
curl -X POST https://us1-merry-cat-32748.upstash.io/subscribe/chat \
  -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934" \
  -H "Accept:text/event-stream"
```

Following request will publish to a channel named `chat`:

```shell
curl -X POST https://us1-merry-cat-32748.upstash.io/publish/chat/hello \
  -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934"
```

The subscriber will receive incoming messages as:

```
data: subscribe,chat,1
data: message,chat,hello
data: message,chat,how are you today?
```

## Security and Authentication

You need to add a header to your API requests as `Authorization: Bearer $TOKEN`
or set the token as a url parameter `_token=$TOKEN`.

```shell
curl -X POST https://us1-merry-cat-32748.upstash.io/info \
  -H "Authorization: Bearer 2553feg6a2d9842h2a0gcdb5f8efe9934"
```

OR

```shell
curl -X POST https://us1-merry-cat-32748.upstash.io/info?_token=2553feg6a2d9842h2a0gcdb5f8efe9934
```

Upstash by default provides two separate access tokens per database: "Standard"
and "Read Only".

*   **Standard** token has full privilege over the database, can execute any
    command.

*   **Read Only** token permits access to the read commands only. Some powerful
    read commands (e.g. SCAN, KEYS) are also restricted with read only token. It
    makes sense to use *Read Only* token when you access Upstash Redis from web
    and mobile clients where the token is exposed to public.

You can get/copy the tokens by clicking copy button next to
`UPSTASH_REDIS_REST_TOKEN` in REST API section of the console. For the *Read
Only* token, just enable the "Read-Only Token" switch.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/restapi/tokens.png" />
</Frame>

<Warning>
  Do not expose your *Standard* token publicly. *Standard* token has full
  privilege over the database. You can expose the *Read Only* token as it has
  access to read commands only. You can revoke both *Standard* and *Read Only*
  tokens by resetting password of your database.
</Warning>

### REST Token for ACL Users

In addition to the tokens provided by default, you can create REST tokens for
the users created via [`ACL SETUSER`](https://redis.io/commands/acl-setuser/)
command. Upstash provides a custom `ACL` subcommand to generate REST tokens:
`ACL RESTTOKEN`. It expects two arguments; username and user's password. And
returns the REST token for the user as a string response.

```
ACL RESTTOKEN <username> <password>
    Generate a REST token for the specified username & password.
    Token will have the same permissions with the user.
```

You can execute `ACL RESTTOKEN` command via `redis-cli`:

```shell
redis-cli> ACL RESTTOKEN default 35fedg8xyu907d84af29222ert
"AYNgAS2553feg6a2d9842h2a0gcdb5f8efe9934DQ="
```

Or via CLI on the Upstash console:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/restapi/acl-resttoken.png" />
</Frame>

If the user doesn't exist or password doesn't match then an error will be
returned.

```shell
redis-cli> ACL RESTTOKEN upstash fakepass
(error) ERR Wrong password or user "upstash" does not exist
```

## Redis Protocol vs REST API

### REST API Pros

*   If you want to access to Upstash database from an environment like CloudFlare
    Workers, WebAssembly, Fastly Compute\@Edge then you can not use Redis protocol
    as it is based on TCP. You can use REST API in those environments.

*   REST API is request (HTTP) based where Redis protocol is connection based. If
    you are running serverless functions (AWS Lambda etc), you may need to manage
    the Redis client's connections. REST API does not have such an issue.

*   Redis protocol requires Redis clients. On the other hand, REST API is
    accessible with any HTTP client.

### Redis Protocol Pros

*   If you have legacy code that relies on Redis clients, the Redis protocol
    allows you to utilize Upstash without requiring any modifications to your
    code.

*   By leveraging the Redis protocol, you can take advantage of the extensive
    Redis ecosystem. For instance, you can seamlessly integrate your Upstash
    database as a session cache for your Express application.

## REST API vs GraphQL API

The REST API generally exhibits lower latency compared to the GraphQL API. In
the case of the REST API, direct access to the database is established. However,
with the GraphQL API, a proxy layer is present, responsible for accepting
connections and translating GraphQL queries into the Redis protocol.

<Note>
  If you do not have a specific GraphQL use case, we recommend REST API instead
  of GraphQL API. We plan to deprecate the GraphQL API in future releases.
</Note>

## Cost and Pricing

Upstash pricing is based on per command/request. So the same pricing listed in
our [pricing](https://upstash.com/pricing/redis) applies to your REST calls too.

## Metrics and Monitoring

In the current version, we do not expose any metrics specific to API calls in
the console. But the metrics of the database backing the API should give a good
summary about the performance of your APIs.

## REST - Redis API Compatibility

| Feature                                                       | REST Support? |                               Notes                               |
| ------------------------------------------------------------- | :-----------: | :---------------------------------------------------------------: |
| [String](https://redis.io/commands/?group=string)             |       ✅       |                                                                   |
| [Bitmap](https://redis.io/commands/?group=bitmap)             |       ✅       |                                                                   |
| [Hash](https://redis.io/commands/?group=hash)                 |       ✅       |                                                                   |
| [List](https://redis.io/commands/?group=list)                 |       ✅       | Blocking commands (BLPOP - BRPOP - BRPOPLPUSH) are not supported. |
| [Set](https://redis.io/commands/?group=set)                   |       ✅       |                                                                   |
| [SortedSet](https://redis.io/commands/?group=sorted_set)      |       ✅       |     Blocking commands (BZPOPMAX - BZPOPMIN) are not supported.    |
| [Geo](https://redis.io/commands/?group=geo)                   |       ✅       |                                                                   |
| [HyperLogLog](https://redis.io/commands/?group=hyperloglog)   |       ✅       |                                                                   |
| [Transactions](https://redis.io/commands/?group=transactions) |       ✅       |              WATCH/UNWATCH/DISCARD are not supported              |
| [Generic](https://redis.io/commands/?group=generic)           |       ✅       |                                                                   |
| [Server](https://redis.io/commands/?group=server)             |       ✅       |                                                                   |
| [Scripting](https://redis.io/commands/?group=scripting)       |       ✅       |                                                                   |
| [Pub/Sub](https://redis.io/commands/?group=pubsub)            |       ⚠️      |               Only PUBLISH and PUBSUB are supported.              |
| [Connection](https://redis.io/commands/?group=connection)     |       ⚠️      |                 Only PING and ECHO are supported.                 |
| [JSON](https://redis.io/commands/?group=json)                 |       ✅       |                                                                   |
| [Streams](https://redis.io/commands/?group=stream)            |       ✅       |                                                                   |
| [Cluster](https://redis.io/commands#cluster)                  |       ❌       |                                                                   |


# Security



Upstash has a set of features to help you secure your data. We will list them
and at the end of the section we will list the best practices to improve
security of database.

## TLS

TLS is an optional feature which you can enable while creating your database.
Once TLS is enabled, the data transfer between the client and database is
encrypted. We strongly recommend enabling TLS for production databases.

## Redis ACL

<Warning>
  As of May 2024, the ACL command is no longer supported for newly created databases on the AWS platform.
  However, it remains available on [fly.io](https://fly.io/docs/reference/redis/), GCP, and AWS databases created before May 2024.
</Warning>

With Redis ACL, you can improve security by restricting a user's access to
commands and keys, so that untrusted clients have no access and trusted clients
have just the minimum required access level to the database. Moreover it
improves operational safety, so that clients or users accessing Redis are not
allowed to damage the data or the configuration due to errors or mistakes. Check
[Redis ACL documentation](https://redis.io/docs/manual/security/acl/). If you
are using the REST API, you can still benefit from ACLs as explained
[here](/redis/features/restapi#rest-token-for-acl-users)

## Database Credentials

When you create a database, a secure password is generated. Upstash keeps the
password encrypted. Use environment variables or your provider's secret
management system (e.g. AWS Secrets Manager, Vercel Secrets) to keep them. Do
not use them hardcoded in your code. If your password is leaked, reset the
password using Upstash console.

## Encryption at Rest

Encryption at REST encrypts the block storage where your data is persisted and
stored. It is enabled by default for Pro plan databases.

## Application Level Encryption

Client side encryption can be used to encrypt data through application
lifecycle. Client-side encryption is used to help protect data in use. This
comes with some limitations. Operations that must operate on the data, such as
increments, comparisons, and searches will not function properly. You can write
client-side encryption logic directly in your own application or use functions
built into clients such as the Java Lettuce cipher codec. We have plans to
support encryption in our SDKs.

## IP Allowlisting

We can restrict the access to your database to a set of IP addresses which will
have access to your database. This is quite a strong way to secure your
database, but it has some limitations. For example you can not know the IP
addresses in serverless platforms such AWS Lambda and Vercel functions.

## TLS Mutual Authentication

mTLS ensures two-way authentication where both client and server authenticate
each other at the same time in the authentication protocol. No client can assess
the database without the certificate. The user downloads uses the certificate
for their client.

## VPC Peering

VPC Peering enables you to connect to Upstash from your own VPC using private
IP. Database will not be accessible from the public network. Database and your
application can run in the same subnet which also minimizes data transfer costs.
VPC Peering is only available for Pro databases.

## Private Link

AWS Private link provides private connectivity between Upstash Database and your
Redis client inside AWS infrastructure. Private link is only available for
Pro databases.


# Compliance



## Upstash Legal & Security Documents

*   [Upstash Terms of Service](https://upstash.com/static/trust/terms.pdf)
*   [Upstash Privacy Policy](https://upstash.com/static/trust/privacy.pdf)
*   [Upstash Data Processing Agreement](https://upstash.com/static/trust/dpa.pdf)
*   [Upstash Technical and Organizational Security Measures](https://upstash.com/static/trust/security-measures.pdf)
*   [Upstash Subcontractors](https://upstash.com/static/trust/subprocessors.pdf)

## Is Upstash SOC2 Compliant?

As of July 2023, Upstash Redis and Kafka are SOC2 compliant. Check our [trust page](https://trust.upstash.com/) for details.

## Is Upstash ISO-27001 Compliant?

We are in process of getting this certification. Contact us
([support@upstash.com](mailto:support@upstash.com)) to learn about the expected
date.

## Is Upstash GDPR Compliant?

Yes. For more information, see our
[Privacy Policy](https://upstash.com/static/trust/privacy.pdf). We acquire DPAs
from each [subcontractor](https://upstash.com/static/trust/subprocessors.pdf)
that we work with.

## Is Upstash HIPAA Compliant?

Upstash is currently not HIPAA compliant. Contact us
([support@upstash.com](mailto:support@upstash.com)) if HIPAA is important for
you and we can share more details.

## Is Upstash PCI Compliant?

Upstash does not store personal credit card information. We use Stripe for
payment processing. Stripe is a certified PCI Service Provider Level 1, which is
the highest level of certification in the payments industry.

## Does Upstash conduct vulnerability scanning and penetration tests?

Yes, we use third party tools and work with pen testers. We share the results
with Enterprise customers. Contact us
([support@upstash.com](mailto:support@upstash.com)) for more information.

## Does Upstash take backups?

Yes, we take regular snapshots of the data cluster to the AWS S3 platform.

## Does Upstash encrypt data?

Customers can enable TLS while creating database/cluster, and we recommend it
for production databases/clusters. Also we encrypt data at rest at request of
customers.


# Frequently Asked Questions



## What is Upstash Redis?

Upstash is a serverless database service compatible with Redis® API.

## What is a Serverless Database?

*   You do not have to manage and provision servers.
*   You do not deal with configuring or maintaining any server.
*   You just use the service and pay what you use. If you are not using it, you should not be paying.

## What are the use cases?

Upstash works for all the common usecases for Redis®. You can use Upstash in your serverless stack. In addition, you can use Upstash as storage (or caching) for your serverless functions. See [Use Cases](/redis/overall/usecases) for more.

## Do you support all Redis® API?

Most of them. See [Redis® API Compatibility](/redis/overall/rediscompatibility) for the list of supported commands.

## Can I use any Redis client?

Yes, Upstash is compatible Redis client protocol.

## Which cloud providers do you support?

Initially we have AWS and GCP. Digital Ocean is planned.

## Which regions do you support in AWS?

We start with AWS-US-EAST-1 (Virginia), GCP-US-CENTRAL-1 (IOWA), AWS-US-WEST-1 (N. California), AWS-EU-WEST-1 (Ireland), AWS-APN-NE-1 (Japan). We will add new regions soon. You can expedite this by telling us your use case and the region you need by emailing to [feedback@upstash.com](mailto:feedback@upstash.com)

## Should my client be hosted in the AWS to use Upstash?

No. Your client can be anywhere but the clients in AWS regions will give you better performance.

## How do you compare Upstash with ElastiCache?

Upstash is serverless. With ElastiCache, you pay even you do not use the database. See [Compare](/redis/overall/compare) for more info.

## How do you compare Upstash with Redis Labs or Compose.io?

Upstash is serverless. With Redis Labs or Compose.io, you always pay a lot when your data size is big but your traffic is low. In Upstash, the pricing is based on per request. See [Compare](/redis/overall/compare) for more info.

## Do you persist data?

Yes, by default we write data to the disk. So in case of a failure you should not lose any data.

## Do you support Redis Cluster?

We support replication in Premium type database. We do not support sharding yet.

## I have database with 10GB data, I pay nothing if I do not use it. Is that correct?

You only pay for the disk storage cost that is \$0.25 per GB. For your case, you will pay \$2.5 monthly.

## What happens when I exceed the request limit on Free Database (10.000 requests per day)?

The exceeding commands return exception.

## When I upgrade my free database, do I lose data?

You do not lose data but clients may disconnect and reconnect.

## Upstash is much cheaper than Elasticache and Redis Labs for big data sizes (> 10GB). How is that possible?

Upstash storage layer is multi tiered. We keep your data in both memory and block storage (disk). The entries that are not accessed frequently are removed from the memory but stored in disk. Latency overhead of idle entries is limited thanks to the SSD based storage. Multi tiered storage allows us to provide more flexible pricing.

## Will my data be safe?

Upstash is a GDPR compliant company. We do not share any user data with third parties. See our [Legal Documents](/common/help/legal) for more information.

## How do you handle the noisy neighbour problem? Do other tenants affect my database?

Databases are isolated on some aspects but still share some hardware resources such as CPU or network. To avoid noisy neighbor influence on these resources, there are specific quotas for each database. When they reach any of these quotas they are throttled using a backoff strategy. When multiple databases sharing the same hardware are close to the limits, our system can add new resources to the pool and/or migrate some of the databases to distribute the load.

Also if a database exceeds its quotas very frequently, we notify users whether they want to upgrade to an upper plan. Databases in enterprise plans are placed either on dedicated or more isolated hardware due to higher resource needs.


# Integration with Third Parties & Partnerships



<Frame>
  <img height="100" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/integration/upstash-integration-diagram.png" />
</Frame>

## Introduction

In this guideline we will outline the steps to integrate Upstash into your platform (GUI or Web App) and allow your users to create and manage Upstash databases without leaving your interfaces. We will explain how to use OAuth2.0 as the underlying foundation to enable this access seamlessly.

If your product or service offering utilizes Redis, Kafka or QStash or if there is a common use case that your end users enable by leveraging these database resources, we invite you to be a partner with us. By integrating Upstash into your platform, you can offer a more complete package for your customers and become a one stop shop. This will also position yourself at the forefront of innovative cloud computing trends such as serverless and expand your customer base.

This is the most commonly used partnership integration model that can be easily implemented by following this guideline. Recently [Cloudflare workers integration](https://blog.cloudflare.com/cloudflare-workers-database-integration-with-upstash/) is implemented through this methodology. For any further questions or partnership discussions please send us an email at [partnerships@upstash.com](mailto:partnerships@upstash.com)

<Info>
  Before starting development to integrate Upstash into your product, please
  send an email to [partnerships@upstash.com](mailto:partnerships@upstash.com) for further assistance and guidance.
</Info>

**General Flow (High level user flow)**

1.  User clicks **`Connect Upstash`** button on your platform’s surface (GUI, Web App)
2.  This initiates the OAuth 2.0 flow, which opens a new browser page displaying the **`Upstash Login Page`**.
3.  If this is an existing user, user logins with their Upstash credentials otherwise they can directly sign up for a new Upstash account.
4.  Browser window redirects to **`Your account has been connected`** page and authentication window automatically closes.
5.  After the user returns to your interface, they see their Upstash Account is now connected.

## Technical Design (SPA - Regular Web Application)

1.  Users click `Connect Upstash` button from Web App.
2.  Web App initiate Upstash OAuth 2.0 flow. Web App can use
    [Auth0 native libraries](https://auth0.com/docs/libraries).

<Note>
  Please reach [partnerships@upstash.com](mailto:partnerships@upstash.com) to receive client id and callback url.
</Note>

3.  After user returns from OAuth 2.0 flow then web app will have JWT token. Web
    App can generate Developer Api key:

```bash
curl -XPOST https://api.upstash.com/apikey \
    -H "Authorization: Bearer JWT_KEY" \
    -H "Content-Type: application/json" \
    -d '{ "name": "APPNAME_API_KEY_TIMESTAMP" }'
```

4.  Web App need to save Developer Api Key to the backend.

## Technical Design ( GUI Apps )

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/integration/oauth2-integration.png" />
</Frame>

1.  User clicks **`Connect Upstash`** button from web app.
2.  Web app initiates Upstash OAuth 2.0 flow and it can use **[Auth0 native libraries](https://auth0.com/docs/libraries)**.
3.  App will open new browser:

```
https://auth.upstash.com/authorize?response_type=code&audience=upstash-api&scope=offline_access&client_id=XXXXXXXXXX&redirect_uri=http%3A%2F%2Flocalhost:3000
```

<Note>Please reach [partnerships@upstash.com](mailto:partnerships@upstash.com) to receive client id.</Note>

4.  After user authenticated Auth0 will redirect user to
    `localhost:3000/?code=XXXXXX`

5.  APP can return some nice html response when Auth0 returns to `localhost:3000`

6.  After getting `code` parameter from the URL query, GUI App will make http
    call to the Auth0 code exchange api. Example CURL request

```bash
curl -XPOST 'https://auth.upstash.com/oauth/token' \
  --header 'content-type: application/x-www-form-urlencoded' \
  --data 'grant_type=authorization_code --data audience=upstash-api' \
  --data 'client_id=XXXXXXXXXXX' \
  --data 'code=XXXXXXXXXXXX' \
  --data 'redirect_uri=localhost:3000'
```

Response:

```json
{
  "access_token": "XXXXXXXXXX",
  "refresh_token": "XXXXXXXXXXX",
  "scope": "offline_access",
  "expires_in": 172800,
  "token_type": "Bearer"
}
```

7.  After 6th Step the response will include `access_token`, it has 3 days TTL.
    GUI App will call Upstash API to get a developer api key:

```bash
curl https://api.upstash.com/apikey -H "Authorization: Bearer JWT_KEY" -d '{ "name" : "APPNAME_API_KEY_TIMESTAMP" }'
```

8.  GUI App will save Developer Api key locally. Then GUI App can call any
    Upstash Developer API [developer.upstash.com/](https://developer.upstash.com/)

## Managing Resources

After obtaining Upstash Developer Api key, your platform surface (web or GUI) can call Upstash API. For example **[Create Database](https://developer.upstash.com/#create-database-global)**, **[List Database](https://developer.upstash.com/#list-databases)**

In this flow, you can ask users for region information and name of the database then can call Create Database API to complete the task

Example CURL request:

```bash
curl -X POST \
  https://api.upstash.com/v2/redis/database \
  -u 'EMAIL:API_KEY' \
  -d '{"name":"myredis", "region":"global", "primary_region":"us-east-1", "read_regions":["us-west-1","us-west-2"], "tls": true}'
```


# Legal



## Upstash Legal Documents

*   [Upstash Terms of Service](https://upstash.com/trust/terms.pdf)
*   [Upstash Privacy Policy](https://upstash.com/trust/privacy.pdf)
*   [Upstash Subcontractors](https://upstash.com/trust/subprocessors.pdf)


# Enterprise Support



Enterprise Support is recommended for customers who use Upstash as part of
their production systems.

Enterprise Support includes the following services:

*   Response time SLA
*   Dedicated Slack/Discord Channels
*   Dedicated real time support: We reserve our engineers for you to help you for
    cases like architecture review, product launch or data migration. Max 10 hours
    per / month.

### Response Time SLA

*   General guidance: 24 hours
*   System impaired: \< 12 hours
*   Production system impaired: \< 4 hours
*   Production system down: \< 1 hour

### Pricing

\$1600 per year + 10% of your monthly bill

<Info>
  To purchase or learn more about Enterprise Support, please contact us at
  [support@upstash.com](mailto:support@upstash.com)
</Info>


# Uptime SLA



This Service Level Agreement ("SLA") applies to the use of the Upstash services,
offered under the terms of our Terms of Service or other agreement with us
governing your use of Upstash. This SLA does not apply to Upstash services in
the Upstash Free Tier. It is clarified that this SLA is subject to the terms of
the Agreement, and does not derogate therefrom (capitalized terms, unless
otherwise indicated herein, have the meaning specified in the Agreement).
Upstash reserves the right to change the terms of this SLA by publishing updated
terms on its website, such change to be effective as of the date of publication.

### Regional and Global Database SLA

Upstash will use commercially reasonable efforts to make regional and global
databases available with a Monthly Uptime Percentage of at least 99.99%.

In the event any of the services do not meet the SLA, you will be eligible to
receive a Service Credit as described below.

| Monthly Uptime Percentage                           | Service Credit Percentage |
| --------------------------------------------------- | ------------------------- |
| Less than 99.99% but equal to or greater than 99.0% | 10%                       |
| Less than 99.0% but equal to or greater than 95.0%  | 30%                       |
| Less than 95.0%                                     | 60%                       |

### SLA Credits

Service Credits are calculated as a percentage of the monthly bill (excluding
one-time payments such as upfront payments) for the service in the affected
region that did not meet the SLA.

Uptime percentages are recorded and published in the
[Upstash Status Page](https://status.upstash.com).

To receive a Service Credit, you should submit a claim by sending an email to
[support@upstash.com](mailto:support@upstash.com). Your credit request should be
received by us before the end of the second billing cycle after the incident
occurred.

We will apply any service credits against future payments for the applicable
services. At our discretion, we may issue the Service Credit to the credit card
you used. Service Credits will not entitle you to any refund or other payment. A
Service Credit will be applicable and issued only if the credit amount for the
applicable monthly billing cycle is greater than one dollar (\$1 USD). Service
Credits may not be transferred or applied to any other account.


# Support & Contact Us



## Community

[Upstash Discord Channel](https://upstash.com/discord) is the best way to
interact with the community.

## Team

You can contact the team
via [support@upstash.com](mailto:support@upstash.com) for technical support as
well as questions and feedback.

## Follow Us

Follow us at [X](https://x.com/upstash).

## Bugs & Issues

You can help us improve Upstash by reporting issues, suggesting new features and
giving general feedback in
our [Community Github Repo](https://github.com/upstash/issues/issues/new).

## Enterprise Support

Get [Enterprise Support](/common/help/prosupport) from the Upstash team.


# Uptime Monitor



## Status Page

You can track the uptime status of Upstash databases in
[Upstash Status Page](https://status.upstash.com)

## Latency Monitor

You can see the average latencies for different regions in
[Upstash Latency Monitoring](https://latency.upstash.com) page


# Connect Your Client



Upstash works with Redis® API, that means you can use any Redis client with
Upstash. At the [Redis Clients](https://redis.io/clients) page you can find the
list of Redis clients in different languages.

Probably, the easiest way to connect to your database is to use `redis-cli`.
Because it is already covered in [Getting Started](../overall/getstarted), we
will skip it here.

## Database

After completing the [getting started](../overall/getstarted) guide, you will
see the database page as below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/getting_started/database.png" />
</Frame>

The information required for Redis clients is displayed here as **Endpoint**,
**Port** and **Password**. Also when you click on `Clipboard` button on **Connect to your database** section, you can copy
the code that is required for your client.

Below, we will provide examples from popular Redis clients, but the information above should help you configure all Redis clients similarly.

<Note>
  TLS is enabled by default for all Upstash Redis databases. It's not possible to disable it.
</Note>

## upstash-redis

<Info>
  Because upstash-redis is HTTP based, we recommend it for Serverless functions.
  Other TCP based clients can cause connection problems in highly concurrent use
  cases.
</Info>

**Library**: [upstash-redis](https://github.com/upstash/upstash-redis)

**Example**:

```typescript
import { Redis } from '@upstash/redis';

const redis = new Redis({ url: 'UPSTASH_REDIS_REST_URL', token: 'UPSTASH_REDIS_REST_TOKEN' });

(async () => {
  try {
    const data = await redis.get('key');
    console.log(data);
  } catch (error) {
    console.error(error);
  }
})();
```

## Node.js

**Library**: [ioredis](https://github.com/luin/ioredis)

**Example**:

```javascript
const Redis = require("ioredis");

let client = new Redis("rediss://:YOUR_PASSWORD@YOUR_ENDPOINT:YOUR_PORT");
await client.set("foo", "bar");
let x = await client.get("foo");
console.log(x);
```

## Python

**Library**: [redis-py](https://github.com/andymccurdy/redis-py)

**Example**:

```python
import redis
r = redis.Redis(
host= 'YOUR_ENDPOINT',
port= 'YOUR_PORT',
password= 'YOUR_PASSWORD', 
ssl=True)
r.set('foo','bar')
print(r.get('foo'))
```

## Java

**Library**: [jedis](https://github.com/xetorthio/jedis)

**Example**:

```java
Jedis jedis = new Jedis("YOUR_ENDPOINT", "YOUR_PORT", true);
jedis.auth("YOUR_PASSWORD");
jedis.set("foo", "bar");
String value = jedis.get("foo");
System.out.println(value);
```

## Go

**Library**: [redigo](https://github.com/gomodule/redigo)

**Example**:

```go
func main() {
  c, err := redis.Dial("tcp", "YOUR_ENDPOINT:YOUR_PORT", redis.DialUseTLS(true))
  if err != nil {
      panic(err)
  }

  _, err = c.Do("AUTH", "YOUR_PASSWORD")
  if err != nil {
      panic(err)
  }

  _, err = c.Do("SET", "foo", "bar")
  if err != nil {
      panic(err)
  }

  value, err := redis.String(c.Do("GET", "foo"))
  if err != nil {
      panic(err)
  }

  println(value)
}
```


# Connect with upstash-redis



[upstash-redis](https://github.com/upstash/redis-js)
is an HTTP/REST based Redis client built on top of
[Upstash REST API](/redis/features/restapi). For more information,
refer to the documentation of Upstash redis client ([TypeScript](/redis/sdks/ts/overview) & [Python](/redis/sdks/py/overview)).

It is the only connectionless (HTTP based) Redis client and designed for:

*   Serverless functions (AWS Lambda ...)
*   Cloudflare Workers (see
    [the example](https://github.com/upstash/upstash-redis/tree/master/examples/workers-with-upstash))
*   Fastly Compute\@Edge
*   Next.js, Jamstack ...
*   Client side web/mobile applications
*   WebAssembly
*   and other environments where HTTP is preferred over TCP.

See
[the list of APIs](https://docs.upstash.com/features/restapi#rest---redis-api-compatibility)
supported.

## Quick Start

### Install

```bash
npm install @upstash/redis
```

### Usage

```typescript
import { Redis } from "@upstash/redis";

const redis = new Redis({
  url: "UPSTASH_REDIS_REST_URL",
  token: "UPSTASH_REDIS_REST_TOKEN",
});

(async () => {
  try {
    const data = await redis.get("key");
    console.log(data);
  } catch (error) {
    console.error(error);
  }
})();
```

If you define `UPSTASH_REDIS_REST_URL` and`UPSTASH_REDIS_REST_TOKEN` environment
variables, you can load them automatically.

```typescript
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv()(async () => {
  try {
    const data = await redis.get("key");
    console.log(data);
  } catch (error) {
    console.error(error);
  }
})();
```


# Datadog - Upstash Redis Integration



This guide will walk you through the steps to seamlessly connect your Datadog account with Upstash for enhanced monitoring and analytics.

<Check>
  **Integration Scope**

  Upstash Datadog Integration only covers Pro databases or those included in the Enterprise Plan.
</Check>

## **Step 1: Log in to Your Datadog Account**

1.  Open your web browser and navigate to [Datadog](https://www.datadoghq.com/).
2.  Log in to your Datadog account.

## **Step 2: Install Upstash Application**

1.  Once logged in, navigate to the "Integrations" page in Datadog.
2.  Search for "Upstash" in the integrations list and click on it.

![integration-tab.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/datadog/integration-tab.png)

Let’s click on the "Install" button to add Upstash to your Datadog account.

![installation.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/datadog/installation.png)

## **Step 3: Connect Accounts**

After installing Upstash, click on the "Connect Accounts" button and Datadog will redirect you to the Upstash site for account integration.

![connect-acc.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/datadog/connect-acc.png)

## **Step 4: Select Account to Integrate**

1.  On the Upstash site, you will be prompted to select the Datadog account you want to integrate.
2.  Choose the appropriate Datadog account from the list.

Upstash Datadog Integration allows you to integrate personal and team based accounts.

**Caveats;**

*   This integration can only be executed only one time.If you would like to extend list of the team in integration please re-establish the integration from scratch.

![personal.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/datadog/personal.png)

![team.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/datadog/team.png)

## **Step 5: Wait for Metrics Availability**

Once you've selected your Datadog account, Upstash will begin the integration process and please be patient while the metrics are being retrieved. This may take a few moments.

And here we go, metrics will be available in Upstash Overview Dashboard !

![upstash-dashboard.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/datadog/upstash-dashboard.png)

## **Step 6: Datadog Integration Removal Process**

Navigate to Integration Tab on your Datadog account,

Once logged in, navigate to the "Integration" tab and continue with Datadog part. If you would like to remove your integration between Upstash and Datadog account press "Remove".

### Confirm Removal:

Upstash will suspend all metric publishing process after the you remove Datadog Integration.

After removing the integration on the Upstash side, it's crucial to go to your Datadog account and remove any related API keys or configurations associated with the integration.

## Pricing

If you choose to integrate Datadog via Upstash, there will be an additional cost of \$5 per month.
This charge will be reflected in your monthly invoice accordingly.

## **Conclusion**

Congratulations! You have successfully integrated your Datadog account with Upstash. You will now have access to enhanced monitoring and analytics for your Datadog metrics.

Feel free to explore Upstash's features and dashboards to gain deeper insights into your system's performance.

If you encounter any issues or have questions, please refer to the Upstash support documentation or contact our support team for assistance.


# EMQX - Upstash Redis Integration



EMQX, a robust open-source MQTT message broker, is engineered for scalable, distributed environments, prioritizing high
availability, throughput, and minimal latency. As a preferred protocol in the IoT landscape, MQTT (Message Queuing
Telemetry Transport) excels in enabling devices to effectively publish and subscribe to messages.

Offered by EMQ, EMQX Cloud is a comprehensively managed MQTT service in the cloud, inherently scalable and secure. Its
design is particularly advantageous for IoT applications, providing dependable MQTT messaging services.

This tutorial guides you on streaming MQTT data to Upstash via data integration. It allows clients to send temperature
and humidity data to EMQX Cloud using MQTT and channel it into Upstash for Redis storage.

## Setting Up Redis Database with Upstash

1.  Log in and create a Redis Database by clicking the **Create Database** button on [Upstash Console](https://console.upstash.com).

2.  Name your database and select a region close to your EMQX Cloud for optimal performance.

3.  Click **Create** to have your serverless Redis Database ready.

![upstash](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_redis_01.png)

### Database Details

Access the database console for the necessary information for further steps.

![upstash](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_redis_02.png)

The above steps, conclude the initial setup for Upstash.

## Establishing Data Integration with Upstash

### Activating EMQX Cloud's NAT Gateway

1.  Log into the EMQX Cloud console and go to the deployment Overview.

2.  Select **NAT Gateway** at the bottom and click **Subscribe Now**.

![NAT](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/public_nat.png)

### Configuring Data Integration

1.  In the EMQX Cloud console, choose **Data Integrations** and select **Upstash for Redis**.

    ![create resource](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_redis_03.png)

2.  Input **Endpoints** info from the Redis detail page into the **Redis Server** field, including the port. Enter the
    password in **Password** and click **Test** to ensure connectivity.
    ![create resource](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_redis_04.png)

3.  Click **New** to add a Redis resource. A new Upstash for Redis will appear under **Configured Resources**.

4.  Formulate a new SQL rule in the **SQL** field. This rule will read from `temp_hum/emqx` and append client\_id, topic,
    timestamp.

    *   `up_timestamp`: Message report time
    *   `client_id`: Publishing client's ID
    *   `temp`: Temperature data
    *   `Hum`: Humidity data

```sql
SELECT
timestamp as up_timestamp,
clientid as client_id,
payload.temp as temp,
payload.hum as hum
FROM
"temp_hum/emqx"
```

![rule sql](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/kafka_create_sql.png)

5.  Execute an SQL test with payload, topic, client info. Successful results confirm the rule's effectiveness.

    ![rule sql](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/kafka_create_sql_test.png)

6.  Proceed to **Next** to link an action. The rule will store the timestamp, client ID, temperature, and humidity in
    Redis. Click **Confirm**.

    ```bash
    HMSET ${client_id} ${up_timestamp} ${temp}
    ```

    ![rule sql](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_redis_05.png)

7.  Post-binding, click **View Details** for the rule SQL and bound actions.

8.  To review rules, select **View Created Rules** in Data Integrations. Check detailed metrics in the **Monitor**
    column.

## Testing the Data Bridge

1.  Simulate temperature and humidity data with [MQTTX](https://mqttx.app/). Add connection address and client
    authentication for the EMQX Dashboard.
    ![MQTTX](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_kafka_06.png)

2.  In Upstash Console, under Data Browser, select a client entry to review messages.

    ![monitor](https://raw.githubusercontent.com/emqx/cloud-docs/master/en_US/rule_engine/_assets/upstash_redis_07.png)


# Get Started with AWS Lambda



You can connect to Upstash database from your Lambda functions using your
favorite Redis client. You do not need any extra configuration. The only thing
to note is you should use the same region for your Lambda function and database
to minimize latency.

If you do not have any experience with AWS Lambda functions, you can follow the
following tutorial. The tutorial explains the required steps to implement an AWS
Lambda function that takes the key/value as parameters from APIGateway then
inserts an entry (key/value) to the database which is on Upstash. We have
implemented the function in Node.js, but the steps and the logic are quite
similar in other languages.

<Note>
  This example uses Redis clients. If you expect many concurrent AWS Lambda
  invocation then we recommend using
  **[upstash-redis](/redis/howto/connectwithupstashredis)** which is HTTP/REST
  based.
</Note>

**Step 1: Create database on Upstash**

If you do not have one, create a database following this
[guide](../overall/getstarted).

**Step 2: Create a Node project**

Create an empty folder for your project and inside the folder create a node
project with the command:

```
npm init
```

Then install the redis client with:

```
npm install ioredis
```

Now create index.js file. Replace the Redis URL in the below code.

<Snippet file="redis/ioredisnote.mdx" />

```javascript
var Redis = require("ioredis");

if (typeof client === "undefined") {
  var client = new Redis("rediss://:YOUR_PASSWORD@YOUR_ENDPOINT:YOUR_PORT");
}
exports.handler = async (event) => {
  await client.set("foo", "bar");
  let result = await client.get("foo");
  let response = {
    statusCode: 200,
    body: JSON.stringify({
      result: result,
    }),
  };
  return response;
};
```

**Step 3: Deploy Your Function**

Our function is ready to deploy. Normally you could copy-paste your function
code to AWS Lambda editor. But here it is not possible because we have an extra
dependency (redis-client). So we will zip and upload our function.

When you are in your project folder, create a zip with this command:

```
zip -r app.zip .
```

Now open your AWS console, from the top-right menu, select the region that you
created your database in Upstash. Then find or search the lambda service, click
on `Create Function` button.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/awslambda/createfunction.png" width="100%" />
</Frame>

Enter a name for your function and select `Node.js 14.x` as runtime. Click
`Create Function`.

Now you are on the function screen, scroll below to `Function Code` section. On
`Code entry type` selection, select `Upload a .zip file`. Upload the `app.zip`
file you have just created and click on the `Save` button on the top-right. You
need to see your code as below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/awslambda/functioncode.png" width="100%" />
</Frame>

Now you can test your code. Click on the `Test` button on the top right. Create
an event like the below:

```
{
  "key": "foo",
  "value": "bar"
}
```

Now, click on Test. You will see something like this:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/awslambda/success.png" width="100%" />
</Frame>

Congratulations, now your lambda function inserts entry to your Upstash
database.

**What can be the next?**

*   You can write and deploy another function to just get values from the
    database.
*   You can learn better ways to deploy your functions such as
    [serverless framework](https://serverless.com/) and
    [AWS SAM](https://aws.amazon.com/serverless/sam/)
*   You can integrate
    [API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html)
    so you can call your function via http.
*   You can learn about how to monitor your functions from CloudWatch as described
    [here](https://docs.aws.amazon.com/lambda/latest/dg//monitoring-functions-logs.html)
    .

#### Redis Connections in AWS Lambda

Although Redis connections are very lightweight, a new connection inside each
Lambda function can cause a notable latency. On the other hand, reusing Redis
connections inside the AWS Lambda functions has its own drawbacks. When AWS
scales out Lambda functions, the number of open connections can rapidly
increase. Fortunately, Upstash detects and terminates the idle and zombie
connections thanks to its smart connection handling algorithm. Since this
algorithm is used; we have been recommending caching your Redis connection in
serverless functions.

<Info>
  See [the blog post](https://blog.upstash.com/serverless-database-connections)
  about the database connections in serverless functions.
</Info>

Below is our findings about various Redis clients' behaviours when connection is
created, a single command is submitted and then connection is closed. **Note
that these commands (AUTH, INFO, PING, QUIT, COMMAND) are not billed.**

| Client                                                | #Commands |   Issued Commands  |
| ----------------------------------------------------- | :-------: | :----------------: |
| [redis-cli](https://redis.io/topics/rediscli)         |     2     |   AUTH - COMMAND   |
| [node-redis](https://github.com/NodeRedis/node-redis) |     3     | AUTH - INFO - QUIT |
| [ioredis](https://github.com/luin/ioredis)            |     3     | AUTH - INFO - QUIT |
| [redis-py](https://github.com/andymccurdy/redis-py)   |     1     |        AUTH        |
| [jedis](https://github.com/xetorthio/jedis)           |     2     |     AUTH - QUIT    |
| [lettuce](https://github.com/lettuce-io/lettuce-core) |     2     |     AUTH - QUIT    |
| [go-redis](https://github.com/go-redis/redis)         |     1     |        AUTH        |


# Get Started with Cloudflare Workers



This tutorial showcases using Redis with REST API in Cloudflare Workers. We will
write a sample edge function (Cloudflare Workers) which will show a custom
greeting depending on the location of the client. We will load the greeting
message from Redis so you can update it without touching the code.

See
[the code](https://github.com/upstash/examples/tree/master/examples/using-cloudflare-workers).

### Why Upstash?

*   Cloudflare Workers does not allow TCP connections. Upstash provides REST API
    on top of the Redis database.
*   Upstash is a serverless offering with per-request pricing which fits for edge
    and serverless functions.
*   Upstash Global database provides low latency all over the world.

### Step-1: Create Redis Database

Create a free Global database from
[Upstash Console](https://console.upstash.com). Find your REST URL and token in
the database details page in the console. Copy them.

Connect your database with redis-cli and add some greetings

```shell
usw1-selected-termite-30690.upstash.io:30690> set GB "Ey up?"
OK
usw1-selected-termite-30690.upstash.io:30690> set US "Yo, what’s up?"
OK
usw1-selected-termite-30690.upstash.io:30690> set TR "Naber dostum?"
OK
usw1-selected-termite-30690.upstash.io:30690> set DE "Was ist los?"
```

### Step-2: Edge Function

The best way to work with Cloudflare Workers is to use
[Wrangler](https://developers.cloudflare.com/workers/get-started/guide). After
installing and configuring wrangler, create a folder for your project inside the
folder run: `wrangler init`

Choose `yes` to create package.json, `no` to typescript and `yes` to create a
worker in src/index.js.

It will create `wrangler.toml`, `package.json` and `src/index.js`.

Append the Upstash REST URL and token to the toml as below:

```toml
# wrangler.toml

# existing config

[vars]
UPSTASH_REDIS_REST_TOKEN = "AX_sASQgODM5ZjExZGEtMmI3Mi00Mjcwk3NDIxMmEwNmNkYjVmOGVmZTk5MzQ="
UPSTASH_REDIS_REST_URL = "https://us1-merry-macaque-31458.upstash.io/"
```

Install upstash-redis: `npm install @upstash/redis`

Replace `src/index.js` with the following:

```javascript
// src/index.js

import { Redis } from "@upstash/redis/cloudflare";

export default {
  async fetch(request, env) {
    const redis = Redis.fromEnv(env);

    const country = request.headers.get("cf-ipcountry");
    if (country) {
      const greeting = await redis.get(country);
      if (greeting) {
        return new Response(greeting);
      }
    }

    return new Response("Hello!");
  },
};
```

The code tries to find out the user's location checking the "cf-ipcountry"
header. Then it loads the correct greeting for that location using the Redis
REST API.

## Run locally

Run `wrangler dev` and open your browser at
[localhost:8787](http://localhost:8787).

## Build and Deploy

Build and deploy your app to Cloudflare by running: `wrangler publish`

The url of your app will be logged:
[https://using-cloudflare-workers.upstash.workers.dev/](https://using-cloudflare-workers.upstash.workers.dev/)

## Typescript example

We also have a typescript example, available
[here](https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers-with-typescript).


# Get Started with Google Cloud Functions



### Prerequisites:

*   A GCP account for Google Cloud functions.
*   Install [Google Cloud SDK](https://cloud.google.com/sdk/docs/install).
*   An Upstash account for Serverless Redis.

### Step 1: Init the Project

*   Create a folder, then run `npm init` inside the folder.

### Step 2: Install a Redis Client

Our only dependency is redis client. Install go-redis via `npm install ioredis`

### Step 3: Create a Redis Database

Create a Redis database from Upstash console. **Select the GCP US-Central-1 as
the region.** Free tier should be enough. It is pretty straight forward but if
you need help, check [getting started](../overall/getstarted) guide. In the
database details page, click the Connect button. You will need the endpoint and
password in the next step.

### Step 4: The function Code

Create index.js as below:

```javascript
var Redis = require("ioredis");

if (typeof client === "undefined") {
  var client = new Redis("rediss://:YOUR_PASSWORD@YOUR_ENDPOINT:YOUR_PORT");
}

exports.helloGET = async (req, res) => {
  let count = await client.incr("counter");
  res.send("Page view:" + count);
};
```

<Snippet file="redis/ioredisnote.mdx" />

The code simply increments a counter in Redis database and returns its value in
json format.

### Step 5: Deployment

Now we are ready to deploy our API. Deploy via:

```shell
gcloud functions deploy helloGET \
--runtime nodejs14 --trigger-http --allow-unauthenticated
```

You will see the URL of your Cloud Function. Click to the URL to check if it is
working properly.

```shell
httpsTrigger:
securityLevel: SECURE_OPTIONAL
url: https://us-central1-functions-317005.cloudfunctions.net/helloGET
```

In case of an issue, you can check the logs of your Cloud Function in the GCP
console as below.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/examples/gcp-error.png" width="100%" />
</Frame>


# Import/Export Data



## Using Upstash Console

You can use the migration wizard in the
[Upstash console](https://console.upstash.com) to import your Redis to Upstash.
In the database list page, click on the `Import` button, you will see the dialog
like below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/import/import.png" width="60%" />
</Frame>

You can move your data from either an Upstash database or a database in another
provider (or on premise).

<Info>
  All the data will be deleted (flushed) in the destination database before the
  migration process starts.
</Info>

## Using upstash-redis-dump

You can also use the
[upstash-redis-dump](https://github.com/upstash/upstash-redis-dump) tool
import/export data from another Redis.

The below is an example how to dump and import data:

```shell
$ upstash-redis-dump -db 0 -host eu1-moving-loon-6379.upstash.io -port 6379 -pass PASSWORD -tls > redis.dump
Database 0: 9 keys dumped
```

See [upstash-redis-dump repo](https://github.com/upstash/upstash-redis-dump) for
more information.


# ioredis note



<Note>
  This example uses ioredis, you can copy the connection string from the `Node`
  tab in the console.
</Note>


# Use IP Allowlist



IP Allowlist can be used to restrict which IP addresses are permitted to access your database by comparing a connection's address with predefined CIDR blocks. This feature enhances database security by allowing connections only from specified IP addresses. For example if you have dedicated production servers with static IP addresses, enabling IP allowlist blocks connections from other addresses.

![allowlist](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/ipallowlist/ipallowlist.png)

## Enabling IP Allowlist

By default, any IP address can be used to connect to your database. You must add at least one IP range to enable the allowlist. You can manage added IP ranges in the `Configuration` section on the database details page. You can either provide

*   IPv4 address, e.g. `37.237.15.43`
*   CIDR block, e.g. `181.49.172.0/24`

<Info>
  Currently, IP Allowlist only supports IPv4 addresses.
</Info>

You can use more than one range to allow multiple clients. Meeting the criteria of just one is enough to establish a connection.

<Note>
  It may take a few minutes for changes to propagate.
</Note>


# Listen Keyspace Notifications



Upstash allows you to listen for keyspace notifications over pubsub channels to
receive events for changes over the keys.

For each event that occurs, two kinds of events are fired over the corresponding
pubsub channels:

*   A keyspace event that will use the pubsub channel for the key, possibly containing
    other events for the same key
*   A keyevent event that will use the pubsub channel for the event, possibly containing
    other events for the different keys

The channel names and their content are of the form:

*   `__keyspace@0__:keyname` channel with the values of the event names for the keyspace
    notifications
*   `__keyevent@0__:eventname` channel with the values of the key names for the keyevent
    notifications

## Enabling Notifications

By default, all keyspace and keyevent notifications are off. To enable it, you can use
the `CONFIG SET` command, and set the `notify-keyspace-events` options to one of the
appropriate flags described below.

<Warning>
  Each keyspace and keyevent notification fired might have an effect on the latency of the
  commands as the events are delivered to the listening clients and cluster members for
  multi-replica deployments. Therefore, it is advised to only enable the minimal subset of the
  notifications that are needed.
</Warning>

| Flag | Description                 |
| ---- | --------------------------- |
| K    | Keyspace events             |
| E    | Keyevent events             |
| g    | Generic command events      |
| \$   | String command events       |
| l    | List command events         |
| s    | Set command events          |
| h    | Hash command events         |
| z    | Sorted set command events   |
| t    | Stream command events       |
| d    | Module(JSON) command events |
| x    | Expiration events           |
| e    | Eviction events             |
| m    | Key miss events             |
| n    | New key events              |
| A    | Alias for g\$lshztxed       |

At least one of the `K` or `E` flags must be present in the option value.

For example, you can use the following command to receive keyspace notifications
only for the hash commands:

<Tabs>
  <Tab title="cURL">
    ```bash
    curl -X POST \
        -d '["CONFIG", "SET", "notify-keyspace-events", "Kh"]' \
        -H "Authorization: Bearer $UPSTASH_REDIS_REST_TOKEN" \
        $UPSTASH_REDIS_REST_URL
    ```
  </Tab>

  <Tab title="redis-cli">
    ```bash
    redis-cli --tls -u $UPSTASH_REDIS_CLI_URL config set notify-keyspace-events Kh
    ```
  </Tab>
</Tabs>

You can listen for all the channels using redis-cli to test the effect of the
above command:

```bash
redis-cli --tls -u $UPSTASH_REDIS_CLI_URL --csv psubscribe '__key*__:*'
```

### Disabling Notifications

You can reuse the `CONFIG SET` command and set `notify-keyspace-events` option to empty string
to disable all keyspace and keyevent notifications.

<Tabs>
  <Tab title="cURL">
    ```bash
    curl -X POST \
        -d '["CONFIG", "SET", "notify-keyspace-events", ""]' \
        -H "Authorization: Bearer $UPSTASH_REDIS_REST_TOKEN" \
        $UPSTASH_REDIS_REST_URL
    ```
  </Tab>

  <Tab title="redis-cli">
    ```bash
    redis-cli --tls -u $UPSTASH_REDIS_CLI_URL config set notify-keyspace-events ""
    ```
  </Tab>
</Tabs>

### Checking Notification Configuration

`CONFIG GET` command can be used the get the current value of the `notify-keyspace-events` option
to see the active keyspace and keyevent notifications configuration.

<Tabs>
  <Tab title="cURL">
    ```bash
    curl -X POST \
        -d '["CONFIG", "GET", "notify-keyspace-events"]' \
        -H "Authorization: Bearer $UPSTASH_REDIS_REST_TOKEN" \
        $UPSTASH_REDIS_REST_URL
    ```
  </Tab>

  <Tab title="redis-cli">
    ```bash
    redis-cli --tls -u $UPSTASH_REDIS_CLI_URL config get notify-keyspace-events
    ```
  </Tab>
</Tabs>


# Metrics and Charts



There are many metrics and charts in Upstash console. In this document, we will
explain what each of these charts refers to. There are two pages where you can
see charts and metrics:

## Database List

The charts on this page give aggregated and total information about the database
and your usage.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/metrics/databaselist.png" width="100%" />
</Frame>

In this chart, all your databases are listed. You can click on the name of the
database that you want to see detailed information. Also, the following
information is listed for each database:

*   The region of the database
*   The current size of the data
*   The current count of active connections: Not that if your connections are
    short-lived then you may see 0 here most of the time.

## Database Detail

The charts on this page show metrics that are specific to the selected database.

### Current Month

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/metrics/current-month.png" width="100%" />
</Frame>

*   This chart shows the daily cost of the database. The chart covers the last 5
    days.

### Daily Request

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/metrics/requests5days.png" width="100%" />
</Frame>

This chart shows the daily total number of requests to the database. The chart
covers the last 5 days.

### Throughput

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/metrics/throughput.png" width="100%" />
</Frame>

Throughput chart shows throughput values for reads, writes and commands (all
commands including reads and writes) per second. The chart covers the last 1
hour and it is updated every 10 seconds.

### Service Time Latency

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/metrics/latency.png" width="100%" />
</Frame>

This chart shows the processing time of the request between it is received by
the server and the response is sent to the caller. It shows the times in max,
mean, min, 99.9 percentile and 99.99 percentile. The chart covers the last 1
hour and it is updated every 10 seconds.

### Data Size

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/metrics/datasize.png" width="100%" />
</Frame>

This chart shows the data size of your database. The chart covers the last 24
hours and it is updated every 10 seconds.

### Connections

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/metrics/connections.png" width="100%" />
</Frame>

This chart shows the number of active client connections. It shows the number of
open connections plus the number of short-lived connections that started and
terminated in 10 seconds period. The chart covers the last 1 hour and it is
updated every 10 seconds.

### Key Space

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/metrics/keyspace.png" width="100%" />
</Frame>

This chart shows the number of keys. The chart covers the last 24 hours and it
is updated every 10 seconds.

### Hits / Misses

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/metrics/hitsmisses.png" width="100%" />
</Frame>

This chart shows the number of hits per second and misses per second. The chart
covers the last 1 hour and it is updated every 10 seconds.


# Monitor your usage



We support the Redis `MONITOR` command, a debugging command that allows you to see all requests processed by your Redis instance in real-time.

## Monitoring Your Usage - Video Guide

In this video, we'll walk through setting up a monitor instance step-by-step.

<Frame>
  <iframe width="560" height="315" src="https://www.youtube.com/embed/tWIm396WAkI?rel=0&disablekb=1" title="YouTube video player" frameborder="0" allow="accelerometer; fullscreen; clipboard-write; encrypted-media; gyroscope" allowfullscreen />
</Frame>

<Note>
  The `MONITOR`command expects a persistent connection and, therefore, does not work over HTTP.
</Note>

In this video, we use `ioredis` to connect to our Upstash Redis database. Using an event handler, we can define what should happen for each executed command against on Redis instance. For example, logging all data to the console.

<RequestExample>
  ```ts Example
  const monitor = await redis.monitor()

  monitor.on("monitor", (time, args, source, database) => {
    console.log(time, args, source, database)
  })
  ```
</RequestExample>


# Read Your Writes



The "Read Your Writes" feature in Upstash Redis ensures that write operations are completed before subsequent read operations occur, maintaining data consistency in your application.

### How It Works

All write operations happen on the primary member and take time to propagate to the read replicas. Imagine that a client attempts to read an item immediately after it’s written. The read may go to a replica that hasn’t synced with the primary yet, resulting in stale data being returned.

RYW consistency solves this by returning a **sync token** after each request, which indicates the primary member’s state. In the next request, this sync token ensures the read replica syncs up to that token before serving the read.

So, the sync token acts as a checkpoint, ensuring that any read operations following a write reflect the most recent changes, even if they are served by a read replica.

Management of the sync token is handled automatically by the official [Typescript (version 1.34.0 and later)](/redis/sdks/ts/overview) and [Python (version 1.2.0 and later)](/redis/sdks/py/overview) SDKs of Upstash. When you initialize a Redis client with these SDKs, the writes made by that client will be respected during subsequent reads from the same client.

For REST users, you can achieve similar behavior by using the `upstash-sync-token` header. Each time you make a request, save the value of the `upstash-sync-token` header from the response and pass it in the `upstash-sync-token` header of your next request. This ensures that subsequent reads reflect the writes.

### Cross-Client Synchronization

Imagine that you are writing some key to Redis and then you read the same key from a different Redis client instance. In this case, the second client’s read request may not reflect the write made by the first client, as the sync tokens are updated independently in the two clients.

Consider these two example functions, each representing separate API endpoints:

```ts
export const writeRequest = async () => {
  const redis = Redis.fromEnv();
  const randomKey = nanoid();
  await redis.set(randomKey, "value");
  return randomKey;
};

export const readRequest = async (randomKey: string) => {
  const redis = Redis.fromEnv();
  const value = await redis.get(randomKey);
  return value;
};
```

If these functions are called in sequence, they will create two separate clients:

```ts
const randomKey = await writeRequest();
await readRequest(randomKey);
```

As explained above, in rare cases, one of your [read replicas](/redis/features/globaldatabase#primary-region-and-read-regions) can serve the `read` request before it receives the `write` update from the primary replica. To avoid this, if you are using `@upstash/redis` version 1.34.1 or later, you can pass the `readYourWritesSyncToken` from the first client to the second:

```ts
export const writeRequest = async () => {
  const redis = Redis.fromEnv();
  const randomKey = nanoid();
  await redis.set(randomKey, "value");

  // Get the token **after** making the write
  const token = redis.readYourWritesSyncToken;
  return { randomKey, token };
};

export const readRequest = async (
  randomKey: string,
  token: string | undefined
) => {
  const redis = Redis.fromEnv();

  // Set the token **before** making the read
  redis.readYourWritesSyncToken = token;

  const value = await redis.get(randomKey);
  return value;
};

const { randomKey, token } = await writeRequest();
await readRequest(randomKey, token);
```

Remember to get the sync token after the write request is completed, as the session token changes with each request.

For REST users or the Upstash Python SDK, a similar approach can be used. In Python, use `Redis._sync_token` instead of `readYourWritesSyncToken`.


# Terraform Provider



You can use Upstash terraform provider to create your resources. API key is
required in order to create resources.

### Configure Provider

Provider requires your email address and api key which can be created in
console.

```
provider "upstash" {
  email = ""
  api_key = ""
}
```

### Create Database

As input you need to give database name, region and type.

```
resource "upstash_database" "mydb" {
  database_name = "testdblstr"
  region = "eu-west-1"
  type = "free"
}
```

You can output database credentials as following

```
output "endpoint" {
  value = "${upstash_database.mydb.endpoint}"
}

output "port" {
  value = "${upstash_database.mydb.port}"
}
output "password" {
  value = "${upstash_database.mydb.password}"
}
```

See our
[Terraform Provider Github Repository](https://github.com/upstash/terraform-provider-upstash)
for details and examples about Upstash Terraform Provider.


# Upgrade Your Database



Free tier has followings restrictions:

*   Max 10.000 requests per day
*   Max 256 MB data size
*   Max 1 database

If you think your database is close to reaching any of these limits, we
recommend you to upgrade to pay-as-you-go plan which includes:

*   No limit on requests per day
*   Data size up to 10 GB

To upgrade your database, you need to have a payment method. You can add a
payment method as described [here](/common/account/addapaymentmethod). After you add
a payment method, Upstash restarts your database and your new database starts
with the pay-as-you-go plan.

See the [quotas](https://upstash.com/pricing/redis) for limits of the
pay-as-you-go plan. If you think, your use case will exceed those quotas,
contact us ([support@upstash.com](mailto:support@upstash.com)) for our [Enterprise Plan](#enterprise-plan)
where you can customize the limits.

<Note>
  During the upgrade process, you will not lose any data but your database will
  experience a downtime about 1-2 seconds. Your existing clients will be
  disconnected. So it is recommended to upgrade your database when there is the
  least activity.
</Note>


# Vercel - Upstash Redis Integration



If you are using [Vercel](https://vercel.com/) then you can integrate Upstash
Redis, Kafka or QStash to your project easily. Upstash is the perfect serverless
solution for your applications thanks to its:

*   Low latency data
*   Per request pricing
*   Durable storage
*   Ease of use

Below are the steps of the integration.

### Add Integration to Your Vercel Account

Visit the [Upstash Integration](https://vercel.com/integrations/upstash) on
Vercel and click the `Add Integration` button. Choose your scope and application
that you want to integrate.

### Select Your Project

Vercel will redirect you to Upstash, where you can select your Vercel project
and Upstash resources that you want to integrate.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel/integration_init.png" width="520" />
</Frame>

If you do not have a Redis database or Kafka cluster yet, you can create one
from the dropdown menu.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel/integration_redis_create.png" width="520" />
</Frame>

Once you have selected all resources, click the `Create` button at the bottom of
the page.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel/integration_filled.png" width="520" />
</Frame>

After all environment variables are created, you will be forwarded to Vercel. Go
to your project settings where you can see all added environment variables.
Depending on the resources you selected, you will see something similar to this:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel/integration_env.png" width="520" />
</Frame>

You need to redeploy your app for the environment variable to be used.

### Create Your Redis Client

If you completed the integration steps above and redeploy your app, the added
environment variables will be accessible inside your Vercel application. You can
now use them in your clients to connect

#### Redis

```ts
const { Redis } = require("@upstash/redis");

module.exports = async (req, res) => {
  /**
   * Redis.fromEnv() will read the following from environment variables:
   * - UPSTASH_REDIS_REST_URL
   * - UPSTASH_REDIS_REST_TOKEN
   */
  const redis = Redis.fromEnv();
  await redis.set("foo", "bar");
  const bar = await redis.get("foo");

  res.json({
    body: `foo: ${bar}`,
  });
};
```

#### Kafka

```ts
import { Kafka } from "@upstash/kafka";

const kafka = new Kafka({
  url: process.env.UPSTASH_KAFKA_REST_URL,
  username: process.env.UPSTASH_KAFKA_REST_USERNAME,
  password: process.env.UPSTASH_KAFKA_REST_PASSWORD,
});

await kafka.producer().produce("my-topic", "my-message");
```

#### QStash

**Client**

```ts
import { Client } from "@upstash/qstash";

const c = new Client({
  token: process.env.QSTASH_TOKEN,
});

const res = await c.publishJSON({
  url: "https://my-api...",
  body: {
    hello: "world",
  },
});
```

**Receiver**

```ts
import { Receiver } from "@upstash/qstash";

const r = new Receiver({
  currentSigningKey: process.env.QSTASH_CURRENT_SIGNING_KEY,
  nextSigningKey: process.env.QSTASH_NEXT_SIGNING_KEY,
});

const isValid = await r.verify(
  signature: "..."
  body: "..."
})
```

### Managing your Integrations

The [Integration Dashboard](https://console.upstash.com/integration/vercel)
allows you to see all your integrations, link new projects or manage existing
ones.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vercel/integration_dashboard.png" />
</Frame>

### Support

If you have any issue you can ask in our
[Discord server](https://discord.gg/w9SenAtbme) or send email at
[support@upstash.com](mailto:support@upstash.com)


# BullMQ with Upstash Redis



You can use BullMQ and Bull with Upstash Redis. BullMQ is a Node.js queue library that is built on top of Bull. It is a Redis-based queue library so you can use Upstash Redis as its storage.

## Install

```bash
npm install bullmq upstash-redis
```

## Usage

```javascript
import { Queue } from 'bullmq';

const myQueue = new Queue('foo', { connection: {
        host: "UPSTASH_REDIS_ENDPOINT",
        port: 6379,
        password: "UPSTASH_REDIS_PASSWORD",
        tls: {}
    }});

async function addJobs() {
    await myQueue.add('myJobName', { foo: 'bar' });
    await myQueue.add('myJobName', { qux: 'baz' });
}

await addJobs();
```

## Billing Optimization

BullMQ accesses Redis regularly, even when there is no queue activity. This can incur extra costs because Upstash charges per request. With feedbacks of BullMQ users, we have revised our billing algorithm. Now, Upstash detects BullMQ usage and charges only for script calls instead of each Redis operation for BullMQ operations. This will avoid increased command count and high costs due to BullMQ library.

Once your billing is in optimized mode, you will see a leaf icon in the console like below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/eco-icon.png" />
</Frame>

**This mechanism is enabled for all tiers except free databases.**


# Prometheus - Upstash Redis Integration



To monitor your Upstash database in Prometheus and visualize metrics in Grafana, follow these steps:

<Check>
  **Integration Scope**

  Upstash Prometheus Integration only covers Pro databases or those included in the Enterprise Plan.
</Check>

## **Step 1: Log in to Your Upstash Account**

1.  Open your web browser and navigate to [Upstash](https://console.upstash.com/).
2.  Navigate to the main dashboard, where you’ll see a list of your databases.

## **Step 2: Select your Database**

1.  Select the database you want to integrate with Prometheus.
2.  This will open the database settings, where you can manage various configuration options for your selected database.

![configuration.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/prometheus/configuration.png)

3.  Enable Prometheus by toggling the switch. This allows you to monitor metrics related to your Upstash database performance, usage, and other key metrics.

## **Step 3: Connect Accounts**

1.  After enabling Prometheus, a monitoring token is generated and displayed.

2.  Copy this token. This token is unique to your database and is required to authenticate Prometheus with the Upstash metrics endpoint.

<Check>
  **Headert Format**

  You should add monitoring token according to this format `Bearer <MONITORING_TOKEN>`
</Check>

![monitoring-token.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/prometheus/monitoring-token.png)

## **Step 4: Configure Prometheus to Scrape Upstash Metrics**

1.  Open your Grafana instance, navigate to the Data Sources section, and select Prometheus as the data source.

![datasource.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/prometheus/datasource.png)

2.  Enter the data source name, set `https://api.upstash.com/monitoring/prometheus` as the data source address, and then add your monitoring token in the HTTP Header section.

![headers.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/prometheus/headers.png)

3.  Then, click <b>Test and Save</b> to verify that the data source is working properly.

![datasource-final.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/prometheus/datasource-final.png)

## **Step 5: Wait for Metrics Availability**

To visualize your Upstash metrics, you can use a pre-built Grafana dashboard.

Select your Prometheus data source when prompted, and complete the import.

Please check this address to access Upstash Grafana Dashboard <a href="https://grafana.com/grafana/dashboards/22257-upstash-redis-dashboard/"> Dashboard </a>

![grafana-dashboard.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/prometheus/grafana-dashboard.png)

## **Conclusion**

You've now integrated your database with Upstash Prometheus, providing access to improved monitoring and analytics.

Feel free to explore Upstash's features and dashboards to gain deeper insights into your system's performance.

If you encounter any issues or have questions, please refer to the Upstash support documentation or contact our support team for assistance.


# Configure Upstash Ratelimit Strapi Plugin



After setting up the plugin, it's possible to customize the ratelimiter algorithm and rates. You can also define different rate limits and rate limit algorithms for different routes.

## General Configurations

<ParamField path="enabled" type="boolean" default="true">
  Enable or disable the plugin.
</ParamField>

## Database Configurations

<ParamField path="token" type="string" required>
  The token to authenticate with the Upstash Redis REST API. You can find this
  credential on Upstash Console with the name `UPSTASH_REDIS_REST_TOKEN`
</ParamField>

<ParamField path="url" type="string" required>
  The URL for the Upstash Redis REST API. You can find this credential on
  Upstash Console with the name `UPSTASH_REDIS_REST_URL`
</ParamField>

<ParamField path="prefix" type="string" default="@strapi">
  The prefix for the rate limit keys. The plugin uses this prefix to store the
  rate limit data in Redis. <br />
  For example, if the prefix is `@strapi`, the key will be
  `@strapi:<method>:<route>:<identifier>`.
</ParamField>

<ParamField path="analytics" type="boolean" default="false">
  Enable analytics for the rate limit. When enabled, the plugin extra insights
  related to your ratelimits. You can use this data to analyze the rate limit
  usage on [Upstash Console](https://console.upstash.com/ratelimit).
</ParamField>

## Strategy

The plugin uses a strategy array to define the rate limits per route. Each strategy object has the following properties:

<ParamField path="methods" type="('GET' | 'POST' | 'DELETE' | 'PUT' | 'PATCH' |'ALL')[]" required>
  An array of HTTP methods to apply the rate limit. <br />
  For example, `["GET", "POST"]`
</ParamField>

<ParamField path="path" type="string" required>
  The path to apply the rate limit. You can use wildcards to match multiple
  routes. For example, `*` matches all routes. <br />
  Some examples: <br />

  *   `path: "/api/restaurants/:id"` <br />
  *   `path: "/api/restaurants"` <br />
</ParamField>

<ParamField path="identifierSource" type="string" required>
  The source to identifiy the user. Requests with the same identifier will be
  rate limited under the same limit. <br />
  Available sources are: <br />

  *   `ip`: The IP address of the user. <br />
  *   `header`: The value of a header key. You should pass the source in the `header.<HEADER_KEY>` format. <br />
      For example, `header.Authorization` will use the value of the `Authorization`
</ParamField>

<ParamField path="debug" type="string">
  Enable debug mode for the route. When enabled, the plugin logs the remaining
  limits and the block status for each request. <br />
</ParamField>

<ParamField path="limiter" type="object" required>
  The limiter configuration for the route. The limiter object has the following
  properties:

  <Card>
    <ParamField path="algorithm" type="'fixed-window' | 'sliding-window' | 'token-bucket'" required>
      The rate limit algorithm to use. For more information related to algorithms, see docs [**here**](https://upstash.com/docs/redis/sdks/ratelimit-ts/algorithms). <br />

      *   `fixed-window`: The fixed-window algorithm divides time into fixed intervals. Each interval has a set limit of allowed requests. When a new interval starts, the count resets. <br />
      *   `sliding-window`:
          The sliding-window algorithm uses a rolling time frame. It considers requests from the past X time units, continuously moving forward. This provides a smoother distribution of requests over time. <br />
      *   `token-bucket`: The token-bucket algorithm uses a bucket that fills with tokens at a steady rate. Each request consumes a token. If the bucket is empty, requests are denied. This allows for bursts of traffic while maintaining a long-term rate limit.<br />
    </ParamField>

    <ParamField path="tokens" type="number" required>
      The number of tokens allowed in the time window. <br />
    </ParamField>

    <ParamField path="window" type="string" required>
      The time window for the rate limit. Available units are `"ms" | "s" | "m" | "h" | "d"` <br />
      For example, `20s` means 20 seconds.
    </ParamField>

    <ParamField path="refillRate" type="number">
      The rate at which the bucket refills. **This property is only used for the token-bucket algorithm.** <br />
    </ParamField>
  </Card>
</ParamField>

## Examples

<CodeGroup>
  ```json Apply rate limit for all routes
  {
     "strapi-plugin-upstash-ratelimit":{
        "enabled":true,
        "resolve":"./src/plugins/strapi-plugin-upstash-ratelimit",
        "config":{
           "enabled":true,
           "token":"process.env.UPSTASH_REDIS_REST_TOKEN",
           "url":"process.env.UPSTASH_REDIS_REST_URL",
           "strategy":[
              {
                 "methods":[
                    "GET",
                    "POST"
                 ],
                 "path":"*",
                 "identifierSource":"header.Authorization",
                 "limiter":{
                    "algorithm":"fixed-window",
                    "tokens":10,
                    "window":"20s"
                 }
              }
           ],
           "prefix":"@strapi"
        }
     }
  }
  ```

  ```json Apply rate limit with IP
  {
    "strapi-plugin-upstash-ratelimit": {
      "enabled": true,
      "resolve": "./src/plugins/strapi-plugin-upstash-ratelimit",
      "config": {
        "enabled": true,
        "token": "process.env.UPSTASH_REDIS_REST_TOKEN",
        "url": "process.env.UPSTASH_REDIS_REST_URL",
        "strategy": [
          {
            "methods": ["GET", "POST"],
            "path": "*",
            "identifierSource": "ip",
            "limiter": {
              "algorithm": "fixed-window",
              "tokens": 10,
              "window": "20s"
            }
          }
        ],
        "prefix": "@strapi"
      }
    }
  }
  ```

  ```json Routes with different rate limit algorithms
  {
    "strapi-plugin-upstash-ratelimit": {
      "enabled": true,
      "resolve": "./src/plugins/strapi-plugin-upstash-ratelimit",
      "config": {
        "enabled": true,
        "token": "process.env.UPSTASH_REDIS_REST_TOKEN",
        "url": "process.env.UPSTASH_REDIS_REST_URL",
        "strategy": [
          {
            "methods": ["GET", "POST"],
            "path": "/api/restaurants/:id",
            "identifierSource": "header.x-author",
            "limiter": {
              "algorithm": "fixed-window",
              "tokens": 10,
              "window": "20s"
            }
          },
          {
            "methods": ["GET"],
            "path": "/api/restaurants",
            "identifierSource": "header.x-author",
            "limiter": {
              "algorithm": "tokenBucket",
              "tokens": 10,
              "window": "20s",
              "refillRate": 1
            }
          }
        ],
        "prefix": "@strapi"
      }
    }
  }
  ```
</CodeGroup>


# Upstash Ratelimit Strapi Integration



Strapi is an open-source, Node.js based, Headless CMS that saves developers a lot of development time, enabling them to build their application backends quickly by decreasing the lines of code necessary.

You can use Upstash's HTTP and Redis based [Ratelimit package](https://github.com/upstash/ratelimit-js) integration with Strapi to protect your APIs from abuse.

## Getting started

### Installation

<CodeGroup>
  ```bash npm
  npm install --save @upstash/strapi-plugin-upstash-ratelimit
  ```

  ```bash yarn
  yarn add @upstash/strapi-plugin-upstash-ratelimit
  ```
</CodeGroup>

### Create database

Create a new redis database on [Upstash Console](https://console.upstash.com/). See [related docs](https://upstash.com/docs/redis/overall/getstarted) for further info related to creating a database.

### Set up environment variables

Get the environment variables from [Upstash Console](https://console.upstash.com/), and set it to `.env` file as below:

```shell .env
UPSTASH_REDIS_REST_TOKEN="<YOUR_TOKEN>"
UPSTASH_REDIS_REST_URL="<YOUR_URL>"
```

### Configure the plugin

You can use

<CodeGroup>
  ```typescript /config/plugins.ts
  export default () => ({
    "strapi-plugin-upstash-ratelimit": {
      enabled: true,
      resolve: "./src/plugins/strapi-plugin-upstash-ratelimit",
      config: {
        enabled: true,
        token: process.env.UPSTASH_REDIS_REST_TOKEN,
        url: process.env.UPSTASH_REDIS_REST_URL,
        strategy: [
          {
            methods: ["GET", "POST"],
            path: "*",
            limiter: {
              algorithm: "fixed-window",
              tokens: 10,
              window: "20s",
            },
          },
        ],
        prefix: "@strapi",
      },
    },
  });
  ```

  ```javascript /config/plugins.js
  module.exports = () => ({
    "strapi-plugin-upstash-ratelimit": {
      enabled: true,
      resolve: "./src/plugins/strapi-plugin-upstash-ratelimit",
      config: {
        enabled: true,
        token: process.env.UPSTASH_REDIS_REST_TOKEN,
        url: process.env.UPSTASH_REDIS_REST_URL,
        strategy: [
          {
            methods: ["GET", "POST"],
            path: "*",
            limiter: {
              algorithm: "fixed-window",
              tokens: 10,
              window: "20s",
            },
          },
        ],
        prefix: "@strapi",
      },
    },
  });
  ```
</CodeGroup>


# Sidekiq with Upstash Redis



You can use Sidekiq with Upstash Redis. Sidekiq is a Ruby based queue library with a Redis-based queue storage so you can use with Upstash Redis.

## Example Application

```bash
bundle init 
bundle add sidekiq
```

```python
require "sidekiq"
require "sidekiq/api"
 
connection_url = ENV['UPSTASH_REDIS_LINK']
 
Sidekiq.configure_client do |config|
    config.redis = {url: connection_url}
end
 
Sidekiq.configure_server do |config|
    config.redis = {url: connection_url}
end
 
class EmailService
    include Sidekiq::Worker
    def perform(id, type)
        # Logic goes here. Let's assume sending email by printing to console.
        puts "Emailed to: " +  id + ": " + "'Congrats on " + type + " plan.'"
    end
end
 
def updateEmail(id, newType)
    jobFound = false
 
    a = Sidekiq::ScheduledSet.new
    a.each do |job|
        if job.args[0] == id
            job.delete
            jobFound = true
        end
    end
 
    if jobFound
        EmailService.perform_async(id, ("starting using our service and upgrading it to " + newType))
    else
        EmailService.perform_async(id, ("upgrading to " + newType))
    end
end
 
def sendEmail(id, type)
    case type
    when "free"
        # if free, delay for 10 seconds.
        EmailService.perform_in("10", id, "free")
    when "paid"
        # if paid, delay for 5 seconds.
        EmailService.perform_in("5", id, "paid")
    when "enterprise"
        # if enterprise, immediately queue.
        EmailService.perform_async(id, "enterprise")
    when "enterprise10k"
        EmailService.perform_async(id, "enterprise10k")
    else
        puts "Only plans are: `free`, `paid` and `enterprise`"
    end
end
 
def clearSchedules()
    Sidekiq::ScheduledSet.new.clear
    Sidekiq::Queue.new.clear
end
```

## Billing Optimization

Sidekiq accesses Redis regularly, even when there is no queue activity. This can incur extra costs because Upstash charges per request. With feedbacks of Sidekiq users, we have revised our billing algorithm. Now, Upstash detects Sidekiq usage and charges only for script calls instead of each Redis operation for Sidekiq operations. This will avoid increased command count and high costs due to Sidekiq library.
We have decided to charge only for script calls instead of all requests. This special pricing is for Sidekiq users. If you use Sidekiq with Upstash Redis, you will only be charged for script calls, not regular requests. This will significantly reduce your costs.

Once your billing is in optimized mode, you will see a leaf icon in the console like below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/eco-icon.png" />
</Frame>

**This mechanism is enabled for all tiers except free databases.**


# Changelog



<Update label="July 2024">
  *   Added REST API support for [`MONITOR`](https://redis.io/docs/latest/commands/monitor/) and [`SUBSCRIBE`](https://redis.io/docs/latest/commands/subscribe/)
      commands using [SSE](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events).
      See [Monitor](../features/restapi#monitor-command) and [Subscribe](../features/restapi#subscribe-command) docs.
  *   Added [`JSON.MSET`](https://redis.io/docs/latest/commands/json.mset/) and [`JSON.MERGE`](https://redis.io/docs/latest/commands/json.merge/) commands
  *   Introduced the `IP Allowlist` feature for enhanced security on newly created databases. By default, all IP addresses will be allowed.
      However, access can be restricted by specifying permitted IP addresses or CIDR ranges.
</Update>

<Update label="June 2024">
  *   Added AWS AP-NorthEast-1 Japan region.
  *   Added an option to return REST response in [`RESP2`](https://redis.io/docs/latest/develop/reference/protocol-spec/) format instead of `JSON`.
      See [REST API docs](https://upstash.com/docs/redis/features/restapi#resp2-format-responses) for more information.
</Update>

<Update label="April 2024">
  *   Implemented [`MONITOR`](https://redis.io/docs/latest/commands/monitor/) command
  *   Implemented Redis [keyspace notifications](https://upstash.com/docs/redis/howto/keyspacenotifications)
  *   Implemented [`WAIT`](https://redis.io/docs/latest/commands/wait/) and [`WAITAOF`](https://redis.io/docs/latest/commands/waitaof/) commands
  *   Added `lag` field to [`XINFO GROUPS`](https://redis.io/docs/latest/commands/xinfo-groups/)
  *   Added [`CLIENT ID`](https://redis.io/docs/latest/commands/client-id/) subcommand
  *   Added password strength check to [`ACL SETUSER`](https://redis.io/docs/latest/commands/acl-setuser/) command
</Update>

<Update label="February 2024">
  *   Fixed JSON commands with empty keys
  *   Fixed a panic on `XTRIM` and `XDEL`
  *   Added `CLIENT SETNAME/NAME/LIST` subcommands
  *   Implemented near exact trim for streams
</Update>

<Update label="September 2023">
  *   Implemented some missing Redis commands:
      *   `DUMP`
      *   `RESTORE`
      *   `ZMPOP`
      *   `BZMPOP`
      *   `LMPOP`
      *   `BLMPOP`
      *   `SINTERCARD`
  *   Added support for `BIT/BYTE` flag to `BITPOS` and `BITCOUNT` commands
  *   Added support for `XX`, `NX`, `GT`, and `LT` arguments to `EXPIRE` commands
  *   Allowed `NX` and `GET` args to be used together in `SET` command
</Update>


# Compare



In this section, we will compare Upstash with alternative cloud based solutions.

## AWS ElastiCache

*   **Serverless Pricing:** Elasticache does not have a serverless pricing model.
    The price does not scale to zero. You need to pay for the instances even when
    you do not use them. Upstash charges per request.
*   **REST API:** Unlike ElastiCache, Upstash has a built-in REST API, so you can
    access from environments where TCP connections are not allowed such as edge
    functions at Cloudflare Workers.
*   **Access:** Elasticache is designed to be used inside AWS VPC. You can access
    Upstash from anywhere.
*   **Durability:** Upstash persists your data to the block storage in addition to
    memory so you can use it as your primary database.

## AWS MemoryDB

*   **Serverless Pricing:** Similar to Elasticache, MemoryDB does not offer a
    serverless pricing model. The pricing does not scale down to zero, and even
    the most affordable instance costs over \$200 per month. This means you are
    required to pay for the instances regardless of usage. In contrast, Upstash
    follows a different approach by charging per request. With Upstash, you only
    incur charges when actively using your Redis database, ensuring that you do
    not have to pay when it's not in use.

*   **REST API:** Unlike MemoryDB, Upstash has a built-in REST API, so you can
    access from environments where TCP connections are not allowed such as edge
    functions at Cloudflare Workers.

*   **Access:** MemoryDB is designed to be used inside AWS VPC. You can access
    Upstash from anywhere.

## Redis Labs

*   **Serverless Pricing:** Redis Labs does not have a serverless pricing model
    either. The price does not scale to zero. You need to pay for the instances
    even when you do not use them. Upstash charges per request, so you only pay
    for your real usage.
*   **REST API:** Unlike Redis Labs, Upstash has a built-in REST API, so you can
    access from environments where TCP connections are not allowed such as edge
    functions at Cloudflare Workers.
*   **Durability:** Upstash persists your data to the block storage instantly in
    addition to the memory, so you can use it as your primary database.

## AWS DynamoDB

*   **Latency:** DynamoDB is a disk based data storage. Both write and read
    latency are much higher than Redis. Check our
    [benchmark app](https://serverless-battleground.vercel.app/) to get an idea.

*   **Complex Pricing:** Initially, DynamoDB may appear cost-effective, but if you
    begin utilizing advanced features such as DAX or Global Tables, you might
    encounter unexpected expenses on your AWS bill. In contrast, Upstash offers a
    more transparent pricing policy, ensuring that you are not taken by surprise.
    With Upstash, there are limits in place to cap your maximum costs, providing
    clarity and preventing any unwelcome surprises in your billing.

*   **Portability:** DynamoDB is exclusive to AWS and cannot be used outside of
    the AWS platform. However, Redis is supported by numerous cloud providers and
    can also be self-hosted. Upstash provides compatibility with Redis, ensuring
    vendor neutrality.

*   **Testability:** Running a local Redis for testing purposes is much easier
    than running a local DynamoDB. Check
    [this](https://stackoverflow.com/questions/26901613/easier-dynamodb-local-testing).

## FaunaDB

*   **Latency:** FaunaDB is a globally consistent database. Consistency at global
    level comes with performance cost. Check our
    [benchmark app](https://serverless-battleground.vercel.app/) to get an idea.

*   **Complex Pricing:** FaunaDB has a complicated pricing. It has 6 different
    dimensions to calculate the price. Check
    [this article](https://docs.fauna.com/fauna/current/concepts/billing) where
    the pricing is explained. If your use case is write heavy and if your requests
    have bigger payloads, then it can become expensive very easily. On the other
    hand, Upstash has different options for different needs and pricing is simple
    for all options. You pay per request in addition to storage cost which is
    generally much smaller amount.

*   **Portability:** FaunaDB is only supported by Fauna Inc. On the other hand,
    you can use Redis almost in all cloud providers as well as you can host Redis
    yourself. Upstash does not lock you to any vendor.

*   **Testability:** Running a local Redis for testing purposes is much easier
    than running a local FaunaDB. Check
    [this](https://dev.to/englishcraig/how-to-set-up-faunadb-for-local-development-5ha7).

## What makes Upstash different?

You have a new project and you do not know how many requests will it receive?
You love the performance and simplicity of Redis. But all Redis Cloud services
charge you per instance or per GB of memory. But maybe your application will not
receive big traffic at first, then why will you pay the full amount?
Unfortunately none of the current Redis cloud products provides a real
`pay-per-use` pricing model.

Let's do a simple calculation. Say I have a 1GB Redis database and I receive 1
million requests per month. For ElastiCache (cache.t3.small, \$0.034 hourly) this
costs at least \$24 not including data transfer and storage cost. For RedisLabs,
the 1GB plan costs \$22 per month. For Upstash the price is \$0.2 per 100k
requests. For 1 million, it is \$2 plus the storage cost that is \$0.25. So for
1GB, 1M request per months, ElastiCache is \$24, RedisLabs is \$22, Upstash is
\$2.25.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/compare/comparecosts.png" />
</Frame>

**What if your product becomes popular and starts to gain high and steady
traffic?**

Most of the serverless products start to lose their spell if the service
receives steady and high traffic as it starts to cost higher than
server/instance based pricing models. To overcome this situation we give you
option to purchase Pro Plan. In Pro plan you can set fixed
price per month with a restriction on max throughput and data size. For high and
steady throughput use cases, enterprise databases cost less than serverless one.
The good thing is you can start your database with pay-as-you-go pricing and
move it enterprise when you want. See [enterprise plans](/redis/overall/enterprise) for more
information.

Even if you choose not to upgrade to the Pro plan, Upstash guarantees
transparent billing without any unexpected surprises. Each Upstash database has
a predefined monthly maximum price, known as the "Ceiling Price." For
pay-as-you-go (PAYG) databases, this ceiling price is set at \$160 per month.
Therefore, even if your application experiences a significant surge in traffic,
such as reaching the front page of HackerNews, your Upstash database will not
exceed a maximum cost of \$160 per month.


# Pro / Enterprise Plans



<Snippet file="redis/enterprise.mdx" />


# Getting Started

Create a Redis compatible database in seconds

<Check>
  **Prerequisite**

  You need an account before creating a database, create one
  [here](https://console.upstash.com).
</Check>

## Create a Database

Once you logged in, you can create a database by clicking on the plus sign at
the top right corner.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/getting_started/create-global.png" />
</Frame>

**Database Name:** Type a name for your database.

**Primary Region and Read Regions:** Choose the regions for your database. For optimal performance, select the Primary region closest to where most of your writes will happen. Select Read Region(s) where most of your reads will happen. We have plans to expand support to additional regions and cloud providers. Feel free to send your requests to [feedback@upstash.com](mailto:feedback@upstash.com) to help us prioritize.

Once you click on Create button, you should see your database up and running as
below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/getting_started/global-db-welcome.png" width="100%" />
</Frame>

## Connect to Your Database

You can connect to your database with any Redis client. For simplicity, we will
use `redis-cli` tool in this guide but please refer to
[Connect Your Client](../howto/connectclient) section for more about connecting
via Redis clients.

`redis-cli` comes packaged with the official Redis distribution. If you do not
have Redis installed, you can get it as described at

<a href="https://redis.io/topics/quickstart" target="_blank">
  {" "}

  Redis Quick Start
</a>

document. Now, you can connect to and run commands on your database as below:

```
> redis-cli --tls -a PASSWORD -h ENDPOINT -p PORT
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
ENDPOINT:PORT> set counter 0
OK
ENDPOINT:PORT> get counter
"0"
ENDPOINT:PORT> incr counter
(int) 1
ENDPOINT:PORT> incr counter
(int) 2
```

Now you will start to see some action on the charts. Please go to
[Metrics and Charts](../howto/metricsandcharts) section for detailed information
about metrics and charts. Note that charts are updated every 10 seconds.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/getting_started/charts.png" width="100%" />
</Frame>

Congratulations! You have created your first database with Upstash.


# Pricing & Limits



Please check our [pricing page](https://upstash.com/pricing/redis) for the most up-to-date information on pricing and limits.


# Python SDK





# Rate Limit SDK





# Typescript SDK





# Redis® API Compatibility



Upstash supports Redis client protocol up to version `6.2`. Following table
shows the list of supported Redis commands:

| Feature                                                       | Supported? |                                                                                                                                                                                                  Supported Commands                                                                                                                                                                                                  |
| ------------------------------------------------------------- | :--------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| [String](https://redis.io/commands/?group=string)             |      ✅     |                                                                                                                       APPEND - DECR - DECRBY - GET - GETDEL - GETEX - GETRANGE - GETSET - INCR - INCRBY - INCRBYFLOAT - MGET - MSET - MSETNX - PSETEX - SET - SETEX - SETNX - SETRANGE - STRLEN                                                                                                                      |
| [Bitmap](https://redis.io/commands/?group=bitmap)             |      ✅     |                                                                                                                                                                         BITCOUNT - BITFIELD - BITFIELD\_RO - BITOP - BITPOS - GETBIT - SETBIT                                                                                                                                                                        |
| [Hash](https://redis.io/commands/?group=hash)                 |      ✅     |                                                                                                                                    HDEL - HEXISTS - HGET - HGETALL - HINCRBY - HINCRBYFLOAT - HKEYS - HLEN - HMGET - HMSET - HSCAN - HSET - HSETNX - HSTRLEN - HRANDFIELD - HVALS                                                                                                                                    |
| [List](https://redis.io/commands/?group=list)                 |      ✅     |                                                                                                               BLMOVE - BLMPOP - BLPOP - BRPOP - BRPOPLPUSH - LINDEX - LINSERT - LLEN - LMOVE - LMPOP - LPOP - LPOS - LPUSH - LPUSHX - LRANGE - LREM - LSET - LTRIM - RPOP - RPOPLPUSH - RPUSH - RPUSHX                                                                                                               |
| [Set](https://redis.io/commands/?group=set)                   |      ✅     |                                                                                                                     SADD - SCARD - SDIFF - SDIFFSTORE - SINTER - SINTERCARD - SINTERSTORE - SISMEMBER - SMEMBERS - SMISMEMBER - SMOVE - SPOP - SRANDMEMBER - SREM - SSCAN - SUNION - SUNIONSTORE                                                                                                                     |
| [SortedSet](https://redis.io/commands/?group=sorted_set)      |      ✅     | BZMPOP - BZPOPMAX - BZPOPMIN - ZADD - ZCARD - ZCOUNT - ZDIFF - ZDIFFSTORE - ZINCRBY - ZINTER - ZINTERCARD - ZINTERSTORE - ZLEXCOUNT - ZMPOP - ZMSCORE - ZPOPMAX - ZPOPMIN - ZRANDMEMBER - ZRANGE - ZRANGESTORE - ZRANGEBYLEX - ZRANGEBYSCORE - ZRANK - ZREM - ZREMRANGEBYLEX - ZREMRANGEBYRANK - ZREMRANGEBYSCORE - ZREVRANGE - ZREVRANGEBYLEX - ZREVRANGEBYSCORE - ZREVRANK - ZSCAN - ZSCORE - ZUNION - ZUNIONSTORE |
| [Geo](https://redis.io/commands/?group=geo)                   |      ✅     |                                                                                                                                       GEOADD - GEODIST - GEOHASH - GEOPOS - GEORADIUS - GEORADIUS\_RO - GEORADIUSBYMEMBER - GEORADIUSBYMEMBER\_RO - GEOSEARCH - GEOSEARCHSTORE                                                                                                                                       |
| [HyperLogLog](https://redis.io/commands/?group=hyperloglog)   |      ✅     |                                                                                                                                                                                               PFADD - PFCOUNT - PFMERGE                                                                                                                                                                                              |
| [Scripting](https://redis.io/commands/?group=scripting)       |      ✅     |                                                                                                                                                                              EVAL - EVALSHA - SCRIPT EXISTS - SCRIPT LOAD - SCRIPT FLUSH                                                                                                                                                                             |
| [Pub/Sub](https://redis.io/commands/?group=pubsub)            |      ✅     |                                                                                                                                                                        SUBSCRIBE - PSUBSCRIBE - UNSUBSCRIBE - PUNSUBSCRIBE - PUBLISH - PUBSUB                                                                                                                                                                        |
| [Transactions](https://redis.io/commands/?group=transactions) |      ✅     |                                                                                                                                                                                       DISCARD - EXEC - MULTI - UNWATCH - WATCH                                                                                                                                                                                       |
| [Generic](https://redis.io/commands/?group=generic)           |      ✅     |                                                                                                        COPY - DEL - DUMP - EXISTS - EXPIRE - EXPIREAT - EXPIRETIME - KEYS - PERSIST - PEXPIRE - PEXPIREAT - PEXPIRETIME - PTTL - RANDOMKEY - RENAME - RENAMENX - RESTORE - SCAN - TOUCH - TTL - TYPE - UNLINK                                                                                                        |
| [Connection](https://redis.io/commands/?group=connection)     |      ✅     |                                                                                                                                                                                  AUTH - HELLO - ECHO - PING - QUIT - RESET - SELECT                                                                                                                                                                                  |
| [Server](https://redis.io/commands/?group=server)             |      ✅     |                                                                                                                                                                                ACL(\*) - DBSIZE - FLUSHALL - FLUSHDB - MONITOR - TIME                                                                                                                                                                                |
| [JSON](https://redis.io/commands/?group=json)                 |      ✅     |                                            JSON.ARRAPPEND - JSON.ARRINSERT - JSON.ARRINDEX - JSON.ARRLEN - JSON.ARRPOP - JSON.ARRTRIM - JSON.CLEAR - JSON.DEL - JSON.FORGET - JSON.GET - JSON.MERGE - JSON.MGET - JSON.MSET - JSON.NUMINCRBY - JSON.NUMMULTBY - JSON.OBJKEYS - JSON.OBJLEN - JSON.RESP - JSON.SET - JSON.STRAPPEND - JSON.STRLEN - JSON.TOGGLE - JSON.TYPE                                           |
| [Streams](https://redis.io/commands/?group=stream)            |      ✅     |                                                                                                                                XACK - XADD - XAUTOCLAIM - XCLAIM - XDEL - XGROUP - XINFO GROUPS - XINFO CONSUMERS - XLEN - XPENDING - XRANGE - XREAD - XREADGROUP - XREVRANGE - XTRIM                                                                                                                                |
| [Cluster](https://redis.io/commands#cluster)                  |      ❌     |                                                                                                                                                                                                                                                                                                                                                                                                                      |

<Warning>
  As of May 2024, the ACL command is no longer supported for newly created databases on the AWS platform.
  However, it remains available on [fly.io](https://fly.io/docs/reference/redis/), GCP, and AWS databases created before May 2024.
</Warning>

We run command integration tests from following Redis clients after each code
change and also periodically:

*   **[Node-Redis](https://github.com/redis/node-redis)**
    [Command Tests](https://github.com/redis/node-redis/tree/v3.1.2/test/commands)
*   **[Jedis](https://github.com/redis/jedis)**
    [Command Tests](https://github.com/redis/jedis/tree/v4.1.1/src/test/java/redis/clients/jedis/commands)
*   **[Lettuce](https://github.com/lettuce-io/lettuce-core)**
    [Command Tests](https://github.com/lettuce-io/lettuce-core/tree/6.1.6.RELEASE/src/test/java/io/lettuce/core/commands)
*   **[Go-Redis](https://github.com/go-redis/redis)**
    [Command Tests](https://github.com/go-redis/redis/blob/master/commands_test.go)
*   **[Redis-py](https://github.com/redis/redis-py)**
    [Command Tests](https://github.com/redis/redis-py/tree/v4.4.0/tests)

Most of the unsupported items are in our roadmap. If you need a feature that we
do not support, please drop a note to
[support@upstash.com](mailto:support@upstash.com). So we can inform you when we
are planning to support it.


# Use Cases



The data store behind Upstash is [compatible](../overall/rediscompatibility)
with almost all Redis® API. So you can use Upstash for the Redis®' popular use
cases such as:

*   General caching
*   Session caching
*   Leaderboards
*   Queues
*   Usage metering (counting)
*   Content filtering

Check Salvatore's [blog](http://antirez.com/post/take-advantage-of-redis-adding-it-to-your-stack.html) post. You can find lots of similar articles about the common use cases of Redis.

## Key Value Store and Caching for Next.js Application

Next.js is increasingly becoming the preferred method for developing dynamic and fast web applications in an agile manner. It owes its popularity to its server-side rendering capabilities and API routes supported by serverless functions, including Vercel serverless and edge functions. Upstash Redis is a great fit with Next.js applications due to its serverless model and its REST-based APIs. The REST API plays a critical role in enabling access from edge functions while also addressing connection issues in serverless functions.

Check the blog post:
[Speed up your Next.js application with Redis](https://upstash.com/blog/nextjs-caching-with-redis)

## Redis for Vercel Functions

Vercel stands out as one of the most popular cloud platform for web developers, offering continuous integration, deployment, CDN and serverless functions. However, when it comes to databases, you'll need to rely on external data services to support dynamic applications.

That's where Upstash comes into play as one of the most favored data solutions within the Vercel platform. Here are some reasons that contribute to Upstash's popularity in the Vercel ecosystem:

*   No connection problems thanks to
    [Upstash SDK](https://github.com/upstash/upstash-redis) built on Upstash REST
    API.
*   Edge runtime does not allow TCP based connections. You can not use regular
    Redis clients. [Upstash SDK](https://github.com/upstash/upstash-redis) works
    on edge runtimes without a problem.
*   Upstash has a [Vercel add on](https://vercel.com/integrations/upstash) where
    you can easily integrate Upstash to your Vercel projects.

## Storage For Lambda Functions (FaaS)

People use Lambda functions for various reasons, with one of the primary advantages being their cost-effectiveness – you only pay for what you actually use, which is great. However, when it comes to needing a storage layer, AWS recommends DynamoDB. DynamoDB does offer a serverless mode, which sounds promising until you encounter its latency when connecting and operating within Lambda Functions. Unfortunately, DynamoDB's latency may not be ideal for Lambda Functions, where every second of latency can have a significant impact on costs. At this point, AWS suggests using ElastiCache for low-latency data storage, which is also a Redis® cache as a service – a positive aspect. However, it's worth noting that ElastiCache is not serverless, and you have to pay based on what you provision, rather than what you use. To be honest, the pricing may not be the most budget-friendly option. This leaves you with two alternatives:

*   DynamoDB: Serverless but high latency
*   ElastiCache: Low latency but not serverless.

Until you meet the Upstash. Our sole mission is to provide a Redis® API
compatible database that you love in the serverless model. In Upstash, you pay
per the number of requests you have sent to your database. So if you are not
using the database you pay almost nothing. (Almost, because we charge for the
storage. It is a very low amount but still it is there.)

We believe that Upstash is the best storage for your Lambda Functions because:

*   Serverless just like Lambda functions itself
*   Designed for low latency data access
*   The lovely simple Redis® API


# AWS Lambda



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/aws-cdk-typescript" horizontal>
  You can find the project source code on GitHub.
</Card>

### Prerequisites

*   Complete all steps in [Getting started with the AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html)

### Project Setup

Create and navigate to a directory named `counter-cdk`. The CDK CLI uses this directory name to name things in your CDK code, so if you decide to use a different name, don't forget to make the appropriate changes when applying this tutorial.

```shell
mkdir counter-cdk && cd counter-cdk
```

Initialize a new CDK project.

```shell
cdk init app --language typescript
```

Install `@upstash/redis`.

```shell
npm install @upstash/redis
```

### Counter Function Setup

Create `/api/counter.ts`.

```ts /api/counter.ts
import { Redis } from '@upstash/redis';

const redis = Redis.fromEnv();

export const handler = async function() {
    const count = await redis.incr("counter");
    return {
        statusCode: 200,
        body: JSON.stringify('Counter: ' + count),
    };
};
```

### Counter Stack Setup

Update `/lib/counter-cdk-stack.ts`.

```ts /lib/counter-cdk-stack.ts
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as nodejs from 'aws-cdk-lib/aws-lambda-nodejs';

export class CounterCdkStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const counterFunction = new nodejs.NodejsFunction(this, 'CounterFunction', {
      entry: 'api/counter.ts',
      handler: 'handler',
      runtime: lambda.Runtime.NODEJS_20_X,
      environment: {
        UPSTASH_REDIS_REST_URL: process.env.UPSTASH_REDIS_REST_URL || '',
        UPSTASH_REDIS_REST_TOKEN: process.env.UPSTASH_REDIS_REST_TOKEN || '',
      },
      bundling: {
        format: nodejs.OutputFormat.ESM,
        target: "node20",
        nodeModules: ['@upstash/redis'],
      },
    });

    const counterFunctionUrl = counterFunction.addFunctionUrl({
      authType: lambda.FunctionUrlAuthType.NONE,
    });

    new cdk.CfnOutput(this, "counterFunctionUrlOutput", {
      value: counterFunctionUrl.url,
    })
  }
}
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and export `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment.

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Deploy

Run in the top folder:

```shell
cdk synth
cdk bootstrap
cdk deploy
```

Visit the output URL.


# Azure Functions



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/azure-functions" horizontal>
  You can find the project source code on GitHub.
</Card>

### Prerequisites

1.  [Create an Azure account.](https://azure.microsoft.com/en-us/free/)
2.  [Set up Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli)
3.  [Install the Azure Functions Core Tools](https://learn.microsoft.com/en-us/azure/azure-functions/create-first-function-cli-typescript)

### Project Setup

Initialize the project:

```shell
func init --typescript
```

Install `@upstash/redis`

```shell
npm install @upstash/redis
```

### Counter Function Setup

Create a new function from template.

```shell
func new --name CounterFunction --template "HTTP trigger" --authlevel "anonymous"
```

Update `/src/functions/CounterFunction.ts`

```ts /src/functions/CounterFunction.ts
import { app, HttpRequest, HttpResponseInit, InvocationContext } from "@azure/functions";
import { Redis } from "@upstash/redis";

const redis = new Redis({
    url: process.env.UPSTASH_REDIS_REST_URL,
    token: process.env.UPSTASH_REDIS_REST_TOKEN
});

export async function CounterFunction(request: HttpRequest, context: InvocationContext): Promise<HttpResponseInit> {
    const count = await redis.incr("counter");

    return { status: 200, body: `Counter: ${count}` };
};

app.http('CounterFunction', {
    methods: ['GET', 'POST'],
    authLevel: 'anonymous',
    handler: CounterFunction
});
```

### Create Azure Resources

You can use the command below to find the `name` of a region near you.

```shell
az account list-locations
```

Create a resource group.

```shell
az group create --name AzureFunctionsQuickstart-rg --location <REGION>
```

Create a storage account.

```shell
az storage account create --name <STORAGE_NAME> --location <REGION> --resource-group AzureFunctionsQuickstart-rg --sku Standard_LRS --allow-blob-public-access false
```

Create your Function App.

```shell
az functionapp create --resource-group AzureFunctionsQuickstart-rg --consumption-plan-location <REGION> --runtime node --runtime-version 18 --functions-version 4 --name <APP_NAME> --storage-account <STORAGE_NAME>
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and set `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` in your Function App's settings.

```shell
az functionapp config appsettings set --name <APP_NAME> --resource-group AzureFunctionsQuickstart-rg --settings UPSTASH_REDIS_REST_URL=<YOUR_URL> UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Deploy

Take a build of your application.

```shell
npm run build
```

Publish your application.

```shell
func azure functionapp publish <APP_NAME>
```

Visit the given Invoke URL.


#  Cloudflare Workers



### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli). Select the global to minimize the
latency from all edge locations.

### Project Setup

We will use **C3 (create-cloudflare-cli)** command-line tool to create our application. You can open a new terminal window and run C3 using the prompt below.

<CodeGroup>
  ```shell npm
  npm create cloudflare@latest
  ```

  ```shell yarn
  yarn create cloudflare@latest
  ```
</CodeGroup>

This will install the `create-cloudflare` package, and lead you through setup. C3 will also install Wrangler in projects by default, which helps us testing and deploying the application.

```text
➜  npm create cloudflare@latest
Need to install the following packages:
  create-cloudflare@2.1.0
Ok to proceed? (y) y

using create-cloudflare version 2.1.0

╭ Create an application with Cloudflare Step 1 of 3
│
├ In which directory do you want to create your application?
│ dir ./cloudflare_starter
│
├ What type of application do you want to create?
│ type "Hello World" Worker
│
├ Do you want to use TypeScript?
│ yes typescript
│
├ Copying files from "hello-world" template
│
├ Do you want to use TypeScript?
│ yes typescript
│
├ Retrieving current workerd compatibility date
│ compatibility date 2023-08-07
│
├ Do you want to use git for version control?
│ yes git
│
╰ Application created
```

We will also install the **Upstash Redis SDK** to connect to Redis.

```bash
npm install @upstash/redis
```

### The Code

Here is a Worker template to configure and test Upstash Redis connection.

<CodeGroup>
  ```ts src/index.ts
  import { Redis } from "@upstash/redis/cloudflare";

  export interface Env {
    UPSTASH_REDIS_REST_URL: string;
    UPSTASH_REDIS_REST_TOKEN: string;
  }

  export default {
    async fetch(
      request: Request,
      env: Env,
      ctx: ExecutionContext
    ): Promise<Response> {
      const redis = Redis.fromEnv(env);
      const count = await redis.incr("counter");
      return new Response(JSON.stringify({ count }));
    },
  };
  ```

  ```js src/index.js
  import { Redis } from "@upstash/redis/cloudflare";

  export default {
    async fetch(request, env, ctx) {
      const redis = Redis.fromEnv(env);

      const count = await redis.incr("counter");

      return new Response(JSON.stringify({ count }));
    },
  };
  ```
</CodeGroup>

### Configure Credentials

There are two methods for setting up the credentials for Upstash Redis client. The recommended way is to use Cloudflare Upstash Integration. Alternatively, you can add the credentials manually.

#### Using the Cloudflare Integration

Access to the [Cloudflare Dashboard](https://dash.cloudflare.com) and login with the same account that you've used while setting up the Worker application. Then, navigate to **Workers & Pages > Overview** section on the sidebar. Here, you'll find your application listed.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/cloudflare-integration/overview.png" />
</Frame>

Clicking on the application will direct you to the application details page, where you can perform the integration process. Switch to the **Settings** tab in the application details, and proceed to **Integrations** section. You will see various Worker integrations listed. To proceed, click the **Add Integration** button associated with the Upstash Redis.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/cloudflare-integration/redis-add-integration.png" />
</Frame>

On the Integration page, connect to your Upstash account. Then, select the related database from the dropdown menu. Finalize the process by pressing Save button.

<Note>
  Please don't make any changes in the secret names on the **Configure secrets**
  step. These credentials will be automatically picked up by
  `Redis.fromEnv(env)` line in the code as `UPSTASH_REDIS_REST_URL` and
  `UPSTASH_REDIS_REST_TOKEN`.
</Note>

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/cloudflare-integration/redis-credentials.png" />
</Frame>

#### Setting up Manually

Navigate to [Upstash Console](https://console.upstash.com) and copy/paste your `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your `wrangler.toml` as below.

```yaml
[vars]
UPSTASH_REDIS_REST_URL="REPLACE_HERE"
UPSTASH_REDIS_REST_TOKEN="REPLACE_HERE"
```

### Test and Deploy

You can test the function locally with `npx wrangler dev`

Deploy your function to Cloudflare with `npx wrangler deploy`

The endpoint of the function will be provided to you, once the deployment is done.

### Repo

Javascript:
[https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers](https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers)

Typescript:
[https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers-with-typescript](https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers-with-typescript)

Wrangler 1:
[https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers-with-wrangler-1](https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers-with-wrangler-1)


# Deno Deploy



This is a step-by-step guide on how to use Upstash Redis to create a view
counter in your Deno deploy project.

### Create a database

Create a Redis database using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli). Select the global to minimize the
latency from all edge locations. Copy the `UPSTASH_REDIS_REST_URL` and
`UPSTASH_REDIS_REST_TOKEN` for the next steps.

### Create a Deno deploy project

Go to [https://dash.deno.com/projects](https://dash.deno.com/projects) and
create a new playground project.

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/redis-deno/create.png)

### 2. Edit the handler function

Then paste the following code into the browser editor:

```ts
import { serve } from "https://deno.land/std@0.142.0/http/server.ts";
import { Redis } from "https://deno.land/x/upstash_redis@v1.14.0/mod.ts";

serve(async (_req: Request) => {
  if (!_req.url.endsWith("favicon.ico")) {
    const redis = new Redis({
      url: "UPSTASH_REDIS_REST_URL",
      token: "UPSTASH_REDIS_REST_TOKEN",
    });

    const counter = await redis.incr("deno-counter");
    return new Response(JSON.stringify({ counter }), { status: 200 });
  }
});
```

### 3. Deploy and Run

Simply click on `Save & Deploy` at the top of the screen.

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/redis-deno/deploy.png)


# DigitalOcean



<Info>
  Upstash has native integration with [DigitalOcean Add-On
  Marketplace](https://marketplace.digitalocean.com/add-ons/upstash-redis).
</Info>

This quickstart shows how to create an Upstash for Redis® Database from
DigitalOcean Add-On Marketplace.

### Database Setup

Creating Upstash for Redis Database requires a DigitalOcean account.

[Login or Sign-up](https://cloud.digitalocean.com/login) for DigitalOcean
account. Then navigate the
[Upstash Redis Marketplace](https://marketplace.digitalocean.com/add-ons/upstash-redis)
page.

Click `Add Upstash Redis` button. Now setup page will open and it will ask
`Database Name / Plan / Region` info.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/digitalocean/img.png" />
</Frame>

After selecting Name, Plan and Region, click `Add Upstash Redis` button.

### Connecting to Database - SSO

After creating database, Overview/Details page will be opened.

Environment variables can be shown in that page.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/digitalocean/img2.png" />
</Frame>

While creating a Droplet, Upstash Addon can be selected and environment
variables are automatically injected to Droplet.

These Steps can be followed: `Create --> Droplets --> Marketplace Add-Ons` then
select the previously created Upstash Redis Addon.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/digitalocean/img3.png" />
</Frame>

Upstash also support Single Sign-On from DigitalOcean to Upstash Console.

So databases created from DigitalOcean can benefit from Upstash Console
features.

In order to access Upstash Console from DigitalOcean just click `Dashboard` link
when you create the Upstash addon.


# Django



### Introduction

In this quickstart tutorial, we will demonstrate how to use Django with Upstash Redis to build a simple web application that increments a counter every time the homepage is accessed.

### Environment Setup

First, install Django and the Upstash Redis client for Python:

```shell
pip install django
pip install upstash-redis
```

### Database Setup

Create a Redis database using the [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and export the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment:

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

You can also use `python-dotenv` to load environment variables from your `.env` file.

### Project Setup

Create a new Django project:

```shell
django-admin startproject myproject
cd myproject
python manage.py startapp myapp
```

In `myproject/settings.py`, add your new app (`myapp`) to the `INSTALLED_APPS` list.

### Application Setup

In `myapp/views.py`, add the following:

```python
from django.http import HttpResponse
from upstash_redis import Redis

redis = Redis.from_env()

def index(request):
    count = redis.incr('counter')
    return HttpResponse(f'Page visited {count} times.')
```

In `myproject/urls.py`, connect the view to a URL pattern:

```python
from django.urls import path
from myapp import views

urlpatterns = [
    path('', views.index),
]
```

### Running the Application

Run the development server:

```shell
python manage.py runserver
```

Visit `http://127.0.0.1:8000/` in your browser, and the counter will increment with each page refresh.

### Code Breakdown

1.  **Redis Setup**: We use the Upstash Redis client to connect to our Redis database using the environment variables `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN`. The `Redis.from_env()` method initializes this connection.

2.  **Increment Counter**: In the `index` view, we increment the `counter` key each time the homepage is accessed. If the key doesn't exist, Redis creates it and starts counting from 1.

3.  **Display the Count**: The updated count is returned as an HTTP response each time the page is loaded.


# Elixir

Tutorial on Using Upstash Redis In Your Phoenix App and Deploying it on Fly.

This tutorial showcases how one can use [fly.io](https://fly.io) to deploy a Phoenix
app using Upstash Redis to store results of external API calls.

See [code](https://github.com/upstash/examples/tree/master/examples/elixir-with-redis) and
[demo](https://elixir-redis.fly.dev/).

### `1` Create a Elixir app with Phoenix

To create an app, run the following command:

```
mix phx.new redix_demo --no-ecto 
```

Phoenix apps are initialized with a datastore. We pass `--no-ecto` flag to disable
the datastore since we will only use Redis. See
[Phoenix documentation](https://hexdocs.pm/phoenix/up_and_running.html) for more details.

Navigate to the new directory by running

```
cd redix_demo
```

### `2` Add Redix

To connect to the Upstash Redis, we will use the
[Redix client](https://github.com/whatyouhide/redix.git) written for Elixir.

To add Redix to our project, we will first update the dependencies of our project. Simply
add the following two entries to the dependencies in the `mix.exs` file
(See [Redix documentation](https://github.com/whatyouhide/redix.git)):

```elixir
defp deps do
  [
    {:redix, "~> 1.1"},
    {:castore, ">= 0.0.0"}
  ]
end
```

Then, run `mix deps.get` to install the new dependencies.

Next, we will add Redix to our app. In our case, we will add a single global Redix instance.
Open the `application.ex` file and find the `children` list in the `start` function.

First, add a method to read the connection parameters from the `REDIS_URL` environment variable.
We choose this name for the environment variable because Fly will create a secret with this name
when we launch the app with a Redis store. Use regex to extract the password, host and port
information from the Redis URL:

```elixir
  def start(_type, _args) do
    [_, password, host, port] = Regex.run(
      ~r{(.+):(.+)@(.+):(\d+)},
      System.get_env("REDIS_URL"),
      capture: :all_but_first
    )
    port = elem(Integer.parse(port), 0)

    # ...
  end
```

Next, add the Redix client to the project by adding it to the `children` array.
([See Redix Documentation for more details](https://hexdocs.pm/redix/real-world-usage.html#single-named-redix-instance))

```elixir
children = [
  # ...
  {
    Redix,
    name: :redix,
    host: host,
    port: port,
    password: password,
    socket_opts: [:inet6]
  }
]
```

Here, we would like to draw attention to the `socket_opts` parameter. If you wish to test
your app locally by creating an Upstash Redis yourself without Fly, you must define Redix
client **without the `socket_opts: [:inet6]` field**.

### `3` Testing the Connection

At this point, our app should now be able to communicate with Redix. To test if this
connection works as expected, we will first add a status page to our app.

To add this page, we will change the default landing page of our Phoenix app. Go to the
`lib/redix_demo_web/controllers/page_html/home.html.heex` file. Replace the content of
the file with:

```html
<.flash_group flash={@flash} />
<div class="container mx-auto px-4">
  <h1 class="text-3xl font-bold mb-4">Redix Demo</h1>

  <form action="/" method="get" class="w-full flex items-center mb-4">
    <input type="text" name="text" placeholder="Location" class="flex-1 px-4 py-2 border border-gray-300 rounded-lg shadow-sm focus:outline-none focus:border-blue-500">
    <button type="submit" class="ml-4 px-4 py-2 bg-blue-500 text-white rounded-lg hover:bg-blue-600">Submit</button>
  </form>

  <%= if @text do %>
    <%= @text %>
  <% end %>

  <%= if @weather do %>
    <div class=" text-lg bg-gray-100 rounded-lg p-4">

      <%= if @location do %>
        <strong>
          Location:
        </strong>
        <%= @location %>
      <% end %>

      <p>
        <strong>
          Weather:
        </strong>
        <%= @weather %> °C
      </p>

    </div>
  <% end %>
</div>
```

This HTML will show different content depending on the parameters we
pass it. It has a form at the top which is where the user will enter
some location. Below, we will show the weather information.

Next, open the `lib/redix_demo_web/router.ex` file. In this file,
URL paths are defined with the `scope` keyword. Update the scope
in the following way:

```
  scope "/", RedixDemoWeb do
    pipe_through :browser

    get "/status", PageController, :status

    get "/", PageController, :home
    get "/:text", PageController, :home
  end
```

Our website will have a `/status` path, which will be rendered with the
`status` method we will define. The website will also render the home
page in `/` and in `/:text`. `/:text` will essentially match any route
and the route will be available to our app as a parameter when rendering.

Finally, we will define the status page in
`lib/redix_demo_web/controllers/page_controller.ex`. We will define a struct
`Payload` and a private method `render_home`. Then, we will define the home
page and the status page:

```elixir
defmodule RedixDemoWeb.PageController do
  use RedixDemoWeb, :controller

  defmodule Payload do
    defstruct text: nil, weather: nil, location: nil
  end

  def status(conn, _params) do
    case Redix.command(:redix, ["PING"]) do
      {:ok, response} ->
        render_home(conn, %Payload{text: "Redis Connection Status: Success! Response to 'PING': '#{response}'"})
      {:error, response} ->
        render_home(conn, %Payload{text: "Redis Connection Status: Error. Reason: #{response.reason}"})
      end
  end

  def home(conn, _params) do
    render_home(conn, %Payload{text: "Enter a location above to get the weather info!"})
  end

  defp render_home(conn, %Payload{} = payload) do
    render(conn, "home.html", text: payload.text, weather: payload.weather, location: payload.location)
  end
end
```

The home page simply renders our home page. The status page renders the same page, but
shows the response of a `PING` request to our Redis server.

We are now ready to deploy the app on Fly!

### `4` Deploy on Fly

To deploy the app on Fly, first
[install Fly CLI](https://fly.io/docs/hands-on/install-flyctl/) and authenticate. Then,
launch the app with:

```
fly launch
```

If you haven't set `REDIS_URL` environment variable in your environment, `fly launch` command will show
an error when compiling the app but don't worry. You can still continue with launching the app.
Fly will add this environment variable itself.

Fly will at some point ask if we want to tweak the settings of the app. Choose yes (`y`):

```
>>> fly launch

Detected a Phoenix app
Creating app in /Users/examples/redix_demo
We're about to launch your Phoenix app on Fly.io. Here's what you're getting:

Organization: C. Arda                (fly launch defaults to the personal org)
Name:         redix_demo             (derived from your directory name)
Region:       Bucharest, Romania     (this is the fastest region for you)
App Machines: shared-cpu-1x, 1GB RAM (most apps need about 1GB of RAM)
Postgres:     <none>                 (not requested)
Redis:        <none>                 (not requested)
Sentry:       false                  (not requested)

? Do you want to tweak these settings before proceeding? (y/N)
```

This will open the settings on the browser. Two settings are relevant to this guide:

*   Region: Upstash is not available in all regions. Choose Amsterdam.
*   Redis: Choose "Redis with Upstash"

If you already have a Redis on Fly you want to use, you may want to not choose the
"Redis with Upstash". Instead, you can get the `REDIS_URL` from [the Upstash Fly console](https://console.upstash.com/flyio/redis)
and add it as a secret with `fly secrets set REDIS_URL=****`. Note that the `REDIS_URL`
will be in `redis://default:****@fly-****.upstash.io:****` format.

Once the app is launched, deploy it with:

```
fly deploy
```

The website will become avaiable after some time. Check the `/status` page to see that
the redis connection is correctly done.

In the rest of our tutorial, we will work on caching the responses from an external api.
If you are only interested in how a Phoenix app with Redis can be deployed on Fly, you
may not need to read the rest of the tutorial.

### `5` Using Redix to Cache External API Responses

Finally, we will now build our website to offer weather information. We will use the API
of [WeatherAPI](https://www.weatherapi.com/) to get the weather information upon user
request. We will cache the results of our calls in Upstash Redis to reduce the number
of calls we make to the external API and to reduce the response time of our app.

In the end, we will have a method `def home(conn, %{"text" => text})` in the
`lib/redix_demo_web/controllers/page_controller.ex` file. To see the final file, find the
[`page_controller.ex` file Upstash examples repository](https://github.com/upstash/examples/blob/main/examples/elixir-with-redis/lib/redix_demo_web/controllers/page_controller.ex).

First, we need to define some private methods to handle the request logic. We start off
with a function to fetch the weather. The method gets the location string and replaces
the empty characters with `%20`. Then it calls `fetch_weather_from_cache` method we will
define. Depending on the result, it either returns the result from cache, or fetches the
result from the api.

```elixir
  defp fetch_weather(location) do
    location = String.replace(location, " ", "%20")
    case fetch_weather_from_cache(location) do
      {:ok, cached_weather} ->
        {:ok, cached_weather}
      {:error, :not_found} ->
        fetch_weather_from_api(location)
      {:error, reason} ->
        {:error, reason}
    end
  end
```

Now, we will define the `fetch_weather_from_cache` method. This method will use
Redix to fetch the weather from the location. If it's not found, we will return
`{:error, :not_found}`. If it's found, we will return after decoding it into a
map.

```elixir
  defp fetch_weather_from_cache(location) do
    case Redix.command(:redix, ["GET", "weather:#{location}"]) do
      {:ok, nil} ->
        {:error, :not_found}
      {:ok, cached_weather_json} ->
        {:ok, Jason.decode!(cached_weather_json)}
      {:error, _reason} ->
        {:error, "Failed to fetch weather data from cache."}
    end
  end
```

Next, we will define the `fetch_weather_from_api` method. This method
requests the weather information from the external API. If the request
is successfull, it also saves the result in the cache with the
`cache_weather_response` method.

```elixir
  defp fetch_weather_from_api(location) do
    weather_api_key = System.get_env("WEATHER_API_KEY")
    url = "http://api.weatherapi.com/v1/current.json?key=#{weather_api_key}&q=#{location}&aqi=no"

    case HTTPoison.get(url) do
      {:ok, %{status_code: 200, body: body}} ->
        weather_info = body
                      |> Jason.decode!()
                      |> get_weather_info()

        # Cache the weather response in Redis for 8 hours
        cache_weather_response(location, Jason.encode!(weather_info))

        {:ok, weather_info}
      {:ok, %{status_code: status_code, body: body}} ->
        {:error, "#{body} (#{status_code})"}
      {:error, _reason} ->
        {:error, "Failed to fetch weather data."}
    end
  end
```

In the `cache_weather_response` method, we simply store the weather
information in our Redis:

```elixir
  defp cache_weather_response(location, weather_data) do
    case Redix.command(:redix, ["SET", "weather:#{location}", weather_data, "EX", 8 * 60 * 60]) do
      {:ok, _} ->
        :ok
      {:error, _reason} ->
        {:error, "Failed to cache weather data."}
    end
  end
```

Finally, we define the `get_weather_info` and `home` methods.

```elixir
  def home(conn, %{"text" => text}) do
    case fetch_weather(text) do
      {:ok, %{"location" => location, "temp" => temp_c, "condition" => condition_text}} ->
        render_home(conn, %Payload{weather: "#{condition_text}, #{temp_c}", location: location})
      {:error, reason} ->
        render_home(conn, %Payload{text: reason})
    end
  end

  defp get_weather_info(%{
    "location" => %{
      "name" => name,
      "region" => region
    },
    "current" => %{
      "temp_c" => temp_c,
      "condition" => %{
        "text" => condition_text
      }
    }
  }) do
    %{"location" => "#{name}, #{region}", "temp" => temp_c, "condition" => condition_text}
  end
```

### `6` Re-deploying the App

To deploy the app after adding the home page logic, only a few steps remain to deploy the
finished app.

First, add `{:httpoison, "~> 1.5"}` dependency to `mix.exs` file and run `mix deps.get`.

Then, get an API key from [WeatherAPI](https://www.weatherapi.com/) and set it as secret in
fly with:

```
fly secrets set WEATHER_API_KEY=****
```

Now, you can run `fly deploy` in your directory to deploy the completed app!


# FastAPI



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/fastapi" horizontal>
  You can find the project source code on GitHub.
</Card>

### Environment Setup

Install FastAPI and `upstash-redis`.

```shell
pip install fastapi
pip install upstash-redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and export the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment.

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### API Setup

Create `main.py`:

```py main.py
from fastapi import FastAPI

from upstash_redis import Redis

app = FastAPI()

redis = Redis.from_env()

@app.get("/")
def read_root():
    count = redis.incr('counter')
    return {"count": count}
```

### Run

Run the app locally with `fastapi dev main.py`, check `http://127.0.0.1:8000/`


# Fastly



### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or
[Upstash CLI](https://github.com/upstash/cli). Select the global to minimize the
latency from all edge locations. Copy the `UPSTASH_REDIS_REST_URL` and
`UPSTASH_REDIS_REST_TOKEN` for the next steps.

### Project Setup

We will use Fastly CLI for deployment, so please install
[Fastly CLI](https://developer.fastly.com/reference/cli/).

Create a folder for your project and run `fastly init`. Select `[2] JavaScript`,
then `[2] Empty starter for JavaScript`

```shell
> fastly compute init

Creating a new Compute@Edge project.

Press ^C at any time to quit.

Name: [fastly-upstash]
Description:
Author: [enes@upstash.com]
Language:
[1] Rust
[2] JavaScript
[3] AssemblyScript (beta)
[4] Other ('bring your own' Wasm binary)
Choose option: [1] 2
Starter kit:
[1] Default starter for JavaScript
    A basic starter kit that demonstrates routing, simple synthetic responses and
    overriding caching rules.
    https://github.com/fastly/compute-starter-kit-javascript-default
[2] Empty starter for JavaScript
    An empty application template for the Fastly Compute@Edge environment which simply
    returns a 200 OK response.
    https://github.com/fastly/compute-starter-kit-javascript-empty
Choose option or paste git URL: [1] 2
```

Install @upstash/redis:

```shell
npm install @upstash/redis
```

Now, we will create a Fastly Compute service by running,
`fastly compute publish`. You need to add your Upstash database's endpoint as a
backend and select 443 as its port.

```shell
> fastly compute publish
✓ Initializing...
✓ Verifying package manifest...
✓ Verifying local javascript toolchain...
✓ Building package using javascript toolchain...
✓ Creating package archive...

SUCCESS: Built package 'fastly-upstash' (pkg/fastly-upstash.tar.gz)


There is no Fastly service associated with this package. To connect to an existing service
add the Service ID to the fastly.toml file, otherwise follow the prompts to create a
service now.

Press ^C at any time to quit.

Create new service: [y/N] y

✓ Initializing...
✓ Creating service...

Domain: [supposedly-included-corgi.edgecompute.app]

Backend (hostname or IP address, or leave blank to stop adding backends): global-concise-scorpion-30984.upstash.io
Backend port number: [80] 443
Backend name: [backend_1] upstash

Backend (hostname or IP address, or leave blank to stop adding backends):

✓ Creating domain 'supposedly-smart-corgi.edgecompute.app'...
✓ Creating backend 'upstash' (host: global-concise-scorpion-30984.upstash.io, port: 443)...
✓ Uploading package...
✓ Activating version...
```

### The Code

Update `src/index.js` as below:

```js
import { Redis } from "@upstash/redis/fastly";

addEventListener("fetch", (event) => event.respondWith(handleRequest(event)));

async function handleRequest(event) {
  const redis = new Redis({
    url: "UPSTASH_REDIS_REST_URL",
    token: "UPSTASH_REDIS_REST_TOKEN",
    backend: "upstash",
  });
  const data = await redis.incr("count");
  return new Response("View Count:" + data, { status: 200 });
}
```

### Deploy

Deploy: `fastly compute deploy`

After deployment, the CLI logs the endpoint. You can check the logs with:
`fastly log-tail --service-id=<YOUR_FASTLY_SERVICE_ID>`

### Run Locally

To run the function locally add the backend to your `fastly.toml` as below:

```toml
[local_server.backends.upstash]
url = "UPSTASH_REDIS_REST_URL"
```

Then run: `fastly compute serve`


# Flask



### Introduction

In this quickstart tutorial, we will explore how to use Flask with Upstash Redis to build a simple web application that increments a counter each time a user accesses the homepage.

### Environment Setup

First, install Flask and the Upstash Redis client for Python.

```shell
pip install flask
pip install upstash-redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and export the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment.

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

You can also use `python-dotenv` to load environment variables from your `.env` file.

### Application Setup

Create `app.py`:

```py app.py
from flask import Flask
from upstash_redis import Redis

app = Flask(__name__)

redis = Redis.from_env()

@app.route('/')
def index():
    count = redis.incr('counter')
    return f'Page visited {count} times.'

if __name__ == '__main__':
    app.run(debug=True)
```

### Running the Application

Run the Flask app locally:

```shell
python app.py
```

Visit `http://127.0.0.1:5000/` in your browser, and you will see the `counter` increment with each refresh.

Code Breakdown

1.  **Redis Setup:** We first import Flask and the Upstash Redis client. Using `Redis.from_env()`, we initialize the connection to our Redis database using the environment variables exported earlier.
2.  **Increment Counter:** Each time the root route (`/`) is accessed, Redis increments the `counter` key. This key-value pair is automatically created in Redis if it does not exist, and its value is incremented on each request.
3.  **Display the Count:** The number of visits is returned in the response as plain text.


# Fly.io



<Info>
  Fly.io has a native integration with Upstash where the databases are hosted in
  Fly. You can still access a Redis from Fly to Upstash but for the best
  latency, we recommend creating Redis (Upstash) inside Fly platform. Check
  [here](https://fly.io/docs/reference/redis/) for details.
</Info>

In this tutorial, we'll walk you through the process of deploying a Redis by
Upstash and connecting it to an application hosted on Fly.io. We'll be using
Node.js and Express for our example application, but the process can be easily
adapted to other languages and frameworks.

### Redis Setup

Create a Redis database using
[Fly CLI](https://fly.io/docs/hands-on/install-flyctl/)

```shell
> flyctl redis create
? Select Organization: upstash (upstash)
? Choose a Redis database name (leave blank to generate one):
? Choose a primary region (can't be changed later) San Jose, California (US) (sjc)

Upstash Redis can evict objects when memory is full. This is useful when caching in Redis. This setting can be changed later.
Learn more at https://fly.io/docs/reference/redis/#memory-limits-and-object-eviction-policies
? Would you like to enable eviction? No
? Optionally, choose one or more replica regions (can be changed later):
? Select an Upstash Redis plan 3G: 3 GB Max Data Size

Your Upstash Redis database silent-tree-6201 is ready.
Apps in the upstash org can connect to at redis://default:978ba2e07tyrt67598acd8ac916a@fly-silent-tree-6201.upstash.io
If you have redis-cli installed, use fly redis connect to connect to your database.
```

### Set up the Node.js application

*   Create a new folder for your project and navigate to it in the terminal.
*   Run `npm init -y` to create a `package.json` file.
*   Install Express and the Redis client: `npm install express redis`
*   Create an `index.js` file in the project folder with the following content:

```js
const express = require("express");
const redis = require("redis");
const { promisify } = require("util");

const app = express();
const client = redis.createClient(process.env.REDIS_URL);

const getAsync = promisify(client.get).bind(client);
const setAsync = promisify(client.set).bind(client);

app.get("/", async (req, res) => {
  const value = await getAsync("counter");
  await setAsync("counter", parseInt(value || 0) + 1);
  res.send(`Hello, visitor number ${value || 0}!`);
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => console.log(`Server running on port ${PORT}`));
```

This code creates a simple Express server that increments a counter in Redis and
returns the visitor number.

### Configure the Fly.io application

*   Run `fly init "your-app-name"` to initialize a new Fly.io application.
*   Choose the "Node.js (14.x)" builder, and accept the defaults for the remaining
    prompts.
*   Open the `fly.toml` file that was generated and add the following environment
    variable under the `[[services]]` section:

```toml
[env]
  REDIS_URL = "<your-upstash-redis-url>"
```

Replace `your-upstash-redis-url` with the Redis URL from your Upstash instance.

### Deploy the application to Fly.io

*   Run fly deploy to build and deploy your application.
*   After the deployment is complete, run fly status to check if the application
    is running.
*   Visit the URL provided in the output (e.g., [https://your-app-name.fly.dev](https://your-app-name.fly.dev)) to
    test your application.

### Conclusion

You have successfully deployed a Node.js application on Fly.io that uses an
Upstash Redis instance as its data store. You can now build and scale your
application as needed, leveraging the benefits of both Fly.io and Upstash.

### Availability of Redis URL for Local Development and Testing

#### Understanding Fly.io and Redis Setup

*   **Redis Instance on Fly.io**: When you create a Redis instance using `fly redis create`, Fly.io establishes a Redis server in its cloud environment, designed specifically for applications running on the Fly.io platform.

*   **Connection String**: This command generates a connection string. However, it's important to note that this string is intended primarily for applications deployed within Fly.io's network. Due to security and network configurations, it's not directly accessible from external networks, like your local development environment.

#### Creating a Tunnel for Local Testing

*   **Fly Redis Connect**: For local access to your Redis instance, use `fly redis connect`. This command establishes a secure tunnel between your local machine and the Redis instance on Fly.io.

    *   **How it Works**:
        *   The tunnel maps a local port to the remote Redis port on Fly.io.
        *   Once established, you can connect to Redis as if it were running locally, typically at `localhost` with the mapped port.

    *   **Setting Up the Tunnel**:
        *   Execute `fly redis connect` in your terminal.
        *   The command provides a local address (e.g., `localhost:10000`).
        *   Use this address as your Redis connection URL in your local development setup.

    *   **Considerations**:
        *   This tunnel is a temporary solution, ideal for development and testing, not for production.
        *   Ensure compatibility with your local firewall and network settings.

#### Additional Notes

*   **Security Considerations**: Exercise caution regarding security. Although the tunnel is secure, it exposes your Redis instance to your local network.
*   **Alternative Approaches**: Some developers opt to run a local Redis instance for development to bypass these complexities.

#### Summary

To connect to a Redis instance hosted on Fly.io from your local machine, a secure tunnel is necessary. This tunnel effectively simulates a local Redis instance, enabling testing and development activities without exposing your Redis instance over the internet.

#### Example Code for Setting Up and Using the Fly.io Redis Tunnel

##### Step 1: Establish the Tunnel

To establish a tunnel between your local machine and the Redis instance on Fly.io, run the following command in your terminal:

```shell
fly redis connect
```

After running this command, you'll receive a local address, such as `localhost:10000`. This address will act as your local Redis endpoint.

##### Step 2: Connect to Redis in Your Application

In your application, you should typically use an environment variable for the Redis URL. When developing locally, set this environment variable to the local address provided by the `fly redis connect` command.

Here's an example in a Node.js application:

```js
const redis = require("redis");

// Local Redis URL for development
const LOCAL_REDIS_URL = 'redis://localhost:10000'; // Replace with your actual local address
const REDIS_URL = process.env.NODE_ENV === 'development' ? LOCAL_REDIS_URL : process.env.REDIS_URL;

const client = redis.createClient({
    url: REDIS_URL
});

client.on("error", function(error) {
  console.error(error);
});

// Rest of your Redis-related code
```

##### Step 3: Running Your Application Locally

Ensure that the Fly.io Redis tunnel is active when you run your application locally. Your application will connect to Redis through this tunnel, simulating a local instance.

```shell
npm start
```

**Important Notes:**

*   The `fly redis connect` tunnel should only be used for development and testing purposes.
*   Replace `LOCAL_REDIS_URL` in the sample code with the actual local address provided by `fly redis connect`.
*   Set your application's environment to 'development' when running locally to use the local Redis URL.


# Google Cloud Functions



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/google-cloud-functions" horizontal>
  You can find the project source code on GitHub.
</Card>

### Prerequisites

1.  [Create a Google Cloud Project.](https://cloud.google.com/resource-manager/docs/creating-managing-projects)
2.  [Enable billing for your project.](https://cloud.google.com/billing/docs/how-to/verify-billing-enabled#console)
3.  Enable Cloud Functions, Cloud Build, Artifact Registry, Cloud Run, Logging, and Pub/Sub APIs.

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli). Copy `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` for the next steps.

### Counter Function Setup & Deploy

1.  Go to [Cloud Functions](https://console.cloud.google.com/functions/list) in Google Cloud Console.
2.  Click **Create Function**.
3.  Setup **Basics and Trigger** Configuration like below:
    ![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/redis-gcloud/basics.png)
4.  Using your `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN`, setup **Runtime environment variables** under **Runtime, build, connections and privacy settings** like below.
    ![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/redis-gcloud/environment.png)
5.  Click **Next**.
6.  Set **Entry point** to `counter`.
7.  Update `index.js`

```js index.js
const { Redis } = require("@upstash/redis");
const functions = require('@google-cloud/functions-framework');

const redis = new Redis({
  url: process.env.UPSTASH_REDIS_REST_URL,
  token: process.env.UPSTASH_REDIS_REST_TOKEN
});

functions.http('counter', async (req, res) => {
  const count = await redis.incr("counter");
  res.send("Counter:" + count);
});
```

8.  Update `package.json` to include `@upstash/redis`.

```json package.json
{
  "dependencies": {
    "@google-cloud/functions-framework": "^3.0.0",
    "@upstash/redis": "^1.31.6"
  }
}
```

9.  Click **Deploy**.
10. Visit the given URL.


# Ion



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/ion" horizontal>
  You can find the project source code on GitHub.
</Card>

### Prerequisites

You need to have AWS credentials configured locally and SST CLI installed.

1.  [Create an AWS account](https://aws.amazon.com/)
2.  [Create an IAM user](https://sst.dev/chapters/create-an-iam-user.html)
3.  [Configure the AWS CLI](https://sst.dev/chapters/configure-the-aws-cli.html)
4.  [Setup SST CLI](https://ion.sst.dev/docs/reference/cli/)

### Project Setup

Let's create a new Next.js application.

```shell
npx create-next-app@latest
cd my-app
```

Let's initialize SST in our app.

```shell
sst init
```

Install the `@upstash/redis` package.

```shell
npm install @upstash/redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Pass the Environment Variables

```ts /sst.config.ts
/// <reference path="./.sst/platform/config.d.ts" />

export default $config({
  app(input) {
    return {
      name: "my-app",
      removal: input?.stage === "production" ? "retain" : "remove",
      home: "aws",
    };
  },
  async run() {
    new sst.aws.Nextjs("MyWeb", {
      environment: {
        UPSTASH_REDIS_REST_URL: process.env.UPSTASH_REDIS_REST_URL || "",
        UPSTASH_REDIS_REST_TOKEN: process.env.UPSTASH_REDIS_REST_TOKEN || "",
      },
    });
  },
});
```

### Home Page Setup

Update `/app/page.tsx`:

```tsx /app/page.tsx
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv();

export default async function Home() {
  const count = await redis.incr("counter");
  return (
    <div className="flex h-screen w-screen items-center justify-center">
      <h1 className="text-4xl font-bold">Counter: {count}</h1>
    </div>
  )
}
```

### Run

Run the SST app.

```shell
npm run dev
```

Check `http://localhost:3000/`

### Deploy

Deploy with SST.

```shell
sst deploy
```

Check the output URL.


# ioredis note



<Note>
  This example uses ioredis, you can copy the connection string from the `Node`
  tab in the console.
</Note>


# Koyeb



Integrate a serverless Upstash Redis database with your Koyeb applications. Combine the serverless features of Koyeb on the application side and Upstash for your key-value storage to deploy and scale applications globally with ease.

This guide explains how to connect an Upstash Redis data store as a database cache with an application running on Koyeb. To successfully follow this documentation, you will need to have:

*   A [Koyeb account](https://app.koyeb.com/) to deploy the application. You can optionally install the [Koyeb CLI](https://www.koyeb.com/docs/quickstart/koyeb-cli) to deploy the application from the command line
*   An [Upstash account](https://console.upstash.com/) to deploy the database
*   [Node.js](https://nodejs.org/en) and `npm` installed on your local machine to create the demo application.

If you already have a freshly created Upstash Redis database running and want to quickly preview how to connect your Upstash database to an application running on Koyeb, use the [Deploy to Koyeb](https://www.koyeb.com/docs/deploy-to-koyeb-button) button below.

[![Deploy to Koyeb](https://www.koyeb.com/static/images/deploy/button.svg)](https://app.koyeb.com/deploy?type=git\&repository=github.com/koyeb/example-koyeb-upstash\&branch=main\&name=example-koyeb-upstash\&env\[UPSTASH_REDIS_REST_URL]=REPLACE_ME\&env\[UPSTASH_REDIS_REST_TOKEN]=REPLACE_ME\&env\[PORT]=8000)

*Make sure to replace the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` environment variables with the values for your Upstash database.*

## Create an Upstash Redis database

To create an Upstash Redis database, sign into your [Upstash account](https://console.upstash.com/).

In the Upstash console, select **Redis** from the top navigation bar. On the Redis page, click **Create database**:

1.  In the **Name** field, choose a name for your database. In this example, we'll use `example-koyeb-upstash`.
2.  Select the **Type** of deployment you want. Because this is demo does not have global requirements, we will use "Regional" in this guide to limit the choices we have to make.
3.  In the **Region** drop-down menu, choose a location that's geographically convenient for your database and users. We use "N. Virginia (us-east-1)".
4.  Select your preferred options. In this example, we will select "TLS (SSL) Enabled" so that connections to the database are encrypted and "Eviction" so that older data will be purged when we run out of space.
5.  Click **Create** to provision the Redis database.

### Retrieve your Upstash URL and token

On your Upstash Redis page, scroll down to the **REST API** section of the page.

Click on the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` buttons to copy their respective values to your clipboard. Paste the copied values to a safe location so that you can reference them later when testing and deploying your application.

Alternatively, you can click on the `@upstash/redis` tab to view a code snippet:

```javascript
import { Redis } from "@upstash/redis";

const redis = new Redis({
  url: "<YOUR_UPSTASH_REDIS_REST_URL>",
  token: "<YOUR_UPSTASH_REDIS_REST_TOKEN>",
});

const data = await redis.set("foo", "bar");
```

When you copy the code block using the provided copy button, the code snippet, along with your database's URL and access token will be copied to your clipboard. While this works well for private or demonstration code, it generally isn't good practice to hard-code sensitive data like tokens within your application. To avoid this, we will configure the application to get these values from the environment instead.

## Create a demo application

Next, you can create a simple Node.js application that uses your Upstash Redis database. The application will use the [Express](https://expressjs.com/) web framework to build and serve a simple page and Upstash's own [`@upstash/redis`](/redis/sdks/ts/overview) package to connect to the database.

### Install the dependencies

Create a new directory for your demo application and navigate to the new location:

```bash
mkdir example-koyeb-upstash
cd example-koyeb-upstash
```

Within the new directory, generate a `package.json` file for the new project using the default settings:

```bash
npm init -y
```

Next, install the `@upstash/redis` package so that you can connect to your Redis database from within the application and the `express` package so that we can build a basic web application:

```bash
npm install @upstash/redis express
```

### Create the application file

Now, create a new file called `index.js` with the following contents:

```javascript
// Note: if you are using Node.js version 17 or lower,
//       change the first line to the following:
// const { Redis } = require ("@upstash/redis/with-fetch");
const { Redis } = require("@upstash/redis");
const express = require("express");

const app = express();
const redis = Redis.fromEnv();

app.get("/", async (req, res) => {
  const value = await redis.get("counter");
  await redis.set("counter", parseInt(value || 0) + 1);
  res.send(`Counter: ${value || 0}`);
});

const PORT = process.env.PORT || 3000;
app.listen(PORT, () => {
  console.log(`Server running on port ${PORT}`);
});
```

**Note:** If you are running Node.js version 17 or lower, you need to adjust the first line of the app to import from the `@upstash/redis/with-fetch` instead of `@upstash/redis`. Node.js versions prior to 18 did not natively support `fetch` API, so you need to change the import path in order to access that functionality.

This above code will introduce a simple `counter` key to your Redis database. It will use this key to store the number of times the page has been accessed and display that value on the page.

### Add the run scripts

Finally, edit the `package.json` file to define the scripts used to run the application. The `dev` script runs the application in debug mode while the `start` script starts the application normally:

```diff
{
  "name": "example-koyeb-upstash",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
+   "dev": "DEBUG=express:* node index.js",
+   "start": "node index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@upstash/redis": "^1.20.6",
    "express": "^4.18.2"
  }
}
```

## Run the demo application locally

Now that the project is set up, you can run the application locally verify that it functions correctly.

In your shell, set and export the variables you copied from your Upstash Redis page:

```bash
export UPSTASH_REDIS_REST_URL="<YOUR_UPSTASH_REDIS_REST_URL>"
export UPSTASH_REDIS_REST_TOKEN="<YOUR_UPSTASH_REDIS_REST_TOKEN>"
```

In the same terminal, you should now be able to test your application by typing:

```bash
npm run dev
```

The application server should start in debug mode, printing information about the process to the display. In your browser, navigate to `127.0.0.1:3000` to see your application. It should show the counter and number of visits you've made: "Counter: 0". The number should increase by one every time you refresh the page.

Press <kbd>CTRL</kbd>-<kbd>c</kbd> to stop the application when you are finished.

## Deploy the application to Koyeb using git-driven deployment

Once you've verified that the project runs locally, create a new Git repository to save your work.

Run the following commands to create a new Git repository within the project's root directory, commit the project files, and push changes to GitHub. Remember to replace the values of `<YOUR_GITHUB_USERNAME>` and `<YOUR_REPOSITORY_NAME>` with your own information:

```bash
git init
echo 'node_modules' >> .gitignore
git add .
git commit -m "Initial commit"
git remote add origin git@github.com:<YOUR_GITHUB_USERNAME>/<YOUR_REPOSITORY_NAME>.git
git push -u origin main
```

You can deploy the demo application to Koyeb and connect it to the Upstash Redis database using the [control panel](#via-the-koyeb-control-panel) or via the [Koyeb CLI](#via-the-koyeb-cli).

### Via the Koyeb control panel

To deploy the using the [control panel](https://app.koyeb.com/), follow these steps:

1.  Click **Create App** in the Koyeb control panel.
2.  Select **GitHub** as the deployment option.
3.  Choose the GitHub **repository** and **branch** containing your application code.
4.  Name your service, for example `upstash-service`.
5.  Click **Advanced** to view additional options. Under **Environment variables**, click **Add Variable** to add two new variables called `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN`. Populate them with the values you copied for your Upstash Redis database.
6.  Name the App, for example `upstash-demo`.
7.  Click the **Deploy** button.

A Koyeb App and Service will be created. Your application will be built and deployed to Koyeb. Once the build has finished, you will be able to access your application running on Koyeb by clicking the URL ending with `.koyeb.app`.

### Via the Koyeb CLI

To deploy the example application using the [Koyeb CLI](https://www.koyeb.com/docs/cli/installation), run the following command in your terminal:

```bash
koyeb app init example-koyeb-upstash \
  --git github.com/<YOUR_GITHUB_USERNAME>/<YOUR_REPOSITORY_NAME> \
  --git-branch main \
  --ports 3000:http \
  --routes /:3000 \
  --env PORT=3000 \
  --env UPSTASH_REDIS_REST_URL="<YOUR_UPSTASH_REDIS_REST_URL>" \
  --env UPSTASH_REDIS_REST_TOKEN="<YOUR_UPSTASH_REDIS_REST_TOKEN>"
```

*Make sure to replace `<YOUR_GITHUB_USERNAME>/<YOUR_REPOSITORY_NAME>` with your GitHub username and repository name and replace `<YOUR_UPSTASH_REDIS_REST_URL>` and `<YOUR_UPSTASH_REDIS_REST_TOKEN>` with the values copied from your Upstash Redis page.*

#### Access deployment logs

To track the app deployment and view the build logs, execute the following command:

```bash
koyeb service logs example-koyeb-upstash/example-koyeb-upstash -t build
```

#### Access your app

Once the deployment of your application has finished, you can retrieve the public domain to access your application by running the following command:

```bash
$ koyeb app get example-koyeb-upstash
ID      	NAME         	        DOMAINS                          	        CREATED AT
85c78d9a	example-koyeb-upstash	["example-koyeb-upstash-myorg.koyeb.app"]	31 May 23 13:08 UTC
```

#### Access runtime logs

With your app running, you can track the runtime logs by running the following command:

```bash
koyeb service logs example-koyeb-upstash/example-koyeb-upstash -t runtime
```

## Deploy the application to Koyeb using a pre-built container

As an alternative to using git-driven deployment, you can deploy a pre-built container from any public or private registry. This can be useful if your application needs specific system dependencies or you need more control over how the build is performed.

To dockerize the application, start by adding a file called `.dockerignore` to the project's root directory. Paste the following contents to limit the files copied to the Docker image:

```
Dockerfile
.dockerignore
.git
node_modules
npm-debug.log
/.cache
.env
README.md
```

Afterwards, create a `Dockerfile` in your project root directory and copy the content below:

```dockerfile
FROM node:18-alpine AS base

FROM base AS deps
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci

FROM base AS runner
WORKDIR /app
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nodejs
COPY --from=deps /app/node_modules ./node_modules
COPY . .
USER node
EXPOSE 3000
ENV PORT 3000
CMD ["npm", "run", "start"]
```

The Dockerfile above provides the minimum requirements to run the sample Node.js application. You can easily extend it depending on your needs.

*Be sure to set the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` environment variables to the values you copied from the Upstash console when you deploy the container in the Koyeb control panel.*

To build and push the Docker image to a registry and deploy it on Koyeb, refer to the [Deploy an app from a Docker image](https://www.koyeb.com/docs/quickstart/deploy-a-docker-application) documentation.

A Koyeb App and Service will be created. Your Docker image will be pulled and deployed to Koyeb. Once the deployment has finished, you will be able to access your application running on Koyeb by clicking the URL ending with `.koyeb.app`.

## Delete the example application and Upstash Redis database

To delete the example application and the Upstash Redis database and avoid incurring any charges, follow these steps:

*   From the [Upstash console](https://console.upstash.com/), select your Redis database and scroll to the bottom of the **Details** page. Click **Delete this database** and follow the instructions.
*   From the [Koyeb control panel](https://app.koyeb.com/), select your App. Click the **Settings** tab, and click the **Danger Zone**. Click **Delete App** and follow the instructions. Alternatively, from the CLI, you can delete your Koyeb App and service by typing `koyeb app delete example-koyeb-upstash`.


# App Router



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/nextjs-app-router" horizontal>
  You can find the project source code on GitHub.
</Card>

### Project Setup

Let's create a new Next.js application with App Router and install `@upstash/redis` package.

```shell
npx create-next-app@latest
cd my-app
npm install @upstash/redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Home Page Setup

Update `/app/page.tsx`:

```tsx /app/page.tsx
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv();

export default async function Home() {
  const count = await redis.incr("counter");
  return (
    <div className="flex h-screen w-screen items-center justify-center">
      <h1 className="text-4xl font-bold">Counter: {count}</h1>
    </div>
  )
}
```

### Run & Deploy

Run the app locally with `npm run dev`, check `http://localhost:3000/`

Deploy your app with `vercel`

<Info>
  You can also integrate your Vercel projects with Upstash using Vercel
  Integration module. Check [this article](../howto/vercelintegration).
</Info>


# Pages Router



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/nextjs-pages-router" horizontal>
  You can find the project source code on GitHub.
</Card>

### Project Setup

Let's create a new Next.js application with Pages Router and install `@upstash/redis` package.

```shell
npx create-next-app@latest
cd my-app
npm install @upstash/redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Home Page Setup

Update `/pages/index.tsx`:

```tsx /pages/index.tsx
import type { InferGetServerSidePropsType, GetServerSideProps } from 'next'
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv();

export const getServerSideProps = (async () => {
  const count = await redis.incr("counter");
  return { props: { count } }
}) satisfies GetServerSideProps<{ count: number }>

export default function Home({
  count,
}: InferGetServerSidePropsType<typeof getServerSideProps>) {
  return (
    <div className="flex h-screen w-screen items-center justify-center">
      <h1 className="text-4xl font-bold">Counter: {count}</h1>
    </div>
  )
}
```

### Run & Deploy

Run the app locally with `npm run dev`, check `http://localhost:3000/`

Deploy your app with `vercel`

<Info>
  You can also integrate your Vercel projects with Upstash using Vercel
  Integration module. Check [this article](../howto/vercelintegration).
</Info>


#  AWS Lambda



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/aws-cdk-python" horizontal>
  You can find the project source code on GitHub.
</Card>

### Prerequisites

*   Complete all steps in [Getting started with the AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html)

### Project Setup

Create and navigate to a directory named `counter-cdk`. CDK CLI uses this directory name to name things in your CDK code, so if you decide to use a different name, don't forget to make the appropriate changes when applying this tutorial.

```shell
mkdir counter-cdk && cd counter-cdk
```

Initialize a new CDK project.

```shell
cdk init app --language typescript
```

### Counter Function Setup

Create a folder named `api` under `lib`

```shell
mkdir lib/api
```

Create `/lib/api/requirements.txt`

```txt /lib/api/requirements.txt
upstash-redis
```

Create `/lib/api/index.py`

```py /lib/api/index.py
from upstash_redis import Redis

redis = Redis.from_env()

def handler(event, context):
    count = redis.incr('counter')
    return {
        'statusCode': 200,
        'body': f'Counter: {count}'
    }
```

### Counter Stack Setup

Update `/lib/counter-cdk-stack.ts`

```ts /lib/counter-cdk-stack.ts
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as path from 'path';

export class CounterCdkStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const counterFunction = new lambda.Function(this, 'CounterFunction', {
      code: lambda.Code.fromAsset(path.join(__dirname, 'api'), {
        bundling: {
          image: lambda.Runtime.PYTHON_3_9.bundlingImage,
          command: [
            'bash', '-c',
            'pip install -r requirements.txt -t /asset-output && cp -au . /asset-output'
          ],
        },
      }),
      runtime: lambda.Runtime.PYTHON_3_9,
      handler: 'index.handler',
      environment: {
        UPSTASH_REDIS_REST_URL: process.env.UPSTASH_REDIS_REST_URL || '',
        UPSTASH_REDIS_REST_TOKEN: process.env.UPSTASH_REDIS_REST_TOKEN || '',
      },
    });

    const counterFunctionUrl = counterFunction.addFunctionUrl({
      authType: lambda.FunctionUrlAuthType.NONE,
    });

    new cdk.CfnOutput(this, "counterFunctionUrlOutput", {
      value: counterFunctionUrl.url,
    })
  }
}
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and export `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment.

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Deploy

Run in the top folder:

```shell
cdk synth
cdk bootstrap
cdk deploy
```

Visit the output url.


# SST v2



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/sst-v2" horizontal>
  You can find the project source code on GitHub.
</Card>

### Prerequisites

You need to have AWS credentials configured locally.

1.  [Create an AWS account](https://aws.amazon.com/)
2.  [Create an IAM user](https://sst.dev/chapters/create-an-iam-user.html)
3.  [Configure the AWS CLI](https://sst.dev/chapters/configure-the-aws-cli.html)

### Project Setup

Let's create a new SST + Next.js application.

```shell
npx create-sst@latest --template standard/nextjs
cd my-sst-app
npm install
```

Install the `@upstash/redis` package.

```shell
npm install @upstash/redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell
npx sst secrets set UPSTASH_REDIS_REST_URL <YOUR_URL>
npx sst secrets set UPSTASH_REDIS_REST_TOKEN <YOUR_TOKEN>
```

### Bind the Secrets

```ts /stacks/Default.ts
import { Config, StackContext, NextjsSite } from "sst/constructs";

export function Default({ stack }: StackContext) {
  const UPSTASH_REDIS_REST_URL = new Config.Secret(stack, "UPSTASH_REDIS_REST_URL");
  const UPSTASH_REDIS_REST_TOKEN = new Config.Secret(stack, "UPSTASH_REDIS_REST_TOKEN");
  const site = new NextjsSite(stack, "site", {
    bind: [UPSTASH_REDIS_REST_URL, UPSTASH_REDIS_REST_TOKEN],
    path: "packages/web",
  });
  stack.addOutputs({
    SiteUrl: site.url,
  });
}
```

### API Setup

```ts /packages/web/pages/api/hello.ts
import { Redis } from "@upstash/redis";
import type { NextApiRequest, NextApiResponse } from "next";
import { Config } from "sst/node/config";

const redis = new Redis({
  url: Config.UPSTASH_REDIS_REST_URL,
  token: Config.UPSTASH_REDIS_REST_TOKEN,
  });

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse,
) {
  const count = await redis.incr("counter");
  res.status(200).json({ count });
}

```

### Run

Run the SST app.

```shell
npm run dev
```

After prompted, run the Next.js app.

```shell
cd packages/web
npm run dev
```

Check `http://localhost:3000/api/hello`

### Deploy

Set the secrets for the prod stage.

```shell
npx sst secrets set --stage prod UPSTASH_REDIS_REST_URL <YOUR_URL>
npx sst secrets set --stage prod UPSTASH_REDIS_REST_TOKEN <YOUR_TOKEN>
```

Deploy with SST.

```shell
npx sst deploy --stage prod
```

Check `<SiteUrl>/api/hello` with the given SiteUrl.


# Supabase Functions



The below is an example for a Redis counter that stores a
[hash](https://redis.io/commands/hincrby/) of Supabase function invocation count
per region.

## Redis database setup

Create a Redis database using the
[Upstash Console](https://console.upstash.com/) or
[Upstash CLI](https://github.com/upstash/cli).

Select the `Global` type to minimize the latency from all edge locations. Copy
the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your .env file.

You'll find them under **Details > REST API > .env**.

```shell
cp supabase/functions/upstash-redis-counter/.env.example supabase/functions/upstash-redis-counter/.env
```

## Code

Make sure you have the latest version of the
[Supabase CLI installed](https://supabase.com/docs/guides/cli#installation).

Create a new function in your project:

```shell
supabase functions new upstash-redis-counter
```

And add the code to the `index.ts` file:

```ts index.ts
import { serve } from "https://deno.land/std@0.177.0/http/server.ts";
import { Redis } from "https://deno.land/x/upstash_redis@v1.19.3/mod.ts";
console.log(`Function "upstash-redis-counter" up and running!`);
serve(async (_req) => {
  try {
    const redis = new Redis({
      url: Deno.env.get("UPSTASH_REDIS_REST_URL")!,
      token: Deno.env.get("UPSTASH_REDIS_REST_TOKEN")!,
    });
    const deno_region = Deno.env.get("DENO_REGION");
    if (deno_region) {
      // Increment region counter
      await redis.hincrby("supa-edge-counter", deno_region, 1);
    } else {
      // Increment localhost counter
      await redis.hincrby("supa-edge-counter", "localhost", 1);
    }
    // Get all values
    const counterHash: Record<string, number> | null = await redis.hgetall(
      "supa-edge-counter"
    );
    const counters = Object.entries(counterHash!)
      .sort(([, a], [, b]) => b - a) // sort desc
      .reduce(
        (r, [k, v]) => ({
          total: r.total + v,
          regions: { ...r.regions, [k]: v },
        }),
        {
          total: 0,
          regions: {},
        }
      );
    return new Response(JSON.stringify({ counters }), { status: 200 });
  } catch (error) {
    return new Response(JSON.stringify({ error: error.message }), {
      status: 200,
    });
  }
});
```

## Run locally

```bash
supabase start
supabase functions serve upstash-redis-counter --no-verify-jwt --env-file supabase/functions/upstash-redis-counter/.env
```

Navigate to [http://localhost:54321/functions/v1/upstash-redis-counter](http://localhost:54321/functions/v1/upstash-redis-counter).

## Deploy

```bash
supabase functions deploy upstash-redis-counter --no-verify-jwt
supabase secrets set --env-file supabase/functions/upstash-redis-counter/.env
```


# App Router



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/vercel-functions-app-router" horizontal>
  You can find the project source code on GitHub.
</Card>

### Project Setup

Let's create a new Next.js application with App Router and install `@upstash/redis` package.

```shell
npx create-next-app@latest
cd my-app
npm install @upstash/redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Function Setup

<Info>
  This is a Vercel Serverless Function. If you want to use Edge Runtime, you can add the `export const runtime = 'edge'` line to this Route Handler.
</Info>

Create `/app/api/hello/route.ts`:

```ts /app/api/hello/route.ts
import { Redis } from "@upstash/redis";
import { NextResponse } from "next/server";

const redis = Redis.fromEnv();

export async function GET() {
    const count = await redis.incr("counter");
    return NextResponse.json({ count });
}

export const dynamic = 'force-dynamic'
```

### Run & Deploy

Run the app locally with `npm run dev`, check `http://localhost:3000/api/hello`

Deploy your app with `vercel`

<Info>
  You can also integrate your Vercel projects with Upstash using Vercel
  Integration module. Check [this article](../howto/vercelintegration).
</Info>


# Pages Router



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/vercel-functions-pages-router" horizontal>
  You can find the project source code on GitHub.
</Card>

<Info>
  This is a quickstart for Vercel Serverless Functions. If you want to use Edge Runtime, Vercel recommends icrementally adopting the App Router.
</Info>

### Project Setup

Let's create a new Next.js application with Pages Router and install `@upstash/redis` package.

```shell
npx create-next-app@latest
cd my-app
npm install @upstash/redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Function Setup

Update `/pages/api/hello.ts`:

```ts /pages/api/hello.ts
import { Redis } from "@upstash/redis";
import type { NextApiRequest, NextApiResponse } from "next";

const redis = Redis.fromEnv();

export default async function handler(
  req: NextApiRequest,
  res: NextApiResponse,
) {
  const count = await redis.incr("counter");
  res.status(200).json({ count });
}
```

### Run & Deploy

Run the app locally with `npm run dev`, check `http://localhost:3000/api/hello`

Deploy your app with `vercel`

<Info>
  You can also integrate your Vercel projects with Upstash using Vercel
  Integration module. Check [this article](../howto/vercelintegration).
</Info>


# Vercel Python Runtime



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/vercel-python-runtime-django" horizontal>
  You can find the project source code on GitHub.
</Card>

<Info>
  This quickstart uses django but you can easily adapt it to Flask, FastAPI or plain Python, see [Vercel Python Templates](https://vercel.com/templates?framework=python).
</Info>

### Project Setup

Let's create a new django application from Vercel's template.

```shell
npx create-next-app vercel-django --example "https://github.com/vercel/examples/tree/main/python/django"
cd vercel-django
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and export `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment.

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Environment Setup

Update `requirements.txt` to include `upstash-redis`.

```txt requirements.txt
Django==4.1.3
upstash-redis
```

We will create a Conda environment with python version `3.12` to match Vercel Python Runtime and avoid conflicts on deployment, you can use any other environment management system.

```shell
conda create --name vercel-django python=3.12
conda activate vercel-django
pip install -r requirements.txt
```

### View Setup

Update `/example/views.py`:

```py /example/views.py
from datetime import datetime

from django.http import HttpResponse

from upstash_redis import Redis

redis = Redis.from_env()

def index(request):
    count = redis.incr('counter')
    html = f'''
    <html>
        <body>
            <h1>Counter: { count }</h1p>
        </body>
    </html>
    '''
    return HttpResponse(html)
```

### Run & Deploy

Run the app locally with `python manage.py runserver`, check `http://localhost:8000/`

Deploy your app with `vercel`

Set `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` in your project's Settings -> Environment Variables. Redeploy from Deployments tab.

<Info>
  You can also integrate your Vercel projects with Upstash using Vercel
  Integration module. Check [this article](../howto/vercelintegration).
</Info>


# ECHO



Returns a message back to you. Useful for debugging the connection.

## Arguments

<ParamField body="message" type="str" required>
  A message to send to the server.
</ParamField>

## Response

<ResponseField type="str" required>
  The same message you sent.
</ResponseField>

<RequestExample>
  ```py Example
  assert redis.echo("hello world") == "hello world"
  ```
</RequestExample>


# PING

Send a ping to the server and get a response if the server is alive.

## Arguments

No arguments

## Response

<ResponseField type="str" required>
  `PONG`
</ResponseField>

<RequestExample>
  ```py Example
  assert redis.ping() == "PONG"
  ```
</RequestExample>


# BITCOUNT

Count the number of set bits.

The `BITCOUNT` command in Redis is used to count the number of set bits (bits with a value of 1) in a range of bytes within a key that is stored as a binary string. It is primarily used for bit-level operations on binary data stored in Redis.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="start" type="int">
  Specify the range of bytes within the binary string to count the set bits. If not provided, it counts set bits in the entire string.

  Either specify both `start` and `end` or neither.
</ParamField>

<ParamField body="end" type="int">
  Specify the range of bytes within the binary string to count the set bits. If not provided, it counts set bits in the entire string.

  Either specify both `start` and `end` or neither.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of set bits in the specified range.
</ResponseField>

<RequestExample>
  ```py Example
  redis.setbit("mykey", 7, 1)
  redis.setbit("mykey", 8, 1)
  redis.setbit("mykey", 9, 1)

  # With range
  assert redis.bitcount("mykey", 0, 10) == 3

  # Without range
  assert redis.bitcount("mykey") == 3
  ```
</RequestExample>


# BITFIELD

Sets or gets parts of a bitfield

The `bitfield` function returns a `BitFieldCommands` instance that can be used
to execute multiple bitfield operations in a single command.

The encoding can be a signed or unsigned integer, by prefixing the type with
`i` or `u`. For example, `i4` is a signed 4-bit integer, and `u8` is an
unsigned 8-bit integer.

```py
redis.set("mykey", "")

# Sets the first 4 bits to 1
# Increments the next 4 bits by 1
result = redis.bitfield("mykey")
        .set("u4", 0, 16)
        .incr("u4", 4, 1)
        .execute()

assert result == [0, 1]
```

## Commands

### `get(type: str, offset: int)`

Returns a value from the bitfield at the given offset.

### `set(type: str, offset: int, value: int)`

Sets a value and returns the old value.

### `incr(type: str, offset: int, increment: int)`

Increments a value and returns the new value.

## Arguments

<ParamField body="key" type="str" required>
  The string key to operate on.
</ParamField>

## Response

<ResponseField type="List[int]" required>
  A list of integers, one for each operation.
</ResponseField>

<RequestExample>
  ```py Get
  redis.set("mykey", "\x05\x06\x07")

  result = redis.bitfield("mykey") \
      .get("u8", 0) \
      .get("u8", 8) \
      .get("u8", 16) \
      .execute()

  assert result == [5, 6, 7]
  ```

  ```py Set
  redis.set("mykey", "")

  result = redis.bitfield("mykey") \
      .set("u4", 0, 16) \
      .set("u4", 4, 1) \
      .execute()
    
  assert result == [0, 1]
  ```

  ```py Incr
  redis.set("mykey", "")

  # Increment offset 0 by 16, return 
  # Increment offset 4 by 1

  result = redis.bitfield("mykey") \
      .incr("u4", 0, 16) \
      .incr("u4", 4, 1) \
      .execute()

  assert result == [0, 1]
  ```
</RequestExample>


# BITOP

Perform bitwise operations between strings.

The `BITOP` command in Redis is used to perform bitwise operations on multiple keys (or Redis strings) and store the result in a destination key. It is primarily used for performing logical AND, OR, XOR, and NOT operations on binary data stored in Redis.

## Arguments

<ParamField body="operation" type="AND | OR | XOR | NOT" required>
  Specifies the type of bitwise operation to perform, which can be one of the
  following: `AND`, `OR`, `XOR`, or `NOT`.
</ParamField>

<ParamField body="destkey" type="str" required>
  The key to store the result of the operation in.
</ParamField>

<ParamField body="keys" type="*List[str]" required>
  One or more keys to perform the operation on.
</ParamField>

## Response

<ResponseField type="int" required>
  The size of the string stored in the destination key.
</ResponseField>

<RequestExample>
  ```py Example
  # key1 = 00000001
  # key2 = 00000010
  redis.setbit("key1", 0, 1)
  redis.setbit("key2", 0, 0)
  redis.setbit("key2", 1, 1)

  assert redis.bitop("AND", "dest", "key1", "key2") == 1

  # result = 00000000
  assert redis.getbit("dest", 0) == 0
  assert redis.getbit("dest", 1) == 0
  ```
</RequestExample>


# BITPOS

Find the position of the first set or clear bit (bit with a value of 1 or 0) in a Redis string key.

## Arguments

<ParamField body="key" type="str" required>
  The key to search in.
</ParamField>

<ParamField body="bit" type="0 | 1" required>
  The key to store the result of the operation in.
</ParamField>

<ParamField body="start" type="int">
  The index to start searching at.
</ParamField>

<ParamField body="end" type="int">
  The index to stop searching at.
</ParamField>

## Response

<ResponseField type="int" required>
  The index of the first occurrence of the bit in the string.
</ResponseField>

<RequestExample>
  ```py Example
  redis.setbit("mykey", 7, 1)
  redis.setbit("mykey", 8, 1)

  assert redis.bitpos("mykey", 1) == 7
  assert redis.bitpos("mykey", 0) == 0

  # With a range
  assert redis.bitpos("mykey", 1, 0, 2) == 0
  assert redis.bitpos("mykey", 1, 2, 3) == -1
  ```

  ```py With Range
  redis.bitpos("key", 1, 5, 20)
  ```
</RequestExample>


# GETBIT

Retrieve a single bit.

## Arguments

<ParamField body="key" type="str" required>
  The key of the bitset
</ParamField>

<ParamField body="offset" type="int" required>
  Specify the offset at which to get the bit.
</ParamField>

## Response

<ResponseField type="int" required>
  The bit value stored at offset.
</ResponseField>

<RequestExample>
  ```py Example
  bit = redis.getbit(key, 4)
  ```
</RequestExample>


# SETBIT

Set a single bit in a string.

## Arguments

<ParamField body="key" type="str" required>
  The key of the bitset
</ParamField>

<ParamField body="offset" type="int" required>
  Specify the offset at which to set the bit.
</ParamField>

<ParamField body="value" type="0 | 1" required>
  The bit to set
</ParamField>

## Response

<ResponseField type="0 | 1" required>
  The original bit value stored at offset.
</ResponseField>

<RequestExample>
  ```py Example
  original_bit = redis.setbit(key, 4, 1)
  ```
</RequestExample>


# DEL

Removes the specified keys. A key is ignored if it does not exist.

## Arguments

<ParamField body="keys" type="*List[str]" required>
  One or more keys to remove.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of keys that were removed.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key1", "Hello")
  redis.set("key2", "World")
  redis.delete("key1", "key2")

  assert redis.get("key1") is None
  assert redis.get("key2") is None
  ```
</RequestExample>


# EXISTS

Check if a key exists.

## Arguments

<ParamField body="keys" type="*List[str]" required>
  One or more keys to check.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of keys that exist
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key1", "Hello")
  redis.set("key2", "World")

  assert redis.exists("key1", "key2") == 2

  redis.delete("key1")

  assert redis.exists("key1", "key2") == 1
  ```
</RequestExample>


# EXPIRE

Sets a timeout on key. The key will automatically be deleted.

## Arguments

<ParamField body="key" type="str" required>
  The key to set the timeout on.
</ParamField>

<ParamField body="seconds" type="int | datetime.timedelta" required>
  The timeout in seconds as int or datetime.timedelta object
</ParamField>

<ParamField body="nx" type="bool">
  Set expiry only when the key has no expiry
</ParamField>

<ParamField body="xx" type="bool">
  Set expiry only when the key has an existing expiry
</ParamField>

<ParamField body="gt" type="bool">
  Set expiry only when the new expiry is greater than current one
</ParamField>

<ParamField body="lt" type="bool">
  Set expiry only when the new expiry is less than current one
</ParamField>

## Response

<ResponseField type="bool">
  `True` if the timeout was set
</ResponseField>

<RequestExample>
  ```py Example
  # With seconds
  redis.set("mykey", "Hello")
  redis.expire("mykey", 5)

  assert redis.get("mykey") == "Hello"

  time.sleep(5)

  assert redis.get("mykey") is None

  # With a timedelta
  redis.set("mykey", "Hello")
  redis.expire("mykey", datetime.timedelta(seconds=5))
  ```
</RequestExample>


# EXPIREAT

Sets a timeout on key. The key will automatically be deleted.

## Arguments

<ParamField body="key" type="str" required>
  The key to set the timeout on.
</ParamField>

<ParamField body="unix_time_seconds" type="int | datetime.datetime" required>
  The timeout in unix seconds timestamp as int or a datetime.datetime object.
</ParamField>

<ParamField body="nx" type="bool">
  Set expiry only when the key has no expiry
</ParamField>

<ParamField body="xx" type="bool">
  Set expiry only when the key has an existing expiry
</ParamField>

<ParamField body="gt" type="bool">
  Set expiry only when the new expiry is greater than current one
</ParamField>

<ParamField body="lt" type="bool">
  Set expiry only when the new expiry is less than current one
</ParamField>

## Response

<ResponseField type="bool">
  `True` if the timeout was set
</ResponseField>

<RequestExample>
  ```py Example
  # With a datetime object
  redis.set("mykey", "Hello")
  redis.expireat("mykey", datetime.datetime.now() + datetime.timedelta(seconds=5))

  # With a unix timestamp
  redis.set("mykey", "Hello")
  redis.expireat("mykey", int(time.time()) + 5)
  ```
</RequestExample>


# KEYS

Returns all keys matching pattern.

<Warning>
  This command may block the DB for a long time, depending on its size. We advice against using it in production. Use [SCAN](/redis/sdks/py/commands/generic/scan) instead.
</Warning>

## Arguments

<ParamField body="match" type="str" required>
  A glob-style pattern. Use `*` to match all keys.
</ParamField>

## Response

<ResponseField type="List[str]">
  Array of keys matching the pattern.
</ResponseField>

<RequestExample>
  ```py Example
  keys = redis.keys("prefix*")
  ```

  ```py Match All
  keys = redis.keys("*")
  ```
</RequestExample>


# PERSIST

Remove any timeout set on the key.

## Arguments

<ParamField body="key" type="str" required>
  The key to persist
</ParamField>

## Response

<ResponseField type="bool">
  `True` if the timeout was set
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key1", "Hello")
  redis.expire("key1", 10)

  assert redis.ttl("key1") == 10

  redis.persist("key1")

  assert redis.ttl("key1") == -1
  ```
</RequestExample>


# PEXPIRE

Sets a timeout on key. After the timeout has expired, the key will automatically be deleted.

## Arguments

<ParamField body="key" type="str" required>
  The key to expire.
</ParamField>

<ParamField body="milliseconds | datetime.timedelta" type="int" required>
  The timeout in milliseconds as int or datetime.timedelta
</ParamField>

<ParamField body="nx" type="bool">
  Set expiry only when the key has no expiry
</ParamField>

<ParamField body="xx" type="bool">
  Set expiry only when the key has an existing expiry
</ParamField>

<ParamField body="gt" type="bool">
  Set expiry only when the new expiry is greater than current one
</ParamField>

<ParamField body="lt" type="bool">
  Set expiry only when the new expiry is less than current one
</ParamField>

## Response

<ResponseField type="bool">
  `True` if the timeout was set
</ResponseField>

<RequestExample>
  ```py Example
  # With milliseconds
  redis.set("mykey", "Hello")
  redis.expire("mykey", 500)

  # With a timedelta
  redis.set("mykey", "Hello")
  redis.expire("mykey", datetime.timedelta(milliseconds=500))
  ```
</RequestExample>


# PEXPIREAT

Sets a timeout on key. After the timeout has expired, the key will automatically be deleted.

## Arguments

<ParamField body="key" type="str" required>
  The key to expire.
</ParamField>

<ParamField body="unix_time_milliseconds" type="int | datetime.datetime" required>
  The timeout in unix milliseconds timestamp as int or a datetime.datetime object.
</ParamField>

<ParamField body="nx" type="bool">
  Set expiry only when the key has no expiry
</ParamField>

<ParamField body="xx" type="bool">
  Set expiry only when the key has an existing expiry
</ParamField>

<ParamField body="gt" type="bool">
  Set expiry only when the new expiry is greater than current one
</ParamField>

<ParamField body="lt" type="bool">
  Set expiry only when the new expiry is less than current one
</ParamField>

## Response

<ResponseField type="bool">
  `True` if the timeout was set
</ResponseField>

<RequestExample>
  ```py Example
  # With a unix timestamp
  redis.set("mykey", "Hello")
  redis.pexpireat("mykey", int(time.time() * 1000) )

  # With a datetime object
  redis.set("mykey", "Hello")
  redis.pexpireat("mykey", datetime.datetime.now() + datetime.timedelta(seconds=5))
  ```
</RequestExample>


# PTTL

Return the expiration in milliseconds of a key.

## Arguments

<ParamField body="key" type="str" required>
  The key
</ParamField>

## Response

<ResponseField type="int" required>
  The number of milliseconds until this expires, negative if the key does not exist or does not have an expiration set.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key1", "Hello")

  assert redis.pttl("key1") == -1

  redis.expire("key1", 1000)

  assert redis.pttl("key1") > 0

  redis.persist("key1")

  assert redis.pttl("key1") == -1
  ```
</RequestExample>


# RANDOMKEY

Returns a random key from database

## Arguments

No arguments

## Response

<ResponseField type="str">
  A random key from database, or `None` when database is empty.
</ResponseField>

<RequestExample>
  ```py Example
  assert redis.randomkey() is None

  redis.set("key1", "Hello")
  redis.set("key2", "World")

  assert redis.randomkey() is not None
  ```
</RequestExample>


# RENAME

Rename a key

Renames a key and overwrites the new key if it already exists.

Throws an exception if the key does not exist.

## Arguments

<ParamField body="source" type="str" required>
  The original key.
</ParamField>

<ParamField body="destination" type="str" required>
  A new name for the key.
</ParamField>

## Response

<ResponseField type="bool" required>
  `True` if key was renamed
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key1", "Hello")
  redis.rename("key1", "key2")

  assert redis.get("key1") is None
  assert redis.get("key2") == "Hello"

  # Renaming a nonexistent key throws an exception
  redis.rename("nonexistent", "key3")
  ```
</RequestExample>


# RENAMENX

Rename a key if it does not already exist.

Renames a key, only if the new key does not exist.

Throws an exception if the key does not exist.

## Arguments

<ParamField body="source" type="str" required>
  The original key.
</ParamField>

<ParamField body="destination" type="str" required>
  A new name for the key.
</ParamField>

## Response

<ResponseField type="bool" required>
  `True` if key was renamed
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key1", "Hello")
  redis.set("key2", "World")

  # Rename failed because "key2" already exists.
  assert redis.renamenx("key1", "key2") == False

  assert redis.renamenx("key1", "key3") == True

  assert redis.get("key1") is None
  assert redis.get("key2") == "World"
  assert redis.get("key3") == "Hello"
  ```
</RequestExample>


# SCAN

Scan the database for keys.

## Arguments

<ParamField body="cursor" type="int" required>
  The cursor, use `0` in the beginning and then use the returned cursor for subsequent calls.
</ParamField>

<ParamField body="match" type="str" required>
  Glob-style pattern to filter by field names.
</ParamField>

<ParamField body="count" type="int" required>
  Number of fields to return per call.
</ParamField>

<ParamField body="type" type="str">
  Filter by type.
  For example `string`, `hash`, `set`, `zset`, `list`, `stream`.
</ParamField>

## Response

<ResponseField type="Tuple[int, List[str]]" required>
  The new cursor and the keys as a tuple.
  If the new cursor is `0` the iteration is complete.

  Use the new cursor for subsequent calls.
</ResponseField>

<RequestExample>
  ```py Example
  # Get all keys

  cursor = 0
  results = []

  while True:
      cursor, keys = redis.scan(cursor, match="*")

      results.extend(keys)
      if cursor == 0:
          break
  ```
</RequestExample>


# TOUCH

Alters the last access time of one or more keys

## Arguments

<ParamField body="keys" type="*List[str]" required>
  One or more keys.
</ParamField>

## Response

<ResponseField type="int">
  The number of keys that were touched.
</ResponseField>

<RequestExample>
  ```py Example
  redis.touch("key1", "key2", "key3")
  ```
</RequestExample>


# TTL

Return the expiration in seconds of a key.

## Arguments

<ParamField body="key" type="str" required>
  The key
</ParamField>

## Response

<ResponseField type="int" required>
  The number of seconds until this expires, negative if the key does not exist or does not have an expiration set.
</ResponseField>

<RequestExample>
  ```py Example
  # Get the TTL of a key
  redis.set("my-key", "value")

  assert redis.ttl("my-key") == -1

  redis.expire("my-key", 10)

  assert redis.ttl("my-key") > 0

  # Non existent key
  assert redis.ttl("non-existent-key") == -2
  ```
</RequestExample>


# TYPE

Get the type of a key.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

## Response

<ResponseField type="str" required>
  The type of the key.

  One of `string` | `list` | `set` | `zset` | `hash` | `none`
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key1", "Hello")

  assert redis.type("key1") == "string"

  redis.lpush("key2", "Hello")

  assert redis.type("key2") == "list"

  assert redis.type("non-existent-key") == "none"
  ```
</RequestExample>


# UNLINK

Removes the specified keys. A key is ignored if it does not exist.

## Arguments

<ParamField body="keys" type="*List[str]" required>
  One or more keys to unlink.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of keys that were unlinked.
</ResponseField>

<RequestExample>
  ```py Basic
  assert redis.unlink("key1", "key2", "key3") == 3
  ```
</RequestExample>


# HDEL

Deletes one or more hash fields.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="fields" type="*List[str]" required>
  One or more fields to delete.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of fields that were removed from the hash.
</ResponseField>

<RequestExample>
  ```py Example
  redis.hset("myhash", "field1", "Hello")
  redis.hset("myhash", "field2", "World")

  assert redis.hdel("myhash", "field1", "field2") == 2
  ```
</RequestExample>


# HEXISTS

Checks if a field exists in a hash.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="field" type="str" required>
  The field to check.
</ParamField>

## Response

<ResponseField type="bool" required>
  `True` if the hash contains `field`. `False` if the hash does not contain `field`, or `key` does not exist.
</ResponseField>

<RequestExample>
  ```py Example
  redis.hset("key", "field", "value")

  assert redis.hexists("key", "field") == True
  ```
</RequestExample>


# HGET

Retrieves the value of a hash field.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="field" type="str" required>
  The field to get.
</ParamField>

## Response

<ResponseField type="Optional[str]">
  The value of the field, or `null`, when field is not present in the hash or key does not exist.
</ResponseField>

<RequestExample>
  ```py Example
  redis.hset("myhash", "field1", "Hello")

  assert redis.hget("myhash", "field1") == "Hello"
  assert redis.hget("myhash", "field2") is None
  ```
</RequestExample>


# HGETALL

Retrieves all fields from a hash.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

## Response

<ResponseField type="Optional[str]" required>
  An object with all fields in the hash.
</ResponseField>

<RequestExample>
  ```py Example
  redis.hset("myhash", values={
  "field1": "Hello",
  "field2": "World"
  })

  assert redis.hgetall("myhash") == {"field1": "Hello", "field2": "World"}
  ```
</RequestExample>


# HINCRBY

Increments the value of a hash field by a given amount

If the hash field does not exist, it is set to 0 before performing the operation.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

<ParamField body="field" type="str" required>
  The field to increment
</ParamField>

<ParamField body="increment" type="int">
  How much to increment the field by. Can be negative to subtract.
</ParamField>

## Response

<ResponseField type="int" required>
  The new value of the field after the increment.
</ResponseField>

<RequestExample>
  ```py Example
  redis.hset("myhash", "field1", 5)

  assert redis.hincrby("myhash", "field1", 10) == 15
  ```
</RequestExample>


# HINCRBYFLOAT

Increments the value of a hash field by a given float value.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

<ParamField body="field" type="str" required>
  The field to increment
</ParamField>

<ParamField body="increment" type="float" required>
  How much to increment the field by. Can be negative to subtract.
</ParamField>

## Response

<ResponseField type="float" required>
  The new value of the field after the increment.
</ResponseField>

<RequestExample>
  ```py Example
  redis.hset("myhash", "field1", 5.5)

  assert redis.hincrbyfloat("myhash", "field1", 10.1) - 15.6 < 0.0001
  ```
</RequestExample>


# HKEYS

Return all field names in the hash stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

## Response

<ResponseField type="List[str]" required>
  The field names of the hash
</ResponseField>

<RequestExample>
  ```py Example
  redis.hset("myhash", values={
      "field1": "Hello",
      "field2": "World"
  })

  assert redis.hkeys("myhash") == ["field1", "field2"]

  ```
</RequestExample>


# HLEN

Returns the number of fields contained in the hash stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

## Response

<ResponseField type="int" required>
  How many fields are in the hash.
</ResponseField>

<RequestExample>
  ```py Example
  assert redis.hlen("myhash") == 0

  redis.hset("myhash", values={
      "field1": "Hello",
      "field2": "World"
  })

  assert redis.hlen("myhash") == 2
  ```
</RequestExample>


# HMGET

Return the requested fields and their values.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

<ParamField body="fields" type="*List[str]" required>
  One or more fields to get.
</ParamField>

## Response

<ResponseField type="List[str | None]" required>
  An object containing the fields and their values.
</ResponseField>

<RequestExample>
  ```py Example
  redis.hset("myhash", values={
      "field1": "Hello",
      "field2": "World"
  })

  assert redis.hmget("myhash", "field1", "field2") == ["Hello", "World"]
  ```
</RequestExample>


# HMSET

Write multiple fields to a hash.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

<ParamField body="fields" type="Dict[str, Any]" required>
  A dictionary of fields and their values.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of fields that were added.
</ResponseField>

<RequestExample>
  ```py Example
  # Set multiple fields
  assert redis.hset("myhash"{
    "field1": "Hello",
    "field2": "World"
  }) == 2
  ```
</RequestExample>


# HRANDFIELD

Return a random field from a hash

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

<ParamField body="count" type="int">
  Optionally return more than one field.
</ParamField>

<ParamField body="withvalues" type="boolean">
  Return the values of the fields as well.
</ParamField>

## Response

<ResponseField type="Record<str, unknown>" required>
  An object containing the fields and their values.
</ResponseField>

<RequestExample>
  ```py Single
  redis.hset("myhash", values={
      "field1": "Hello",
      "field2": "World"
  })

  assert redis.hrandfield("myhash") in ["field1", "field2"]
  ```

  ```py Multiple
  redis.hset("myhash", values={
      "field1": "Hello",
      "field2": "World"
  })

  assert redis.hrandfield("myhash", count=2) in [
      ["field1", "field2"],
      ["field2", "field1"]
  ]
  ```

  ```py With Values
  redis.hset("myhash", values={
      "field1": "Hello",
      "field2": "World"
  })

  assert redis.hrandfield("myhash", count=1, withvalues=True) in [
      {"field1": "Hello"},
      {"field2": "World"}
  ]
  ```
</RequestExample>


# HSCAN

Scan a hash for fields.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

<ParamField body="cursor" type="int" required>
  The cursor, use `0` in the beginning and then use the returned cursor for subsequent calls.
</ParamField>

<ParamField body="match" type="str">
  Glob-style pattern to filter by field names.
</ParamField>

<ParamField body="count" type="int">
  Number of fields to return per call.
</ParamField>

## Response

<ResponseField type="Tuple[number, List[str]]" required>
  The new cursor and the fields.
  If the new cursor is `0` the iteration is complete.
</ResponseField>

<RequestExample>
  ```py Basic
  # Get all members of a hash.

  cursor = 0
  results = []

  while True:
      cursor, keys = redis.hscan("myhash", cursor, match="*")

      results.extend(keys)
      if cursor == 0:
          break
  ```
</RequestExample>


# HSET

Write one or more fields to a hash.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

<ParamField body="field" type="str">
  Field to set
</ParamField>

<ParamField body="value" type="str">
  Value to set
</ParamField>

<ParamField body="fields" type="Dict[str, Any]">
  An object of fields and their values.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of fields that were added.
</ResponseField>

<RequestExample>
  ```py Single
  # Set a single field
  assert redis.hset("myhash", "field1", "Hello") == 1
  ```

  ```py Multiple
  # Set multiple fields
  assert redis.hset("myhash", values={
    "field1": "Hello",
    "field2": "World"
  }) == 2
  ```
</RequestExample>


# HSETNX

Write a field to a hash but only if the field does not exist.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

<ParamField body="field" type="str" required>
  The name of the field.
</ParamField>

<ParamField body="value" type="Any" required>
  The value to set.
</ParamField>

## Response

<ResponseField type="bool" required>
  `True` if the field was set, `False` if it already existed.
</ResponseField>

<RequestExample>
  ```py Example
  assert redis.hsetnx("myhash", "field1", "Hello") == True
  assert redis.hsetnx("myhash", "field1", "World") == False
  ```
</RequestExample>


# HSTRLEN

Returns the string length of a value in a hash.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

<ParamField body="field" type="str" required>
  The name of the field.
</ParamField>

## Response

<ResponseField type="int" required>
  `0` if the hash or field does not exist. Otherwise the length of the string.
</ResponseField>

<RequestExample>
  ```py Example
  length = redis.hstrlen("key", "field")
  ```
</RequestExample>


# HVALS

Returns all values in the hash stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key of the hash.
</ParamField>

## Response

<ResponseField type="List[str]" required>
  All values in the hash, or an empty list when key does not exist.
</ResponseField>

<RequestExample>
  ```py Example
  redis.hset("myhash", values={
    "field1": "Hello",
    "field2": "World"
  })

  assert redis.hvals("myhash") == ["Hello", "World"]
  ```
</RequestExample>


# JSON.ARRAPPEND

Append values to the array at path in the JSON document at key.

<Tip>
  To specify a string as an array value to append, wrap the quoted string with an additional set of single quotes. Example: '"silver"'.
</Tip>

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

<ParamField body="value" type="...TValue[]" required>
  One or more values to append to the array.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the array after the appending.
</ResponseField>

<RequestExample>
  ```py Example
  redis.json.arrappend("key", "$.path.to.array", "a")
  ```
</RequestExample>


# JSON.ARRINDEX

Search for the first occurrence of a JSON value in an array.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

<ParamField body="value" type="TValue" required>
  The value to search for.
</ParamField>

<ParamField body="start" type="int" default={0}>
  The start index.
</ParamField>

<ParamField body="stop" type="int" default={0}>
  The stop index.
</ParamField>

## Response

<ResponseField type="int" required>
  The index of the first occurrence of the value in the array, or -1 if not found.
</ResponseField>

<RequestExample>
  ```py Example
  index = redis.json.arrindex("key", "$.path.to.array", "a")
  ```
</RequestExample>


# JSON.ARRINSERT

Insert the json values into the array at path before the index (shifts to the right).

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

<ParamField body="index" type="int" required>
  The index where to insert the values.
</ParamField>

<ParamField body="values" type="...TValue[]" required>
  One or more values to append to the array.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the array after the insertion.
</ResponseField>

<RequestExample>
  ```py Example
  length = redis.json.arrinsert("key", "$.path.to.array", 2, "a", "b")
  ```
</RequestExample>


# JSON.ARRLEN

Report the length of the JSON array at `path` in `key`.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the array.
</ResponseField>

<RequestExample>
  ```py Example
  length = redis.json.arrlen("key", "$.path.to.array")
  ```
</RequestExample>


# JSON.ARRPOP

Remove and return an element from the index in the array. By default the last element from an array is popped.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str">
  The path of the array.
</ParamField>

<ParamField body="index" type="int" default={-1}>
  The index of the element to pop.
</ParamField>

## Response

<ResponseField type="TValue | null" required>
  The popped element or null if the array is empty.
</ResponseField>

<RequestExample>
  ```py Example
  element = redis.json("key", "$.path.to.array")
  ```

  ```py First
  firstElement = redis.json("key", "$.path.to.array", 0)
  ```
</RequestExample>


# JSON.ARRTRIM

Trim an array so that it contains only the specified inclusive range of elements.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

<ParamField body="start" type="int" required>
  The start index of the range.
</ParamField>

<ParamField body="stop" type="int" required>
  The stop index of the range.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the array after the trimming.
</ResponseField>

<RequestExample>
  ```py Example
  length = redis.json.arrtrim("key", "$.path.to.array", 2, 10)
  ```
</RequestExample>


# JSON.CLEAR

Clear container values (arrays/objects) and set numeric values to 0.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path to clear
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the array after the appending.
</ResponseField>

<RequestExample>
  ```py Example
  redis.json.clear("key")
  ```

  ```py With path
  redis.json.clear("key", "$.my.key")
  ```
</RequestExample>


# JSON.DEL

Delete a key from a JSON document.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path to delete
</ParamField>

<ParamField body="value" type="...TValue[]" required>
  One or more values to append to the array.
</ParamField>

## Response

<ResponseField type="int" required>
  How many paths were deleted.
</ResponseField>

<RequestExample>
  ```py Example
  redis.json.del("key", "$.path.to.value")
  ```
</RequestExample>


# JSON.FORGET

Delete a key from a JSON document.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path to forget.
</ParamField>

<ParamField body="value" type="...TValue[]" required>
  One or more values to append to the array.
</ParamField>

## Response

<ResponseField type="int" required>
  How many paths were deleted.
</ResponseField>

<RequestExample>
  ```py Example
  redis.json.forget("key", "$.path.to.value")
  ```
</RequestExample>


# JSON.GET

Get a single value from a JSON document.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="options">
  <Expandable>
    <ParamField body="indent" type="str">
      Sets the indentation string for nested levels.
    </ParamField>

    <ParamField body="newline" type="str">
      Sets the string that's printed at the end of each line.
    </ParamField>

    <ParamField body="space" type="str">
      Sets the string that is put between a key and a value.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField body="paths" type="*List[str]" default="$">
  One or more paths to retrieve from the JSON document.
</ParamField>

## Response

<ResponseField type="TValue | null" required>
  The value at the specified path or `null` if the path does not exist.
</ResponseField>

<RequestExample>
  ```py Example
  value = redis.json.get("key", "$.path.to.somewhere")
  ```

  ```py With Options
  value = redis.json.get("key", {
      indent: "  ",
      newline: "\n",
      space: " ",
  }, "$.path.to.somewhere")
  ```
</RequestExample>


# JSON.MGET

Get the same path from multiple JSON documents.

## Arguments

<ParamField body="keys" type="*List[str]" required>
  One or more keys of JSON documents.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path to get from the JSON document.
</ParamField>

## Response

<ResponseField type="TValue[]" required>
  The values at the specified path or `null` if the path does not exist.
</ResponseField>

<RequestExample>
  ```py Example
  values = redis.json.mget(["key1", "key2"],  "$.path.to.somewhere")
  ```
</RequestExample>


# JSON.NUMINCRBY

Increment the number value stored at `path` by number.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

<ParamField body="increment" type="number" required>
  The number to increment by.
</ParamField>

## Response

<ResponseField type="int" required>
  The new value after incrementing
</ResponseField>

<RequestExample>
  ```py Example
  newValue = redis.json.numincrby("key", "$.path.to.array", 2)
  ```
</RequestExample>


# JSON.NUMMULTBY

Multiply the number value stored at `path` by number.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

<ParamField body="multiply" type="number" required>
  The number to multiply by.
</ParamField>

## Response

<ResponseField type="int" required>
  The new value after multiplying
</ResponseField>

<RequestExample>
  ```py Example
  newValue = redis.json.nummultby("key", "$.path.to.array", 2)
  ```
</RequestExample>


# JSON.OBJKEYS

Return the keys in the object that`s referenced by path.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

## Response

<ResponseField type="List[str]" required>
  The keys of the object at the path.
</ResponseField>

<RequestExample>
  ```py Example
  keys = redis.json.objkeys("key", "$.path")
  ```
</RequestExample>


# JSON.OBJLEN

Report the number of keys in the JSON object at `path` in `key`.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

## Response

<ResponseField type="int[]" required>
  The number of in the objects.
</ResponseField>

<RequestExample>
  ```py Example
  lengths = redis.json.objlen("key", "$.path")
  ```
</RequestExample>


# JSON.SET

Set the JSON value at path in key.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

<ParamField body="value" type="TValue" required>
  The value to set.
</ParamField>

## Response

<ResponseField type="OK" required>
  `OK`
</ResponseField>

<RequestExample>
  ```py Example
  Set the JSON value at path in key.
  redis.json.set(key, "$.path", value)
  ```

  ```py NX
  value = ...
  redis.json.set(key, "$.path", value, { nx:true })
  ```

  ```py XX
  value = ...
  redis.json.set(key, "$.path", value, { xx:true })
  ```
</RequestExample>


# JSON.STRAPPEND

Append the json-string values to the string at path.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the value.
</ParamField>

<ParamField body="value" type="str" required>
  The value to append to the existing string.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the array after the appending.
</ResponseField>

<RequestExample>
  ```py Example
  redis.json.strappend("key", "$.path.to.array", "abc")
  ```
</RequestExample>


# JSON.STRLEN

Report the length of the JSON String at path in key

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the array.
</ParamField>

<ParamField body="value" type="...TValue[]" required>
  One or more values to append to the array.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the array after the appending.
</ResponseField>

<RequestExample>
  ```py Example
  redis.json("key", "$.path.to.array", "a")
  ```
</RequestExample>


# JSON.TOGGLE

Toggle a boolean value stored at `path`.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the boolean.
</ParamField>

## Response

<ResponseField type="boolean" required>
  The new value of the boolean.
</ResponseField>

<RequestExample>
  ```py Example
  bool = redis.json.toggle("key", "$.path.to.array")
  ```
</RequestExample>


# JSON.TYPE

Report the type of JSON value at `path`.

## Arguments

<ParamField body="key" type="str" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="str" default="$">
  The path of the value.
</ParamField>

## Response

<ResponseField type="str | null" required>
  The type of the value at `path` or `null` if the value does not exist.
</ResponseField>

<RequestExample>
  ```py Example
  myType = redis.json.type("key", "$.path.to.value")
  ```
</RequestExample>


# LINDEX

Returns the element at index index in the list stored at key.

The index is zero-based, so 0 means the first element, 1 the second element and so on. Negative indices can be used to designate elements starting at the tail of the list.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="index" type="int" required>
  The index of the element to return, zero-based.
</ParamField>

## Response

<ResponseField type="Optional[str]" required>
  The value of the element at index index in the list. If the index is out of range, `None` is returned.
</ResponseField>

<RequestExample>
  ```py Example
  redis.rpush("key", "a", "b", "c")

  assert redis.lindex("key", 0) == "a"
  ```
</RequestExample>


# LINSERT

Insert an element before or after another element in a list

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="direction" type="&#x22;BEFORE&#x22; | &#x22;AFTER&#x22;" required>
  Whether to insert the element before or after pivot.
</ParamField>

<ParamField body="pivot" type="Any" required>
  The element to insert before or after.
</ParamField>

<ParamField body="value" type="Any" required>
  The element to insert.
</ParamField>

## Response

<ResponseField type="int" required>
  The list length after insertion, `0` when the list doesn't exist or `-1` when pivot was not found.
</ResponseField>

<RequestExample>
  ```py Example
  redis.rpush("key", "a", "b", "c")
  redis.linsert("key", "before", "b", "x")
  ```
</RequestExample>


# LLEN

Returns the length of the list stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the list at key.
</ResponseField>

<RequestExample>
  ```py Example
  redis.rpush("key", "a", "b", "c")

  assert redis.llen("key") == 3
  ```
</RequestExample>


# LMOVE

Move an element from one list to another.

## Arguments

<ParamField body="source" type="str" required>
  The key of the source list.
</ParamField>

<ParamField body="destination" type="str" required>
  The key of the destination list.
</ParamField>

<ParamField body="wherefrom" type="&#x22;left&#x22; | &#x22;right&#x22;" required>
  The side of the source list from which the element was popped.
</ParamField>

<ParamField body="whereto" type="&#x22;left&#x22; | &#x22;right&#x22;" required>
  The side of the destination list to which the element was pushed.
</ParamField>

## Response

<ResponseField type="str" required>
  The element that was moved.
</ResponseField>

<RequestExample>
  ```py Example
  redis.rpush("source", "one", "two", "three")
  redis.lpush("destination", "four", "five", "six")

  assert redis.lmove("source", "destination", "RIGHT", "LEFT") == "three"

  assert redis.lrange("source", 0, -1) == ["one", "two"]
  ```
</RequestExample>


# LPOP

Remove and return the first element(s) of a list

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="count" type="int">
  How many elements to pop. If not specified, a single element is popped.
</ParamField>

## Response

<ResponseField type="str | List[str] None" required>
  The popped element(s). If `count` was specified, an array of elements is
  returned, otherwise a single element is returned. If the list is empty, `None`
  is returned.
</ResponseField>

<RequestExample>
  ```py Single 
  redis.rpush("mylist", "one", "two", "three")

  assert redis.lpop("mylist") == "one"
  ```

  ```py Multiple
  redis.rpush("mylist", "one", "two", "three")

  assert redis.lpop("mylist", 2) == ["one", "two"]
  ```
</RequestExample>


# LPOS

Returns the index of matching elements inside a list.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="element" type="unknown" required>
  The element to match.
</ParamField>

<ParamField body="rank" type="int">
  Which match to return. 1 to return the first match, 2 to return the second match, and so on.
  1 by default.
</ParamField>

<ParamField body="count" type="int">
  The maximum number of elements to match. If specified, an array of elements
  is returned instead of a single element.
</ParamField>

<ParamField body="maxlen" type="int">
  Limit the number of comparisons to perform.
</ParamField>

## Response

<ResponseField type="int | List[int]" required>
  The index of the matching element or an array of indexes if `count` is
  specified.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.rpush("key", "a", "b", "c"); 

  assert redis.lpos("key", "b") == 1
  ```

  ```py With Rank 
  redis.rpush("key", "a", "b", "c", "b"); 

  assert redis.lpos("key", "b", rank=2) == 3
  ```

  ```py With Count
  redis.rpush("key", "a", "b", "b")

  assert redis.lpos("key", "b", count=2) == [1, 2]
  ```
</RequestExample>


# LPUSH

Push an element at the head of the list.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="elements" type="*List[Any]" required>
  One or more elements to push at the head of the list.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the list after the push operation.
</ResponseField>

<RequestExample>
  ```py Example 
  assert redis.lpush("mylist", "one", "two", "three") == 3

  assert lrange("mylist", 0, -1) == ["three", "two", "one"]
  ```
</RequestExample>


# LPUSHX

Push an element at the head of the list only if the list exists.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="elements" type="*List[str]" required>
  One or more elements to push at the head of the list.
</ParamField>

## Response

<ResponseField type="number" required>
  The length of the list after the push operation.

  `0` if the list did not exist and thus no element was pushed.
</ResponseField>

<RequestExample>
  ```py Example 
  # Initialize the list
  redis.lpush("mylist", "one")

  assert redis.lpushx("mylist", "two", "three") == 3

  assert lrange("mylist", 0, -1) == ["three", "two", "one"]

  # Non existing key
  assert redis.lpushx("non-existent-list", "one") == 0
  ```
</RequestExample>


# LRANGE

Returns the specified elements of the list stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="start" type="int" required>
  The starting index of the range to return.

  Use negative numbers to specify offsets starting at the end of the list.
</ParamField>

<ParamField body="end" type="int" required>
  The ending index of the range to return.

  Use negative numbers to specify offsets starting at the end of the list.
</ParamField>

## Response

<ResponseField type="List[str]">
  The list of elements in the specified range.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.rpush("mylist", "one", "two", "three")

  assert redis.lrange("mylist", 0, 1) == ["one", "two"]

  assert redis.lrange("mylist", 0, -1) == ["one", "two", "three"]
  ```
</RequestExample>


# LREM

Remove the first `count` occurrences of an element from a list.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="count" type="number" required>
  How many occurrences of the element to remove.
</ParamField>

<ParamField body="element" type="Any" required>
  The element to remove
</ParamField>

## Response

<ResponseField type="int" required>
  The number of elements removed.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.rpush("mylist", "one", "two", "three", "two", "one")

  assert redis.lrem("mylist", 2, "two") == 2

  assert redis.lrange("mylist", 0, -1) == ["one", "three", "one"]
  ```
</RequestExample>


# LSET

Set a value at a specific index.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="index" type="number" required>
  At which index to set the value.
</ParamField>

<ParamField body="element" type="str" required>
  The value to set.
</ParamField>

## Response

<ResponseField type="bool" required>
  Returns `True` if the index was in range and the value was set.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.rpush("mylist", "one", "two", "three")

  assert redis.lset("mylist", 1, "Hello") == True

  assert redis.lrange("mylist", 0, -1) == ["one", "Hello", "three"]

  assert redis.lset("mylist", 5, "Hello") == False

  assert redis.lrange("mylist", 0, -1) == ["one", "Hello", "three"]
  ```
</RequestExample>


# LTRIM

Trim a list to the specified range

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="start" type="int" required>
  The index of the first element to keep.
</ParamField>

<ParamField body="stop" type="int" required>
  The index of the first element to keep.
</ParamField>

## Response

<ResponseField type="bool" required>
  Returns `True` if the list was trimmed, `False` otherwise.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.rpush("mylist", "one", "two", "three")

  assert redis.ltrim("mylist", 0, 1) == True

  assert redis.lrange("mylist", 0, -1) == ["one", "two"]
  ```
</RequestExample>


# RPOP

Remove and return the last element(s) of a list

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="count" type="int">
  How many elements to pop. If not specified, a single element is popped.
</ParamField>

## Response

<ResponseField type="TValue | TValue[] | null" required>
  The popped element(s). If `count` was specified, an array of elements is
  returned, otherwise a single element is returned. If the list is empty, `null`
  is returned.
</ResponseField>

<RequestExample>
  ```py Single 
  redis.rpush("mylist", "one", "two", "three")

  assert redis.rpop("mylist") == "three"
  ```

  ```py Multiple 
  redis.rpush("mylist", "one", "two", "three")

  assert redis.rpop("mylist", 2) == ["three", "two"]
  ```
</RequestExample>


# RPUSH

Push an element at the end of the list.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="elements" type="*List[str]" required>
  One or more elements to push at the end of the list.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the list after the push operation.
</ResponseField>

<RequestExample>
  ```py Example 
  assert redis.rpush("mylist", "one", "two", "three") == 3

  assert lrange("mylist", 0, -1) == ["one", "two", "three"]
  ```
</RequestExample>


# RPUSHX

Push an element at the end of the list only if the list exists.

## Arguments

<ParamField body="key" type="str" required>
  The key of the list.
</ParamField>

<ParamField body="elements" type="*List[str]" required>
  One or more elements to push at the end of the list.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the list after the push operation.

  `0` if the list did not exist and thus no element was pushed.
</ResponseField>

<RequestExample>
  ```py Example 
  assert redis.rpushx("mylist", "one", "two", "three") == 3

  assert lrange("mylist", 0, -1) == ["one", "two", "three"]

  # Non existing key
  assert redis.rpushx("non-existent-list", "one") == 0
  ```
</RequestExample>


# Overview

Available Commands in upstash-redis

<AccordionGroup>
  <Accordion title="Auth">
    <CardGroup cols={3}>
      <Card title="ECHO" href="/redis/sdks/py/commands/auth/echo">
        Echo the given string.
      </Card>

      <Card title="PING" href="/redis/sdks/py/commands/auth/ping">
        Ping the server.
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Bitmap">
    <CardGroup cols={3}>
      <Card title="BITCOUNT" href="/redis/sdks/py/commands/bitmap/bitcount">
        Count set bits in a string.
      </Card>

      <Card title="BITFIELD" href="/redis/sdks/py/commands/bitmap/bitfield">
        Perform bitwise operations between strings.
      </Card>

      <Card title="BITOP" href="/redis/sdks/py/commands/bitmap/bitop">
        Perform bitwise operations between strings.
      </Card>

      <Card title="BITPOS" href="/redis/sdks/py/commands/bitmap/bitpos">
        Find first bit set or clear in a string.
      </Card>

      <Card title="GETBIT" href="/redis/sdks/py/commands/bitmap/getbit">
        Returns the bit value at offset in the string value stored at key.
      </Card>

      <Card title="SETBIT" href="/redis/sdks/py/commands/bitmap/setbit">
        Sets or clears the bit at offset in the string value stored at key.
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Generic">
    <CardGroup cols={3}>
      <Card title="DEL" href="/redis/sdks/py/commands/generic/del">
        Delete one or multiple keys.
      </Card>

      <Card title="EXISTS" href="/redis/sdks/py/commands/generic/exists">
        Determine if a key exists.
      </Card>

      <Card title="EXPIRE" href="/redis/sdks/py/commands/generic/expire">
        Set a key's time to live in seconds.
      </Card>

      <Card title="EXPIREAT" href="/redis/sdks/py/commands/generic/expireat">
        Set the expiration for a key as a UNIX timestamp.
      </Card>

      <Card title="KEYS" href="/redis/sdks/py/commands/generic/keys">
        Find all keys matching the given pattern.
      </Card>

      <Card title="PERSIST" href="/redis/sdks/py/commands/generic/persist">
        Remove the expiration from a key.
      </Card>

      <Card title="PEXPIRE" href="/redis/sdks/py/commands/generic/pexpire">
        Set a key's time to live in milliseconds.
      </Card>

      <Card title="PEXPIREAT" href="/redis/sdks/py/commands/generic/pexpireat">
        Set the expiration for a key as a UNIX timestamp specified in milliseconds.
      </Card>

      <Card title="PTTL" href="/redis/sdks/py/commands/generic/pttl">
        Get the time to live for a key in milliseconds.
      </Card>

      <Card title="RANDOMKEY" href="/redis/sdks/py/commands/generic/randomkey">
        Return a random key from the keyspace.
      </Card>

      <Card title="RENAME" href="/redis/sdks/py/commands/generic/rename">
        Rename a key.
      </Card>

      <Card title="RENAMENX" href="/redis/sdks/py/commands/generic/renamenx">
        Rename a key, only if the new key does not exist.
      </Card>

      {/* <Card title="RESTORE" href="/redis/sdks/py/commands/generic/restore">
          Create a key using the provided serialized value, previously obtained using DUMP.
      </Card> */}

      <Card title="SCAN" href="/redis/sdks/py/commands/generic/scan">
        Incrementally iterate the keys space.
      </Card>

      {/* <Card title="SORT" href="/redis/sdks/py/commands/generic/sort">
          Sort the elements in a list, set or sorted set.
      </Card> */}

      <Card title="TOUCH" href="/redis/sdks/py/commands/generic/touch">
        Alters the last access time of a key(s). Returns the number of existing keys specified.
      </Card>

      <Card title="TTL" href="/redis/sdks/py/commands/generic/ttl">
        Get the time to live for a key.
      </Card>

      <Card title="TYPE" href="/redis/sdks/py/commands/generic/type">
        Determine the type stored at key.
      </Card>

      <Card title="UNLINK" href="/redis/sdks/py/commands/generic/unlink">
        Delete one or more keys.
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Hash">
    <CardGroup cols={3}>
      <Card title="HDEL" href="/redis/sdks/py/commands/hash/hdel" />

      <Card title="HEXISTS" href="/redis/sdks/py/commands/hash/hexists" />

      <Card title="HGET" href="/redis/sdks/py/commands/hash/hget" />

      <Card title="HGETALL" href="/redis/sdks/py/commands/hash/hgetall" />

      <Card title="HINCRBY" href="/redis/sdks/py/commands/hash/hincrby" />

      <Card title="HINCRBYFLOAT" href="/redis/sdks/py/commands/hash/hincrbyfloat" />

      <Card title="HKEYS" href="/redis/sdks/py/commands/hash/hkeys" />

      <Card title="HLEN" href="/redis/sdks/py/commands/hash/hlen" />

      <Card title="HMGET" href="/redis/sdks/py/commands/hash/hmget" />

      <Card title="HRANDFIELD" href="/redis/sdks/py/commands/hash/hrandfield" />

      <Card title="HSCAN" href="/redis/sdks/py/commands/hash/hscan" />

      <Card title="HSET" href="/redis/sdks/py/commands/hash/hset" />

      <Card title="HSET" href="/redis/sdks/py/commands/hash/hmset" />

      <Card title="HSETNX" href="/redis/sdks/py/commands/hash/hsetnx" />

      <Card title="HSTRLEN" href="/redis/sdks/py/commands/hash/hstrlen" />

      <Card title="HVALS" href="/redis/sdks/py/commands/hash/hvals" />
    </CardGroup>
  </Accordion>

  <Accordion title="List">
    <CardGroup cols={3}>
      <Card title="LINDEX" href="/redis/sdks/py/commands/list/lindex" />

      <Card title="LINSERT" href="/redis/sdks/py/commands/list/linsert" />

      <Card title="LLEN" href="/redis/sdks/py/commands/list/llen" />

      <Card title="LMOVE" href="/redis/sdks/py/commands/list/lmove" />

      <Card title="LPOP" href="/redis/sdks/py/commands/list/lpop" />

      <Card title="LPOS" href="/redis/sdks/py/commands/list/lpos" />

      <Card title="LPUSH" href="/redis/sdks/py/commands/list/lpush" />

      <Card title="LPUSHX" href="/redis/sdks/py/commands/list/lpushx" />

      <Card title="LRANGE" href="/redis/sdks/py/commands/list/lrange" />

      <Card title="LREM" href="/redis/sdks/py/commands/list/lrem" />

      <Card title="LSET" href="/redis/sdks/py/commands/list/lset" />

      <Card title="LTRIM" href="/redis/sdks/py/commands/list/ltrim" />

      <Card title="RPOP" href="/redis/sdks/py/commands/list/rpop" />

      <Card title="RPUSH" href="/redis/sdks/py/commands/list/rpush" />

      <Card title="RPUSHX" href="/redis/sdks/py/commands/list/rpushx" />
    </CardGroup>
  </Accordion>

  <Accordion title="PubSub">
    <CardGroup cols={3}>
      <Card title="PUBLISH" href="/redis/sdks/py/commands/pubsub/publish">
        Publish messages to many clients
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Scripts">
    <CardGroup cols={3}>
      <Card title="EVAL" href="/redis/sdks/py/commands/scripts/eval" />

      <Card title="EVELSHA" href="/redis/sdks/py/commands/scripts/evalsha" />

      <Card title="SCRIPT EXISTS" href="/redis/sdks/py/commands/scripts/script_exists" />

      <Card title="SCRIPT FLUSH" href="/redis/sdks/py/commands/scripts/script_flush" />

      <Card title="SCRIPT LOAD" href="/redis/sdks/py/commands/scripts/script_load" />
    </CardGroup>
  </Accordion>

  <Accordion title="Server">
    <CardGroup cols={3}>
      <Card title="DBSIZE" href="/redis/sdks/py/commands/server/dbsize" />

      <Card title="FLUSHALL" href="/redis/sdks/py/commands/server/flushall" />

      <Card title="FLUSHDB" href="/redis/sdks/py/commands/server/flushdb" />
    </CardGroup>
  </Accordion>

  <Accordion title="Set">
    <CardGroup cols={3}>
      <Card title="SADD" href="/redis/sdks/py/commands/set/sadd" />

      <Card title="SCARD" href="/redis/sdks/py/commands/set/scard" />

      <Card title="SDIFF" href="/redis/sdks/py/commands/set/sdiff" />

      <Card title="SDIFFSTORE" href="/redis/sdks/py/commands/set/sdiffstore" />

      <Card title="SINTER" href="/redis/sdks/py/commands/set/sinter" />

      <Card title="SINTERSTORE" href="/redis/sdks/py/commands/set/sinterstore" />

      <Card title="SISMEMBER" href="/redis/sdks/py/commands/set/sismember" />

      <Card title="SMEMBERS" href="/redis/sdks/py/commands/set/smembers" />

      <Card title="SMISMEMBER" href="/redis/sdks/py/commands/set/smismember" />

      <Card title="SMOVE" href="/redis/sdks/py/commands/set/smove" />

      <Card title="SPOP" href="/redis/sdks/py/commands/set/spop" />

      <Card title="SRANDMEMBER" href="/redis/sdks/py/commands/set/srandmember" />

      <Card title="SREM" href="/redis/sdks/py/commands/set/srem" />

      <Card title="SSCAN" href="/redis/sdks/py/commands/set/sscan" />

      <Card title="SUNION" href="/redis/sdks/py/commands/set/sunion" />

      <Card title="SUNIONSTORE" href="/redis/sdks/py/commands/set/sunionstore" />
    </CardGroup>
  </Accordion>

  <Accordion title="Sorted Set">
    <CardGroup cols={3}>
      <Card title="ZADD" href="/redis/sdks/py/commands/zset/zadd" />

      <Card title="ZCARD" href="/redis/sdks/py/commands/zset/zcard" />

      <Card title="ZCOUNT" href="/redis/sdks/py/commands/zset/zcount" />

      <Card title="ZDIFF" href="/redis/sdks/py/commands/zset/zdiff" />

      <Card title="ZDIFFSTORE" href="/redis/sdks/py/commands/zset/zdiffstore" />

      <Card title="ZINCRBY" href="/redis/sdks/py/commands/zset/zincrby" />

      <Card title="ZINTER" href="/redis/sdks/py/commands/zset/zinter" />

      <Card title="ZINTERSTORE" href="/redis/sdks/py/commands/zset/zinterstore" />

      <Card title="ZLEXCOUNT" href="/redis/sdks/py/commands/zset/zlexcount" />

      <Card title="ZMSCORE" href="/redis/sdks/py/commands/zset/zmscore" />

      <Card title="ZPOPMAX" href="/redis/sdks/py/commands/zset/zpopmax" />

      <Card title="ZPOPMIN" href="/redis/sdks/py/commands/zset/zpopmin" />

      <Card title="ZRANDMEMBER" href="/redis/sdks/py/commands/zset/zrandmember" />

      <Card title="ZRANGE" href="/redis/sdks/py/commands/zset/zrange" />

      <Card title="ZRANK" href="/redis/sdks/py/commands/zset/zrank" />

      <Card title="ZREM" href="/redis/sdks/py/commands/zset/zrem" />

      <Card title="ZREMRANGEBYLEX" href="/redis/sdks/py/commands/zset/zremrangebylex" />

      <Card title="ZREMRANGEBYRANK" href="/redis/sdks/py/commands/zset/zremrangebyrank" />

      <Card title="ZREMRANGEBYSCORE" href="/redis/sdks/py/commands/zset/zremrangebyscore" />

      <Card title="ZREVRANK" href="/redis/sdks/py/commands/zset/zrevrank" />

      <Card title="ZSCAN" href="/redis/sdks/py/commands/zset/zscan" />

      <Card title="ZSCORE" href="/redis/sdks/py/commands/zset/zscore" />

      <Card title="ZUNION" href="/redis/sdks/py/commands/zset/zunion" />

      <Card title="ZUNIONSTORE" href="/redis/sdks/py/commands/zset/zunionstore" />
    </CardGroup>
  </Accordion>

  <Accordion title="String">
    <CardGroup cols={3}>
      <Card title="APPEND" href="/redis/sdks/py/commands/string/append">
        Append a value to a string stored at key.
      </Card>

      <Card title="DECR" href="/redis/sdks/py/commands/string/decr">
        Decrement the integer value of a key by one.
      </Card>

      <Card title="DECRBY" href="/redis/sdks/py/commands/string/decrby">
        Decrement the integer value of a key by the given number.
      </Card>

      <Card title="GET" href="/redis/sdks/py/commands/string/get">
        Get the value of a key.
      </Card>

      <Card title="GETDEL" href="/redis/sdks/py/commands/string/getdel">
        Get the value of a key and delete the key.
      </Card>

      <Card title="GETRANGE" href="/redis/sdks/py/commands/string/getrange">
        Get a substring of the string stored at a key.
      </Card>

      <Card title="GETSET" href="/redis/sdks/py/commands/string/getset">
        Set the string value of a key and return its old value.
      </Card>

      <Card title="INCR" href="/redis/sdks/py/commands/string/incr">
        Increment the integer value of a key by one.
      </Card>

      <Card title="INCRBY" href="/redis/sdks/py/commands/string/incrby">
        Increment the integer value of a key by the given amount.
      </Card>

      <Card title="INCRBYFLOAT" href="/redis/sdks/py/commands/string/incrbyfloat">
        Increment the float value of a key by the given amount.
      </Card>

      <Card title="MGET" href="/redis/sdks/py/commands/string/mget">
        Get the values of all the given keys.
      </Card>

      <Card title="MSET" href="/redis/sdks/py/commands/string/mset">
        Set multiple keys to multiple values.
      </Card>

      <Card title="MSETNX" href="/redis/sdks/py/commands/string/msetnx">
        Set multiple keys to multiple values, only if none of the keys exist.
      </Card>

      <Card title="PSETEX" href="/redis/sdks/py/commands/string/psetex">
        Set the value and expiration in milliseconds of a key.
      </Card>

      <Card title="SET" href="/redis/sdks/py/commands/string/set">
        Set the string value of a key.
      </Card>

      <Card title="SETEX" href="/redis/sdks/py/commands/string/setex">
        Set the value and expiration of a key.
      </Card>

      <Card title="SETNX" href="/redis/sdks/py/commands/string/setnx">
        Set the value of a key, only if the key does not exist.
      </Card>

      <Card title="SETRANGE" href="/redis/sdks/py/commands/string/setrange">
        Overwrite part of a string at key starting at the specified offset.
      </Card>

      <Card title="STRLEN" href="/redis/sdks/py/commands/string/strlen">
        Get the length of the value stored in a key.
      </Card>

      <Card title="SUBSTR" href="/redis/sdks/py/commands/string/substr">
        Get a substring of the string stored at a key.
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Transactions">
    <Card title="TRANSACTION" href="/redis/sdks/py/features#pipelines-and-transactions">
      Run multiple commands in a transaction.
    </Card>
  </Accordion>
</AccordionGroup>


# PUBLISH

Publish a message to a channel

## Arguments

<ParamField body="channel" type="str" required>
  The channel to publish to.
</ParamField>

<ParamField body="message" type="str">
  The message to publish.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of clients who received the message.
</ResponseField>

<RequestExample>
  ```py Example
  listeners = redis.publish("my-topic", "my-message")
  ```
</RequestExample>


# EVAL

Evaluate a Lua script server side.

## Arguments

<ParamField body="script" type="str" required>
  The lua script to run.
</ParamField>

<ParamField body="keys" type="List[str]" required>
  All of the keys accessed in the script
</ParamField>

<ParamField body="args" type="unknown[]" required>
  All of the arguments you passed to the script
</ParamField>

## Response

<ResponseField type="Any" required>
  The result of the script.
</ResponseField>

<RequestExample>
  ```py Example
  script = """
  local value = redis.call("GET", KEYS[1])
  return value
  """

  redis.set("mykey", "Hello")

  assert redis.eval(script, keys=["mykey"]) == "Hello"
  ```

  ```py Accepting arguments
  assert redis.eval("return ARGV[1]", args=["Hello"]) == "Hello"
  ```
</RequestExample>


# EVALSHA

Evaluate a cached Lua script server side.

`EVALSHA` is like `EVAL` but instead of sending the script over the wire every time, you reference the script by its SHA1 hash. This is useful for caching scripts on the server side.

## Arguments

<ParamField body="sha" type="str" required>
  The sha1 hash of the script.
</ParamField>

<ParamField body="keys" type="List[str]" required>
  All of the keys accessed in the script
</ParamField>

<ParamField body="args" type="List[str]" required>
  All of the arguments you passed to the script
</ParamField>

## Response

<ResponseField type="?" required>
  The result of the script.
</ResponseField>

<RequestExample>
  ```py Example
  result = redis.eval("fb67a0c03b48ddbf8b4c9b011e779563bdbc28cb", args=["hello"])
  assert result = "hello"
  ```
</RequestExample>


# SCRIPT EXISTS

Check if scripts exist in the script cache.

## Arguments

<ParamField body="hashes" type="List[str]" required>
  The sha1 of the scripts to check.
</ParamField>

## Response

<ResponseField type="List[bool]" required>
  A list of booleans indicating if the script exists in the script cache.
</ResponseField>

<RequestExample>
  ```py Example
  # Script 1 exists
  # Script 0 does not
  await redis.scriptExists("<sha1>", "<sha2>") == [1, 0]
  ```
</RequestExample>


# SCRIPT FLUSH

Removes all scripts from the script cache.

## Arguments

<ParamField body="flush_type" type="&#x22;ASYNC&#x22; | &#x22;SYNC&#x22;" required>
  Whether to perform the flush asynchronously or synchronously.
</ParamField>

<RequestExample>
  ```py Example
  redis.script_flush(flush_type="ASYNC")
  ```
</RequestExample>


# SCRIPT LOAD

Load the specified Lua script into the script cache.

## Arguments

<ParamField body="script" type="str" required>
  The script to load.
</ParamField>

## Response

<ResponseField type="str" required>
  The sha1 of the script.
</ResponseField>

<RequestExample>
  ```py Example
  sha1 = redis.script_load("return 1")

  assert redis.evalsha(sha1) == 1
  ```
</RequestExample>


# DBSIZE

Count the number of keys in the database.

## Arguments

This command has no arguments

## Response

<ResponseField type="int" required>
  The number of keys in the database
</ResponseField>

<RequestExample>
  ```py Example
  redis.dbsize()
  ```
</RequestExample>


# FLUSHALL



<Warning>
  Deletes all keys permanently. Use with caution!
</Warning>

## Arguments

<ParamField body="flush_type" type="&#x22;ASYNC&#x22; | &#x22;SYNC&#x22;">
  Whether to perform the operation asynchronously.
  Defaults to synchronous.
</ParamField>

<RequestExample>
  ```py Sync
  redis.flushall()
  ```

  ```py Async
  redis.flushall(flush_type="ASYNC")
  ```
</RequestExample>


# FLUSHDB



<Warning>
  Deletes all keys permanently. Use with caution!
</Warning>

## Arguments

<ParamField body="flush_type" type="&#x22;ASYNC&#x22; | &#x22;SYNC&#x22;">
  Whether to perform the operation asynchronously.
  Defaults to synchronous.
</ParamField>

<RequestExample>
  ```py Sync
  redis.flushall()
  ```

  ```py Async
  redis.flushall(flush_type="ASYNC")
  ```
</RequestExample>


# SADD

Adds one or more members to a set.

## Arguments

<ParamField body="key" type="str" required>
  The key of the set.
</ParamField>

<ParamField body="members" type="...TValue[]" required>
  One or more members to add to the set.
</ParamField>

## Response

<ResponseField type="number" required>
  The number of elements that were added to the set, not including all the elements already present in the set.
</ResponseField>

<RequestExample>
  ```py Example 
  assert redis.sadd("key", "a", "b", "c") == 3
  ```
</RequestExample>


# SCARD

Return how many members are in a set

## Arguments

<ParamField body="key" type="str" required>
  The key of the set.
</ParamField>

## Response

<ResponseField type="int" required>
  How many members are in the set.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("key", "a", "b", "c"); 

  assert redis.scard("key") == 3
  ```
</RequestExample>


# SDIFF

Return the difference between sets

## Arguments

<ParamField body="keys" type="*List[str]" required>
  The keys of the sets to perform the difference operation on.
</ParamField>

## Response

<ResponseField type="set[str]" required>
  The resulting set.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("set1", "a", "b", "c"); 
  redis.sadd("set2", "c", "d", "e"); 

  assert redis.sdiff("set1", "set2") == {"a", "b"}
  ```
</RequestExample>


# SDIFFSTORE

Write the difference between sets to a new set

## Arguments

<ParamField body="destination" type="str" required>
  The key of the set to store the resulting set in.
</ParamField>

<ParamField body="keys" type="*List[str]" required>
  The keys of the sets to perform the difference operation on.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("key1", "a", "b", "c")

  redis.sadd("key2", "c", "d", "e")

  # Store the result in a new set
  assert redis.sdiffstore("res", "key1", "key2") == 2

  assert redis.smembers("set") == {"a", "b"}
  ```
</RequestExample>


# SINTER

Return the intersection between sets

## Arguments

<ParamField body="keys" type="*List[str]" required>
  The keys of the sets to perform the intersection operation on.
</ParamField>

## Response

<ResponseField type="set[str]" required>
  The resulting set.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("set1", "a", "b", "c"); 
  redis.sadd("set2", "c", "d", "e"); 

  assert redis.sinter("set1", "set2") == {"c"}
  ```
</RequestExample>


# SINTER

Return the intersection between sets and store the resulting set in a key

## Arguments

<ParamField body="destination" type="str" required>
  The key of the set to store the resulting set in.
</ParamField>

<ParamField body="keys" type="*List[str]" required>
  The keys of the sets to perform the intersection operation on.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("set1", "a", "b", "c"); 

  redis.sadd("set2", "c", "d", "e"); 

  assert redis.sinter("destination", "set1", "set2") == 1
  ```
</RequestExample>


# SISMEMBER

Check if a member exists in a set

## Arguments

<ParamField body="key" type="str" required>
  The key of the set to check.
</ParamField>

<ParamField body="member" type="str">
  The member to check for.
</ParamField>

## Response

<ResponseField type="bool" required>
  `True` if the member exists in the set, `False` if not.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("set", "a", "b", "c")

  assert redis.sismember("set", "a") == True
  ```
</RequestExample>


# SMEMBERS

Return all the members of a set

## Arguments

<ParamField body="key" type="str" required>
  The key of the set.
</ParamField>

## Response

<ResponseField type="set[str]" required>
  The members of the set.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("set", "a", "b", "c"); 
  assert redis.smembers("set") == {"a", "b", "c"}
  ```
</RequestExample>


# SMISMEMBER

Check if multiple members exist in a set

## Arguments

<ParamField body="key" type="str" required>
  The key of the set to check.
</ParamField>

<ParamField body="members" type="TMember[]">
  The members to check
</ParamField>

## Response

<ResponseField type="List[bool]" required>
  An array of `True` and `False` values.
  `True` if the member exists in the set, `False` if not.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("myset", "one", "two", "three")

  assert redis.smismember("myset", "one", "four") == [True, False]

  assert redis.smismember("myset", "four", "five") == [False, False]
  ```
</RequestExample>


# SMOVE

Move a member from one set to another

## Arguments

<ParamField body="source" type="str" required>
  The key of the set to move the member from.
</ParamField>

<ParamField body="destination" type="str" required>
  The key of the set to move the member to.
</ParamField>

<ParamField body="member" type="str">
  The members to move
</ParamField>

## Response

<ResponseField type="bool" required>
  `True` if the member was moved, `False` if it was not.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("src", "one", "two", "three")

  redis.sadd("dest", "four")

  assert redis.smove("src", "dest", "three") == True

  assert redis.smembers("source") == {"one", "two"}

  assert redis.smembers("destination") == {"three", "four"}
  ```
</RequestExample>


# SPOP

Removes and returns one or more random members from a set.

## Arguments

<ParamField body="key" type="str" required>
  The key of the set.
</ParamField>

<ParamField body="count" type="int">
  How many members to remove and return.
</ParamField>

## Response

<ResponseField type="str | set[str]" required>
  The popped member.
  If `count` is specified, a set of members is returned.
</ResponseField>

<RequestExample>
  ```py Single
  redis.sadd("myset", "one", "two", "three")

  assert redis.spop("myset") in {"one", "two", "three"}
  ```

  ```py With Count 
  redis.sadd("myset", "one", "two", "three")

  assert redis.spop("myset", 2) in {"one", "two", "three"}
  ```
</RequestExample>


# SRANDMEMBER

Returns one or more random members from a set.

## Arguments

<ParamField body="key" type="str" required>
  The key of the set.
</ParamField>

<ParamField body="count" type="number" default={1}>
  How many members to return.
</ParamField>

## Response

<ResponseField type="TMember | TMember[]" required>
  The random member.
  If `count` is specified, an array of members is returned.
</ResponseField>

<RequestExample>
  ```py Single
  redis.sadd("myset", "one", "two", "three")

  assert redis.srandmember("myset") in {"one", "two", "three"}
  ```

  ```py With Count 
  redis.sadd("myset", "one", "two", "three")

  assert redis.srandmember("myset", 2) in {"one", "two", "three"}
  ```
</RequestExample>


# SREM

Remove one or more members from a set

## Arguments

<ParamField body="key" type="str" required>
  The key of the set to remove the member from.
</ParamField>

<ParamField body="members" type="*List[str]">
  One or more members to remove from the set.
</ParamField>

## Response

<ResponseField type="int" required>
  How many members were removed
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("myset", "one", "two", "three")

  assert redis.srem("myset", "one", "four") == 1
  ```
</RequestExample>


# SSCAN

Scan a set

## Arguments

<ParamField body="key" type="str" required>
  The key of the set.
</ParamField>

<ParamField body="cursor" type="number">
  The cursor, use `0` in the beginning and then use the returned cursor for subsequent calls.
</ParamField>

<ParamField body="options" type="Object">
  <ParamField body="match" type="str">
    Glob-style pattern to filter by members.
  </ParamField>

  <ParamField body="count" type="number">
    Number of members to return per call.
  </ParamField>
</ParamField>

## Response

<ResponseField type="Tuple[number, TMember[]]" required>
  The new cursor and the members.
  If the new cursor is `0` the iteration is complete.
</ResponseField>

<RequestExample>
  ```py Example
  # Get all members of a set.

  cursor = 0
  results = set()

  while True:
      cursor, keys = redis.sscan("myset", cursor, match="*")

      results.extend(keys)
      if cursor == 0:
          break
  ```
</RequestExample>


# SUNION

Return the union between sets

## Arguments

<ParamField body="keys" type="*List[str]" required>
  The keys of the sets to perform the union operation on.
</ParamField>

## Response

<ResponseField type="set[str]" required>
  The resulting set
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("key1", "a", "b", "c")

  redis.sadd("key2", "c", "d", "e")

  assert redis.sunion("key1", "key2") == {"a", "b", "c", "d", "e"}
  ```
</RequestExample>


# SUNIONSTORE

Return the union between sets and store the resulting set in a key

## Arguments

<ParamField body="destination" type="str" required>
  The key of the set to store the resulting set in.
</ParamField>

<ParamField body="keys" type="*List[str]" required>
  The keys of the sets to perform the union operation on.
</ParamField>

## Response

<ResponseField type="set[str]" required>
  The members of the resulting set.
</ResponseField>

<RequestExample>
  ```py Example 
  redis.sadd("set1", "a", "b", "c"); 
  redis.sadd("set2", "c", "d", "e"); 
  redis.sunionstore("destination", "set1", "set2")
  ```
</RequestExample>


# XADD

Appends one or more new entries to a stream.

## Arguments

<ParamField body="key" type="str" required>
  The key to of the stream.
</ParamField>

<ParamField body="id" type="str | *" required>
  The stream entry ID. If `*` is passed, a new ID will be generated
  automatically.
</ParamField>

<ParamField body="entries" type="Record<str, unknown>" required>
  Key-value data to be appended to the stream.
</ParamField>

<ParamField body="options">
  <ParamField body="nomkStream" type="boolean">
    Prevent creating the stream if it does not exist.
  </ParamField>

  <ParamField body="trim">
    <ParamField body="type" type="'MAXLEN' | 'MINID'" required>
      The trim mode.
    </ParamField>

    {" "}

    <ParamField body="threshold" type="number | str" required>
      The threshold value for the trim mode.
    </ParamField>

    <ParamField body="comparison" type="~ | =" required>
      The comparison operator for the trim mode.
    </ParamField>

    <ParamField body="limit" type="number">
      Limit how many entries will be trimmed at most.
    </ParamField>
  </ParamField>
</ParamField>

## Response

<ResponseField type="str">The ID of the newly added entry.</ResponseField>

<RequestExample>
  ```py Example
  redis.xadd(key, "*", { name: "John Doe", age: 30 })

  ```

  ```py Trimming
  redis.xadd(key, "*", { name: "John Doe", age: 30 }, {
    trim: {
      type: "MAXLEN",
      threshold: 1000,
      comparison: "=",
    },
  })

  ```
</RequestExample>


# XRANGE

Returns stream entries matching a given range of IDs.

## Arguments

<ParamField body="key" type="str" required>
  The key to of the stream.
</ParamField>

<ParamField body="start" type="str" required>
  The stream entry ID to start from.
</ParamField>

<ParamField body="end" type="str" required>
  The stream entry ID to end at.
</ParamField>

<ParamField body="count" type="int">
  The maximum number of entries to return.
</ParamField>

## Response

<ResponseField type="Record<streamId, Record<field, value>>">
  An object of stream entries, keyed by their stream ID
</ResponseField>

<RequestExample>
  ```py Example
  entries = redis.xrange(key, "-", "+")
  print(entries)
  # {
  #   "1548149259438-0": {
  #     "field1": "value1",
  #     "field2": "value2"
  #   },
  #   "1548149259438-1": {
  #     "field1": "value3",
  #     "field2": "value4"
  #   }
  # }

  ```
</RequestExample>


# APPEND

Append a value to a string stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="value" required>
  The value to append.
</ParamField>

## Response

<ResponseField type="int" required>
  How many characters were added to the string.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", "Hello")

  assert redis.append("key", " World") == 11

  assert redis.get("key") == "Hello World"
  ```
</RequestExample>


# DECR

Decrement the integer value of a key by one

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="str" required>
  The key to decrement.
</ParamField>

## Response

<ResponseField type="int" required>
  The value at the key after the decrementing.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", 6)

  assert redis.decr("key") == 5
  ```
</RequestExample>


# DECRBY

Decrement the integer value of a key by a given number.

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="str" required>
  The key to decrement.
</ParamField>

<ParamField body="decrement" type="int" required>
  The amount to decrement by.
</ParamField>

## Response

<ResponseField type="int" required>
  The value at the key after the decrementing.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", 6)

  assert redis.decrby("key", 4) == 2
  ```
</RequestExample>


# GET

Return the value of the specified key or `None` if the key doesn't exist.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

## Response

<ResponseField required>
  The response is the value stored at the key or `None` if the key doesn't exist.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", "value")

  assert redis.get("key") == "value"
  ```
</RequestExample>


# GETDEL

Return the value of the specified key and delete the key.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

## Response

<ResponseField required>
  The response is the value stored at the key or `None` if the key doesn't exist.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", "value")

  assert redis.getdel("key") == "value"

  assert redis.get("key") == None
  ```
</RequestExample>


# GETRANGE

Return a substring of value at the specified key.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="start" type="int" required>
  The start index of the substring.
</ParamField>

<ParamField body="end" type="int" required>
  The end index of the substring.
</ParamField>

## Response

<ResponseField type="str" required>
  The substring.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", "Hello World")

  assert redis.getrange("key", 0, 4) == "Hello"
  ```
</RequestExample>


# GETSET

Return the value of the specified key and replace it with a new value.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="value" type="Any" required>
  The new value to store.
</ParamField>

## Response

<ResponseField required>
  The response is the value stored at the key or `None` if the key doesn't exist.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", "old-value")

  assert redis.getset("key", "newvalue") == "old-value"
  ```
</RequestExample>


# INCR

Increment the integer value of a key by one

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="str" required>
  The key to increment.
</ParamField>

## Response

<ResponseField type="int" required>
  The value at the key after the incrementing.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", 6)

  assert redis.incr("key") == 7
  ```
</RequestExample>


# INCRBY

Increment the integer value of a key by a given number.

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="str" required>
  The key to decrement.
</ParamField>

<ParamField body="increment" type="int" required>
  The amount to increment by.
</ParamField>

## Response

<ResponseField type="int" required>
  The value at the key after the incrementing.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", 6)

  assert redis.incrby("key", 4) == 10
  ```
</RequestExample>


# INCRBYFLOAT

Increment the float value of a key by a given number.

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="str" required>
  The key to decrement.
</ParamField>

<ParamField body="increment" type="float" required>
  The amount to increment by.
</ParamField>

## Response

<ResponseField type="float" required>
  The value at the key after the incrementing.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", 6)

  # returns 10.5
  redis.incrbyfloat("key", 4,5)
  ```
</RequestExample>


# MGET

Load multiple keys from Redis in one go.

For billing purposes, this counts as a single command.

## Arguments

<ParamField body="keys" type="*List[str]" required>
  Multiple keys to load from Redis.
</ParamField>

## Response

<ResponseField type="List[str]" required>
  An array of values corresponding to the keys passed in. If a key doesn't exist, the value will be `None`.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key1", "value1")

  redis.set("key2", "value2")

  assert redis.mget("key1", "key2") == ["value1", "value2"]
  ```
</RequestExample>


# MSET

Set multiple keys in one go.

For billing purposes, this counts as a single command.

## Arguments

<ParamField type="Dict[str, Any]" required>
  An object where the keys are the keys to set, and the values are the values to set.
</ParamField>

## Response

<ResponseField type="bool" required>
  `True` if the operation succeeded.
</ResponseField>

<RequestExample>
  ```py Example
  redis.mset({
    "key1": "value1",
    "key2": "value2"
  })
  ```
</RequestExample>


# MSETNX

Set multiple keys in one go unless they exist already.

For billing purposes, this counts as a single command.

## Arguments

<ParamField type="Record<str, TValue>" required>
  An object where the keys are the keys to set, and the values are the values to set.
</ParamField>

## Response

<ResponseField required>
  `1` if all keys were set, `0` if at least one key was not set.
</ResponseField>

<RequestExample>
  ```py Example
  redis.msetnx({
      key1: 1,
      key2: "hello",
      key3: { a: 1, b: "hello" },
  })
  ```
</RequestExample>


# SET

Set a key to hold a string value.

## Arguments

<ParamField body="key" type="str" required>
  The key
</ParamField>

<ParamField body="value" type="TValue" required>
  The value, if this is not a string, we will use `JSON.stringify` to convert it
  to a string.
</ParamField>

<ParamField body="get" type="bool">
  Instead of returning `True`, this will cause the command to return the old
  value stored at key, or `None` when key did not exist.
</ParamField>

<ParamField body="ex" type="int">
  Sets an expiration (in seconds) to the key.
</ParamField>

<ParamField body="px" type="int">
  Sets an expiration (in milliseconds) to the key.
</ParamField>

<ParamField body="exat" type="int">
  Set the UNIX timestamp in seconds until the key expires.
</ParamField>

<ParamField body="pxat" type="int">
  Set the UNIX timestamp in milliseconds until the key expires.
</ParamField>

<ParamField body="keepttl" type="bool">
  Keeps the old expiration if the key already exists.
</ParamField>

<ParamField body="nx" type="bool">
  Only set the key if it does not already exist.
</ParamField>

<ParamField body="xx" type="bool">
  Only set the key if it already exists.
</ParamField>

## Response

<ResponseField required>
  `True` if the key was set.
  If `get` is specified, this will return the old value stored at key, or `None` when
  the key did not exist.
</ResponseField>

<RequestExample>
  ```py Basic
  assert redis.set("key", "value") == True

  assert redis.get("key") == "value"
  ```

  ```py With nx and xx
  # Only set the key if it does not already exist.
  assert redis.set("key", "value", nx=True) == False

  # Only set the key if it already exists.
  assert redis.set("key", "value", xx=True) == True
  ```

  ```py With expiration
  # Set the key to expire in 10 seconds.
  assert redis.set("key", "value", ex=10) == True

  # Set the key to expire in 10000 milliseconds.
  assert redis.set("key", "value", px=10000) == True
  ```

  ```py With old value
  # Get the old value stored at the key.
  assert redis.set("key", "new-value", get=True) == "old-value"
  ```
</RequestExample>


# SETRANGE

Writes the value of key at offset.

The SETRANGE command in Redis is used to modify a portion of the value of a key by replacing a substring within the key's existing value. It allows you to update part of the string value associated with a specific key at a specified offset.

## Arguments

<ParamField body="key" type="str" required>
  The name of the Redis key for which you want to modify the value.
</ParamField>

<ParamField body="offset" type="int" required>
  The zero-based index in the value where you want to start replacing characters.
</ParamField>

<ParamField body="value" type="str" required>
  The new string that you want to insert at the specified offset in the existing value.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the value after it was modified.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", "Hello World")

  assert redis.setrange("key", 6, "Redis") == 11

  assert redis.get("key") == "Hello Redis"
  ```
</RequestExample>


# STRLEN

Return the length of a string stored at a key.

The \`STRLEN\`\` command in Redis is used to find the length of the string value associated with a key. In Redis, keys can be associated with various data types, and one of these data types is the "string." The STRLEN command specifically operates on keys that are associated with string values.

## Arguments

<ParamField body="key" type="str" required>
  The name of the Redis key.
</ParamField>

## Response

<ResponseField type="int" required>
  The length of the value.
</ResponseField>

<RequestExample>
  ```py Example
  redis.set("key", "Hello World")

  assert redis.strlen("key") == 11
  ```
</RequestExample>


# ZADD

Add a member to a sorted set, or update its score if it already exists.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set.
</ParamField>

<ParamField body="scores" type="Dict[str, float]" required>
  A dictionary of elements and their scores.
</ParamField>

<ParamField body="xx" type="bool">
  Only update elements that already exist. Never add elements.
</ParamField>

<ParamField body="xx" type="bool">
  Only add new elements. Never update elements.
</ParamField>

<ParamField body="gt" type="bool">
  Update scores if the new score is greater than the old score.
</ParamField>

<ParamField body="lt" type="bool">
  Update scores if the new score is less than the old score.
</ParamField>

<ParamField body="ch" type="bool">
  Return the number of elements changed instead.
</ParamField>

<ParamField body="incr" type="bool">
  When this option is specified `ZADD` acts like `ZINCRBY`. Only one score-element pair can be specified in this mode.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of elements added to the sorted sets, not including elements already existing for which the score was updated.

  If `ch` was specified, the number of elements that were updated.

  If `incr` was specified, the new score of `member`.
</ResponseField>

<RequestExample>
  ```py Simple
  # Add three elements
  assert redis.zadd("myset", {
      "one": 1,
      "two": 2,
      "three": 3
  }) == 3

  # No element is added since "one" and "two" already exist
  assert redis.zadd("myset", {
      "one": 1,
      "two": 2
  }, nx=True) == 0

  # New element is not added since it does not exist
  assert redis.zadd("myset", {
      "new-element": 1
  }, xx=True) == 0

  # Only "three" is updated since new score was greater
  assert redis.zadd("myset", {
      "three": 10, "two": 0
  }, gt=True) == 1

  # Only "three" is updated since new score was greater
  assert redis.zadd("myset", {
      "three": 10,
      "two": 0
  }, gt=True) == 1
  ```
</RequestExample>


# ZCARD

Returns the number of elements in the sorted set stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of elements in the sorted set.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"one": 1, "two": 2, "three": 3})

  assert redis.zcard("myset") == 3
  ```
</RequestExample>


# ZCOUNT

Returns the number of elements in the sorted set stored at key filterd by score.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="min" type="int | str" required>
  The minimum score to filter by.

  Use `-inf` to effectively ignore this filter.

  Use `(number` to include the value.\`
</ParamField>

<ParamField body="max" type="int | str" required>
  The maximum score to filter by.

  Use `+inf` to effectively ignore this filter.

  Use `number)` to include the value.\`
</ParamField>

## Response

<ResponseField type="int" required>
  The number of elements where score is between min and max.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("key", 
      { score: 1, member: "one"}, 
      { score: 2, member: "two" },
  )
  elements = redis.zcount("key", 1, "+inf")
  print(elements); # 1
  ```
</RequestExample>


# ZDIFF

Returns the difference between sets.

## Arguments

<ParamField body="keys" type="List[str]" required>
  The keys of the sets to compare.
</ParamField>

<ParamField body="withscores" type="bool" default="false">
  Whether to include scores in the result.
</ParamField>

## Response

<ResponseField type="List[str] | List[Tuple[str, float]]">
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```py Simple
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"c": 3, "d": 4, "e": 5})

  result = redis.zdiff(["key1", "key2"])

  assert result == ["a", "b"]
  ```

  ```py With scores
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"c": 3, "d": 4, "e": 5})

  result = redis.zdiff(["key1", "key2"], withscores=True)

  assert result == [("a", 1), ("b", 2)]
  ```
</RequestExample>


# ZDIFFSTORE

Writes the difference between sets to a new key.

## Arguments

<ParamField body="destination" type="str" required>
  The key to write the difference to.
</ParamField>

<ParamField body="keys" type="List[str]" required>
  The keys to compare.
</ParamField>

## Response

<ResponseField type="int">
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"c": 3, "d": 4, "e": 5})

  # a and b
  assert redis.zdiffstore("dest", ["key1", "key2"]) == 2
  ```
</RequestExample>


# ZINCRBY

Increment the score of a member.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set.
</ParamField>

<ParamField body="increment" type="int" required>
  The increment to add to the score.
</ParamField>

<ParamField body="member" type="str" required>
  The member to increment.
</ParamField>

## Response

<ResponseField type="float" required>
  The new score of `member` after the increment operation.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"one": 1, "two": 2, "three": 3})

  assert redis.zincrby("myset", 2, "one") == 3
  ```
</RequestExample>


# ZINTER

Returns the intersection between sets.

## Arguments

<ParamField body="keys" type="List[str]" required>
  The keys of the sets to compare.
</ParamField>

<ParamField body="weights" type="List[float]" default="None">
  The weights to apply to the sets.
</ParamField>

<ParamField body="aggregate" type="&#x22;SUM&#x22; | &#x22;MIN&#x22; | &#x22;MAX&#x22;" default="sum">
  The aggregation function to apply to the sets.
</ParamField>

<ParamField body="withscores" type="bool" default="false">
  Whether to include scores in the result.
</ParamField>

## Response

<ResponseField type="List[str] | List[Tuple[str, float]]">
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```py Simple
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"c": 3, "d": 4, "e": 5})

  result = redis.zinter(["key1", "key2"])

  assert result == ["c"]
  ```

  ```py Aggregation
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"a": 3, "b": 4, "c": 5})

  result = redis.zinter(["key1", "key2"], withscores=True, aggregate="SUM")

  assert result == [("a", 4), ("b", 6), ("c", 8)]
  ```

  ```py Weights
  redis.zadd("key1", {"a": 1})

  redis.zadd("key2", {"a": 1})

  result = redis.zinter(["key1", "key2"],
                        withscores=True,
                        aggregate="SUM",
                        weights=[2, 3])

  assert result == [("a", 5)]
  ```
</RequestExample>


# ZINTERSTORE

Calculates the intersection of sets and stores the result in a key

## Arguments

<ParamField body="destination" type="str" required>
  The key to store the result in.
</ParamField>

<ParamField body="keys" type="List[str]" required>
  The keys of the sets to compare.
</ParamField>

<ParamField body="weights" type="List[float]" default="None">
  The weights to apply to the sets.
</ParamField>

<ParamField body="aggregate" type="&#x22;SUM&#x22; | &#x22;MIN&#x22; | &#x22;MAX&#x22;" default="sum">
  The aggregation function to apply to the sets.
</ParamField>

<ParamField body="withscores" type="bool" default="false">
  Whether to include scores in the result.
</ParamField>

## Response

## Response

<ResponseField required>
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```py Simple
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"c": 3, "d": 4, "e": 5})

  result = redis.zinterstore("dest", ["key1", "key2"])

  assert result == 1
  ```

  ```py Aggregation
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"a": 3, "b": 4, "c": 5})

  result = redis.zinterstore("dest", ["key1", "key2"], withscores=True, aggregate="SUM")

  assert result == 3
  ```

  ```py Weights
  redis.zadd("key1", {"a": 1})

  redis.zadd("key2", {"a": 1})

  result = redis.zinterstore("dest", ["key1", "key2"],
                        withscores=True,
                        aggregate="SUM",
                        weights=[2, 3])

  assert result == 1
  ```
</RequestExample>


# ZLEXCOUNT

Returns the number of elements in the sorted set stored at key filterd by lex.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="min" type="str" required>
  The lower lexicographical bound to filter by.

  Use `-` to disable the lower bound.
</ParamField>

<ParamField body="max" type="str" required>
  The upper lexicographical bound to filter by.

  Use `+` to disable the upper bound.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of matched.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zlexcount("myset", "-", "+") == 3
  ```
</RequestExample>


# ZMSCORE

Returns the scores of multiple members.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set.
</ParamField>

## Response

<ResponseField body="members" type="List[str]" required>
  The members of the sorted set.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zlexcount("myset", "-", "+") == 3
  ```
</RequestExample>


# ZPOPMAX

Removes and returns up to count members with the highest scores in the sorted set stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set
</ParamField>

<ParamField body="count" type="int">
  The number of members to pop
</ParamField>

## Response

<ResponseField type="List[Tuple[str, float]]">
  A list of tuples containing the popped members and their scores
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zpopmax("myset") == [("c", 3)]
  ```
</RequestExample>


# ZPOPMIN

Removes and returns up to count members with the lowest scores in the sorted set stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set
</ParamField>

<ParamField body="count" type="int">
  The number of members to pop
</ParamField>

## Response

<ResponseField type="List[Tuple[str, float]]">
  A list of tuples containing the popped members and their scores
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zpopmin("myset") == [("a", 1)]
  ```
</RequestExample>


# ZRANDMEMBER

Returns one or more random members from a sorted set, optionally with their scores.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set
</ParamField>

<ParamField body="count" type="int">
  The number of members to return
</ParamField>

<ParamField body="withscores" type="bool">
  Whether to return the scores along with the members
</ParamField>

## Response

<ResponseField type="str | Tuple[str, float] | List[str] | List[Tuple[str, float]]">
  The random member(s) from the sorted set

  If no count is specified, a single member is returned. If count is specified, a list of members is returned.

  If withscores, members are returned as a tuple of (member, score).
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"one": 1, "two": 2, "three": 3})

  # "one"
  redis.zrandmember("myset")

  # ["one", "three"]
  redis.zrandmember("myset", 2)
  ```
</RequestExample>


# ZRANGE

Returns the specified range of elements in the sorted set stored at key.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="min" type="float | str" required>
  The minimum value to include.
</ParamField>

<ParamField body="max" type="float | str" required>
  The maximum value to include.
</ParamField>

"-inf" and "+inf" are also valid values for the ranges

<ParamField body="withscores" type="bool">
  Whether to include the scores in the response.
</ParamField>

<ParamField body="rev" type="bool">
  Whether to reverse the order of the response.
</ParamField>

<ParamField body="sortby" type="&#x22;BYSCORE&#x22; | &#x22;BYLEX&#x22;">
  If bylex
</ParamField>

<ParamField body="offset" type="int">
  The offset to start from.
</ParamField>

<ParamField body="count" type="int">
  The number of elements to return.
</ParamField>

## Response

<ResponseField type="List[str] | List[Tuple[str, float]]">
  The values in the specified range.

  If `withscores` is true, the members will be tuples of the form `(member, score)`.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zrange("myset", 0, 1) == ["a", "b"]
  ```

  ```py Reverse
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zrange("myset", 0, 1, rev=True) == ["c", "b"]

  ```

  ```py Sorted
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zrange("myset", 0, 1, sortby="BYSCORE") == ["a", "b"]

  ```

  ```py With scores
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zrange("myset", 0, 1, withscores=True) == [("a", 1), ("b", 2)]
  ```
</RequestExample>


# ZRANK

Returns the rank of a member

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="member" type="TMember" required>
  The member to get the rank of.
</ParamField>

## Response

<ResponseField type="int" required>
  The rank of the member.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zrank("myset", "a") == 0

  assert redis.zrank("myset", "d") == None

  assert redis.zrank("myset", "b") == 1

  assert redis.zrank("myset", "c") == 2
  ```
</RequestExample>


# ZREM

Remove one or more members from a sorted set

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set
</ParamField>

<ParamField body="members" type="*List[str]" required>
  One or more members to remove
</ParamField>

## Response

<ResponseField required>
  The number of members removed from the sorted set.
</ResponseField>

<RequestExample>
  ```py Single
  redis.zadd("myset", {"one": 1, "two": 2, "three": 3})

  assert redis.zrem("myset", "one", "four") == 1
  ```
</RequestExample>


# ZREMRANGEBYLEX

Remove all members in a sorted set between the given lexicographical range.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set
</ParamField>

<ParamField body="min" type="str" required>
  The minimum lexicographical value to remove.
</ParamField>

<ParamField body="min" type="str" required>
  The maximum lexicographical value to remove.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of elements removed from the sorted set.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zremrangebylex("key", "alpha", "omega")
  ```
</RequestExample>


# ZREMRANGEBYRANK

Remove all members in a sorted set between the given ranks.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set
</ParamField>

<ParamField body="min" type="str | float" required>
  The minimum rank to remove.
</ParamField>

<ParamField body="min" type="str | float" required>
  The maximum rank to remove.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of elements removed from the sorted set.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zremrangebyrank("key", 4, 20)
  ```
</RequestExample>


# ZREMRANGEBYSCORE

Remove all members in a sorted set between the given scores.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set
</ParamField>

<ParamField body="min" type="str | float" required>
  The minimum score to remove.
</ParamField>

<ParamField body="min" type="str | float" required>
  The maximum score to remove.
</ParamField>

## Response

<ResponseField type="int" required>
  The number of elements removed from the sorted set.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zremrangebyscore("key", 2, 5)
  ```
</RequestExample>


# ZREVRANK

Returns the rank of a member in a sorted set, with scores ordered from high to low.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

<ParamField body="member" type="str" required>
  The member to get the reverse rank of.
</ParamField>

## Response

<ResponseField type="int" required>
  The reverse rank of the member.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zrevrank("myset", "a") == 2
  ```
</RequestExample>


# ZSCAN

Scan a sorted set

Return a paginated list of members and their scores of an ordered set matching a pattern.

## Arguments

<ParamField body="key" type="str" required>
  The key of the sorted set.
</ParamField>

<ParamField body="cursor" type="int" required>
  The cursor, use `0` in the beginning and then use the returned cursor for subsequent calls.
</ParamField>

<ParamField body="match" type="str">
  Glob-style pattern to filter by members.
</ParamField>

<ParamField body="count" type="int">
  Number of members to return per call.
</ParamField>

## Response

<ResponseField type="Tuple[int, List[str]]" required>
  The new cursor and keys as a tuple.
  If the new cursor is `0` the iteration is complete.
</ResponseField>

<RequestExample>
  ```py Example
  # Get all elements of an ordered set.

  cursor = 0
  results = []

  while True:
      cursor, keys = redis.zscan("myzset", cursor, match="*")

      results.extend(keys)
      if cursor == 0:
          break

  for key, score in results:
      print(key, score)
  ```
</RequestExample>


# ZSCORE

Returns the scores of a member.

## Arguments

<ParamField body="key" type="str" required>
  The key to get.
</ParamField>

## Response

<ResponseField body="member" type="TMember" required>
  A member of the sortedset.
</ResponseField>

<RequestExample>
  ```py Example
  redis.zadd("myset", {"a": 1, "b": 2, "c": 3})

  assert redis.zscore("myset", "a") == 1
  ```
</RequestExample>


# ZINTER

Returns the intersection between sets.

## Arguments

<ParamField body="keys" type="List[str]" required>
  The keys of the sets to compare.
</ParamField>

<ParamField body="weights" type="List[float]" default="None">
  The weights to apply to the sets.
</ParamField>

<ParamField body="aggregate" type="&#x22;SUM&#x22; | &#x22;MIN&#x22; | &#x22;MAX&#x22;" default="sum">
  The aggregation function to apply to the sets.
</ParamField>

<ParamField body="withscores" type="bool" default="false">
  Whether to include scores in the result.
</ParamField>

## Response

<ResponseField type="List[str] | List[Tuple[str, float]]">
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```py Simple
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"c": 3, "d": 4, "e": 5})

  result = redis.zunion(["key1", "key2"])

  assert result == ["a", "b", "c", "d", "e"]
  ```

  ```py Aggregation
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"a": 3, "b": 4, "c": 5})

  result = redis.zunion(["key1", "key2"], withscores=True, aggregate="SUM")

  assert result == [("a", 4), ("b", 6), ("c", 8)]
  ```

  ```py Weights
  redis.zadd("key1", {"a": 1})

  redis.zadd("key2", {"a": 1})

  result = redis.zunion(["key1", "key2"],
                        withscores=True,
                        aggregate="SUM",
                        weights=[2, 3])

  assert result == [("a", 5)]
  ```
</RequestExample>


# ZUNIONSTORE

Writes the union between sets to a new key.

## Arguments

<ParamField body="destination" type="str" required>
  The key to store the resulting set in.
</ParamField>

<ParamField body="keys" type="List[str]" required>
  The keys of the sets to compare.
</ParamField>

<ParamField body="weights" type="List[float]" default="None">
  The weights to apply to the sets.
</ParamField>

<ParamField body="aggregate" type="&#x22;SUM&#x22; | &#x22;MIN&#x22; | &#x22;MAX&#x22;" default="sum">
  The aggregation function to apply to the sets.
</ParamField>

<ParamField body="withscores" type="bool" default="false">
  Whether to include scores in the result.
</ParamField>

## Response

<ResponseField required>
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```py Simple
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"c": 3, "d": 4, "e": 5})

  result = redis.zunionstore(["key1", "key2"])

  assert result == 5
  ```

  ```py Aggregation
  redis.zadd("key1", {"a": 1, "b": 2, "c": 3})

  redis.zadd("key2", {"a": 3, "b": 4, "c": 5})

  result = redis.zunionstore(["key1", "key2"], withscores=True, aggregate="SUM")

  assert result == [("a", 4), ("b", 6), ("c", 8)]
  ```

  ```py Weights
  redis.zadd("key1", {"a": 1})

  redis.zadd("key2", {"a": 1})

  result = redis.zunionstore(["key1", "key2"],
                        withscores=True,
                        aggregate="SUM",
                        weights=[2, 3])

  assert result == [("a", 5)]
  ```
</RequestExample>


# Features



### BITFIELD and BITFIELD\_RO

One particular case is represented by these two chained commands, which are
available as functions that return an instance of the `BITFIELD` and,
respectively, `BITFIELD_RO` classes. Use the `execute` function to run the
commands.

```python
redis.bitfield("test_key") \
  .incrby(encoding="i8", offset=100, increment=100) \
  .overflow("SAT") \
  .incrby(encoding="i8", offset=100, increment=100) \
  .execute()

redis.bitfield_ro("test_key_2") \
  .get(encoding="u8", offset=0) \
  .get(encoding="u8", offset="#1") \
  .execute()
```

### Custom commands

If you want to run a command that hasn't been implemented, you can use the
`execute` function of your client instance and pass the command as a `list`.

```python
redis.execute(["XLEN", "test_stream"])
```

# Encoding

Although Redis can store invalid JSON data, there might be problems with the
deserialization. To avoid this, the Upstash REST proxy is capable of encoding
the data as base64 on the server and then sending it to the client to be
decoded.

For very large data, this can add a few milliseconds in latency. So, if you're
sure that your data is valid JSON, you can set `rest_encoding` to `None`.

# Retry mechanism

upstash-redis has a fallback mechanism in case of network or API issues. By
default, if a request fails it'll retry once, 3 seconds after the error. If you
want to customize that, set `rest_retries` and `rest_retry_interval` (in
seconds).

# Pipelines & Transactions

If you want to submit commands in batches to reduce the number of roundtrips, you can utilize pipelining or
transactions. The difference between pipelines and transactions is that transactions are atomic: no other
command is executed during that transaction. In pipelines there is no such guarantee.

To use a pipeline, simply call the `pipeline` method:

```python
pipeline = redis.pipeline()

pipeline.set("foo", 1)
pipeline.incr("foo")
pipeline.get("foo")

result = pipeline.exec()

print(result)
# prints [True, 2, '2']
```

For transaction, use `mutli`:

```python
pipeline = redis.multi()

pipeline.set("foo", 1)
pipeline.incr("foo")
pipeline.get("foo")

result = pipeline.exec()

print(result)
# prints [True, 2, '2']
```

You can also chain the commands:

```python
pipeline = redis.pipeline()

pipeline.set("foo", 1).incr("foo").get("foo")
result = pipeline.exec()

print(result)
# prints [True, 2, '2']
```


# Getting Started



## Install

### PyPI

```bash
pip install upstash-redis
```

## Usage

To be able to use upstash-redis, you need to create a database on
[Upstash](https://console.upstash.com/) and grab `UPSTASH_REDIS_REST_URL` and
`UPSTASH_REDIS_REST_TOKEN` from the console.

```python
# for sync client
from upstash_redis import Redis

redis = Redis(url="UPSTASH_REDIS_REST_URL", token="UPSTASH_REDIS_REST_TOKEN")

# for async client
from upstash_redis.asyncio import Redis

redis = Redis(url="UPSTASH_REDIS_REST_URL", token="UPSTASH_REDIS_REST_TOKEN")
```

Or, if you want to automatically load the credentials from the environment:

```python
# for sync use
from upstash_redis import Redis
redis = Redis.from_env()

# for async use
from upstash_redis.asyncio import Redis
redis = Redis.from_env()
```

If you are in a serverless environment that allows it, it's recommended to
initialise the client outside the request handler to be reused while your
function is still hot.

Running commands might look like this:

```python
from upstash_redis import Redis

redis = Redis.from_env()

def main():
  redis.set("a", "b")
  print(redis.get("a"))

# or for async context:

from upstash_redis.asyncio import Redis

redis = Redis.from_env()

async def main():
  await redis.set("a", "b")
  print(await redis.get("a"))
```


# Overview



`upstash-redis` is a connectionless, HTTP-based Redis client for Python,
designed to be used in serverless and serverful environments such as:

*   AWS Lambda
*   Vercel Serverless
*   Google Cloud Functions
*   and other environments where HTTP is preferred over TCP.

Inspired by other Redis clients like
[@upstash/redis](https://github.com/upstash/upstash-redis) and
[redis-py](https://github.com/redis/redis-py), the goal of this SDK is to
provide a simple way to use Redis over the
[Upstash REST API](https://docs.upstash.com/redis/features/restapi).

The SDK is currently compatible with Python 3.8 and above.

You can find the Github Repository [here](https://github.com/upstash/redis-python).


# Ratelimiting Algorithms



## Fixed Window

This algorithm divides time into fixed durations/windows. For example each
window is 10 seconds long. When a new request comes in, the current time is used
to determine the window and a counter is increased. If the counter is larger
than the set limit, the request is rejected.

### Pros

*   Very cheap in terms of data size and computation
*   Newer requests are not starved due to a high burst in the past

### Cons

*   Can cause high bursts at the window boundaries to leak through
*   Causes request stampedes if many users are trying to access your server,
    whenever a new window begins

### Usage

```python
from upstash_ratelimit import Ratelimit, FixedWindow
from upstash_redis import Redis

ratelimit = Ratelimit(
    redis=Redis.from_env(),
    limiter=FixedWindow(max_requests=10, window=10),
)
```

## Sliding Window

Builds on top of fixed window but instead of a fixed window, we use a rolling
window. Take this example: We have a rate limit of 10 requests per 1 minute. We
divide time into 1 minute slices, just like in the fixed window algorithm.
Window 1 will be from 00:00:00 to 00:01:00 (HH:MM:SS). Let's assume it is
currently 00:01:15 and we have received 4 requests in the first window and 5
requests so far in the current window. The approximation to determine if the
request should pass works like this:

```python
limit = 10

# 4 request from the old window, weighted + requests in current window
rate = 4 * ((60 - 15) / 60) + 5 = 8

return rate < limit # True means we should allow the request
```

### Pros

*   Solves the issue near boundary from fixed window.

### Cons

*   More expensive in terms of storage and computation
*   It's only an approximation because it assumes a uniform request flow in the
    previous window

### Usage

```python
from upstash_ratelimit import Ratelimit, SlidingWindow
from upstash_redis import Redis

ratelimit = Ratelimit(
    redis=Redis.from_env(),
    limiter=SlidingWindow(max_requests=10, window=10),
)
```

## Token Bucket

Consider a bucket filled with maximum number of tokens that refills constantly
at a rate per interval. Every request will remove one token from the bucket and
if there is no token to take, the request is rejected.

### Pros

*   Bursts of requests are smoothed out and you can process them at a constant
    rate.
*   Allows setting a higher initial burst limit by setting maximum number of
    tokens higher than the refill rate

### Cons

*   Expensive in terms of computation

### Usage

```python
from upstash_ratelimit import Ratelimit, TokenBucket
from upstash_redis import Redis

ratelimit = Ratelimit(
    redis=Redis.from_env(),
    limiter=TokenBucket(max_tokens=10, refill_rate=5, interval=10),
)
```


# Features



## Block until ready

You also have the option to try and wait for a request to pass in the given
timeout.

It is very similar to the `limit` method and takes an identifier and returns the
same response. However if the current limit has already been exceeded, it will
automatically wait until the next window starts and will try again. Setting the
timeout parameter (in seconds) will cause the method to block a finite amount of
time.

```python
from upstash_ratelimit import Ratelimit, SlidingWindow
from upstash_redis import Redis

# Create a new ratelimiter, that allows 10 requests per 10 seconds
ratelimit = Ratelimit(
    redis=Redis.from_env(),
    limiter=SlidingWindow(max_requests=10, window=10),
)

response = ratelimit.block_until_ready("id", timeout=30)

if not response.allowed:
    print("Unable to process, even after 30 seconds")
else:
    do_expensive_calculation()
    print("Here you go!")
```

## Using multiple limits

Sometimes you might want to apply different limits to different users. For
example you might want to allow 10 requests per 10 seconds for free users, but
60 requests per 10 seconds for paid users.

Here's how you could do that:

```python
from upstash_ratelimit import Ratelimit, SlidingWindow
from upstash_redis import Redis

class MultiRL:
    def __init__(self) -> None:
        redis = Redis.from_env()
        self.free = Ratelimit(
            redis=redis,
            limiter=SlidingWindow(max_requests=10, window=10),
            prefix="ratelimit:free",
        )

        self.paid = Ratelimit(
            redis=redis,
            limiter=SlidingWindow(max_requests=60, window=10),
            prefix="ratelimit:paid",
        )

# Create a new ratelimiter, that allows 10 requests per 10 seconds
ratelimit = MultiRL()

ratelimit.free.limit("userIP")
ratelimit.paid.limit("userIP")
```

## Custom Rates

When rate limiting, you may want different requests to consume different amounts of tokens.
This could be useful when processing batches of requests where you want to rate limit based
on items in the batch or when you want to rate limit based on the number of tokens.

To achieve this, you can simply pass `rate` parameter when calling the limit method:

```python

from upstash_ratelimit import Ratelimit, FixedWindow
from upstash_redis import Redis

ratelimit = Ratelimit(
    redis=Redis.from_env(),
    limiter=FixedWindow(max_requests=10, window=10),
)

# pass rate as 5 to subtract 5 from the number of
# allowed requests in the window:
identifier = "api"
response = ratelimit.limit(identifier, rate=5)
```


# Getting Started



## Install

```bash
pip install upstash-ratelimit
```

## Create database

To be able to use upstash-ratelimit, you need to create a database on
[Upstash](https://console.upstash.com/).

## Usage

For possible Redis client configurations, have a look at the
[Redis SDK repository](https://github.com/upstash/redis-python).

> This library supports asyncio as well. To use it, import the asyncio-based
> variant from the `upstash_ratelimit.asyncio` module.

```python
from upstash_ratelimit import Ratelimit, FixedWindow
from upstash_redis import Redis

# Create a new ratelimiter, that allows 10 requests per 10 seconds
ratelimit = Ratelimit(
    redis=Redis.from_env(),
    limiter=FixedWindow(max_requests=10, window=10),
    # Optional prefix for the keys used in Redis. This is useful
    # if you want to share a Redis instance with other applications
    # and want to avoid key collisions. The default prefix is
    # "@upstash/ratelimit"
    prefix="@upstash/ratelimit",
)

# Use a constant string to limit all requests with a single ratelimit
# Or use a user ID, API key or IP address for individual limits.
identifier = "api"
response = ratelimit.limit(identifier)

if not response.allowed:
    print("Unable to process at this time")
else:
    do_expensive_calculation()
    print("Here you go!")
```

The `limit` method also returns the following metadata:

```python
@dataclasses.dataclass
class Response:
    allowed: bool
    """
    Whether the request may pass(`True`) or exceeded the limit(`False`)
    """

    limit: int
    """
    Maximum number of requests allowed within a window.
    """

    remaining: int
    """
    How many requests the user has left within the current window.
    """

    reset: float
    """
    Unix timestamp in seconds when the limits are reset
    """
```


# Overview



`upstash-ratelimit` is a connectionless rate limiting library for Python,
designed to be used in serverless environments such as:

*   AWS Lambda
*   Vercel Serverless
*   Google Cloud Functions
*   and other environments where HTTP is preferred over TCP.

The SDK is currently compatible with Python 3.8 and above.

You can find the Github Repository [here](https://github.com/upstash/ratelimit-python).


# Ratelimiting Algorithms



We provide different algorithms to use out of the box. Each has pros and cons.

## Fixed Window

This algorithm divides time into fixed durations/windows. For example each
window is 10 seconds long. When a new request comes in, the current time is used
to determine the window and a counter is increased. If the counter is larger
than the set limit, the request is rejected.

### Pros

*   Very cheap in terms of data size and computation
*   Newer requests are not starved due to a high burst in the past

### Cons

*   Can cause high bursts at the window boundaries to leak through
*   Causes request stampedes if many users are trying to access your server,
    whenever a new window begins

### Usage

Create a new ratelimiter, that allows 10 requests per 10 seconds.

<Tabs>
  <Tab title="Regional">
    ```ts
    const ratelimit = new Ratelimit({
      redis: Redis.fromEnv(),
      limiter: Ratelimit.fixedWindow(10, "10 s"),
    });
    ```
  </Tab>

  <Tab title="Multi Regional">
    ```ts
    const ratelimit = new MultiRegionRatelimit({
      redis: [
        new Redis({
          /* auth */
        }),
        new Redis({
          /* auth */
        })
      ],
      limiter: MultiRegionRatelimit.fixedWindow(10, "10 s"),
    });
    ```
  </Tab>
</Tabs>

## Sliding Window

Builds on top of fixed window but instead of a fixed window, we use a rolling
window. Take this example: We have a rate limit of 10 requests per 1 minute. We
divide time into 1 minute slices, just like in the fixed window algorithm.
Window 1 will be from 00:00:00 to 00:01:00 (HH:MM:SS). Let's assume it is
currently 00:01:15 and we have received 4 requests in the first window and 5
requests so far in the current window. The approximation to determine if the
request should pass works like this:

```ts
limit = 10

// 4 request from the old window, weighted + requests in current window
rate = 4 * ((60 - 15) / 60) + 5 = 8

return rate < limit // True means we should allow the request
```

### Pros

*   Solves the issue near boundary from fixed window.

### Cons

*   More expensive in terms of storage and computation
*   Is only an approximation, because it assumes a uniform request flow in the
    previous window, but this is fine in most cases

### Usage

Create a new ratelimiter, that allows 10 requests per 10 seconds.

<Tabs>
  <Tab title="Regional">
    ```ts
    const ratelimit = new Ratelimit({
      redis: Redis.fromEnv(),
      limiter: Ratelimit.slidingWindow(10, "10 s"),
    });
    ```
  </Tab>

  <Tab title="Multi Regional">
    **Warning:** Using sliding window algorithm with the multiregion setup results in large number of
    commands in Redis and long request processing times. If you want to keep the number of commands
    low, we recommend using the [fixed window algorithm in multi region setup](https://upstash.com/docs/redis/sdks/ratelimit-ts/algorithms#fixed-window).

    ```ts
    const ratelimit = new MultiRegionRatelimit({
      redis: [
        new Redis({
          /* auth */
        }),
        new Redis({
          /* auth */
        })
      ],
      limiter: MultiRegionRatelimit.slidingWindow(10, "10 s"),
    });
    ```
  </Tab>
</Tabs>

## Token Bucket

Consider a bucket filled with `{maxTokens}` tokens that refills constantly at
`{refillRate}` per `{interval}`. Every request will remove one token from the
bucket and if there is no token to take, the request is rejected.

### Pros

*   Bursts of requests are smoothed out and you can process them at a constant
    rate.
*   Allows to set a higher initial burst limit by setting `maxTokens` higher than
    `refillRate`

### Cons

*   Expensive in terms of computation

### Usage

Create a new bucket, that refills 5 tokens every 10 seconds and has a maximum
size of 10.

<Tabs>
  <Tab title="Regional">
    ```ts
    const ratelimit = new Ratelimit({
      redis: Redis.fromEnv(),
      limiter: Ratelimit.tokenBucket(5, "10 s", 10),
      analytics: true,
    });
    ```
  </Tab>

  <Tab title="Multi Regional">
    *Not yet supported for `MultiRegionRatelimit`*
  </Tab>
</Tabs>


# Costs



This page details the cost of the Ratelimit algorithms in terms of the number of Redis commands. Note that these are calculated for Regional Ratelimits. For [Multi Region Ratelimit](https://upstash.com/docs/redis/sdks/ratelimit-ts/features#multi-region), costs will be higher. Additionally, if a Global Upstash Redis is used as the database, number of commands should be calculated as `(1+readRegionCount) * writeCommandCount + readCommandCount` and plus 1 if analytics is enabled.

The Rate Limit SDK minimizes Redis calls to reduce latency overhead and cost. Number of commands executed by the Rate Limit algorithm depends on the chosen algorithm, as well as the state of the algorithm and the caching.

#### Algorithm State

By state of the algorithm, we refer to the entry in our Redis store regarding some identifier `ip1`. You can imagine that there is a state for every identifier. We name these states in the following manner for the purpose of attributing costs to each one:

| State        | Success | Explanation                                                              |
| ------------ | ------- | ------------------------------------------------------------------------ |
| First        | true    | First time the Ratelimit was called with identifier `ip1`                |
| Intermediate | true    | Second or some other time the Ratelimit was called with identifier `ip1` |
| Rate-Limited | false   | Requests with identifier `ip1` which are rate limited.                   |

For instance, first time we call the algorithm with `ip1`, `PEXPIRE` is called so that the key expires after some time. In the following calls, we still use the same script but don't call `PEXPIRE`. In the rate-limited state, we may avoid using Redis altogether if we can make use of the cache.

#### Cache Result

We distinguish the two cases when the identifier `ip1` is found in cache, resulting in a "hit" and the case when the identifier `ip1` is not found in the cache, resulting in a "miss". The cache only exists in the runtime environment and is independent of the Redis database. The state of the cache is especially relevant for serverless contexts, where the cache will usually be empty because of a cold start.

| Result | Explanation                                                                                             |
| ------ | ------------------------------------------------------------------------------------------------------- |
| Hit    | Identifier `ip1` is found in the runtime cache                                                          |
| Miss   | Identifier `ip1` is not found in cache or the value in the cache doesn't block (rate-limit) the request |

An identifier is saved in the cache only when a request is rate limited after a call to the Redis database. The request to Redis returns a timestamp for the time when such a request won't be rate limited anymore. We save this timestamp in the cache and this allows us to reject any request before this timestamp without having to consult the Redis database.

See the [section on caching](https://upstash.com/docs/redis/sdks/ratelimit-ts/features) for more details.

# Costs

### `limit()`

#### Fixed Window

| Cache Result | Algorithm State | Command Count | Commands            |
| ------------ | --------------- | ------------- | ------------------- |
| Hit/Miss     | First           | 3             | EVAL, INCR, PEXPIRE |
| Hit/Miss     | Intermediate    | 2             | EVAL, INCR          |
| Miss         | Rate-Limited    | 2             | EVAL, INCR          |
| Hit          | Rate-Limited    | 0             | *utilized cache*    |

#### Sliding Window

| Cache Result | Algorithm State | Command Count | Commands                      |
| ------------ | --------------- | ------------- | ----------------------------- |
| Hit/Miss     | First           | 5             | EVAL, GET, GET, INCR, PEXPIRE |
| Hit/Miss     | Intermediate    | 4             | EVAL, GET, GET, INCR          |
| Miss         | Rate-Limited    | 3             | EVAL, GET, GET                |
| Hit          | Rate-Limited    | 0             | *utilized cache*              |

#### Token Bucket

| Cache Result | Algorithm State    | Command Count | Commands                   |
| ------------ | ------------------ | ------------- | -------------------------- |
| Hit/Miss     | First/Intermediate | 4             | EVAL, HMGET, HSET, PEXPIRE |
| Miss         | Rate-Limited       | 2             | EVAL, HMGET                |
| Hit          | Rate-Limited       | 0             | *utilized cache*           |

### `getRemaining()`

This method doesn't use the cache or it doesn't have a state it depends on. Therefore, every call
results in the same number of commands in Redis.

| Algorithm      | Command Count | Commands       |
| -------------- | ------------- | -------------- |
| Fixed Window   | 2             | EVAL, GET      |
| Sliding Window | 3             | EVAL, GET, GET |
| Token Bucket   | 2             | EVAL, HMGET    |

### `resetUsedTokens()`

This method starts with a `SCAN` command and deletes every key that matches with `DEL` commands:

| Algorithm      | Command Count | Commands             |
| -------------- | ------------- | -------------------- |
| Fixed Window   | 3             | EVAL, SCAN, DEL      |
| Sliding Window | 4             | EVAL, SCAN, DEL, DEL |
| Token Bucket   | 3             | EVAL, SCAN, DEL      |

### `blockUntilReady()`

Works the same as `limit()`.

# Deny List

Enabling deny lists introduces a cost of 2 additional command per `limit` call.

Values passed in `identifier`, `ip`, `userAgent` and `country` are checked with a single `SMISMEMBER` command.
The other command is TTL which is for checking the status of the current ip deny list to figure out whether
it is expired, valid or disabled.

If [Auto IP deny list](https://upstash.com/docs/redis/sdks/ratelimit-ts/features#auto-ip-deny-list) is enabled,
the Ratelimit SDK will update the ip deny list everyday, in the first `limit` invocation after 2 AM UTC.
This will consume 9 commands per day.

If a value is found in the deny list at redis, the client saves this value in the cache and denies
any further requests with that value for a minute without calling Redis (except for analytics).

# Analytics

If analytics is enabled, all calls of `limit` will result in 1 more command since `ZINCRBY` will be called to update the analytics.


# Features



## Caching

For extreme load or denial of service attacks, it might be too expensive to call
redis for every incoming request, just to find out it should be blocked because
they have exceeded the limit.

You can use an ephemeral in memory cache by passing some variable of type
`Map<string, number>` as the `ephemeralCache` option:

```ts
const cache = new Map(); // must be outside of your serverless function handler

// ...

const ratelimit = new Ratelimit({
  // ...
  ephemeralCache: cache,
});
```

By default, `ephemeralCache` will be initialized with `new Map()` if no value is provided
as the `ephemeralCache` parameter. To disable the cache, one must pass `ephemeralCache: false`.

If enabled, the ratelimiter will keep track of the blocked identifiers and their
reset timestamps. When a request is received with some identifier `ip1` before the reset time of
`ip1`, the request will be denied without having to call Redis. [`reason` field of the
limit response will be `cacheBlock`](https://upstash.com/docs/redis/sdks/ratelimit-ts/methods#limit)

In serverless environments this is only possible if you create the cache or ratelimiter
instance outside of your handler function. While the function is still hot, the
ratelimiter can block requests without having to request data from Redis, thus
saving time and money.

See the section on how caching impacts the cost in the
[costs page](https://upstash.com/docs/redis/sdks/ratelimit-ts/costs#cache-result).

## Timeout

You can define an optional timeout in milliseconds, after which the request will
be allowed to pass regardless of what the current limit is. This can be useful
if you don't want network issues to cause your application to reject requests.

```ts
const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(10, "10 s"),
  timeout: 1000, // 1 second
  analytics: true,
});
```

Default value of the timeout is 5 seconds if no `timeout` is provided. When response
is success because of a timeout, this is shown in
[the `reason` field of the limit method](https://upstash.com/docs/redis/sdks/ratelimit-ts/methods#limit).

## Analytics & Dashboard

Another feature of the rate limiting algorithm is to collect analytics.

Bu default, analytics is disabled. To enable analytics, simply set the `analytics` parameter to `true`:

```js
const ratelimit = new Ratelimit({
  redis,
  analytics: true,
  limiter: Ratelimit.slidingWindow(60, "10s"),
});
```

Everytime we call `ratelimit.limit()`, analytics will be sent to the Redis database
([see costs page](https://upstash.com/docs/redis/sdks/ratelimit-ts/costs#analytics))
and information about the hour, identifier and the number of rate limit success and
failiures will be collected. This information can be viewed from the Upstash console.

If you are using rate limiting in Cloudflare Workers, Vercel Edge or a similar environment,
you need to make sure that the analytics request is delivered correctly to the Redis.
Otherwise, you may observe lower numbers than the actual number of calls.

To make sure that the request completes, you can use the `pending` field returned by
the `limit` method. See the
[Asynchronous synchronization between databases](https://upstash.com/docs/redis/sdks/ratelimit-ts/features#asynchronous-synchronization-between-databases)
section to see how `pending` can be used.

### Dashboard

If the analytics is enabled, you can find information about how many requests were made
with which identifiers and how many of the requests were blocked from the [Rate Limit
dashboard in Upstash Console](https://console.upstash.com/ratelimit).

To find to the dashboard, simply click the three dots and choose the "Rate Limit Analytics" tab:

![navigate.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/ratelimit/navigate.png)

In the dashboard, you can find information on how many requests were accepted, how many were blocked
and how many were received in total. Additionally, you can see requests over time; top allowed, rate limited
and denied requests.

![dashboard.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/ratelimit/dashboard.png)

**Allowed requests** show the identifiers of the requests which succeded. **Rate limited requests** show the
identifiers of the which were blocked because they surpassed the limit. **Denied requests** show the identifier,
user agent, country or the ip address which caused the request to fail.

If you are using a custom prefix, you need to use the same in the dashboard’s top left corner.

## Using Multiple Limits

Sometimes you might want to apply different limits to different users. For
example you might want to allow 10 requests per 10 seconds for free users, but
60 requests per 10 seconds for paid users.

Here's how you could do that:

```ts
import { Redis } from "@upstash/redis";
import { Ratelimit } from "@upstash/ratelimit";

const redis = Redis.fromEnv();

const ratelimit = {
  free: new Ratelimit({
    redis,
    analytics: true,
    prefix: "ratelimit:free",
    limiter: Ratelimit.slidingWindow(10, "10s"),
  }),
  paid: new Ratelimit({
    redis,
    analytics: true,
    prefix: "ratelimit:paid",
    limiter: Ratelimit.slidingWindow(60, "10s"),
  }),
};

await ratelimit.free.limit(ip);
// or for a paid user you might have an email or userId available:
await ratelimit.paid.limit(userId);
```

## Custom Rates

When we call `limit`, it subtracts 1 from the number of calls/tokens available in
the timeframe by default. But there are use cases where we may want to subtract different
numbers depending on the request.

Consider a case where we receive some input from the user either alone or in batches.
If we want to rate limit based on the number of inputs the user can send, we need a way of
specifying what value to subtract.

This is possible thanks to the `rate` parameter. Simply call the `limit` method like the
following:

```ts
const { success } = await ratelimit.limit("identifier", { rate: batchSize });
```

This way, the algorithm will subtract `batchSize` instead of 1.

## Multi Region

Let's assume you have customers in the US and Europe. In this case you can
create 2 seperate global redis databases on [Upstash](https://console.upstash.com)
(one with its primary in US and the other in Europe) and your users will enjoy
the latency of whichever db is closest to them.

Using a single Redis instance with replicas in different regions can not offer
the same performance as `MultiRegionRatelimit` because all write commands have
to go through the primary, increasing latency in other regions.

Using a single redis instance has the downside of providing low latencies only
to the part of your userbase closest to the deployed db. That's why we also
built `MultiRegionRatelimit` which replicates the state across multiple redis
databases as well as offering lower latencies to more of your users.

`MultiRegionRatelimit` does this by checking the current limit in the closest db
and returning immediately. Only afterwards will the state be asynchronously
replicated to the other databases leveraging
[CRDTs](https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type). Due
to the nature of distributed systems, there is no way to guarantee the set
ratelimit is not exceeded by a small margin. This is the tradeoff for reduced
global latency.

### Usage

The api is the same, except for asking for multiple redis instances:

```ts
import { MultiRegionRatelimit } from "@upstash/ratelimit"; // for deno: see above
import { Redis } from "@upstash/redis";

// Create a new ratelimiter, that allows 10 requests per 10 seconds
const ratelimit = new MultiRegionRatelimit({
  redis: [
    new Redis({
      /* auth */
    }),
    new Redis({
      /* auth */
    }),
    new Redis({
      /* auth */
    }),
  ],
  limiter: MultiRegionRatelimit.slidingWindow(10, "10 s"),
  analytics: true,
});

// Use a constant string to limit all requests with a single ratelimit
// Or use a userID, apiKey or ip address for individual limits.
const identifier = "api";
const { success } = await ratelimit.limit(identifier);
```

### Asynchronous synchronization between databases

The MultiRegion setup will do some synchronization between databases after
returning the current limit. This can lead to problems on Cloudflare Workers and
therefore Vercel Edge functions, because dangling promises must be taken care
of:

```ts
const { pending } = await ratelimit.limit("id");
context.waitUntil(pending);
```

See more information on `context.waitUntil` at
[Cloudflare](https://developers.cloudflare.com/workers/runtime-apis/context/#waituntil)
and [Vercel](https://vercel.com/docs/functions/edge-middleware/middleware-api#waituntil).
You can also utilize [`waitUntil` from Vercel Functions API](https://vercel.com/docs/functions/functions-api-reference#waituntil).


# Getting Started



## Create your Redis instance

For the rate limit to work, we need to create an Upstash Redis and get its credentials. To create an Upstash Redis, you can follow the [Upstash Redis "Get Started" guide](https://upstash.com/docs/redis/overall/getstarted).

## Add Ratelimit to Your Project

Once we have a Redis instance, next step is adding the rate limit to your project in its most basic form.

### Install Ratelimit

First, we need to install `@upstash/ratelimit`:

<Tabs>
  <Tab title="npm">
    ```bash
    npm install @upstash/ratelimit
    ```
  </Tab>

  <Tab title="deno">
    ```ts
    import { Ratelimit } from "https://cdn.skypack.dev/@upstash/ratelimit@latest";
    ```
  </Tab>
</Tabs>

### Add Ratelimit to Your Endpoint

Next step is to add Ratelimit to your endpoint. In the example below, you can see how to initialize a Ratelimit and use it:

```ts
import { Ratelimit } from "@upstash/ratelimit"; // for deno: see above
import { Redis } from "@upstash/redis";

// Create a new ratelimiter, that allows 10 requests per 10 seconds
const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(10, "10 s"),
  analytics: true,
  /**
   * Optional prefix for the keys used in redis. This is useful if you want to share a redis
   * instance with other applications and want to avoid key collisions. The default prefix is
   * "@upstash/ratelimit"
   */
  prefix: "@upstash/ratelimit",
});

// Use a constant string to limit all requests with a single ratelimit
// Or use a userID, apiKey or ip address for individual limits.
const identifier = "api";
const { success } = await ratelimit.limit(identifier);

if (!success) {
  return "Unable to process at this time";
}
doExpensiveCalculation();
return "Here you go!";
```

For Cloudflare Workers and Fastly Compute\@Edge, you can use the following imports:

```ts
import { Redis } from "@upstash/redis/cloudflare"; // for cloudflare workers and pages
import { Redis } from "@upstash/redis/fastly"; // for fastly compute@edge
```

In this example, we initialize a Ratelimit with an Upstash Redis. The Uptash Redis instance is created from the environment variables and passed to the Ratelimit instance. Then, we check the access rate using the `ratelimit.limit(identifier)` method. If the `success` field is true, we allow the expensive calculation to go through.

For more examples, see the [Examples](https://upstash.com/docs/redis/sdks/ratelimit-ts/overview#examples).

### Set Environment Variables

Final step is to update the environment variables so that the Ratelimit can communicate with the Upstash Redis. `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` environment variables must be set in order for the `Redis.fromEnv()` command to work. You can get the values of these environment variables from the [Upstash Console](https://console.upstash.com/redis) by navigating to the page of the Redis instance you created.

An alternative of using the `Redis.fromEnv()` method is to pass the variables yourself. This can be useful if you save these environment variables with a different name:

```ts
new Redis({
  url: "https://****.upstash.io",
  token: "********",
});
```

Here is how you can set the environment variables in different cases:

<Tabs>
  <Tab title="Vercel">
    Go to the "Settings" tab in your project. In the menu to the left, click "Environment Variables". Add `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` environment variables and their values.
  </Tab>

  <Tab title="Cloudflare Worker (Wrangler)">
    Run:

    ```
    npx wrangler secret put UPSTASH_REDIS_REST_URL
    ```

    When prompted, enter the value of `UPSTASH_REDIS_REST_URL` when prompted. Do the same for `UPSTASH_REDIS_REST_TOKEN`:

    ```
    npx wrangler secret put UPSTASH_REDIS_REST_TOKEN
    ```
  </Tab>

  <Tab title="Local Nextjs Project">
    Go to the `.env.local` file and add the environment variables:

    ```
    UPSTASH_REDIS_REST_URL=****
    UPSTASH_REDIS_REST_TOKEN=****
    ```
  </Tab>
</Tabs>

## Serverless Environments

When we use ratelimit in a serverless environment like CloudFlare Workers or Vercel Edge,
we need to be careful about making sure that the rate limiting operations complete correctly
before the runtime ends after returning the response.

This is important in two cases where we do some operations in the backgroung asynchronously after `limit` is called:

1.  Using MultiRegion: synchronize Redis instances in different regions
2.  Enabling analytics: send analytics to Redis

In these cases, we need to wait for these operations to finish before sending the response to the user. Otherwise, the runtime will end and we won't be able to complete our chores.

In order to wait for these operations to finish, use the `pending` promise:

```ts
const { pending } = await ratelimit.limit("id");
context.waitUntil(pending);
```

See more information on `context.waitUntil` at
[Cloudflare](https://developers.cloudflare.com/workers/runtime-apis/context/#waituntil)
and [Vercel](https://vercel.com/docs/functions/edge-middleware/middleware-api#waituntil).
You can also utilize [`waitUntil` from Vercel Functions API](https://vercel.com/docs/functions/functions-api-reference#waituntil).

## Customizing the Ratelimit Algorithm

There are several algorithms we can use for rate limiting. Explore the different rate-limiting algorithms available; how they work, their advantages and disadvantages in the [Algorithms page](https://upstash.com/docs/redis/sdks/ratelimit-ts/algorithms). You can learn about the **cost in terms of the number of commands**, by referring to the [Costs page](https://upstash.com/docs/redis/sdks/ratelimit-ts/costs).

## Methods

In our example, we only used the `limit` method. There are other methods we can use in the Ratelimit. These are:

*   `blockUntilReady`: Process a request only when the rate-limiting algorithm allows it.
*   `resetUsedTokens`: Reset the rate limiter state for some identifier.
*   `getRemaining`: Get the remaining tokens/requests left for some identifier.

To learn more about these methods, refer to the [Methods page](https://upstash.com/docs/redis/sdks/ratelimit-ts/methods).

## Features

To configure the your Ratelimit according to your needs, you can make use of several features:

<CardGroup cols={2}>
  <Card title="Caching" icon="shield-halved" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#caching">
    Handle blocked requests without having to call your Redis Database
  </Card>

  <Card title="Timeout" icon="stopwatch" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#timeout">
    If the Redis call of the ratelimit is not resolved in some timeframe, allow
    the request by default
  </Card>

  <Card title="Analytics & Dashboard" icon="magnifying-glass-chart" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#analytics-and-dashboard">
    Collect information on which identifiers made how many requests and how many
    were blocked
  </Card>

  <Card title="Traffic Protection" icon="lock" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/traffic-protection">
    Create a deny list to block requests based on user agents, countries, IP
    addresses an more
  </Card>

  <Card title="Custom Rates" icon="chart-simple" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#custom-rates">
    Consume different amounts of tokens in different requests (example: limiting
    based on request/response size)
  </Card>

  <Card title="Multi Region" icon="globe" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#multi-region">
    Utilize several Redis databases in different regions to serve users faster
  </Card>

  <Card title="Multiple Limits" icon="gears" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#using-multiple-limits">
    Use different limits for different kinds of requests (example: paid and free
    users)
  </Card>
</CardGroup>

For more information about the features, see the [Features page](https://upstash.com/docs/redis/sdks/ratelimit-ts/features).


# Configure Upstash Ratelimit Strapi Plugin



After setting up the plugin, it's possible to customize the ratelimiter algorithm and rates. You can also define different rate limits and rate limit algorithms for different routes.

## General Configurations

<ParamField path="enabled" type="boolean" default="true">
  Enable or disable the plugin.
</ParamField>

## Database Configurations

<ParamField path="token" type="string" required>
  The token to authenticate with the Upstash Redis REST API. You can find this
  credential on Upstash Console with the name `UPSTASH_REDIS_REST_TOKEN`
</ParamField>

<ParamField path="url" type="string" required>
  The URL for the Upstash Redis REST API. You can find this credential on
  Upstash Console with the name `UPSTASH_REDIS_REST_URL`
</ParamField>

<ParamField path="prefix" type="string" default="@strapi">
  The prefix for the rate limit keys. The plugin uses this prefix to store the
  rate limit data in Redis. <br />
  For example, if the prefix is `@strapi`, the key will be
  `@strapi:<method>:<route>:<identifier>`.
</ParamField>

<ParamField path="analytics" type="boolean" default="false">
  Enable analytics for the rate limit. When enabled, the plugin extra insights
  related to your ratelimits. You can use this data to analyze the rate limit
  usage on [Upstash Console](https://console.upstash.com/ratelimit).
</ParamField>

## Strategy

The plugin uses a strategy array to define the rate limits per route. Each strategy object has the following properties:

<ParamField path="methods" type="('GET' | 'POST' | 'DELETE' | 'PUT' | 'PATCH' |'ALL')[]" required>
  An array of HTTP methods to apply the rate limit. <br />
  For example, `["GET", "POST"]`
</ParamField>

<ParamField path="path" type="string" required>
  The path to apply the rate limit. You can use wildcards to match multiple
  routes. For example, `*` matches all routes. <br />
  Some examples: <br />

  *   `path: "/api/restaurants/:id"` <br />
  *   `path: "/api/restaurants"` <br />
</ParamField>

<ParamField path="identifierSource" type="string" required>
  The source to identifiy the user. Requests with the same identifier will be
  rate limited under the same limit. <br />
  Available sources are: <br />

  *   `ip`: The IP address of the user. <br />
  *   `header`: The value of a header key. You should pass the source in the `header.<HEADER_KEY>` format. <br />
      For example, `header.Authorization` will use the value of the `Authorization`
</ParamField>

<ParamField path="debug" type="string">
  Enable debug mode for the route. When enabled, the plugin logs the remaining
  limits and the block status for each request. <br />
</ParamField>

<ParamField path="limiter" type="object" required>
  The limiter configuration for the route. The limiter object has the following
  properties:

  <Card>
    <ParamField path="algorithm" type="'fixed-window' | 'sliding-window' | 'token-bucket'" required>
      The rate limit algorithm to use. For more information related to algorithms, see docs [**here**](https://upstash.com/docs/redis/sdks/ratelimit-ts/algorithms). <br />

      *   `fixed-window`: The fixed-window algorithm divides time into fixed intervals. Each interval has a set limit of allowed requests. When a new interval starts, the count resets. <br />
      *   `sliding-window`:
          The sliding-window algorithm uses a rolling time frame. It considers requests from the past X time units, continuously moving forward. This provides a smoother distribution of requests over time. <br />
      *   `token-bucket`: The token-bucket algorithm uses a bucket that fills with tokens at a steady rate. Each request consumes a token. If the bucket is empty, requests are denied. This allows for bursts of traffic while maintaining a long-term rate limit.<br />
    </ParamField>

    <ParamField path="tokens" type="number" required>
      The number of tokens allowed in the time window. <br />
    </ParamField>

    <ParamField path="window" type="string" required>
      The time window for the rate limit. Available units are `"ms" | "s" | "m" | "h" | "d"` <br />
      For example, `20s` means 20 seconds.
    </ParamField>

    <ParamField path="refillRate" type="number">
      The rate at which the bucket refills. **This property is only used for the token-bucket algorithm.** <br />
    </ParamField>
  </Card>
</ParamField>

## Examples

<CodeGroup>
  ```json Apply rate limit for all routes
  {
     "strapi-plugin-upstash-ratelimit":{
        "enabled":true,
        "resolve":"./src/plugins/strapi-plugin-upstash-ratelimit",
        "config":{
           "enabled":true,
           "token":"process.env.UPSTASH_REDIS_REST_TOKEN",
           "url":"process.env.UPSTASH_REDIS_REST_URL",
           "strategy":[
              {
                 "methods":[
                    "GET",
                    "POST"
                 ],
                 "path":"*",
                 "identifierSource":"header.Authorization",
                 "limiter":{
                    "algorithm":"fixed-window",
                    "tokens":10,
                    "window":"20s"
                 }
              }
           ],
           "prefix":"@strapi"
        }
     }
  }
  ```

  ```json Apply rate limit with IP
  {
    "strapi-plugin-upstash-ratelimit": {
      "enabled": true,
      "resolve": "./src/plugins/strapi-plugin-upstash-ratelimit",
      "config": {
        "enabled": true,
        "token": "process.env.UPSTASH_REDIS_REST_TOKEN",
        "url": "process.env.UPSTASH_REDIS_REST_URL",
        "strategy": [
          {
            "methods": ["GET", "POST"],
            "path": "*",
            "identifierSource": "ip",
            "limiter": {
              "algorithm": "fixed-window",
              "tokens": 10,
              "window": "20s"
            }
          }
        ],
        "prefix": "@strapi"
      }
    }
  }
  ```

  ```json Routes with different rate limit algorithms
  {
    "strapi-plugin-upstash-ratelimit": {
      "enabled": true,
      "resolve": "./src/plugins/strapi-plugin-upstash-ratelimit",
      "config": {
        "enabled": true,
        "token": "process.env.UPSTASH_REDIS_REST_TOKEN",
        "url": "process.env.UPSTASH_REDIS_REST_URL",
        "strategy": [
          {
            "methods": ["GET", "POST"],
            "path": "/api/restaurants/:id",
            "identifierSource": "header.x-author",
            "limiter": {
              "algorithm": "fixed-window",
              "tokens": 10,
              "window": "20s"
            }
          },
          {
            "methods": ["GET"],
            "path": "/api/restaurants",
            "identifierSource": "header.x-author",
            "limiter": {
              "algorithm": "tokenBucket",
              "tokens": 10,
              "window": "20s",
              "refillRate": 1
            }
          }
        ],
        "prefix": "@strapi"
      }
    }
  }
  ```
</CodeGroup>


# Upstash Ratelimit Strapi Integration



Strapi is an open-source, Node.js based, Headless CMS that saves developers a lot of development time, enabling them to build their application backends quickly by decreasing the lines of code necessary.

You can use Upstash's HTTP and Redis based [Ratelimit package](https://github.com/upstash/ratelimit-js) integration with Strapi to protect your APIs from abuse.

## Getting started

### Installation

<CodeGroup>
  ```bash npm
  npm install --save @upstash/strapi-plugin-upstash-ratelimit
  ```

  ```bash yarn
  yarn add @upstash/strapi-plugin-upstash-ratelimit
  ```
</CodeGroup>

### Create database

Create a new redis database on [Upstash Console](https://console.upstash.com/). See [related docs](https://upstash.com/docs/redis/overall/getstarted) for further info related to creating a database.

### Set up environment variables

Get the environment variables from [Upstash Console](https://console.upstash.com/), and set it to `.env` file as below:

```shell .env
UPSTASH_REDIS_REST_TOKEN="<YOUR_TOKEN>"
UPSTASH_REDIS_REST_URL="<YOUR_URL>"
```

### Configure the plugin

You can use

<CodeGroup>
  ```typescript /config/plugins.ts
  export default () => ({
    "strapi-plugin-upstash-ratelimit": {
      enabled: true,
      resolve: "./src/plugins/strapi-plugin-upstash-ratelimit",
      config: {
        enabled: true,
        token: process.env.UPSTASH_REDIS_REST_TOKEN,
        url: process.env.UPSTASH_REDIS_REST_URL,
        strategy: [
          {
            methods: ["GET", "POST"],
            path: "*",
            limiter: {
              algorithm: "fixed-window",
              tokens: 10,
              window: "20s",
            },
          },
        ],
        prefix: "@strapi",
      },
    },
  });
  ```

  ```javascript /config/plugins.js
  module.exports = () => ({
    "strapi-plugin-upstash-ratelimit": {
      enabled: true,
      resolve: "./src/plugins/strapi-plugin-upstash-ratelimit",
      config: {
        enabled: true,
        token: process.env.UPSTASH_REDIS_REST_TOKEN,
        url: process.env.UPSTASH_REDIS_REST_URL,
        strategy: [
          {
            methods: ["GET", "POST"],
            path: "*",
            limiter: {
              algorithm: "fixed-window",
              tokens: 10,
              window: "20s",
            },
          },
        ],
        prefix: "@strapi",
      },
    },
  });
  ```
</CodeGroup>


# Methods



This page contains information on what methods are available in Ratelimit and how they can be used. For information on the
cost of these operations in term of number of Redis commands, refer to the [Costs page](https://upstash.com/docs/redis/sdks/ratelimit-ts/costs).

## `limit`

The `limit` method is the heart of the Ratelimit algorithm.

```ts
ratelimit.limit(
  identifier: string,
  req?: {
    geo?: Geo;
    rate?: number,
    ip?: string,
    userAgent?: string,
    country?: string
  },
): Promise<RatelimitResponse>
```

It receives an identifier to rate limit. Additionally, it can be passed a `req` parameter which can contain either a
`geo` or a `rate` field. `geo` field is passed to the analytics but is not in use currently. The `rate` field determines
the amount of tokens/requests to subtract from the state of the algorithm with regards to the provided identifier.

The `limit` method returns some more metadata that might be useful to you:

````ts
export type RatelimitResponse = {
  /**
   * Whether the request may pass(true) or exceeded the limit(false)
   */
  success: boolean;
  /**
   * Maximum number of requests allowed within a window.
   */
  limit: number;
  /**
   * How many requests the user has left within the current window.
   */
  remaining: number;
  /**
   * Unix timestamp in milliseconds when the limits are reset.
   */
  reset: number;

  /**
   * For the MultiRegion setup we do some synchronizing in the background, after returning the current limit.
   * Or when analytics is enabled, we send the analytics asynchronously after returning the limit.
   * In most case you can simply ignore this.
   *
   * On Vercel Edge or Cloudflare workers, you need to explicitly handle the pending Promise like this:
   *
   * ```ts
   * const { pending } = await ratelimit.limit("id")
   * context.waitUntil(pending)
   * ```
   *
   * See `waitUntil` documentation in
   * [Cloudflare](https://developers.cloudflare.com/workers/runtime-apis/handlers/fetch/#contextwaituntil)
   * and [Vercel](https://vercel.com/docs/functions/edge-middleware/middleware-api#waituntil)
   * for more details.
   * ```
   */
  pending: Promise<unknown>;

  /**
   * Reason behind the result in `success` field.
   * - Is set to "timeout" when request times out
   * - Is set to "cacheBlock" when an identifier is blocked through cache without calling redis because it was
   *    rate limited previously.
   * - Is set to "denyList" when identifier or one of ip/user-agent/country parameters is in deny list. To enable
   *    deny list, see `enableProtection` parameter. To edit the deny list, see the Upstash Ratelimit Dashboard
   *    at https://console.upstash.com/ratelimit.
   * - Is set to undefined if rate limit check had to use Redis. This happens in cases when `success` field in
   *    the response is true. It can also happen the first time sucecss is false.
   */
  reason?: RatelimitResponseType;

  /**
   * The value which was in the deny list if reason: "denyList"
   */
  deniedValue?: string;
};
````

## `blockUntilReady`

In case you don't want to reject a request immediately but wait until it can be
processed, we also provide

```ts
ratelimit.blockUntilReady(
  identifier: string,
  timeout: number
): Promise<RatelimitResponse>
```

It is very similar to the `limit` method and takes an identifier and returns the
same response. However if the current limit has already been exceeded, it will
automatically wait until the next window starts and will try again. Setting the
timeout parameter (in milliseconds) will cause the returned Promise to resolve
in a finite amount of time.

```ts
// Create a new ratelimiter, that allows 10 requests per 10 seconds
const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(10, "10 s"),
  analytics: true,
});

// `blockUntilReady` returns a promise that resolves as soon as the request is allowed to be processed, or after 30 seconds
const { success } = await ratelimit.blockUntilReady("id", 30_000);

if (!success) {
  return "Unable to process, even after 30 seconds";
}
doExpensiveCalculation();
return "Here you go!";
```

<Warning>
  In **Cloudflare**, `blockUntilReady` will not work as intended due to
  `Date.now()` not behaving the same as in Node environments.

  <br />

  <br />

  **For more information, check**: <br />
  [https://developers.cloudflare.com/workers/runtime-apis/web-standards](https://developers.cloudflare.com/workers/runtime-apis/web-standards)
</Warning>

## `resetUsedTokens`

This method resets the state of the algorithm with respect to some identifier:

```ts
ratelimit.resetUsedTokens(identifier: string): Promise<void>
```

## `getRemaining`

This method returns the remaining tokens/requests available for some identifier:

```ts
ratelimit.getRemaining(identifier: string): Promise<{
  remaining: number;
  reset: number;
}>
```

`remaining` is the remaining tokens. `reset` is the reset timestamp.


# Overview



# Upstash Rate Limit

[![npm (scoped)](https://img.shields.io/npm/v/@upstash/ratelimit)](https://www.npmjs.com/package/ratelimit)

It is the only connectionless (HTTP based) rate limiting library and designed
for:

*   Serverless functions (AWS Lambda, Vercel ...)
*   Cloudflare Workers
*   Vercel Edge
*   Fastly Compute\@Edge
*   Next.js, Jamstack ...
*   Client side web/mobile applications
*   WebAssembly
*   and other environments where HTTP is preferred over TCP.

## Quick Links:

*   [Github Repository](https://github.com/upstash/ratelimit)
*   [Getting Started](https://upstash.com/docs/redis/sdks/ratelimit-ts/gettingstarted)
*   [Costs](https://upstash.com/docs/redis/sdks/ratelimit-ts/costs)

## Features

<CardGroup cols={2}>
  <Card title="Caching" icon="shield-halved" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#caching">
    Handle blocked requests without having to call your Redis Database
  </Card>

  <Card title="Timeout" icon="stopwatch" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#timeout">
    If the Redis call of the ratelimit is not resolved in some timeframe, allow
    the request by default
  </Card>

  <Card title="Analytics & Dashboard" icon="magnifying-glass-chart" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#analytics-and-dashboard">
    Collect information on which identifiers made how many requests and how many
    were blocked
  </Card>

  <Card title="Traffic Protection" icon="lock" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/traffic-protection">
    Create a deny list to block requests based on user agents, countries, IP
    addresses and more
  </Card>

  <Card title="Custom Rates" icon="chart-simple" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#custom-rates">
    Consume different amounts of tokens in different requests (example: limiting
    based on request/response size)
  </Card>

  <Card title="Multi Region" icon="globe" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#multi-region">
    Utilize several Redis databases in different regions to serve users faster
  </Card>

  <Card title="Multiple Limits" icon="gears" href="https://upstash.com/docs/redis/sdks/ratelimit-ts/features#using-multiple-limits">
    Use different limits for different kinds of requests (example: paid and free
    users)
  </Card>
</CardGroup>

For more information about the features, see the [Features tab](https://upstash.com/docs/redis/sdks/ratelimit-ts/features).

## Examples

<CardGroup cols={2}>
  <Card title="Nextjs" href="https://github.com/upstash/ratelimit/tree/main/examples/nextjs">
    Rate limit an API in a Nextjs project
  </Card>

  <Card title="Nextjs with Middleware" href="https://github.com/upstash/ratelimit/tree/main/examples/nextjs-middleware">
    Rate limit an API with a Middleware in a Nextjs project
  </Card>

  <Card title="Vercel Edge" href="https://github.com/upstash/ratelimit/tree/main/examples/vercel-edge">
    Rate limit an Vercel Edge Function
  </Card>

  <Card title="Enabling Protection" href="https://github.com/upstash/ratelimit/tree/main/examples/enable-protection">
    Use Deny Lists to Protect Your Website
  </Card>

  <Card title="Cloudflare Pages" href="https://github.com/upstash/ratelimit/tree/main/examples/cloudflare-pages">
    Rate limit access to your Cloudflare Pages app
  </Card>

  <Card title="Cloudflare Workers" href="https://github.com/upstash/ratelimit/tree/main/examples/cloudflare-workers">
    Rate limit access to your Cloudflare Workers
  </Card>

  <Card title="Remix" href="https://github.com/upstash/ratelimit/tree/main/examples/remix">
    Rate limit access to a Remix App
  </Card>

  <Card title="Rate limit using Vercel KV" href="https://github.com/upstash/ratelimit/tree/main/examples/with-vercel-kv">
    Rate limit a Nexjs app using Vercel KV
  </Card>

  <Card title="Deno App" href="https://github.com/upstash/ratelimit/tree/main/examples/deno">
    Rate limit your deno app
  </Card>

  <Card title="Rate limit your Chatbot" href="https://upstash.com/blog/degree-guru#rate-limiting">
    Limiting requests to a Chatbot endpoint which streams LLM outputs
  </Card>
</CardGroup>


# Traffic Protection



### Deny List

Imagine that you want to block requests from certain countries or from some
user agents. In this case, you can make use of deny lists introduced in
ratelimit version 1.2.1.

Deny lists allow you to block based on IP addresses, user agents, countries
and [identifiers](https://upstash.com/docs/redis/sdks/ratelimit-ts/methods#limit).

To enable checking the deny list in your Ratelimit client, simply pass
`enableProtection` as `true`:

```tsx
const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(10, "10 s"),
  enableProtection: true
  analytics: true,
});
```

When `limit` is called, the client will check whether any of these values
are in the deny list and block the request if so.

```tsx
const { success, pending, reason, deniedValue } = ratelimit.limit("userId", {
  ip: "ip-address",
  userAgent: "user-agent",
  country: "country",
});

await pending; // await pending if you have analytics enabled

console.log(success, reason, deniedValue);
// prints: false, "denyList", "ip-address"
```

If a request fails because a value was in deny list, `reason` field will
be `"denyList"`. `deniedValue` will contain the value in the deny list.
See [limit method](https://upstash.com/docs/redis/sdks/ratelimit-ts/methods#limit)
for more detailts.

Client also keeps a **cache** of denied values. When a value is found
in the deny list, the client stores this value in the cache. If this value
is encountered in the following requests, it is **denied without calling
Redis at all**. Items are stored in the cache for a minute. This means that if
you add a new value to the deny list, it will immediately take affect but when you
remove a value, it can take up to a minute for clients to start
accepting the value. This can significantly reduce the number of calls to Redis.

Contents of the deny lists are managed from the [Ratelimit Dashboard](https://upstash.com/docs/redis/sdks/ratelimit-ts/features#dashboard).
You can use the dashboard to add items to the deny list or remove them.
If you have analytics enabled, you can also view the number of denied
requests per country/ip address/user agent/identifier on the dashboard.

![denylist.png](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/ratelimit/denylist.png)

Note that we look for exact match when checking a value to see if it's in
the deny lists. **Pattern matching is not supported**.

### Auto IP Deny List

The Auto IP Deny List feature enables the automatic blocking of IP addresses
identified as malicious through open-source IP deny lists. This functionality
uses the [ipsum repository on GitHub](https://github.com/stamparm/ipsum),
which aggregates data from over 30 different deny lists.

To enable protection, set the enableProtection parameter to true. Once activated,
your SDK will automatically block IP addresses by leveraging the IP deny lists
when you provide the request IPs in the limit method.

```ts
const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(10, "10 s"),
  enableProtection: true,
});
```

The IP deny list is updated daily at 2 AM UTC. Upon expiration, the
first call to limit after 2 AM UTC will trigger an update, downloading
the latest IPs from GitHub and refreshing the list in your Redis
instance. The update process occurs asynchronously, allowing you to
return the result to the user while the IP list updates in the
background. To ensure the update completes successfully, utilize the
pending field:

```ts
const { success, pending } = await ratelimit.limit(
  content,
  {ip: "ip-address"}
);

await pending;
```

For more information on effectively using pending, refer to the
["Asynchronous synchronization between databases" section](https://upstash.com/docs/redis/sdks/ratelimit-ts/features#asynchronous-synchronization-between-databases).

Blocked IPs will be listed in the "Denied" section of the Ratelimit
dashboard, providing a clear overview of the addresses that have
been automatically blocked.

If you prefer to disable the Auto IP Deny List feature while still
using the deny lists, you can do so via the [Ratelimit dashboard on
the Upstash Console](https://console.upstash.com/ratelimit).


# Advanced



## Disable automatic serialization

Your data is (de)serialized as `json` by default. This works for most use cases
but you can disable it if you want:

```ts
const redis = new Redis({
  // ...
  automaticDeserialization: false,
});

// or
const redis = Redis.fromEnv({
  automaticDeserialization: false,
});
```

This probably breaks quite a few types, but it's a first step in that direction.
Please report bugs and broken types
[here](https://github.com/upstash/upstash-redis/issues/49).

## Keep-Alive

`@upstash/redis` optimizes performance by reusing connections wherever possible, reducing latency.
This is achieved by keeping the client in memory instead of reinitializing it with each new function invocation.
As a result, when a hot lambda function receives a new request, it uses the already initialized client, allowing for the reuse of existing connections to Upstash.

<Tip>This functionality is enabled by default.</Tip>


# ECHO



Returns a message back to you. Useful for debugging the connection.

## Arguments

<ParamField body="message" type="string" required>
  A message to send to the server.
</ParamField>

## Response

<ResponseField type="string" required>
  The same message you sent.
</ResponseField>

<RequestExample>
  ```ts Example
  const response = await redis.echo("hello world");
  console.log(response); // "hello world"
  ```
</RequestExample>


# PING

Send a ping to the server and get a response if the server is alive.

## Arguments

No arguments

## Response

<ResponseField type="string" required>
  `PONG`
</ResponseField>

<RequestExample>
  ```ts Example
  const response = await redis.ping();
  console.log(response); // "PONG"
  ```
</RequestExample>


# BITCOUNT

Count the number of set bits.

The `BITCOUNT` command in Redis is used to count the number of set bits (bits with a value of 1) in a range of bytes within a key that is stored as a binary string. It is primarily used for bit-level operations on binary data stored in Redis.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="start" type="integer">
  Specify the range of bytes within the binary string to count the set bits. If not provided, it counts set bits in the entire string.

  Either specify both `start` and `end` or neither.
</ParamField>

<ParamField body="end" type="integer">
  Specify the range of bytes within the binary string to count the set bits. If not provided, it counts set bits in the entire string.

  Either specify both `start` and `end` or neither.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of set bits in the specified range.
</ResponseField>

<RequestExample>
  ```ts Example
  const bits = await redis.bitcount(key);
  ```

  ```ts With Range
  const bits = await redis.bitcount(key, 5, 10);
  ```
</RequestExample>


# BITOP

Perform bitwise operations between strings.

The `BITOP` command in Redis is used to perform bitwise operations on multiple keys (or Redis strings) and store the result in a destination key. It is primarily used for performing logical AND, OR, XOR, and NOT operations on binary data stored in Redis.

## Arguments

<ParamField body="operation" type="AND | OR | XOR | NOT" required>
  Specifies the type of bitwise operation to perform, which can be one of the following: `AND`, `OR`, `XOR`, or `NOT`.
</ParamField>

<ParamField body="destinationKey" type="string" required>
  The key to store the result of the operation in.
</ParamField>

<ParamField body="sourceKeys" type="...string[]" required>
  One or more keys to perform the operation on.
</ParamField>

## Response

<ResponseField type="integer" required>
  The size of the string stored in the destination key.
</ResponseField>

<RequestExample>
  ```ts Example
   await redis.bitop("AND", "destKey", "sourceKey1", "sourceKey2");
  ```
</RequestExample>


# BITPOS

Find the position of the first set or clear bit (bit with a value of 1 or 0) in a Redis string key.

## Arguments

<ParamField body="key" type="string" required>
  The key to search in.
</ParamField>

<ParamField body="bit" type="0 | 1" required>
  The key to store the result of the operation in.
</ParamField>

<ParamField body="start" type="number">
  The index to start searching at.
</ParamField>

<ParamField body="end" type="number">
  The index to stop searching at.
</ParamField>

## Response

<ResponseField type="integer" required>
  The index of the first occurrence of the bit in the string.
</ResponseField>

<RequestExample>
  ```ts Example
   await redis.bitpos("key", 1);
  ```

  ```ts With Range
   await redis.bitpos("key", 1, 5, 20);
  ```
</RequestExample>


# GETBIT

Retrieve a single bit.

## Arguments

<ParamField body="key" type="string" required>
  The key of the bitset
</ParamField>

<ParamField body="offset" type="integer" required>
  Specify the offset at which to get the bit.
</ParamField>

## Response

<ResponseField type="integer" required>
  The bit value stored at offset.
</ResponseField>

<RequestExample>
  ```ts Example
  const bit = await redis.getbit(key, 4);
  ```
</RequestExample>


# SETBIT

Set a single bit in a string.

## Arguments

<ParamField body="key" type="string" required>
  The key of the bitset
</ParamField>

<ParamField body="offset" type="integer" required>
  Specify the offset at which to set the bit.
</ParamField>

<ParamField body="value" type="0 | 1" required>
  The bit to set
</ParamField>

## Response

<ResponseField type="0 | 1" required>
  The original bit value stored at offset.
</ResponseField>

<RequestExample>
  ```ts Example
  const originalBit = await redis.setbit(key, 4, 1);
  ```
</RequestExample>


# DEL

Removes the specified keys. A key is ignored if it does not exist.

## Arguments

<ParamField body="keys" type="...string[]" required>
  One or more keys to remove.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of keys that were removed.
</ResponseField>

<RequestExample>
  ```ts Basic
  await redis.del("key1", "key2");
  ```

  ```ts Array
  // in case you have an array of keys
  const keys = ["key1", "key2"];
  await redis.del(...keys)

  ```
</RequestExample>


# EXISTS

Check if a key exists.

## Arguments

<ParamField body="keys" type="...string[]" required>
  One or more keys to check.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of keys that exist
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("key1", "value1")
  await redis.set("key2", "value2")
  const keys = await redis.exists("key1", "key2", "key3");
  console.log(keys) // 2
  ```
</RequestExample>


# EXPIRE

Sets a timeout on key. The key will automatically be deleted.

## Arguments

<ParamField body="key" type="string" required>
  The key to set the timeout on.
</ParamField>

<ParamField body="seconds" type="integer">
  How many seconds until the key should be deleted.
</ParamField>

## Response

<ResponseField type="integer" required>
  `1` if the timeout was set, `0` otherwise
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("mykey", "Hello");
  await redis.expire("mykey", 10);
  ```
</RequestExample>


# EXPIREAT

Sets a timeout on key. The key will automatically be deleted.

## Arguments

<ParamField body="key" type="string" required>
  The key to set the timeout on.
</ParamField>

<ParamField body="unix" type="integer">
  A unix timestamp in seconds at which point the key will expire.
</ParamField>

## Response

<ResponseField type="integer" required>
  `1` if the timeout was set, `0` otherwise
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("mykey", "Hello");
  const tenSecondsFromNow = Math.floor(Date.now() / 1000) + 10;
  await redis.expireat("mykey", tenSecondsFromNow);
  ```
</RequestExample>


# KEYS

Returns all keys matching pattern.

<Warning>
  This command may block the DB for a long time, depending on its size. We advice against using it in production. Use [SCAN](/redis/sdks/ts/commands/generic/scan) instead.
</Warning>

## Arguments

<ParamField body="match" type="string" required>
  A glob-style pattern. Use `*` to match all keys.
</ParamField>

## Response

<ResponseField type="string[]" required>
  Array of keys matching the pattern.
</ResponseField>

<RequestExample>
  ```ts Example
  const keys = await redis.keys("prefix*");
  ```

  ```ts Match All
  const keys = await redis.keys("*");
  ```
</RequestExample>


# PERSIST

Remove any timeout set on the key.

## Arguments

<ParamField body="key" type="string" required>
  The key to persist
</ParamField>

## Response

<ResponseField type="integer" required>
  `1` if the timeout was removed, `0` if `key` does not exist or does not have an associated timeout.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.persist(key);
  ```
</RequestExample>


# PEXPIRE

Sets a timeout on key. After the timeout has expired, the key will automatically be deleted.

## Arguments

<ParamField body="key" type="string" required>
  The key to expire.
</ParamField>

<ParamField body="milliseconds" type="integer">
  The number of milliseconds until the key expires.
</ParamField>

## Response

<ResponseField type="integer" required>
  `1` if the timeout was applied, `0` if `key` does not exist.
</ResponseField>

<RequestExample>
  ```ts Example
   await redis.pexpire(key, 60_000); // 1 minute
  ```
</RequestExample>


# PEXPIREAT

Sets a timeout on key. After the timeout has expired, the key will automatically be deleted.

## Arguments

<ParamField body="key" type="string" required>
  The key to expire.
</ParamField>

<ParamField body="unixmilli" type="integer">
  The unix timestamp in milliseconds at which the key will expire.
</ParamField>

## Response

<ResponseField type="integer" required>
  `1` if the timeout was applied, `0` if `key` does not exist.
</ResponseField>

<RequestExample>
  ```ts Example
  const 10MinutesFromNow = Date.now() + 10 * 60 * 1000;
   await redis.pexpireat(key, 10MinutesFromNow);
  ```
</RequestExample>


# PTTL

Return the expiration in milliseconds of a key.

## Arguments

<ParamField body="key" type="string" required>
  The key
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of milliseconds until this expires, negative if the key does not exist or does not have an expiration set.
</ResponseField>

<RequestExample>
  ```ts Example
  const millis = await redis.pttl(key);
  ```
</RequestExample>


# RANDOMKEY

Returns a random key from database

## Arguments

No arguments

## Response

<ResponseField type="string" required>
  A random key from database, or `null` when database is empty.
</ResponseField>

<RequestExample>
  ```ts Example
  const key = await redis.randomkey();
  ```
</RequestExample>


# RENAME

Rename a key

## Arguments

<ParamField body="source" type="string" required>
  The original key.
</ParamField>

<ParamField body="destination" type="string" required>
  A new name for the key.
</ParamField>

## Response

<ResponseField type="string" required>
  `OK`
</ResponseField>

<RequestExample>
  ```ts Example
   await redis.rename("old", "new");
  ```
</RequestExample>


# RENAMENX

Rename a key if it does not already exist.

## Arguments

<ParamField body="source" type="string" required>
  The original key.
</ParamField>

<ParamField body="destination" type="string" required>
  A new name for the key.
</ParamField>

## Response

<ResponseField type="0 | 1" required>
  `1` if key was renamed, `0` if key was not renamed.
</ResponseField>

<RequestExample>
  ```ts Example
  const renamed = await redis.rename("old", "new");
  ```
</RequestExample>


# SCAN

Scan the database for keys.

## Arguments

<ParamField body="cursor" type="number">
  The cursor, use `0` in the beginning and then use the returned cursor for subsequent calls.
</ParamField>

<ParamField body="options" type="Object">
  <ParamField body="match" type="string">
    Glob-style pattern to filter by field names.
  </ParamField>

  <ParamField body="count" type="number">
    Number of fields to return per call.
  </ParamField>

  <ParamField body="type" type="string">
    Filter by type.
    For example `string`, `hash`, `set`, `zset`, `list`, `stream`.
  </ParamField>
</ParamField>

## Response

<ResponseField type="[number, string[]]" required>
  The new cursor and the keys.
  If the new cursor is `0` the iteration is complete.
</ResponseField>

<RequestExample>
  ```ts Basic
  const [newCursor, keys] = await redis.scan(0, {match: "*"});
  ```
</RequestExample>


# TOUCH

Alters the last access time of one or more keys

## Arguments

<ParamField body="keys" type="...string[]" required>
  One or more keys.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of keys that were touched.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.touch("key1", "key2", "key3");
  ```
</RequestExample>


# TTL

Return the expiration in seconds of a key.

## Arguments

<ParamField body="key" type="string" required>
  The key
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of seconds until this expires, negative if the key does not exist or does not have an expiration set.
</ResponseField>

<RequestExample>
  ```ts Example
  const seconds = await redis.pttl(key);
  ```
</RequestExample>


# TYPE

Get the type of a key.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

## Response

<ResponseField type="string" required>
  The type of the key.

  One of `string` | `list` | `set` | `zset` | `hash` | `none`
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("key", "value");
  const t = await redis.type("key");
  console.log(t) // "string"
  ```
</RequestExample>


# UNLINK

Removes the specified keys. A key is ignored if it does not exist.

## Arguments

<ParamField body="keys" type="...string[]" required>
  One or more keys to unlink.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of keys that were unlinked.
</ResponseField>

<RequestExample>
  ```ts Basic
  await redis.unlink("key1", "key2");
  ```

  ```ts Array
  // in case you have an array of keys
  const keys = ["key1", "key2"];
  await redis.unlink(...keys)

  ```
</RequestExample>


# HDEL

Deletes one or more hash fields.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="fields" required>
  One or more fields to delete.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of fields that were removed from the hash.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hdel(key, 'field1', 'field2');
  // returns 5
  ```
</RequestExample>


# HEXISTS

Checks if a field exists in a hash.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="field" type="string" required>
  The field to check.
</ParamField>

## Response

<ResponseField type="integer" required>
  `1` if the hash contains `field`. `0` if the hash does not contain `field`, or `key` does not exist.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", "field", "value");
  const exists = await redis.hexists("key", "field");
  console.log(exists); // 1

  ```
</RequestExample>


# HGET

Retrieves the value of a hash field.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="field" type="string" required>
  The field to get.
</ParamField>

## Response

<ResponseField type="TValue | null" required>
  The value of the field, or `null`, when field is not present in the hash or key does not exist.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", {field: "value"});
  const field = await redis.hget("key", "field");
  console.log(field); // "value"
  ```
</RequestExample>


# HGETALL

Retrieves all fields from a hash.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

## Response

<ResponseField type="TValue | null" required>
  An object with all fields in the hash.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", {
    field1: "value1",
    field2: "value2",
    });
  const hash = await redis.hgetall("key");
  console.log(hash); // { field1: "value1", field2: "value2" }
  ```
</RequestExample>


# HINCRBY

Increments the value of a hash field by a given amount

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

<ParamField body="field" type="string" required>
  The field to increment
</ParamField>

<ParamField body="increment" type="integer" required>
  How much to increment the field by. Can be negative to subtract.
</ParamField>

## Response

<ResponseField type="integer" required>
  The new value of the field after the increment.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", {
    field: 20,
    });
  const after = await redis.hincrby("key", "field", 2);
  console.log(after); // 22
  ```
</RequestExample>


# HINCRBYFLOAT

Increments the value of a hash field by a given float value.

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

<ParamField body="field" type="string" required>
  The field to increment
</ParamField>

<ParamField body="increment" type="float" required>
  How much to increment the field by. Can be negative to subtract.
</ParamField>

## Response

<ResponseField type="float" required>
  The new value of the field after the increment.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", {
    field: 20,
    });
  const after = await redis.hincrby("key", "field", 2.5);
  console.log(after); // 22.5
  ```
</RequestExample>


# HKEYS

Return all field names in the hash stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

## Response

<ResponseField type="string[]" required>
  The field names of the hash
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    });
  const fields = await redis.hkeys("key");
  console.log(fields); // ["id", "username"]
  ```
</RequestExample>


# HLEN

Returns the number of fields contained in the hash stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

## Response

<ResponseField type="integer" required>
  How many fields are in the hash.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    });
  const fields = await redis.hlen("key");
  console.log(fields); // 2
  ```
</RequestExample>


# HMGET

Return the requested fields and their values.

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

<ParamField body="fields" type="...string[]" required>
  One or more fields to get.
</ParamField>

## Response

<ResponseField type="Record<string, unknown>" required>
  An object containing the fields and their values.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    name: "andreas"
    });
  const fields = await redis.hmget("key", "username", "name");
  console.log(fields); // { username: "chronark", name: "andreas" }
  ```
</RequestExample>


# HRANDFIELD

Return a random field from a hash

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

<ParamField body="count" type="integer">
  Optionally return more than one field.
</ParamField>

<ParamField body="withValues" type="boolean">
  Return the values of the fields as well.
</ParamField>

## Response

<ResponseField type="Record<string, unknown>" required>
  An object containing the fields and their values.
</ResponseField>

<RequestExample>
  ```ts Basic
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    name: "andreas"
   });
  const randomField = await redis.hrandfield("key");
  console.log(randomField); // one of "id", "username" or  "name"
  ```

  ```ts Multiple Fields
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    name: "andreas",
  });
  const randomFields = await redis.hrandfield("key", 2);
  console.log(randomFields); // ["id", "username"] or any other combination
  ```

  ```ts With Values
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    name: "andreas",
  });
  const randomFields = await redis.hrandfield("key", 2, true);
  console.log(randomFields); // { id: "1", username: "chronark" } or any other combination
  ```
</RequestExample>


# HSCAN

Scan a hash for fields.

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

<ParamField body="cursor" type="number">
  The cursor, use `0` in the beginning and then use the returned cursor for subsequent calls.
</ParamField>

<ParamField body="options" type="Object">
  <ParamField body="match" type="string">
    Glob-style pattern to filter by field names.
  </ParamField>

  <ParamField body="count" type="number">
    Number of fields to return per call.
  </ParamField>
</ParamField>

## Response

<ResponseField type="[number, string[]]" required>
  The new cursor and the fields array in format `[field, value, field, value]`.
  If the new cursor is `0` the iteration is complete.
</ResponseField>

<RequestExample>
  ```ts Basic
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    name: "andreas"
   });
  const [newCursor, fields] = await redis.hscan("key", 0);
  console.log(newCursor); // likely `0` since this is a very small hash
  console.log(fields); // ["id", 1, "username", "chronark", "name", "andreas"]
  ```

  ```ts Match
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    name: "andreas",
  });
  const [newCursor, fields] = await redis.hscan("key", 0, { match: "user*" });
  console.log(newCursor); // likely `0` since this is a very small hash
  console.log(fields); // ["username", "chronark"]
  ```

  ```ts Count
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    name: "andreas",
  });
  const [newCursor, fields] = await redis.hscan("key", 0, { count: 2 });
  console.log(fields); // ["id", 1, "name", "andreas", "username", "chronark"]
  ```
</RequestExample>


# HSET

Write one or more fields to a hash.

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

<ParamField body="fields" type="{ [fieldName]: TValue }" required>
  An object of fields and their values.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of fields that were added.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", {
    id: 1,
    username: "chronark",
    name: "andreas"
    });
  ```
</RequestExample>


# HSETNX

Write a field to a hash but only if the field does not exist.

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

<ParamField body="field" type="string" required>
  The name of the field.
</ParamField>

<ParamField body="value" type="TValue" required>
  Any value, if it's not a string it will be serialized to JSON.
</ParamField>

## Response

<ResponseField type="integer" required>
  `1` if the field was set, `0` if it already existed.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hsetnx("key", "id", 1)
  ```
</RequestExample>


# HSTRLEN

Returns the string length of a value in a hash.

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

<ParamField body="field" type="string" required>
  The name of the field.
</ParamField>

## Response

<ResponseField type="integer" required>
  `0` if the hash or field does not exist. Otherwise the length of the string.
</ResponseField>

<RequestExample>
  ```ts Example
  const length = await redis.hstrlen("key", "field")
  ```
</RequestExample>


# HVALS

Returns all values in the hash stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key of the hash.
</ParamField>

## Response

<ResponseField type="unknown[]" required>
  All values in the hash, or an empty list when key does not exist.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.hset("key", {
    field1: "Hello",
    field2: "World",
  })
  const values = await redis.hvals("key")
  console.log(values) // ["Hello", "World"]
  ```
</RequestExample>


# JSON.ARRAPPEND

Append values to the array at path in the JSON document at key.

<Tip>
  To specify a string as an array value to append, wrap the quoted string with an additional set of single quotes. Example: '"silver"'.
</Tip>

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

<ParamField body="value" type="...TValue[]" required>
  One or more values to append to the array.
</ParamField>

## Response

<ResponseField type="integer" required>
  The length of the array after the appending.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.json.arrappend("key", "$.path.to.array", "a");
  ```
</RequestExample>


# JSON.ARRINDEX

Search for the first occurrence of a JSON value in an array.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

<ParamField body="value" type="TValue" required>
  The value to search for.
</ParamField>

<ParamField body="start" type="integer" default={0}>
  The start index.
</ParamField>

<ParamField body="stop" type="integer" default={0}>
  The stop index.
</ParamField>

## Response

<ResponseField type="integer" required>
  The index of the first occurrence of the value in the array, or -1 if not found.
</ResponseField>

<RequestExample>
  ```ts Example
  const index = await redis.json.arrindex("key", "$.path.to.array", "a");
  ```
</RequestExample>


# JSON.ARRINSERT

Insert the json values into the array at path before the index (shifts to the right).

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

<ParamField body="index" type="integer" required>
  The index where to insert the values.
</ParamField>

<ParamField body="values" type="...TValue[]" required>
  One or more values to append to the array.
</ParamField>

## Response

<ResponseField type="integer" required>
  The length of the array after the insertion.
</ResponseField>

<RequestExample>
  ```ts Example
  const length = await redis.json.arrinsert("key", "$.path.to.array", 2, "a", "b");
  ```
</RequestExample>


# JSON.ARRLEN

Report the length of the JSON array at `path` in `key`.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

## Response

<ResponseField type="integer" required>
  The length of the array.
</ResponseField>

<RequestExample>
  ```ts Example
  const length = await redis.json.arrlen("key", "$.path.to.array");
  ```
</RequestExample>


# JSON.ARRPOP

Remove and return an element from the index in the array. By default the last element from an array is popped.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string">
  The path of the array.
</ParamField>

<ParamField body="index" type="number" default={-1}>
  The index of the element to pop.
</ParamField>

## Response

<ResponseField type="TValue | null" required>
  The popped element or null if the array is empty.
</ResponseField>

<RequestExample>
  ```ts Example
  const element = await redis.json("key", "$.path.to.array");
  ```

  ```ts First
  const firstElement = await redis.json("key", "$.path.to.array", 0);
  ```
</RequestExample>


# JSON.ARRTRIM

Trim an array so that it contains only the specified inclusive range of elements.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

<ParamField body="start" type="integer" required>
  The start index of the range.
</ParamField>

<ParamField body="stop" type="integer" required>
  The stop index of the range.
</ParamField>

## Response

<ResponseField type="integer" required>
  The length of the array after the trimming.
</ResponseField>

<RequestExample>
  ```ts Example
  const length = await redis.json.arrtrim("key", "$.path.to.array", 2, 10);
  ```
</RequestExample>


# JSON.CLEAR

Clear container values (arrays/objects) and set numeric values to 0.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path to clear
</ParamField>

## Response

<ResponseField type="integer" required>
  The length of the array after the appending.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.json.clear("key");
  ```

  ```ts With path
  await redis.json.clear("key", "$.my.key");
  ```
</RequestExample>


# JSON.DEL

Delete a key from a JSON document.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path to delete
</ParamField>

<ParamField body="value" type="...TValue[]" required>
  One or more values to append to the array.
</ParamField>

## Response

<ResponseField type="integer" required>
  How many paths were deleted.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.json.del("key", "$.path.to.value");
  ```
</RequestExample>


# JSON.FORGET

Delete a key from a JSON document.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path to forget.
</ParamField>

<ParamField body="value" type="...TValue[]" required>
  One or more values to append to the array.
</ParamField>

## Response

<ResponseField type="integer" required>
  How many paths were deleted.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.json.forget("key", "$.path.to.value");
  ```
</RequestExample>


# JSON.GET

Get a single value from a JSON document.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="options">
  <Expandable>
    <ParamField body="indent" type="string">
      Sets the indentation string for nested levels.
    </ParamField>

    <ParamField body="newline" type="string">
      Sets the string that's printed at the end of each line.
    </ParamField>

    <ParamField body="space" type="string">
      Sets the string that is put between a key and a value.
    </ParamField>
  </Expandable>
</ParamField>

<ParamField body="paths" type="...string[]" default="$">
  One or more paths to retrieve from the JSON document.
</ParamField>

## Response

<ResponseField type="TValue | null" required>
  The value at the specified path or `null` if the path does not exist.
</ResponseField>

<RequestExample>
  ```ts Example
  const value = await redis.json.get("key", "$.path.to.somewhere");
  ```

  ```ts With Options
  const value = await redis.json.get("key", {
      indent: "  ",
      newline: "\n",
      space: " ",
  }, "$.path.to.somewhere");
  ```
</RequestExample>


# JSON.MGET

Get the same path from multiple JSON documents.

## Arguments

<ParamField body="keys" type="...string[]" required>
  One or more keys of JSON documents.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path to get from the JSON document.
</ParamField>

## Response

<ResponseField type="TValue[]" required>
  The values at the specified path or `null` if the path does not exist.
</ResponseField>

<RequestExample>
  ```ts Example
  const values = await redis.json.mget(["key1", "key2"],  "$.path.to.somewhere");
  ```
</RequestExample>


# JSON.NUMINCRBY

Increment the number value stored at `path` by number.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

<ParamField body="increment" type="number" required>
  The number to increment by.
</ParamField>

## Response

<ResponseField type="integer" required>
  The new value after incrementing
</ResponseField>

<RequestExample>
  ```ts Example
  const newValue = await redis.json.numincrby("key", "$.path.to.array", 2);
  ```
</RequestExample>


# JSON.NUMMULTBY

Multiply the number value stored at `path` by number.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

<ParamField body="multiply" type="number" required>
  The number to multiply by.
</ParamField>

## Response

<ResponseField type="integer" required>
  The new value after multiplying
</ResponseField>

<RequestExample>
  ```ts Example
  const newValue = await redis.json.nummultby("key", "$.path.to.array", 2);
  ```
</RequestExample>


# JSON.OBJKEYS

Return the keys in the object that`s referenced by path.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

## Response

<ResponseField type="string[]" required>
  The keys of the object at the path.
</ResponseField>

<RequestExample>
  ```ts Example
  const keys = await redis.json.objkeys("key", "$.path");
  ```
</RequestExample>


# JSON.OBJLEN

Report the number of keys in the JSON object at `path` in `key`.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

## Response

<ResponseField type="integer[]" required>
  The number of in the objects.
</ResponseField>

<RequestExample>
  ```ts Example
  const lengths = await redis.json.objlen("key", "$.path");
  ```
</RequestExample>


# JSON.SET

Set the JSON value at path in key.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

<ParamField body="value" type="TValue" required>
  The value to set.
</ParamField>

## Response

<ResponseField type="OK" required>
  `OK`
</ResponseField>

<RequestExample>
  ```ts Example
  Set the JSON value at path in key.
  redis.json.set(key, "$.path", value);
  ```

  ```ts NX
  const value = ...
  redis.json.set(key, "$.path", value, { nx:true });
  ```

  ```ts XX
  const value = ...
  redis.json.set(key, "$.path", value, { xx:true });
  ```
</RequestExample>


# JSON.STRAPPEND

Append the json-string values to the string at path.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the value.
</ParamField>

<ParamField body="value" type="string" required>
  The value to append to the existing string.
</ParamField>

## Response

<ResponseField type="integer" required>
  The length of the array after the appending.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.json.strappend("key", "$.path.to.array", "abc");
  ```
</RequestExample>


# JSON.STRLEN

Report the length of the JSON String at path in key

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the array.
</ParamField>

## Response

<ResponseField type="array" required>
  JSON.STRLEN returns by recursive descent an array of integer replies for each path, the array's length, or nil, if the matching JSON value is not a string.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.json("key", "$.path.to.array", "a");
  ```
</RequestExample>


# JSON.TOGGLE

Toggle a boolean value stored at `path`.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the boolean.
</ParamField>

## Response

<ResponseField type="boolean" required>
  The new value of the boolean.
</ResponseField>

<RequestExample>
  ```ts Example
  const bool = await redis.json.toggle("key", "$.path.to.array");
  ```
</RequestExample>


# JSON.TYPE

Report the type of JSON value at `path`.

## Arguments

<ParamField body="key" type="string" required>
  The key of the json entry.
</ParamField>

<ParamField body="path" type="string" default="$">
  The path of the value.
</ParamField>

## Response

<ResponseField type="string | null" required>
  The type of the value at `path` or `null` if the value does not exist.
</ResponseField>

<RequestExample>
  ```ts Example
  const myType = await redis.json.type("key", "$.path.to.value");
  ```
</RequestExample>


# LINDEX

Returns the element at index index in the list stored at key.

The index is zero-based, so 0 means the first element, 1 the second element and so on. Negative indices can be used to designate elements starting at the tail of the list.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="index" type="number" required>
  The index of the element to return, zero-based.
</ParamField>

## Response

<ResponseField type="TValue | null" required>
  The value of the element at index index in the list. If the index is out of range, `null` is returned.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.rpush("key", "a", "b", "c");
  const element = await redis.lindex("key", 0);
  console.log(element); // "a"
  ```
</RequestExample>


# LINSERT

Insert an element before or after another element in a list

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="direction" type="before | after" required>
  Whether to insert the element before or after pivot.
</ParamField>

<ParamField body="pivot" type="TValue" required>
  The element to insert before or after.
</ParamField>

<ParamField body="value" type="TValue" required>
  The element to insert.
</ParamField>

## Response

<ResponseField type="integer" required>
  The list length after insertion, `0` when the list doesn't exist or `-1` when pivot was not found.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.rpush("key", "a", "b", "c");
  await redis.linsert("key", "before", "b", "x");
  ```
</RequestExample>


# LLEN

Returns the length of the list stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

## Response

<ResponseField type="number" required>
  The length of the list at key.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.rpush("key", "a", "b", "c");
  const length = await redis.llen("key");
  console.log(length); // 3
  ```
</RequestExample>


# LMOVE

Move an element from one list to another.

## Arguments

<ParamField body="source" type="string" required>
  The key of the source list.
</ParamField>

<ParamField body="destination" type="string" required>
  The key of the destination list.
</ParamField>

<ParamField body="from" type="&#x22;left&#x22; | &#x22;right&#x22;" required>
  The side of the source list from which the element was popped.
</ParamField>

<ParamField body="to" type="&#x22;left&#x22; | &#x22;right&#x22;" required>
  The side of the destination list to which the element was pushed.
</ParamField>

## Response

<ResponseField type="TValue" required>
  The element that was moved.
</ResponseField>

<RequestExample>
  ```ts Example
   await redis.rpush("source", "a", "b", "c"); 
   const element = await redis.move("source", "destination", "left", "left");  
  ```
</RequestExample>


# LPOP

Remove and return the first element(s) of a list

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="count" type="integer">
  How many elements to pop. If not specified, a single element is popped.
</ParamField>

## Response

<ResponseField type="TValue | TValue[] | null" required>
  The popped element(s). If `count` was specified, an array of elements is
  returned, otherwise a single element is returned. If the list is empty, `null`
  is returned.
</ResponseField>

<RequestExample>
  ```ts Single 
  await redis.rpush("key", "a", "b", "c"); 
  const element = await redis.lpop("key");
  console.log(element); // "a"
  ```

  ```ts Multiple 
  await redis.rpush("key", "a", "b", "c"); 
  const element = await redis.lpop("key", 2);
  console.log(element); // ["a", "b"]
  ```
</RequestExample>


# LPOS

Returns the index of matching elements inside a list.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="element" type="unknown" required>
  The element to match.
</ParamField>

<ParamField body="opts">
  <ParamField body="rank" type="number">
    The rank of the element to match. If specified, the element at the given
    rank is matched instead of the first element.
  </ParamField>

  <ParamField body="count" type="number">
    The maximum number of elements to match. If specified, an array of elements
    is returned instead of a single element.
  </ParamField>

  <ParamField body="maxLen" type="number">
    Limit the number of comparisons to perform.
  </ParamField>
</ParamField>

## Response

<ResponseField type="number | number[]" required>
  The index of the matching element or an array of indexes if `opts.count` is
  specified.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.rpush("key", "a", "b", "c"); 
  const index = await redis.lpos("key", "b");
  console.log(index); // 1
  ```

  ```ts With Rank 
  await redis.rpush("key", "a", "b", "c", "b"); 
  const index = await redis.lpos("key", "b", { rank: 2 });
  console.log(index); // 3
  ```

  ```ts With Count
  await redis.rpush("key", "a", "b", "b");
  const positions = await redis.lpos("key", "b", { count: 2 });
  console.log(positions); // [1, 2]
  ```
</RequestExample>


# LPUSH

Push an element at the head of the list.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="elements" type="...TValue[]" required>
  One or more elements to push at the head of the list.
</ParamField>

## Response

<ResponseField type="number" required>
  The length of the list after the push operation.
</ResponseField>

<RequestExample>
  ```ts Example 
  const length1 = await redis.lpush("key", "a", "b", "c"); 
  console.log(length1); // 3
  const length2 = await redis.lpush("key", "d"); 
  console.log(length2); // 4
  ```
</RequestExample>


# LPUSHX

Push an element at the head of the list only if the list exists.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="elements" type="...TValue[]" required>
  One or more elements to push at the head of the list.
</ParamField>

## Response

<ResponseField type="number" required>
  The length of the list after the push operation.

  `0` if the list did not exist and thus no element was pushed.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.lpush("key", "a", "b", "c"); 
  const length = await redis.lpushx("key", "d"); 
  console.log(length); // 4
  ```

  ```ts Without existing list 
  const length = await redis.lpushx("key", "a"); 
  console.log(length); // 0
  ```
</RequestExample>


# LRANGE

Returns the specified elements of the list stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="start" type="integer" required>
  The starting index of the range to return.

  Use negative numbers to specify offsets starting at the end of the list.
</ParamField>

<ParamField body="end" type="integer" required>
  The ending index of the range to return.

  Use negative numbers to specify offsets starting at the end of the list.
</ParamField>

## Response

<ResponseField type="TValue[]" required>
  The list of elements in the specified range.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.lpush("key", "a", "b", "c"); 
  const elements = await redis.lrange("key", 1, 2); 
  console.log(elements) // ["b", "c"]
  ```
</RequestExample>


# LREM

Remove the first `count` occurrences of an element from a list.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="count" type="number" required>
  How many occurrences of the element to remove.
</ParamField>

<ParamField body="element" type="TValue" required>
  The element to remove
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of elements removed.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.lpush("key", "a", "a", "b", "b", "c"); 
  const removed = await redis.lrem("key", 4, "b"); 
  console.log(removed) // 2
  ```
</RequestExample>


# LSET

Set a value at a specific index.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="index" type="number" required>
  At which index to set the value.
</ParamField>

<ParamField body="data" type="TValue" required>
  The value to set.
</ParamField>

## Response

<ResponseField type="OK" required>
  `OK`
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.lpush("key", "a", "b", "c"); 
  await redis.lset("key", 1, "d"); 

  // list is now ["a", "d", "c"]
  ```
</RequestExample>


# LTRIM

Trim a list to the specified range

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="start" type="number" required>
  The index of the first element to keep.
</ParamField>

<ParamField body="end" type="TValue" required>
  The index of the first element to keep.
</ParamField>

## Response

<ResponseField type="OK" required>
  `OK`
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.lpush("key", "a", "b", "c", "d"); 
  await redis.ltrim("key", 1, 2); 
  // the list is now ["b", "c"]
  ```
</RequestExample>


# RPOP

Remove and return the last element(s) of a list

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="count" type="integer">
  How many elements to pop. If not specified, a single element is popped.
</ParamField>

## Response

<ResponseField type="TValue | TValue[] | null" required>
  The popped element(s). If `count` was specified, an array of elements is
  returned, otherwise a single element is returned. If the list is empty, `null`
  is returned.
</ResponseField>

<RequestExample>
  ```ts Single 
  await redis.rpush("key", "a", "b", "c"); 
  const element = await redis.rpop("key");
  console.log(element); // "c"
  ```

  ```ts Multiple 
  await redis.rpush("key", "a", "b", "c"); 
  const element = await redis.rpop("key", 2);
  console.log(element); // ["c", "b"]
  ```
</RequestExample>


# RPUSH

Push an element at the end of the list.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="elements" type="...TValue[]" required>
  One or more elements to push at the end of the list.
</ParamField>

## Response

<ResponseField type="number" required>
  The length of the list after the push operation.
</ResponseField>

<RequestExample>
  ```ts Example 
  const length1 = await redis.rpush("key", "a", "b", "c"); 
  console.log(length1); // 3
  const length2 = await redis.rpush("key", "d"); 
  console.log(length2); // 4
  ```
</RequestExample>


# RPUSHX

Push an element at the end of the list only if the list exists.

## Arguments

<ParamField body="key" type="string" required>
  The key of the list.
</ParamField>

<ParamField body="elements" type="...TValue[]" required>
  One or more elements to push at the end of the list.
</ParamField>

## Response

<ResponseField type="number" required>
  The length of the list after the push operation.

  `0` if the list did not exist and thus no element was pushed.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.lpush("key", "a", "b", "c"); 
  const length = await redis.rpushx("key", "d"); 
  console.log(length); // 4
  ```

  ```ts Without existing list 
  const length = await redis.rpushx("key", "a"); 
  console.log(length); // 0
  ```
</RequestExample>


# Overview

Available Commands in @upstash/redis

<AccordionGroup>
  <Accordion title="Auth">
    <CardGroup cols={3}>
      <Card title="ECHO" href="/redis/sdks/ts/commands/auth/echo">
        Echo the given string.
      </Card>

      <Card title="PING" href="/redis/sdks/ts/commands/auth/ping">
        Ping the server.
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Bitmap">
    <CardGroup cols={3}>
      <Card title="BITCOUNT" href="/redis/sdks/ts/commands/bitmap/bitcount">
        Count set bits in a string.
      </Card>

      <Card title="BITOP" href="/redis/sdks/ts/commands/bitmap/bitop">
        Perform bitwise operations between strings.
      </Card>

      <Card title="BITPOS" href="/redis/sdks/ts/commands/bitmap/bitpos">
        Find first bit set or clear in a string.
      </Card>

      <Card title="GETBIT" href="/redis/sdks/ts/commands/bitmap/getbit">
        Returns the bit value at offset in the string value stored at key.
      </Card>

      <Card title="SETBIT" href="/redis/sdks/ts/commands/bitmap/setbit">
        Sets or clears the bit at offset in the string value stored at key.
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Generic">
    <CardGroup cols={3}>
      <Card title="DEL" href="/redis/sdks/ts/commands/generic/del">
        Delete one or multiple keys.
      </Card>

      <Card title="EXISTS" href="/redis/sdks/ts/commands/generic/exists">
        Determine if a key exists.
      </Card>

      <Card title="EXPIRE" href="/redis/sdks/ts/commands/generic/expire">
        Set a key's time to live in seconds.
      </Card>

      <Card title="EXPIREAT" href="/redis/sdks/ts/commands/generic/expireat">
        Set the expiration for a key as a UNIX timestamp.
      </Card>

      <Card title="KEYS" href="/redis/sdks/ts/commands/generic/keys">
        Find all keys matching the given pattern.
      </Card>

      <Card title="PERSIST" href="/redis/sdks/ts/commands/generic/persist">
        Remove the expiration from a key.
      </Card>

      <Card title="PEXPIRE" href="/redis/sdks/ts/commands/generic/pexpire">
        Set a key's time to live in milliseconds.
      </Card>

      <Card title="PEXPIREAT" href="/redis/sdks/ts/commands/generic/pexpireat">
        Set the expiration for a key as a UNIX timestamp specified in milliseconds.
      </Card>

      <Card title="PTTL" href="/redis/sdks/ts/commands/generic/pttl">
        Get the time to live for a key in milliseconds.
      </Card>

      <Card title="RANDOMKEY" href="/redis/sdks/ts/commands/generic/randomkey">
        Return a random key from the keyspace.
      </Card>

      <Card title="RENAME" href="/redis/sdks/ts/commands/generic/rename">
        Rename a key.
      </Card>

      <Card title="RENAMENX" href="/redis/sdks/ts/commands/generic/renamenx">
        Rename a key, only if the new key does not exist.
      </Card>

      {/* <Card title="RESTORE" href="/redis/sdks/ts/commands/generic/restore">
          Create a key using the provided serialized value, previously obtained using DUMP.
      </Card> */}

      <Card title="SCAN" href="/redis/sdks/ts/commands/generic/scan">
        Incrementally iterate the keys space.
      </Card>

      {/* <Card title="SORT" href="/redis/sdks/ts/commands/generic/sort">
          Sort the elements in a list, set or sorted set.
      </Card> */}

      <Card title="TOUCH" href="/redis/sdks/ts/commands/generic/touch">
        Alters the last access time of a key(s). Returns the number of existing keys specified.
      </Card>

      <Card title="TTL" href="/redis/sdks/ts/commands/generic/ttl">
        Get the time to live for a key.
      </Card>

      <Card title="TYPE" href="/redis/sdks/ts/commands/generic/type">
        Determine the type stored at key.
      </Card>

      <Card title="UNLINK" href="/redis/sdks/ts/commands/generic/unlink">
        Delete one or more keys.
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Hash">
    <CardGroup cols={3}>
      <Card title="HDEL" href="/redis/sdks/ts/commands/hash/hdel" />

      <Card title="HEXISTS" href="/redis/sdks/ts/commands/hash/hexists" />

      <Card title="HGET" href="/redis/sdks/ts/commands/hash/hget" />

      <Card title="HGETALL" href="/redis/sdks/ts/commands/hash/hgetall" />

      <Card title="HINCRBY" href="/redis/sdks/ts/commands/hash/hincrby" />

      <Card title="HINCRBYFLOAT" href="/redis/sdks/ts/commands/hash/hincrbyfloat" />

      <Card title="HKEYS" href="/redis/sdks/ts/commands/hash/hkeys" />

      <Card title="HLEN" href="/redis/sdks/ts/commands/hash/hlen" />

      <Card title="HMGET" href="/redis/sdks/ts/commands/hash/hmget" />

      <Card title="HRANDFIELD" href="/redis/sdks/ts/commands/hash/hrandfield" />

      <Card title="HSCAN" href="/redis/sdks/ts/commands/hash/hscan" />

      <Card title="HSET" href="/redis/sdks/ts/commands/hash/hset" />

      <Card title="HSETNX" href="/redis/sdks/ts/commands/hash/hsetnx" />

      <Card title="HSTRLEN" href="/redis/sdks/ts/commands/hash/hstrlen" />

      <Card title="HVALS" href="/redis/sdks/ts/commands/hash/hvals" />
    </CardGroup>
  </Accordion>

  <Accordion title="JSON">
    <CardGroup cols={3}>
      <Card title="ARRAPPEND" href="/redis/sdks/ts/commands/json/arrappend" />

      <Card title="ARRINDEX" href="/redis/sdks/ts/commands/json/arrindex" />

      <Card title="ARRINSERT" href="/redis/sdks/ts/commands/json/arrinsert" />

      <Card title="ARRLEN" href="/redis/sdks/ts/commands/json/arrlen" />

      <Card title="ARRPOP" href="/redis/sdks/ts/commands/json/arrpop" />

      <Card title="ARRTRIM" href="/redis/sdks/ts/commands/json/arrtrim" />

      <Card title="CLEAR" href="/redis/sdks/ts/commands/json/clear" />

      <Card title="DEL" href="/redis/sdks/ts/commands/json/del" />

      <Card title="FORGET" href="/redis/sdks/ts/commands/json/forget" />

      <Card title="GET" href="/redis/sdks/ts/commands/json/get" />

      <Card title="MGET" href="/redis/sdks/ts/commands/json/mget" />

      <Card title="NUMINCRBY" href="/redis/sdks/ts/commands/json/numincrby" />

      <Card title="NUMMULTBY" href="/redis/sdks/ts/commands/json/nummultby" />

      <Card title="OBJKEYS" href="/redis/sdks/ts/commands/json/objkeys" />

      <Card title="OBJLEN" href="/redis/sdks/ts/commands/json/objlen" />

      <Card title="SET" href="/redis/sdks/ts/commands/json/set" />

      <Card title="STRAPPEND" href="/redis/sdks/ts/commands/json/strappend" />

      <Card title="STRLEN" href="/redis/sdks/ts/commands/json/strlen" />

      <Card title="TOGGLE" href="/redis/sdks/ts/commands/json/toggle" />

      <Card title="TYPE" href="/redis/sdks/ts/commands/json/type" />
    </CardGroup>
  </Accordion>

  <Accordion title="List">
    <CardGroup cols={3}>
      <Card title="LINDEX" href="/redis/sdks/ts/commands/list/lindex" />

      <Card title="LINSERT" href="/redis/sdks/ts/commands/list/linsert" />

      <Card title="LLEN" href="/redis/sdks/ts/commands/list/llen" />

      <Card title="LMOVE" href="/redis/sdks/ts/commands/list/lmove" />

      <Card title="LPOP" href="/redis/sdks/ts/commands/list/lpop" />

      <Card title="LPOS" href="/redis/sdks/ts/commands/list/lpos" />

      <Card title="LPUSH" href="/redis/sdks/ts/commands/list/lpush" />

      <Card title="LPUSHX" href="/redis/sdks/ts/commands/list/lpushx" />

      <Card title="LRANGE" href="/redis/sdks/ts/commands/list/lrange" />

      <Card title="LREM" href="/redis/sdks/ts/commands/list/lrem" />

      <Card title="LSET" href="/redis/sdks/ts/commands/list/lset" />

      <Card title="LTRIM" href="/redis/sdks/ts/commands/list/ltrim" />

      <Card title="RPOP" href="/redis/sdks/ts/commands/list/rpop" />

      <Card title="RPUSH" href="/redis/sdks/ts/commands/list/rpush" />

      <Card title="RPUSHX" href="/redis/sdks/ts/commands/list/rpushx" />
    </CardGroup>
  </Accordion>

  <Accordion title="PubSub">
    <CardGroup cols={3}>
      <Card title="PUBLISH" href="/redis/sdks/ts/commands/pubsub/publish">
        Publish messages to many clients
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Scripts">
    <CardGroup cols={3}>
      <Card title="EVAL" href="/redis/sdks/ts/commands/scripts/eval" />

      <Card title="EVELSHA" href="/redis/sdks/ts/commands/scripts/evalsha" />

      <Card title="SCRIPT EXISTS" href="/redis/sdks/ts/commands/scripts/script_exists" />

      <Card title="SCRIPT FLUSH" href="/redis/sdks/ts/commands/scripts/script_flush" />

      <Card title="SCRIPT LOAD" href="/redis/sdks/ts/commands/scripts/script_load" />
    </CardGroup>
  </Accordion>

  <Accordion title="Server">
    <CardGroup cols={3}>
      <Card title="DBSIZE" href="/redis/sdks/ts/commands/server/dbsize" />

      <Card title="FLUSHALL" href="/redis/sdks/ts/commands/server/flushall" />

      <Card title="FLUSHDB" href="/redis/sdks/ts/commands/server/flushdb" />
    </CardGroup>
  </Accordion>

  <Accordion title="Set">
    <CardGroup cols={3}>
      <Card title="SADD" href="/redis/sdks/ts/commands/set/sadd" />

      <Card title="SCARD" href="/redis/sdks/ts/commands/set/scard" />

      <Card title="SDIFF" href="/redis/sdks/ts/commands/set/sdiff" />

      <Card title="SDIFFSTORE" href="/redis/sdks/ts/commands/set/sdiffstore" />

      <Card title="SINTER" href="/redis/sdks/ts/commands/set/sinter" />

      <Card title="SINTERSTORE" href="/redis/sdks/ts/commands/set/sinterstore" />

      <Card title="SISMEMBER" href="/redis/sdks/ts/commands/set/sismember" />

      <Card title="SMEMBERS" href="/redis/sdks/ts/commands/set/smembers" />

      <Card title="SMISMEMBER" href="/redis/sdks/ts/commands/set/smismember" />

      <Card title="SMOVE" href="/redis/sdks/ts/commands/set/smove" />

      <Card title="SPOP" href="/redis/sdks/ts/commands/set/spop" />

      <Card title="SRANDMEMBER" href="/redis/sdks/ts/commands/set/srandmember" />

      <Card title="SREM" href="/redis/sdks/ts/commands/set/srem" />

      <Card title="SSCAN" href="/redis/sdks/ts/commands/set/sscan" />

      <Card title="SUNION" href="/redis/sdks/ts/commands/set/sunion" />

      <Card title="SUNIONSTORE" href="/redis/sdks/ts/commands/set/sunionstore" />
    </CardGroup>
  </Accordion>

  <Accordion title="Sorted Set">
    <CardGroup cols={3}>
      <Card title="ZADD" href="/redis/sdks/ts/commands/zset/zadd" />

      <Card title="ZCARD" href="/redis/sdks/ts/commands/zset/zcard" />

      <Card title="ZCOUNT" href="/redis/sdks/ts/commands/zset/zcount" />

      <Card title="ZDIFFSTORE" href="/redis/sdks/ts/commands/zset/zdiffstore" />

      <Card title="ZINCRBY" href="/redis/sdks/ts/commands/zset/zincrby" />

      <Card title="ZINTERSTORE" href="/redis/sdks/ts/commands/zset/zinterstore" />

      <Card title="ZLEXCOUNT" href="/redis/sdks/ts/commands/zset/zlexcount" />

      <Card title="ZMSCORE" href="/redis/sdks/ts/commands/zset/zmscore" />

      <Card title="ZPOPMAX" href="/redis/sdks/ts/commands/zset/zpopmax" />

      <Card title="ZPOPMIN" href="/redis/sdks/ts/commands/zset/zpopmin" />

      <Card title="ZRANGE" href="/redis/sdks/ts/commands/zset/zrange" />

      <Card title="ZRANK" href="/redis/sdks/ts/commands/zset/zrank" />

      <Card title="ZREM" href="/redis/sdks/ts/commands/zset/zrem" />

      <Card title="ZREMRANGEBYLEX" href="/redis/sdks/ts/commands/zset/zremrangebylex" />

      <Card title="ZREMRANGEBYRANK" href="/redis/sdks/ts/commands/zset/zremrangebyrank" />

      <Card title="ZREMRANGEBYSCORE" href="/redis/sdks/ts/commands/zset/zremrangebyscore" />

      <Card title="ZREVRANK" href="/redis/sdks/ts/commands/zset/zrevrank" />

      <Card title="ZSCAN" href="/redis/sdks/ts/commands/zset/zscan" />

      <Card title="ZSCORE" href="/redis/sdks/ts/commands/zset/zscore" />

      <Card title="ZUNIONSTORE" href="/redis/sdks/ts/commands/zset/zunionstore" />
    </CardGroup>
  </Accordion>

  <Accordion title="Stream">
    <CardGroup cols={3}>
      <Card title="XADD" href="/redis/sdks/ts/commands/stream/xadd">
        Appends a new entry to a stream.
      </Card>

      <Card title="XRANGE" href="/redis/sdks/ts/commands/stream/xrange">
        Return a range of elements in a stream, with IDs matching the specified IDs interval.
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="String">
    <CardGroup cols={3}>
      <Card title="APPEND" href="/redis/sdks/ts/commands/string/append">
        Append a value to a string stored at key.
      </Card>

      <Card title="DECR" href="/redis/sdks/ts/commands/string/decr">
        Decrement the integer value of a key by one.
      </Card>

      <Card title="DECRBY" href="/redis/sdks/ts/commands/string/decrby">
        Decrement the integer value of a key by the given number.
      </Card>

      <Card title="GET" href="/redis/sdks/ts/commands/string/get">
        Get the value of a key.
      </Card>

      <Card title="GETDEL" href="/redis/sdks/ts/commands/string/getdel">
        Get the value of a key and delete the key.
      </Card>

      <Card title="GETRANGE" href="/redis/sdks/ts/commands/string/getrange">
        Get a substring of the string stored at a key.
      </Card>

      <Card title="GETSET" href="/redis/sdks/ts/commands/string/getset">
        Set the string value of a key and return its old value.
      </Card>

      <Card title="INCR" href="/redis/sdks/ts/commands/string/incr">
        Increment the integer value of a key by one.
      </Card>

      <Card title="INCRBY" href="/redis/sdks/ts/commands/string/incrby">
        Increment the integer value of a key by the given amount.
      </Card>

      <Card title="INCRBYFLOAT" href="/redis/sdks/ts/commands/string/incrbyfloat">
        Increment the float value of a key by the given amount.
      </Card>

      <Card title="MGET" href="/redis/sdks/ts/commands/string/mget">
        Get the values of all the given keys.
      </Card>

      <Card title="MSET" href="/redis/sdks/ts/commands/string/mset">
        Set multiple keys to multiple values.
      </Card>

      <Card title="MSETNX" href="/redis/sdks/ts/commands/string/msetnx">
        Set multiple keys to multiple values, only if none of the keys exist.
      </Card>

      <Card title="PSETEX" href="/redis/sdks/ts/commands/string/psetex">
        Set the value and expiration in milliseconds of a key.
      </Card>

      <Card title="SET" href="/redis/sdks/ts/commands/string/set">
        Set the string value of a key.
      </Card>

      <Card title="SETEX" href="/redis/sdks/ts/commands/string/setex">
        Set the value and expiration of a key.
      </Card>

      <Card title="SETNX" href="/redis/sdks/ts/commands/string/setnx">
        Set the value of a key, only if the key does not exist.
      </Card>

      <Card title="SETRANGE" href="/redis/sdks/ts/commands/string/setrange">
        Overwrite part of a string at key starting at the specified offset.
      </Card>

      <Card title="STRLEN" href="/redis/sdks/ts/commands/string/strlen">
        Get the length of the value stored in a key.
      </Card>

      <Card title="SUBSTR" href="/redis/sdks/ts/commands/string/substr">
        Get a substring of the string stored at a key.
      </Card>
    </CardGroup>
  </Accordion>

  <Accordion title="Transactions">
    <Card title="TRANSACTION" href="/redis/sdks/ts/commands/transaction">
      Run multiple commands in a transaction.
    </Card>
  </Accordion>
</AccordionGroup>


# PUBLISH

Publish a message to a channel

## Arguments

<ParamField body="channel" type="string" required>
  The channel to publish to.
</ParamField>

<ParamField body="message" type="TMessage">
  The message to publish.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of clients who received the message.
</ResponseField>

<RequestExample>
  ```ts Example
  const listeners = await redis.publish("my-topic", "my-message");
  ```
</RequestExample>


# EVAL

Evaluate a Lua script server side.

## Arguments

<ParamField body="script" type="string" required>
  The lua script to run.
</ParamField>

<ParamField body="keys" type="string[]" required>
  All of the keys accessed in the script
</ParamField>

<ParamField body="args" type="unknown[]" required>
  All of the arguments you passed to the script
</ParamField>

## Response

<ResponseField type="any" required>
  The result of the script.
</ResponseField>

<RequestExample>
  ```ts Example
  const script = `
      return ARGV[1]
  `
  const result = await redis.eval(script, [], ["hello"]);
  console.log(result) // "hello"

  ```
</RequestExample>


# EVALSHA

Evaluate a cached Lua script server side.

`EVALSHA` is like `EVAL` but instead of sending the script over the wire every time, you reference the script by its SHA1 hash. This is useful for caching scripts on the server side.

## Arguments

<ParamField body="sha" type="string" required>
  The sha1 hash of the script.
</ParamField>

<ParamField body="keys" type="string[]" required>
  All of the keys accessed in the script
</ParamField>

<ParamField body="args" type="unknown[]" required>
  All of the arguments you passed to the script
</ParamField>

## Response

<ResponseField type="?" required>
  The result of the script.
</ResponseField>

<RequestExample>
  ```ts Example

  const result = await redis.evalsha("fb67a0c03b48ddbf8b4c9b011e779563bdbc28cb", [], ["hello"]);
  console.log(result) // "hello"

  ```
</RequestExample>


# SCRIPT EXISTS

Check if scripts exist in the script cache.

## Arguments

<ParamField body="hashes" type="string[]" required>
  The sha1 of the scripts to check.
</ParamField>

## Response

<ResponseField type="number[]" required>
  An array of numbers. `1` if the script exists, otherwise `0`.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.scriptExists("<sha1>", "<sha2>")

  // Returns 1 
  // [1, 0]
  ```
</RequestExample>


# SCRIPT FLUSH

Removes all scripts from the script cache.

## Arguments

<ParamField body="options" type="Object">
  <ParamField body="async" type="boolean">
    Performs the flush asynchronously.
  </ParamField>

  <ParamField body="sync" type="boolean">
    Performs the flush synchronously.
  </ParamField>
</ParamField>

<RequestExample>
  ```ts Example
  await redis.scriptFlush();
  ```

  ```ts With options
  await redis.scriptFlush({
    async: true,
  });
  ```
</RequestExample>


# SCRIPT LOAD

Load the specified Lua script into the script cache.

## Arguments

<ParamField body="script" type="string" required>
  The script to load.
</ParamField>

## Response

<ResponseField type="string" required>
  The sha1 of the script.
</ResponseField>

<RequestExample>
  ```ts Example
  const script = `
    local value = redis.call('GET', KEYS[1])
    return value
  `;
  const sha1 = await redis.scriptLoad(script);

  ```
</RequestExample>


# DBSIZE

Count the number of keys in the database.

## Arguments

This command has no arguments

## Response

<ResponseField type="integer" required>
  The number of keys in the database
</ResponseField>

<RequestExample>
  ```ts Example
  const keys = await redis.dbsize();
  console.log(keys) // 20
  ```
</RequestExample>


# FLUSHALL



<Warning>
  Deletes all keys permanently. Use with caution!
</Warning>

## Arguments

<ParamField body="async" type="boolean">
  Whether to perform the operation asynchronously.
  Defaults to synchronous.
</ParamField>

<RequestExample>
  ```ts Sync
  await redis.flushall();
  ```

  ```ts Async
  await redis.flushall({async: true})
  ```
</RequestExample>


# FLUSHDB



<Warning>
  Deletes all keys permanently. Use with caution!
</Warning>

## Arguments

<ParamField body="async" type="boolean">
  Whether to perform the operation asynchronously.
  Defaults to synchronous.
</ParamField>

<RequestExample>
  ```ts Sync
  await redis.flushdb();
  ```

  ```ts Async
  await redis.flushdb({async: true})
  ```
</RequestExample>


# SADD

Adds one or more members to a set.

## Arguments

<ParamField body="key" type="string" required>
  The key of the set.
</ParamField>

<ParamField body="members" type="...TValue[]" required>
  One or more members to add to the set.
</ParamField>

## Response

<ResponseField type="number" required>
  The number of elements that were added to the set, not including all the elements already present in the set.
</ResponseField>

<RequestExample>
  ```ts Example 
  // 3
  await redis.sadd("key", "a", "b", "c"); 

  // 0
  await redis.sadd("key", "a", "b"); 
  ```
</RequestExample>


# SCARD

Return how many members are in a set

## Arguments

<ParamField body="key" type="string" required>
  The key of the set.
</ParamField>

## Response

<ResponseField type="number" required>
  How many members are in the set.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("key", "a", "b", "c"); 
  const cardinality = await redis.scard("key");
  console.log(cardinality); // 3
  ```
</RequestExample>


# SDIFF

Return the difference between sets

## Arguments

<ParamField body="keys" type="...string[]" required>
  The keys of the sets to perform the difference operation on.
</ParamField>

## Response

<ResponseField type="TValue[]" required>
  The members of the resulting set.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set1", "a", "b", "c"); 
  await redis.sadd("set2", "c", "d", "e"); 
  const diff =  await redis.sdiff("set1", "set2");
  console.log(diff); // ["a", "b"]
  ```
</RequestExample>


# SDIFFSTORE

Write the difference between sets to a new set

## Arguments

<ParamField body="destination" type="string" required>
  The key of the set to store the resulting set in.
</ParamField>

<ParamField body="keys" type="...string[]" required>
  The keys of the sets to perform the difference operation on.
</ParamField>

## Response

<ResponseField type="TValue[]" required>
  The members of the resulting set.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set1", "a", "b", "c"); 
  await redis.sadd("set2", "c", "d", "e"); 
  await redis.sdiff("dest", "set1", "set2");
  console.log(diff); // ["a", "b"]
  ```
</RequestExample>


# SINTER

Return the intersection between sets

## Arguments

<ParamField body="keys" type="...string[]" required>
  The keys of the sets to perform the intersection operation on.
</ParamField>

## Response

<ResponseField type="TValue[]" required>
  The members of the resulting set.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set1", "a", "b", "c"); 
  await redis.sadd("set2", "c", "d", "e"); 
  const intersection =  await redis.sinter("set1", "set2");
  console.log(intersection); // ["c"]
  ```
</RequestExample>


# SINTERSTORE

Return the intersection between sets and store the resulting set in a key

## Arguments

<ParamField body="destination" type="string" required>
  The key of the set to store the resulting set in.
</ParamField>

<ParamField body="keys" type="...string[]" required>
  The keys of the sets to perform the intersection operation on.
</ParamField>

## Response

<ResponseField type="TValue[]" required>
  The members of the resulting set.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set1", "a", "b", "c"); 
  await redis.sadd("set2", "c", "d", "e"); 
  await redis.sinterstore("destination", "set1", "set2");
  ```
</RequestExample>


# SISMEMBER

Check if a member exists in a set

## Arguments

<ParamField body="key" type="string" required>
  The key of the set to check.
</ParamField>

<ParamField body="member" type="TMember">
  The member to check for.
</ParamField>

## Response

<ResponseField type="0 | 1" required>
  `1` if the member exists in the set, `0` if not.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set", "a", "b", "c"); 
  const isMember =  await redis.sismember("set", "a");
  console.log(isMember); // 1
  ```
</RequestExample>


# SMEMBERS

Return all the members of a set

## Arguments

<ParamField body="key" type="string" required>
  The key of the set.
</ParamField>

## Response

<ResponseField type="TMember[]" required>
  The members of the set.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set", "a", "b", "c"); 
  const members =  await redis.smembers("set");
  console.log(members); // ["a", "b", "c"]
  ```
</RequestExample>


# SMISMEMBER

Check if multiple members exist in a set

## Arguments

<ParamField body="key" type="string" required>
  The key of the set to check.
</ParamField>

<ParamField body="members" type="TMember[]">
  The members to check
</ParamField>

## Response

<ResponseField type="(0 | 1)[]" required>
  An array of `0` and `1` values.
  `1` if the member exists in the set, `0` if not.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set", "a", "b", "c"); 
  const members =  await redis.smismember("set", ["a", "b", "d"]);
  console.log(members); // [1, 1, 0]
  ```
</RequestExample>


# SMOVE

Move a member from one set to another

## Arguments

<ParamField body="source" type="string" required>
  The key of the set to move the member from.
</ParamField>

<ParamField body="destination" type="string" required>
  The key of the set to move the member to.
</ParamField>

<ParamField body="member" type="TMember">
  The members to move
</ParamField>

## Response

<ResponseField type="0 | 1" required>
  `1` if the member was moved, `0` if not.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("original", "a", "b", "c"); 
  const moved =  await redis.smove("original", "destination", "a");
  // moved:       1
  // original:    ["b", "c"]
  // destination: ["a"]
  ```
</RequestExample>


# SPOP

Removes and returns one or more random members from a set.

## Arguments

<ParamField body="key" type="string" required>
  The key of the set.
</ParamField>

<ParamField body="count" type="number" default={1}>
  How many members to remove and return.
</ParamField>

## Response

<ResponseField type="TMember | TMember[]" required>
  The popped member.
  If `count` is specified, an array of members is returned.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set", "a", "b", "c"); 
  const popped = await redis.spop("set");
  console.log(popped); // "a"
  ```

  ```ts With Count 
  await redis.sadd("set", "a", "b", "c"); 
  const popped = await redis.spop("set", 2);
  console.log(popped); // ["a", "b"]
  ```
</RequestExample>


# SRANDMEMBER

Returns one or more random members from a set.

## Arguments

<ParamField body="key" type="string" required>
  The key of the set.
</ParamField>

<ParamField body="count" type="number" default={1}>
  How many members to return.
</ParamField>

## Response

<ResponseField type="TMember | TMember[]" required>
  The random member.
  If `count` is specified, an array of members is returned.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set", "a", "b", "c"); 
  const member = await redis.srandmember("set");
  console.log(member); // "a"
  ```

  ```ts With Count 
  await redis.sadd("set", "a", "b", "c"); 
  const members = await redis.srandmember("set", 2);
  console.log(members); // ["a", "b"]
  ```
</RequestExample>


# SREM

Remove one or more members from a set

## Arguments

<ParamField body="key" type="string" required>
  The key of the set to remove the member from.
</ParamField>

<ParamField body="members" type="...TMember[]">
  One or more members to remove from the set.
</ParamField>

## Response

<ResponseField type="integer" required>
  How many members were removed
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set", "a", "b", "c"); 
  const removed = await redis.srem("set", "a", "b", "d");
  console.log(removed); // 2
  ```
</RequestExample>


# SSCAN

Scan a set

## Arguments

<ParamField body="key" type="string" required>
  The key of the set.
</ParamField>

<ParamField body="cursor" type="number">
  The cursor, use `0` in the beginning and then use the returned cursor for subsequent calls.
</ParamField>

<ParamField body="options" type="Object">
  <ParamField body="match" type="string">
    Glob-style pattern to filter by members.
  </ParamField>

  <ParamField body="count" type="number">
    Number of members to return per call.
  </ParamField>
</ParamField>

## Response

<ResponseField type="[number, TMember[]]" required>
  The new cursor and the members.
  If the new cursor is `0` the iteration is complete.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.sadd("key", "a", "ab","b", "c");
  const [newCursor, fields] = await redis.sscan("key", 0, { match: "a*"});
  console.log(newCursor); // likely `0` since this is a very small set
  console.log(fields); // ["a", "ab"]
  ```
</RequestExample>


# SUNION

Return the union between sets

## Arguments

<ParamField body="keys" type="...string[]" required>
  The keys of the sets to perform the union operation on.
</ParamField>

## Response

<ResponseField type="TValue[]" required>
  The members of the resulting set.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set1", "a", "b", "c"); 
  await redis.sadd("set2", "c", "d", "e"); 
  const union =  await redis.sunion("set1", "set2");
  console.log(union); // ["a", "b", "c", "d", "e"]
  ```
</RequestExample>


# SUNIONSTORE

Return the union between sets and store the resulting set in a key

## Arguments

<ParamField body="destination" type="string" required>
  The key of the set to store the resulting set in.
</ParamField>

<ParamField body="keys" type="...string[]" required>
  The keys of the sets to perform the union operation on.
</ParamField>

## Response

<ResponseField type="TValue[]" required>
  The members of the resulting set.
</ResponseField>

<RequestExample>
  ```ts Example 
  await redis.sadd("set1", "a", "b", "c"); 
  await redis.sadd("set2", "c", "d", "e"); 
  await redis.sunionstore("destination", "set1", "set2");
  ```
</RequestExample>


# XADD

Appends one or more new entries to a stream.

## Arguments

<ParamField body="key" type="string" required>
  The key to of the stream.
</ParamField>

<ParamField body="id" type="string | *" required>
  The stream entry ID. If `*` is passed, a new ID will be generated
  automatically.
</ParamField>

<ParamField body="entries" type="Record<string, unknown>" required>
  Key-value data to be appended to the stream.
</ParamField>

<ParamField body="options">
  <ParamField body="nomkStream" type="boolean">
    Prevent creating the stream if it does not exist.
  </ParamField>

  <ParamField body="trim">
    <ParamField body="type" type="'MAXLEN' | 'MINID'" required>
      The trim mode.
    </ParamField>

    {" "}

    <ParamField body="threshold" type="number | string" required>
      The threshold value for the trim mode.
    </ParamField>

    <ParamField body="comparison" type="~ | =" required>
      The comparison operator for the trim mode.
    </ParamField>

    <ParamField body="limit" type="number">
      Limit how many entries will be trimmed at most.
    </ParamField>
  </ParamField>
</ParamField>

## Response

<ResponseField type="string">The ID of the newly added entry.</ResponseField>

<RequestExample>
  ```ts Example
  await redis.xadd(key, "*", { name: "John Doe", age: 30 });

  ```

  ```ts Trimming
  await redis.xadd(key, "*", { name: "John Doe", age: 30 }, {
    trim: {
      type: "MAXLEN",
      threshold: 1000,
      comparison: "=",
    },
  });

  ```
</RequestExample>


# XRANGE

Returns stream entries matching a given range of IDs.

## Arguments

<ParamField body="key" type="string" required>
  The key to of the stream.
</ParamField>

<ParamField body="start" type="string" required>
  The stream entry ID to start from.
</ParamField>

<ParamField body="end" type="string" required>
  The stream entry ID to end at.
</ParamField>

<ParamField body="count" type="integer">
  The maximum number of entries to return.
</ParamField>

## Response

<ResponseField type="Record<streamId, Record<field, value>>">
  An object of stream entries, keyed by their stream ID
</ResponseField>

<RequestExample>
  ```ts Example
  const entries = redis.xrange(key, "-", "+");
  console.log(entries)
  // {
  //   "1548149259438-0": {
  //     "field1": "value1",
  //     "field2": "value2"
  //   },
  //   "1548149259438-1": {
  //     "field1": "value3",
  //     "field2": "value4"
  //   }
  // }

  ```
</RequestExample>


# String Commands



## MGET

Load multiple keys at once. For billing purposes, this counts as a single command.

If a key is not found, it will be returned as `null`, so you might end up with `null` values in your response array.

```ts
const values = await redis.mget("key1", "key2", "key3");
```

## MSET

Set multiple values at once. For billing purposes, this counts as a single command.

```ts
await redis.mset({
  key1: { a: 1 },
  key2: "value2",
  key3: true,
});
```

## MSETNX

```ts
```

## PSETEX

```ts
```

## SET

```ts
```

## SETEX

```ts
```

## SETNX

```ts
```

## SETRANGE

```ts
```

## STRLEN

```ts
```

## SUBSTR

```ts
```


# APPEND

Append a value to a string stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="value" required>
  The value to append.
</ParamField>

## Response

<ResponseField type="integer" required>
  How many characters were added to the string.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.append(key, "Hello");
  // returns 5
  ```
</RequestExample>


# DECR

Decrement the integer value of a key by one

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="string" required>
  The key to decrement.
</ParamField>

## Response

<ResponseField type="integer" required>
  The value at the key after the decrementing.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("key", 6);
  await redis.decr("key");
  // returns 5
  ```
</RequestExample>


# DECRBY

Decrement the integer value of a key by a given number.

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="string" required>
  The key to decrement.
</ParamField>

<ParamField body="decrementBy" type="integer" required>
  The amount to decrement by.
</ParamField>

## Response

<ResponseField type="integer" required>
  The value at the key after the decrementing.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("key", 6);
  await redis.decrby("key", 4);
  // returns 2
  ```
</RequestExample>


# GET

Return the value of the specified key or `null` if the key doesn't exist.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

## Response

<ResponseField required>
  The response is the value stored at the key or `null` if the key doesn't exist.
</ResponseField>

<RequestExample>
  ```ts Example
  type MyType = {
      a: number;
      b: string;
  }
  const value = await redis.get<MyType>("key");
  if (!value) {
      // key doesn't exist
  } else {
      // value is of type MyType
  }
  ```
</RequestExample>


# GETDEL

Return the value of the specified key and delete the key.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

## Response

<ResponseField required>
  The response is the value stored at the key or `null` if the key doesn't exist.
</ResponseField>

<RequestExample>
  ```ts Example
  type MyType = {
      a: number;
      b: string;
  }
  await redis.getdel<MyType>("key");
  // returns {a: 1, b: "2"}
  ```
</RequestExample>


# GETRANGE

Return a substring of value at the specified key.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="start" type="integer" required>
  The start index of the substring.
</ParamField>

<ParamField body="end" type="integer" required>
  The end index of the substring.
</ParamField>

## Response

<ResponseField type="string" required>
  The substring.
</ResponseField>

<RequestExample>
  ```ts Example
  const substring = await redis.getrange("key", 2, 4);
  ```
</RequestExample>


# GETSET

Return the value of the specified key and replace it with a new value.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="newValue" required>
  The new value to store.
</ParamField>

## Response

<ResponseField required>
  The response is the value stored at the key or `null` if the key doesn't exist.
</ResponseField>

<RequestExample>
  ```ts Example

  const oldValue = await redis.getset("key", newValue);
  ```
</RequestExample>


# INCR

Increment the integer value of a key by one

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="string" required>
  The key to increment.
</ParamField>

## Response

<ResponseField type="integer" required>
  The value at the key after the incrementing.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("key", 6);
  await redis.incr("key");
  // returns 7
  ```
</RequestExample>


# INCRBY

Increment the integer value of a key by a given number.

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="string" required>
  The key to decrement.
</ParamField>

<ParamField body="incrementBy" type="integer" required>
  The amount to increment by.
</ParamField>

## Response

<ResponseField type="integer" required>
  The value at the key after the incrementing.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("key", 6);
  await redis.incrby("key", 4);
  // returns 10
  ```
</RequestExample>


# INCRBYFLOAT

Increment the float value of a key by a given number.

If a key does not exist, it is initialized as 0 before performing the operation. An error is returned if the key contains a value of the wrong type or contains a string that can not be represented as integer.

## Arguments

<ParamField body="key" type="string" required>
  The key to decrement.
</ParamField>

<ParamField body="incrementBy" type="float" required>
  The amount to increment by.
</ParamField>

## Response

<ResponseField type="float" required>
  The value at the key after the incrementing.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("key", 6);
  await redis.incrbyfloat("key", 4,5);
  // returns 10.5
  ```
</RequestExample>


# MGET

Load multiple keys from Redis in one go.

For billing purposes, this counts as a single command.

## Arguments

<ParamField body="keys" type="...string" required>
  Multiple keys to load from Redis.
</ParamField>

## Response

<ResponseField type="T[]" required>
  An array of values corresponding to the keys passed in. If a key doesn't exist, the value will be `null`.
</ResponseField>

<RequestExample>
  ```ts Example
  type MyType = {
      a: number;
      b: string;
  }
  const values = await redis.mget<MyType>("key1", "key2", "key3");
  // values.length -> 3
  ```
</RequestExample>


# MSET

Set multiple keys in one go.

For billing purposes, this counts as a single command.

## Arguments

<ParamField type="Record<string, TValue>" required>
  An object where the keys are the keys to set, and the values are the values to set.
</ParamField>

## Response

<ResponseField type="string" required>
  "OK"
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.mset({
      key1: 1,
      key2: "hello",
      key3: { a: 1, b: "hello" },
  });
  ```
</RequestExample>


# MSETNX

Set multiple keys in one go unless they exist already.

For billing purposes, this counts as a single command.

## Arguments

<ParamField type="Record<string, TValue>" required>
  An object where the keys are the keys to set, and the values are the values to set.
</ParamField>

## Response

<ResponseField required>
  `True` if all keys were set, `False` if at least one key was not set.
</ResponseField>

<RequestExample>
  ```ts Example
  redis.msetnx({
    "key1": "value1",
    "key2": "value2"
  })
  ```
</RequestExample>


# SET

Set a key to hold a string value.

## Arguments

<ParamField body="key" type="string" required>
  The key
</ParamField>

<ParamField body="value" type="TValue" required>
  The value, if this is not a string, we will use `JSON.stringify` to convert it
  to a string.
</ParamField>

<ParamField body="opts" type="object">
  You can pass a few options to the command.

  <Expandable>
    <ParamField body="get" type="boolean">
      Instead of returning `"OK"`, this will cause the command to return the old
      value stored at key, or `null` when key did not exist.
    </ParamField>

    <ParamField body="ex" type="integer">
      Adds an expiration (in seconds) to the key.
    </ParamField>

    <ParamField body="px" type="integer">
      Adds an expiration (in milliseconds) to the key.
    </ParamField>

    <ParamField body="exat" type="integer">
      Expires the key after the given timestamp (in seconds).
    </ParamField>

    <ParamField body="pxat" type="integer">
      Expires the key after the given timestamp (in milliseconds).
    </ParamField>

    <ParamField body="keepTtl" type="boolean">
      Keeps the old expiration if the key already exists.
    </ParamField>

    <ParamField body="nx" type="boolean">
      Only set the key if it does not already exist.
    </ParamField>

    <ParamField body="xx" type="boolean">
      Only set the key if it already exists.
    </ParamField>
  </Expandable>
</ParamField>

## Response

<ResponseField required>
  `"OK"`
</ResponseField>

<RequestExample>
  ```ts Basic
  await redis.set("my-key", {my: "value"});
  ```

  ```ts Expire in 60 seconds
  await redis.set("my-key", {my: "value"}, {
    ex: 60
  });
  ```

  ```ts Only update
  await redis.set("my-key", {my: "value"}, {
    xx: true
  });
  ```
</RequestExample>


# SETRANGE

Writes the value of key at offset.

The SETRANGE command in Redis is used to modify a portion of the value of a key by replacing a substring within the key's existing value. It allows you to update part of the string value associated with a specific key at a specified offset.

## Arguments

<ParamField body="key" type="string" required>
  The name of the Redis key for which you want to modify the value.
</ParamField>

<ParamField body="offset" type="integer" required>
  The zero-based index in the value where you want to start replacing characters.
</ParamField>

<ParamField body="value" type="string" required>
  The new string that you want to insert at the specified offset in the existing value.
</ParamField>

## Response

<ResponseField type="integer" required>
  The length of the value after it was modified.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("key", "helloworld")
  const length = await redis.setrange("key", 5, "redis");
  console.log(length); // 10

  // The value of "key" is now "helloredis"
  ```
</RequestExample>


# STRLEN

Return the length of a string stored at a key.

The \`STRLEN\`\` command in Redis is used to find the length of the string value associated with a key. In Redis, keys can be associated with various data types, and one of these data types is the "string." The STRLEN command specifically operates on keys that are associated with string values.

## Arguments

<ParamField body="key" type="string" required>
  The name of the Redis key.
</ParamField>

## Response

<ResponseField type="integer" required>
  The length of the value.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.set("key", "helloworld")
  const length = await redis.strlen("key");
  console.log(length); // 10
  ```
</RequestExample>


# Transactions

Transactions

You can use transactions or pipelines with the `multi` or `pipeline` method.

Transactions are executed atomically, while pipelines are not. In pipelines you can execute multiple commands at once, but other commands from other clients can be executed in between.

<CodeGroup>
  ```ts Pipeline
  const p = redis.pipeline();
  p.set("foo", "bar");
  p.get("foo");
  const res = await p.exec();
  ```

  ```ts Transaction
  const tx = redis.multi();
  tx.set("foo", "bar");
  tx.get("foo");
  const res = await tx.exec();
  ```
</CodeGroup>

For more information on pipelines and transactions, see
[the Pipeline page](https://docs.upstash.com/redis/sdks/ts/pipelining/pipeline-transaction).


# ZADD

Add a member to a sorted set, or update its score if it already exists.

## Arguments

<ParamField body="key" type="string" required>
  The key of the sorted set.
</ParamField>

<ParamField body="options">
  <ParamField body="xx" type="boolean">
    Only update elements that already exist. Never add elements.
  </ParamField>

  <ParamField body="nx" type="boolean">
    Only add new elements. Never update elements.
  </ParamField>

  <ParamField body="ch" type="boolean">
    Return the number of elements added or updated.
  </ParamField>

  <ParamField body="incr" type="boolean">
    When this option is specified ZADD acts like ZINCRBY. Only one score-element pair can be specified in this mode.
  </ParamField>
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of elements added to the sorted sets, not including elements already existing for which the score was updated.

  If `ch` was specified, the number of elements that were updated.

  If `incr` was specified, the new score of `member`.
</ResponseField>

<RequestExample>
  ```ts Simple

  await redis.zadd(
      "key", 
      { score: 2, member: "member" }, 
      { score: 3, member: "member2"},
  );
  ```

  ```ts XX 
  await redis.zadd(
      "key",
      { xx: true },
      { score: 2, member: "member" },
  )
  ```

  ```ts NX 
  await redis.zadd(
      "key",
      { nx: true },
      { score: 2, member: "member" },
  )
  ```

  ```ts CH 
  await redis.zadd(
      "key",
      { ch: true },
      { score: 2, member: "member" },
  )
  ```

  ```ts INCR 
  await redis.zadd(
      "key",
      { cincrh: true },
      { score: 2, member: "member" },
  )
  ```
</RequestExample>


# ZCARD

Returns the number of elements in the sorted set stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of elements in the sorted set.
</ResponseField>

<RequestExample>
  ```ts Example

  await redis.zadd("key", 
      { score: 1, member: "one"}, 
      { score: 2, member: "two" },
  );
  const elements = await redis.zrank("key");
  console.log(elements); // 2
  ```
</RequestExample>


# ZCOUNT

Returns the number of elements in the sorted set stored at key filterd by score.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="min" type="integer | string" required>
  The minimum score to filter by.

  Use `-inf` to effectively ignore this filter.

  Use `(number` to include the value.\`
</ParamField>

<ParamField body="max" type="integer | string" required>
  The maximum score to filter by.

  Use `+inf` to effectively ignore this filter.

  Use `number)` to include the value.\`
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of elements where score is between min and max.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.zadd("key", 
      { score: 1, member: "one"}, 
      { score: 2, member: "two" },
  );
  const elements = await redis.zcount("key", 1, "+inf");
  console.log(elements); // 1
  ```
</RequestExample>


# ZDIFFSTORE

Writes the difference between sets to a new key.

## Arguments

<ParamField body="destination" type="string" required>
  The key to write the difference to.
</ParamField>

<ParamField body="keys" type="integer" required>
  How many keys to compare.
</ParamField>

<ParamField body="keys" type="...string[]" required>
  The keys to compare.
</ParamField>

## Response

<ResponseField required>
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```ts Example
  const values = await redis.zdiffstore("destination", 2, "key1", "key2");
  ```
</RequestExample>


# ZINCRBY

Increment the score of a member.

## Arguments

<ParamField body="key" type="string" required>
  The key of the sorted set.
</ParamField>

<ParamField body="increment" type="integer" required>
  The increment to add to the score.
</ParamField>

<ParamField body="member" type="TMember" required>
  The member to increment.
</ParamField>

## Response

<ResponseField type="integer" required>
  The new score of `member` after the increment operation.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.zadd("key", 1, "member");
  const value = await redis.zincrby("key", 2, "member");
  console.log(value); // 3
  ```
</RequestExample>


# ZINTERSTORE

Writes the intersection between sets to a new key.

## Arguments

<ParamField body="destination" type="string" required>
  The key to write the intersection to.
</ParamField>

<ParamField body="keys" type="integer" required>
  How many keys to compare.
</ParamField>

<ParamField body="keys" type="string | string[]" required>
  The keys to compare.
</ParamField>

<ParamField body="options">
  <ParamField body="aggregate" type="sum | min | max">
    The aggregation method.
  </ParamField>

  <ParamField body="weight" type="number">
    The weight to apply to each key.
  </ParamField>

  <ParamField body="weights" type="number[]">
    The weights to apply to each key.
  </ParamField>
</ParamField>

## Response

<ResponseField required>
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```ts Simple
  await redis.zadd(
      "key1", 
      { score: 1, member: "member1" },
  )
  await redis.zadd(
      "key2",
      { score: 1, member: "member1" },
      { score: 2, member: "member2" },
  )

  const res = await redis.zinterstore("destination", 2, ["key1", "key2"]);
  console.log(res) // 1
  ```

  ```ts With Weights
  await redis.zadd(
      "key1", 
      { score: 1, member: "member1" },
  )
  await redis.zadd(
      "key2",
      { score: 1, member: "member1" },
      { score: 2, member: "member2" },
  )
  const res = await redis.zinterstore(
      "destination",
      2,
      ["key1", "key2"],
      { weights: [2, 3] },
  );
  console.log(res) // 1
  ```

  ```ts Aggregate
  await redis.zadd(
      "key1", 
      { score: 1, member: "member1" },
  )
  await redis.zadd(
      "key2",
      { score: 1, member: "member1" },
      { score: 2, member: "member2" },
  )
  const res = await redis.zinterstore(
      "destination",
      2,
      ["key1", "key2"],
      { aggregate: "sum" },
  );
  console.log(res) // 1
  ```
</RequestExample>


# ZLEXCOUNT

Returns the number of elements in the sorted set stored at key filtered by lex.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="min" type="string" required>
  The lower lexicographical bound to filter by.

  Use `-` to disable the lower bound.
</ParamField>

<ParamField body="max" type="string" required>
  The upper lexicographical bound to filter by.

  Use `+` to disable the upper bound.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of matched.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.zadd("key", 
      { score: 1, member: "one"}, 
      { score: 2, member: "two" },
  );
  const elements = await redis.zlexcount("key", "two", "+");
  console.log(elements); // 1
  ```
</RequestExample>


# ZMSCORE

Returns the scores of multiple members.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

## Response

<ResponseField body="members" type="TMember[]" required>
  The members of the sorted set.
</ResponseField>

<RequestExample>
  ```ts Example

  await redis.zadd("key", 
      { score: 1, member: "m1" },
      { score: 2, member: "m2" },
      { score: 3, member: "m3" },
      { score: 4, member: "m4" },
  )

  const scores = await redis.zmscore("key", ["m2", "m4"])
  console.log(scores) // [2, 4]
  ```
</RequestExample>


# ZPOPMAX

Removes and returns up to count members with the highest scores in the sorted set stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key of the sorted set
</ParamField>

## Response

<ResponseField body="count" type="integer">
  The number of elements removed. Defaults to 1.
</ResponseField>

<RequestExample>
  ```ts Example
  const popped = await redis.zpopmax("key", 4);
  ```
</RequestExample>


# ZPOPMIN

Removes and returns up to count members with the lowest scores in the sorted set stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key of the sorted set
</ParamField>

## Response

<ResponseField body="count" type="integer">
  The number of elements removed. Defaults to 1.
</ResponseField>

<RequestExample>
  ```ts Example
  const popped = await redis.zpopmin("key", 4);
  ```
</RequestExample>


# ZRANGE

Returns the specified range of elements in the sorted set stored at key.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="min" type="number | string" required>
  The lower bound of the range.
</ParamField>

<ParamField body="max" type="number | string" required>
  The upper bound of the range.
</ParamField>

<ParamField body="options">
  <Expandable>
    <ParamField body="withScores" type="boolean">
      Whether to include the scores in the response.
    </ParamField>

    <ParamField body="rev" type="boolean">
      Whether to reverse the order of the response.
    </ParamField>

    <ParamField body="byScore" type="boolean">
      Whether to use the score as the sort order.
    </ParamField>

    <ParamField body="byLex" type="boolean">
      Whether to use lexicographical ordering.
    </ParamField>

    <ParamField body="offset" type="number">
      The offset to start from.
    </ParamField>

    <ParamField body="count" type="number">
      The number of elements to return.
    </ParamField>
  </Expandable>
</ParamField>

## Response

<ResponseField type="TMember[]">
  The values in the specified range.

  If `withScores` is true, the response will have interleaved members and scores: `[TMember, number, TMember, number, ...]`
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.zadd("key", 
      { score: 1, member: "m1" },
      { score: 2, member: "m2" },
  )
  const res = await redis.zrange("key", 1, 3)
  console.log(res) // ["m2"]
  ```

  ```ts WithScores
  await redis.zadd("key", 
      { score: 1, member: "m1" },
      { score: 2, member: "m2" },
  )
  const res = await redis.zrange("key", 1, 3, { withScores: true })
  console.log(res) // ["m2", 2]
  ```

  ```ts ByScore
  await redis.zadd("key", 
      { score: 1, member: "m1" },
      { score: 2, member: "m2" },
      { score: 3, member: "m3" },
  )
  const res = await redis.zrange("key", 1, 2, { byScore: true })
  console.log(res) // ["m1", "m2"]
  ```
</RequestExample>


# ZRANK

Returns the rank of a member

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="member" type="TMember" required>
  The member to get the rank of.
</ParamField>

## Response

<ResponseField type="integer" required>
  The rank of the member.
</ResponseField>

<RequestExample>
  ```ts Example
  const rank = await redis.rank("key", "member");
  ```
</RequestExample>


# ZREM

Remove one or more members from a sorted set

## Arguments

<ParamField body="key" type="string" required>
  The key of the sorted set
</ParamField>

<ParamField body="members" type="...TMember[]" required>
  One or more members to remove
</ParamField>

## Response

<ResponseField required>
  The number of members removed from the sorted set.
</ResponseField>

<RequestExample>
  ```ts Single
  await redis.zrem("key", "member");
  ```

  ```ts Multiple
  await redis.zrem("key", "member1", "member2");
  ```
</RequestExample>


# ZREMRANGEBYLEX

Remove all members in a sorted set between the given lexicographical range.

## Arguments

<ParamField body="key" type="string" required>
  The key of the sorted set
</ParamField>

<ParamField body="min" type="string" required>
  The minimum lexicographical value to remove.
</ParamField>

<ParamField body="min" type="string" required>
  The maximum lexicographical value to remove.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of elements removed from the sorted set.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.zremrangebylex("key", "alpha", "omega")
  ```
</RequestExample>


# ZREMRANGEBYRANK

Remove all members in a sorted set between the given ranks.

## Arguments

<ParamField body="key" type="string" required>
  The key of the sorted set
</ParamField>

<ParamField body="min" type="number" required>
  The minimum rank to remove.
</ParamField>

<ParamField body="min" type="number" required>
  The maximum rank to remove.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of elements removed from the sorted set.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.zremrangebyrank("key", 4, 20)
  ```
</RequestExample>


# ZREMRANGEBYSCORE

Remove all members in a sorted set between the given scores.

## Arguments

<ParamField body="key" type="string" required>
  The key of the sorted set
</ParamField>

<ParamField body="min" type="number" required>
  The minimum score to remove.
</ParamField>

<ParamField body="min" type="number" required>
  The maximum score to remove.
</ParamField>

## Response

<ResponseField type="integer" required>
  The number of elements removed from the sorted set.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.zremrangebyscore("key", 2, 5)
  ```
</RequestExample>


# ZREVRANK

Returns the rank of a member in a sorted set, with scores ordered from high to low.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

<ParamField body="member" type="TMember" required>
  The member to get the reverse rank of.
</ParamField>

## Response

<ResponseField type="integer" required>
  The reverse rank of the member.
</ResponseField>

<RequestExample>
  ```ts Example
  const rank = await redis.rank("key", "member");
  ```
</RequestExample>


# ZSCAN

Scan a sorted set

## Arguments

<ParamField body="key" type="string" required>
  The key of the sorted set.
</ParamField>

<ParamField body="cursor" type="number">
  The cursor, use `0` in the beginning and then use the returned cursor for subsequent calls.
</ParamField>

<ParamField body="options" type="Object">
  <ParamField body="match" type="string">
    Glob-style pattern to filter by members.
  </ParamField>

  <ParamField body="count" type="number">
    Number of members to return per call.
  </ParamField>
</ParamField>

## Response

<ResponseField type="[number, TMember[]]" required>
  The new cursor and the members.
  If the new cursor is `0` the iteration is complete.
</ResponseField>

<RequestExample>
  ```ts Example
  await redis.zadd("key", 
      { score: 1, member: "a" },
      { score: 2, member: "ab" },
      { score: 3, member: "b" },
      { score: 4, member: "c" },
      { score: 5, member: "d" },
  )
  const [newCursor, members] = await redis.sscan("key", 0, { match: "a*"});
  console.log(newCursor); // likely `0` since this is a very small set
  console.log(members); // ["a", "ab"]
  ```

  ```ts withCount
  await redis.zadd("key", 
      { score: 1, member: "a" },
      { score: 2, member: "ab" },
      { score: 3, member: "b" },
      { score: 4, member: "c" },
      { score: 5, member: "d" },
  )
  const [newCursor, members] = await redis.sscan("key", 0, { match: "a*", count: 1});
  console.log(newCursor); // likely `0` since this is a very small set
  console.log(members); // ["a"]
  ```
</RequestExample>


# ZSCORE

Returns the scores of a member.

## Arguments

<ParamField body="key" type="string" required>
  The key to get.
</ParamField>

## Response

<ResponseField body="member" type="TMember" required>
  A member of the sortedset.
</ResponseField>

<RequestExample>
  ```ts Example

  await redis.zadd("key", 
      { score: 1, member: "m1" },
      { score: 2, member: "m2" },
      { score: 3, member: "m3" },
      { score: 4, member: "m4" },
  )

  const score = await redis.zscore("key", "m2")
  console.log(score) // 2
  ```
</RequestExample>


# ZUNIONSTORE

Writes the union between sets to a new key.

## Arguments

<ParamField body="destination" type="string" required>
  The key to write the union to.
</ParamField>

<ParamField body="keys" type="integer" required>
  How many keys to compare.
</ParamField>

<ParamField body="keys" type="string | string[]" required>
  The keys to compare.
</ParamField>

<ParamField body="options">
  <ParamField body="aggregate" type="sum | min | max">
    The aggregation method.
  </ParamField>

  <ParamField body="weight" type="number">
    The weight to apply to each key.
  </ParamField>

  <ParamField body="weights" type="number[]">
    The weights to apply to each key.
  </ParamField>
</ParamField>

## Response

<ResponseField required>
  The number of elements in the resulting set.
</ResponseField>

<RequestExample>
  ```ts Simple
  await redis.zadd(
      "key1", 
      { score: 1, member: "member1" },
  )
  await redis.zadd(
      "key2",
      { score: 1, member: "member1" },
      { score: 2, member: "member2" },
  )

  const res = await redis.zunionstore("destination", 2, ["key1", "key2"]);
  console.log(res) // 2
  ```

  ```ts With Weights
  await redis.zadd(
      "key1", 
      { score: 1, member: "member1" },
  )
  await redis.zadd(
      "key2",
      { score: 1, member: "member1" },
      { score: 2, member: "member2" },
  )
  const res = await redis.zunionstore(
      "destination",
      2,
      ["key1", "key2"],
      { weights: [2, 3] },
  );
  console.log(res) // 2
  ```

  ```ts Aggregate
  await redis.zadd(
      "key1", 
      { score: 1, member: "member1" },
  )
  await redis.zadd(
      "key2",
      { score: 1, member: "member1" },
      { score: 2, member: "member2" },
  )
  const res = await redis.zunionstore(
      "destination",
      2,
      ["key1", "key2"],
      { aggregate: "sum" },
  );
  console.log(res) // 2
  ```
</RequestExample>


# Deployment



We support various platforms, such as nodejs, cloudflare and fastly. Platforms
differ slightly when it comes to environment variables and their `fetch` api.
Please use the correct import when deploying to special platforms.

## Node.js / Browser

Examples: Vercel, Netlify, AWS Lambda

If you are running on nodejs you can set `UPSTASH_REDIS_REST_URL` and
`UPSTASH_REDIS_REST_TOKEN` as environment variable and create a redis instance
like this:

```ts
import { Redis } from "@upstash/redis"

const redis = new Redis({
  url: <UPSTASH_REDIS_REST_URL>,
  token: <UPSTASH_REDIS_REST_TOKEN>,
})

// or load directly from env
const redis = Redis.fromEnv()
```

<Info>
  If you are running on nodejs v17 and earlier, `fetch` will not be natively
  supported. Platforms like Vercel, Netlify, Deno, Fastly etc. provide a polyfill
  for you. But if you are running on bare node, you need to either specify a
  polyfill yourself or change the import path slightly:

  ```typescript
  import { Redis } from "@upstash/redis/with-fetch";
  ```
</Info>

*   [Code example](https://github.com/upstash/upstash-redis/blob/main/examples/nodejs)

## Cloudflare Workers

Cloudflare handles environment variables differently than Node.js. Please add
`UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` using
`wrangler secret put ...` or in the cloudflare dashboard.

Afterwards you can create a redis instance:

```ts
import { Redis } from "@upstash/redis/cloudflare"

const redis = new Redis({
  url: <UPSTASH_REDIS_REST_URL>,
  token: <UPSTASH_REDIS_REST_TOKEN>,
})


// or load directly from global env

// service worker
const redis = Redis.fromEnv()


// module worker
export default {
  async fetch(request: Request, env: Bindings) {
    const redis = Redis.fromEnv(env)
    // ...
  }
}
```

*   [Code example](https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers)
*   [Code example typescript](https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers-with-typescript)
*   [Code example Wrangler 1](https://github.com/upstash/upstash-redis/tree/main/examples/cloudflare-workers-with-wrangler-1)
*   [Documentation](https://docs.upstash.com/redis/tutorials/cloudflare_workers_with_redis)

## Fastly

Fastly introduces a concept called
[backend](https://developer.fastly.com/reference/api/services/backend/). You
need to configure a backend in your `fastly.toml`. An example can be found
[here](https://github.com/upstash/upstash-redis/blob/main/examples/fastly/fastly.toml).
Until the fastly api stabilizes we recommend creating an instance manually:

```ts
import { Redis } from "@upstash/redis/fastly"

const redis = new Redis({
  url: <UPSTASH_REDIS_REST_URL>,
  token: <UPSTASH_REDIS_REST_TOKEN>,
  backend: <BACKEND_NAME>,
})
```

*   [Code example](https://github.com/upstash/upstash-redis/tree/main/examples/fastly)
*   [Documentation](https://blog.upstash.com/fastly-compute-edge-with-redis)

## Deno

Examples: [Deno Deploy](https://deno.com/deploy),
[Netlify Edge](https://www.netlify.com/products/edge/)

```ts
import { Redis } from "https://deno.land/x/upstash_redis/mod.ts"

const redis = new Redis({
  url: <UPSTASH_REDIS_REST_URL>,
  token: <UPSTASH_REDIS_REST_TOKEN>,
})

// or
const redis = Redis.fromEnv();
```


# Developing or Testing



When developing or testing your application, you might not want or can not use
Upstash over the internet. In this case, you can use a community project called
[Serverless Redis HTTP (SRH)](https://github.com/hiett/serverless-redis-http)
created by [Scott Hiett](https://x.com/hiettdigital).

SRH is a Redis proxy and connection pooler that uses HTTP rather than the Redis
binary protocol. The aim of this project is to be entirely compatible with
Upstash, and work with any Upstash supported Redis version.

We are working with Scott together to keep SRH up to date with the latest
Upstash features.

## Use cases for SRH:

*   For usage in your CI pipelines, creating Upstash databases is tedious, or you
    have lots of parallel runs.
    *   See [Using in GitHub Actions](#in-github-actions) on how to quickly get SRH
        setup for this context.
*   For usage inside of Kubernetes, or any network whereby the Redis server is not
    exposed to the internet.
    *   See [Using in Docker Compose](#via-docker-compose) for the various setup
        options directly using the Docker Container.
*   For local development environments, where you have a local Redis server
    running, or require offline access.
    *   See [Using the Docker Command](#via-docker-command), or
        [Using Docker Compose](#via-docker-compose).

## Setting up SRH

### Via Docker command

If you have a locally running Redis server, you can simply start an SRH
container that connects to it. In this example, SRH will be running on port
`8080`.

```bash
docker run \
    -it -d -p 8080:80 --name srh \
    -e SRH_MODE=env \
    -e SRH_TOKEN=your_token_here \
    -e SRH_CONNECTION_STRING="redis://your_server_here:6379" \
    hiett/serverless-redis-http:latest
```

### Via Docker Compose

If you wish to run in Kubernetes, this should contain all the basics would need
to set that up. However, be sure to read the Configuration Options, because you
can create a setup whereby multiple Redis servers are proxied.

```yml
version: "3"
services:
  redis:
    image: redis
    ports:
      - "6379:6379"
  serverless-redis-http:
    ports:
      - "8079:80"
    image: hiett/serverless-redis-http:latest
    environment:
      SRH_MODE: env
      SRH_TOKEN: example_token
      SRH_CONNECTION_STRING: "redis://redis:6379" # Using `redis` hostname since they're in the same Docker network.
```

### In GitHub Actions

SRH works nicely in GitHub Actions because you can run it as a container in a
job's services. Simply start a Redis server, and then SRH alongside it. You
don't need to worry about a race condition of the Redis instance not being
ready, because SRH doesn't create a Redis connection until the first command
comes in.

```yml
name: Test @upstash/redis compatibility
on:
  push:
  workflow_dispatch:

env:
  SRH_TOKEN: example_token

jobs:
  container-job:
    runs-on: ubuntu-latest
    container: denoland/deno
    services:
      redis:
        image: redis/redis-stack-server:6.2.6-v6 # 6.2 is the Upstash compatible Redis version
      srh:
        image: hiett/serverless-redis-http:latest
        env:
          SRH_MODE: env # We are using env mode because we are only connecting to one server.
          SRH_TOKEN: ${{ env.SRH_TOKEN }}
          SRH_CONNECTION_STRING: redis://redis:6379

    steps:
      # You can place your normal testing steps here. In this example, we are running SRH against the upstash/upstash-redis test suite.
      - name: Checkout code
        uses: actions/checkout@v3
        with:
          repository: upstash/upstash-redis

      - name: Run @upstash/redis Test Suite
        run: deno test -A ./pkg
        env:
          UPSTASH_REDIS_REST_URL: http://srh:80
          UPSTASH_REDIS_REST_TOKEN: ${{ env.SRH_TOKEN }}
```

A huge thanks goes out to [Scott](https://hiett.dev/) for creating this project,
and for his continued efforts to keep it up to date with Upstash.


# Get Started



`@upstash/redis` is written in Deno and can be imported from
[deno.land](https://deno.land)

```ts
import { Redis } from "https://deno.land/x/upstash_redis/mod.ts";
```

We transpile the package into an npm compatible package as well:

```bash
npm install @upstash/redis
```

```bash
yarn add @upstash/redis
```

```bash
pnpm add @upstash/redis
```

## Basic Usage:

```ts
import { Redis } from "@upstash/redis"

const redis = new Redis({
  url: <UPSTASH_REDIS_REST_URL>,
  token: <UPSTASH_REDIS_REST_TOKEN>,
})

// string
await redis.set('key', 'value');
let data = await redis.get('key');
console.log(data)

await redis.set('key2', 'value2', {ex: 1});

// sorted set
await redis.zadd('scores', { score: 1, member: 'team1' })
data = await redis.zrange('scores', 0, 100 )
console.log(data)

// list
await redis.lpush('elements', 'magnesium')
data = await redis.lrange('elements', 0, 100 )
console.log(data)

// hash
await redis.hset('people', {name: 'joe'})
data = await redis.hget('people', 'name' )
console.log(data)

// sets
await redis.sadd('animals', 'cat')
data  = await redis.spop('animals', 1)
console.log(data)
```


# Overview



`@upstash/redis` is an HTTP/REST based Redis client for TypeScript, built on top
of [Upstash REST API](https://docs.upstash.com/features/restapi).

[![Tests](https://github.com/upstash/upstash-redis/actions/workflows/tests.yaml/badge.svg)](https://github.com/upstash/upstash-redis/actions/workflows/tests.yaml)
![npm (scoped)](https://img.shields.io/npm/v/@upstash/redis)
![npm bundle size](https://img.shields.io/bundlephobia/minzip/@upstash/redis)

You can find the Github Repository [here](https://github.com/upstash/upstash-redis).

It is the only connectionless (HTTP based) Redis client and designed for:

*   Serverless functions (AWS Lambda ...)
*   Cloudflare Workers (see
    [the example](https://github.com/upstash/upstash-redis/tree/master/examples/cloudflare-workers))
*   Fastly Compute\@Edge (see
    [the example](https://github.com/upstash/upstash-redis/tree/master/examples/fastly))
*   Next.js, Jamstack ...
*   Client side web/mobile applications
*   WebAssembly
*   and other environments where HTTP is preferred over TCP.

See
[the list of APIs](https://docs.upstash.com/features/restapi#rest---redis-api-compatibility)
supported.


# Auto-Pipelining



### Auto Pipelining

Auto pipelining allows you to use the Redis client as usual
while in the background it tries to send requests in batches
whenever possible.

In a nutshell, the client will accumulate commands in a pipeline
and wait for a short amount of time for more commands to arrive.
When there are no more commands, it will execute them as a batch.

To enable the feature, simply pass `enableAutoPipelining: true`
when creating the Redis client:

<CodeGroup>
  ```ts Redis
  import { Redis } from "@upstash/redis";

  const redis = Redis.fromEnv({
    latencyLogging: false,
    enableAutoPipelining: true
  });
  ```

  ```ts fromEnv
  import { Redis } from "@upstash/redis";

  const redis = new Redis({
    url: <UPSTASH_REDIS_REST_URL>,
    token: <UPSTASH_REDIS_REST_TOKEN>,
    enableAutoPipelining: true
  })
  ```
</CodeGroup>

This is especially useful in cases when we want to make async
requests or when we want to make requests in batches.

```ts
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv({
  latencyLogging: false,
  enableAutoPipelining: true
});

// async call to redis. Not executed right away, instead
// added to the pipeline
redis.hincrby("Brooklyn", "visited", 1);

// making requests in batches
const brooklynInfo = Promise.all([
  redis.hget("Brooklyn", "coordinates"),
  redis.hget("Brooklyn", "population")
]);

// when we call await, the three commands are executed
// as a pipeline automatically. A single PIPELINE command
// is executed instead of three requests and the results
// are returned:
const [ coordinates, population ] = await brooklynInfo;
```

The benefit of auto pipelining is that it reduces the number
of HTTP requests made like pipelining and transaction while
being extremely simple to enable and use. It's especially
useful in cases like Vercel Edge and [Cloudflare Workers, where the number of
simultaneous requests is limited by 6](https://developers.cloudflare.com/workers/platform/limits/#account-plan-limits).

To learn more about how auto pipelining can be utilized in a
project, see
[the auto-pipeline example project under `upstash-redis` repository](https://github.com/upstash/upstash-redis/tree/main/examples/auto-pipeline)

### How it Works

For auto pipeline to work, the client keeps an active pipeline
and adds incoming commands to this pipeline. After the command
is added to the pipeline, execution of the pipeline is delayed
by releasing the control of the Node thread.

The pipeline executes when one of these two conditions are met:
No more commands are being added or at least one of the commands
added is being 'awaited'.

This means that if you are awaiting every time you run a command,
you won't benefit much from auto pipelining since each await will
trigger a pipeline:

```ts
const foo = await redis.get("foo") // makes a PIPELINE call
const bar = await redis.get("bar") // makes another PIPELINE call
```

In these cases, we suggest using `Promise.all`:

```ts
// makes a single PIPELINE call:
const [ foo, bar ] = await Promise.all([
  redis.get("foo"),
  redis.get("bar")
])
```

In addition to resulting in a single PIPELINE call, the commands
in `Promise.all` are executed in the order they are written!


# Pipeline & Transaction



### Pipeline

Pipelining commands allows you to send a single http request with multiple
commands. Keep in mind, that the execution of pipelines is not atomic and the
execution of other commands can interleave.

```ts
import { Redis } from "@upstash/redis";

const redis = new Redis({
  /* auth */
});

const p = redis.pipeline();

// Now you can chain multiple commands to create your pipeline:

p.set("key", 2);
p.incr("key");

// or inline:
p.hset("key2", "field", { hello: "world" }).hvals("key2");

// Execute the pipeline once you are done building it:
// `exec` returns an array where each element represents the response of a command in the pipeline.
// You can optionally provide a type like this to get a typed response.
const res = await p.exec<[Type1, Type2, Type3]>();
```

For more information about pipelines using REST see
[here](https://blog.upstash.com/pipeline).

If you wish to benefit from pipeline automatically,
you can simply enable auto-pipelining to make your redis client
handle the commands in batches in the background. See
[the Auto-pipelining page](https://docs.upstash.com/redis/sdks/ts/pipelining/auto-pipeline).

### Transaction

Remember that the pipeline is able to send multiple commands at once but
can't execute them atomically. With transactions, you can make the commands
execute atomically.

```ts
import { Redis } from "@upstash/redis";

const redis = new Redis({
  /* auth */
});

const p = redis.multi();

p.set("key", 2);
p.incr("key");

// or inline:
p.hset("key2", "field", { hello: "world" }).hvals("key2");

// execute the transaction
const res = await p.exec<[Type1, Type2, Type3]>();
```


# Retries



By default `@upstash/redis` will retry sending you request when network errors
occur. It will retry 5 times with a backoff of
`(retryCount) => Math.exp(retryCount) * 50` milliseconds.

You can customize this in the `Redis` constructor:

```ts
new Redis({
  url: UPSTASH_REDIS_REST_URL,
  token: UPSTASH_REDIS_REST_TOKEN,
  retry: {
    retries: 5,
    backoff: (retryCount) => Math.exp(retryCount) * 50,
  },
});
```

The exact type definition can be found
[here](https://github.com/upstash/upstash-redis/blob/4948b049e0d580d1de0a4cbfeac5565d7e035cc4/pkg/http.ts#LL31C1-L49C5).


# Troubleshooting



## ReferenceError: fetch is not defined

#### Problem

If you are running on nodejs v17 and earlier, fetch will not be natively
supported. Platforms like Vercel, Netlify, Deno, Fastly etc. provide a polyfill
for you. But if you are running on bare node, you need to add a polyfill.

#### Solution

```bash
npm i isomorphic-fetch
```

```ts
import { Redis } from "@upstash/redis";
import "isomorphic-fetch";

const redis = new Redis({
  /*...*/
});
```

## Hashed Response

The response from a server is not what you expect but looks like a hash?

```ts
await redis.set("key", "value");
const data = await redis.get("key");
console.log(data);

// dmFsdWU=
```

#### Problem

By default `@upstash/redis` will request responses from the server to be base64
encoded. This is to prevent issues with some edge cases when storing data where
the http response fails to be deserialized using `res.json()`

This solves the problem for almost all edge cases, but it can cause new issues.

#### Solution

You can disable this behavior by setting `responseEncoding` to `false` in the
options.

```ts
const redis = new Redis({
  // ...
  responseEncoding: false,
});
```

This should no longer be necessary, but if you are still experiencing issues
with this, please let us know:

*   [Discord](https://discord.gg/w9SenAtbme)
*   [X](https://x.com/upstash)
*   [GitHub](https://github.com/upstash/upstash-redis/issues/new)

## Large numbers are returned as string

You are trying to load a large number and it is returned as a string instead.

```ts
await redis.set("key", "101600000000150081467");
const res = await redis("get");

// "101600000000150081467"
```

#### Problem

Javascript can not handle numbers larger than `2^53 -1` safely and would return
wrong results when trying to deserialize them. In these cases the default
deserializer will return them as string instead. This might cause a mismatch
with your custom types.

#### Solution

Please be aware that this is a limitation of javascript and take special care
when handling large numbers.


# Unexpected Increase in Command Count



### Symptom

You notice an increasing command count for your Redis database in the Upstash Console, even when there are no connected clients.

### Diagnosis

The Upstash Console interacts with your Redis database to provide its functionality, which can result in an increased command count. This behavior is normal and expected. Here's a breakdown of why this occurs:

1.  **Data Browser functionality:**
    The Data Browser tab sends various commands to list and display your keys, including:
    *   SCAN: To iterate through the keyspace
    *   GET: To retrieve values for keys
    *   TTL: To check the time-to-live for keys

2.  **Rate Limiting check:**
    The Console checks if your database is being used for Rate Limiting. This involves sending EXISTS commands for rate limiting-related keys.

3.  **Other Console features:**
    Additional features in the Console may send commands to your database to retrieve or display information.

### Verification

You can use the Monitor tab in the Upstash Console to observe which commands are being sent by the Console itself. This can help you distinguish between Console-generated commands and those from your application or other clients.
Also, Usage tab contains 'Top Commands Usage' graph which shows the exact command history.

### Conclusion

The increasing command count you're seeing is likely due to the Console's normal operations and should not be a cause for concern. These commands do not significantly impact your database's performance or your usage limits.

If you have any further questions or concerns about command usage, please don't hesitate to contact Upstash support.


# ERR DB capacity quota exceeded



### Symptom

The client gets an exception similar to:

```
ReplyError: ERR DB capacity quota exceeded
```

### Diagnosis

Your total database size exceeds the max data size limit, which is `256Mb` for
"Free" and `10Gb` for "Pay as you go" databases. When this limit is reached,
write requests may be rejected. Read and delete requests will not be affected.

### Solution-1

You can manually delete some entries to allow further writes. Additionally you
can consider setting TTL (expiration time) for your keys or enable
[eviction](../features/eviction) for your database.

### Solution-2

You can upgrade your database to Pro for higher limits.


# Error read ECONNRESET



### Symptom

The client can not connect to the database throwing an exception similar to:

```
[ioredis] Unhandled error event: Error: read ECONNRESET
    at TCP.onStreamRead (node:internal/stream_base_commons:211:20)
```

### Diagnosis

The server is TLS enabled but your connection (client) is not.

### Solution

Check your connection parameters and ensure you enable TLS.

If you are using a Redis URL then it should start with `rediss://`.

You can copy the correct client configuration from Upstash console clicking on
**Redis Connect** button.


# WRONGPASS invalid or missing auth token



### Symptom

The database rejects your request with an error similar to:

```
UpstashError: WRONGPASS invalid or missing auth token
```

### Diagnosis

The server rejects your request because the auth token is missing or invalid.

Most likely you have forgotten to set it in your environment variables, or you
are using a wrong token.

The connection password can only be used in traditional Redis clients. If you
want to connect over HTTP, you need to use the HTTP auth token.

### Solution

1.  Check that you have set the `UPSTASH_REDIS_REST_TOKEN` in your environment
    variables and it is loaded correctly by your application at runtime.

2.  Make sure you are using the correct HTTP auth token. You can copy the correct
    client configuration from the
    [Upstash console](https://console.upstash.com/redis) by copying the snippet
    from the `Connect to your database` -> `@upstash/redis` tab

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/troubleshooting/rest/console_upstash_redis.png)

Or scroll further down to the `REST API` section and copy the
`UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` from there.

![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/troubleshooting/rest/console_rest_api.png)


# ERR max concurrent connections exceeded



### Symptom

New clients can not connect to the database throwing an exception similar to:

```
"message" : "[ioredis] Unhandled error event:
ReplyError: ERR max concurrent connections exceeded\r
at Object.onceWrapper (events.js:286:20)\r
at Socket.emit (events.js:203:15)\r    at Socket.EventEmitter.emit (domain.js:448:20)\r
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1093:10)\n"
```

### Diagnosis

You have reached the concurrent connection limit.

### Solution-1

You need to manage connections more efficiently. If you are using serverless
functions, you can create the Redis client inside the function and close the
connection when you are done with the database as below.

<Note>
  This solution may have a latency overhead (about 4 ms). See [the blog
  post](https://blog.upstash.com/serverless-database-connections) for more.
</Note>

```javascript
exports.handler = async (event) => {
  const client = new Redis(process.env.REDIS_URL);
  /*
    do stuff with redis
     */
  await client.quit();
  /*
  do other stuff
   */
  return {
    response: "response",
  };
};
```

### Solution-2

You can use [@upstash/redis](https://github.com/upstash/upstash-redis) client
which is REST based so it does not have any connection related problems.

### Solution-3

You can upgrade your database to Pro for higher limits.

<Info>
  See [the blog post](https://blog.upstash.com/serverless-database-connections)
  about the database connections in serverless functions.
</Info>


# ERR max daily request limit exceeded



### Symptom

The client gets an exception similar to:

```
ReplyError: ERR max daily request limit exceeded
```

### Diagnosis

Your database exceeds the max daily request count limit which is `10000` for the
Free Tier(`2500` for Global).

### Solution-1

You can refactor your application to send less number of commands.

### Solution-2

You can upgrade your database to pay-as-you-go plan by entering a payment
method. When you entered your credit card, your database will be upgraded
automatically.

See [here](https://docs.upstash.com/howto/upgradedatabase) for more information.


# ERR max key size exceeded



### Symptom

The client gets an exception similar to:

```
ReplyError: ERR max key size exceeded. Limit: X bytes, Actual: Z bytes
```

### Diagnosis

Size of the key in the request exceeds the max key size limit, which is `32Kb`.

### Solution

This is a hardcoded limit and cannot be configured per database. You should
reduce the key size.


# ERR max single record size exceeded



### Symptom

The client gets an exception similar to:

```
ReplyError: ERR max single record size exceeded
```

### Diagnosis

An entry size exceeds the max record size limit which is `100Mb` for "Free" and
"Pay as you go" databases. You may reach this limit either by inserting a single
huge value or appending many small values to an entry. This entry can be a
String, List, Set, Hash etc. Read (`GET`, `LRANGE`, `HMGET`, `ZRANGE` etc) and
delete (`DEL`, `LPOP`, `HDEL`, `SREM` etc) requests will not be affected.

### Solution-1

You can split your data into smaller chunks and store them as separate entries
with different keys.

### Solution-2

You can upgrade your database to Pro as it has higher limits. Also
you can submit quota increase request in the console or contact
[support@upstash.com](mailto:support@upstash.com) about the options with higher max record size limit.


# ERR max request size exceeded



### Symptom

The client gets an exception similar to:

```
ReplyError: ERR max request size exceeded
```

### Diagnosis

Your command exceeds the max request size which is `1Mb` for "Free" and "Pay as
you go" databases.

### Solution-1

You can split your data into smaller chunks and send them in separate commands.

### Solution-2

You can upgrade your database to Pro as it has higher limits. Also
you can submit quota increase request in the console or you can contact
[support@upstash.com](mailto:support@upstash.com) about the options with higher max request size limit.

<Note>
  max-request-size-limit is about the size of a single request. Your data
  structure (like list, set) can exceed the max request size limit without any
  problem. If you try to load all elements in the list with a single request
  then it can throw the max-request-size-limit exception.
</Note>


# NOAUTH Authentication Required



### Symptom

The client can not connect to the database throwing an exception similar to:

```
[ioredis] Unhandled error event:
ReplyError: NOAUTH Authentication required
```

### Diagnosis

The server does not let you connect because the password is missing in your
connection parameters.

### Solution

Check your connection parameters and ensure they contain the password. If you
are using ioredis (Redis client) with a Redis URL, check the URL format. ioredis
requires a colon before the password. The format for IORedis - TLS enabled

```
rediss://:YOUR_PASSWORD@YOUR_ENDPOINT:YOUR_PORT
```

The format for IORedis - TLS disabled

```
redis://:YOUR_PASSWORD@YOUR_ENDPOINT:YOUR_PORT
```

You can copy the correct client configuration from Upstash console clicking on
**Redis Connect** button.


# ERR XReadGroup is cancelled



### Symptom

The client gets an exception similar to:

```
ReplyError: ERR XReadGroup is cancelled. Pending Entries List limit per consumer is about to be reached. Limit: 1000, Current PEL size: 90, Requested Read: 20, Key: mstream, Group: group1, Consumer: consumer1.
```

### Diagnosis

Pending Entries List of the stream for the consumer is full. For each consumer
in a consumer group, there is a pending entries list. This list keeps the
messages that are delivered to a consumer but not yet acknowledged via
[XACK](https://redis.io/commands/xack/). This list is populated via
[XREADGROUP](https://redis.io/commands/xreadgroup/).

### Solution

Acknowledge the consumed messages via [XACK](https://redis.io/commands/xack/)
from the list of the associated group and consumer.


# Deploy a Serverless API with AWS CDK and AWS Lambda



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/aws-cdk-typescript" horizontal>
  You can find the project source code on GitHub.
</Card>

In this tutorial, we will implement a Serverless API using AWS Lambda and we
will deploy it using AWS CDK. We will use Typescript as the CDK language. It
will be a view counter where we keep the state in Redis.

### What is AWS CDK?

AWS CDK is an interesting project which allows you to provision and deploy AWS
infrastructure with code. Currently TypeScript, JavaScript, Python, Java,
C#/.Net and Go are supported. You can compare AWS CDK with following technologies:

*   AWS CloudFormation
*   AWS SAM
*   Serverless Framework

The above projects allows you to set up the infrastructure with configuration
files (yaml, json) while with AWS CDK, you set up the resources with code. For
more information about CDK see the related
[AWS Docs](https://docs.aws.amazon.com/cdk/latest/guide/home.html).

### Prerequisites

*   Complete all steps in [Getting started with the AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html)

### Project Setup

Create and navigate to a directory named `counter-cdk`. The CDK CLI uses this directory name to name things in your CDK code, so if you decide to use a different name, don't forget to make the appropriate changes when applying this tutorial.

```shell
mkdir counter-cdk && cd counter-cdk
```

Initialize a new CDK project.

```shell
cdk init app --language typescript
```

Install `@upstash/redis`.

```shell
npm install @upstash/redis
```

### Counter Function Setup

Create `/api/counter.ts`.

```ts /api/counter.ts
import { Redis } from '@upstash/redis';

const redis = Redis.fromEnv();

export const handler = async function() {
    const count = await redis.incr("counter");
    return {
        statusCode: 200,
        body: JSON.stringify('Counter: ' + count),
    };
};
```

### Counter Stack Setup

Update `/lib/counter-cdk-stack.ts`.

```ts /lib/counter-cdk-stack.ts
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as nodejs from 'aws-cdk-lib/aws-lambda-nodejs';

export class CounterCdkStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const counterFunction = new nodejs.NodejsFunction(this, 'CounterFunction', {
      entry: 'api/counter.ts',
      handler: 'handler',
      runtime: lambda.Runtime.NODEJS_20_X,
      environment: {
        UPSTASH_REDIS_REST_URL: process.env.UPSTASH_REDIS_REST_URL || '',
        UPSTASH_REDIS_REST_TOKEN: process.env.UPSTASH_REDIS_REST_TOKEN || '',
      },
      bundling: {
        format: nodejs.OutputFormat.ESM,
        target: "node20",
        nodeModules: ['@upstash/redis'],
      },
    });

    const counterFunctionUrl = counterFunction.addFunctionUrl({
      authType: lambda.FunctionUrlAuthType.NONE,
    });

    new cdk.CfnOutput(this, "counterFunctionUrlOutput", {
      value: counterFunctionUrl.url,
    })
  }
}
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and export `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment.

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Deploy

Run in the top folder:

```shell
cdk synth
cdk bootstrap
cdk deploy
```

Visit the output URL.


# Autocomplete API with Serverless Redis



This tutorial implements an autocomplete API powered by serverless Redis. See
[the demo](https://auto-complete-example.vercel.app/) and
[API endpoint](https://wfgz7cju24.execute-api.us-east-1.amazonaws.com/query?term=ca)
and
[the source code](https://github.com/upstash/examples/tree/main/examples/auto-complete-api).

We will keep country names in a Redis Sorted set. In Redis sorted set, elements
with the same score are sorted lexicographically. So in our case, all country
names will have the same score, 0. We keep all prefixes of country and use ZRANK
to find the terms to suggest. See
[this blog post](https://oldblog.antirez.com/post/autocomplete-with-redis.html)
for the details of the algorithm.

### Step 1: Project Setup

<Note>
  I will use Serverless framework for this tutorial. You can also use [AWS
  SAM](/redis/tutorials/using_aws_sam)
</Note>

If you do not have it already install serverless framework via:
`npm install -g serverless`

In any folder run `serverless` as below:

```text
>> serverless

Serverless: No project detected. Do you want to create a new one? Yes
Serverless: What do you want to make? AWS Node.js
Serverless: What do you want to call this project? test-upstash

Project successfully created in 'test-upstash' folder.

You can monitor, troubleshoot, and test your new service with a free Serverless account.

Serverless: Would you like to enable this? No
You can run the “serverless” command again if you change your mind later.
```

Inside the project folder create a node project with the command:

```
npm init
```

Then install the redis client with:

```
npm install ioredis
```

### Step 2: API Implementation

Edit handler.js file as below. See
[the blog post](https://oldblog.antirez.com/post/autocomplete-with-redis.html)
for the details of the algorithm.

```javascript
var Redis = require("ioredis");
if (typeof client === "undefined") {
  var client = new Redis(process.env.REDIS_URL);
}
const headers = {
  "Access-Control-Allow-Origin": "*",
  "Access-Control-Allow-Credentials": true,
};

module.exports.query = async (event, context, callback) => {
  if (!event.queryStringParameters || !event.queryStringParameters.term) {
    return {
      statusCode: 400,
      headers: headers,
      body: JSON.stringify({
        message: "Invalid parameters. Term needed as query param.",
      }),
    };
  }
  let term = event.queryStringParameters.term.toUpperCase();
  let res = [];
  let rank = await client.zrank("terms", term);
  if (rank != null) {
    let temp = await client.zrange("terms", rank, rank + 100);
    for (const el of temp) {
      if (!el.startsWith(term)) {
        break;
      }
      if (el.endsWith("*")) {
        res.push(el.substring(0, el.length - 1));
      }
    }
  }
  return {
    statusCode: 200,
    headers: headers,
    body: JSON.stringify({
      message: "Query:" + event.queryStringParameters.term,
      result: res,
    }),
  };
};
```

### Step 3: Create database on Upstash

If you do not have one, create a database following this
[guide](../overall/getstarted). Copy the Redis URL by clicking `Redis Connect`
button inside database page. Copy the URL for ioredis as we use ioredis in our
application. Create .env file and paste your Redis URL:

```text
REDIS_URL=YOUR_REDIS_URL
```

<Snippet file="redis/ioredisnote.mdx" />

### Step 4: Initialize Database

We will initialize the database with country names. Copy and run initdb.js
script from
[here](https://github.com/upstash/examples/tree/main/examples/auto-complete-api/initdb.js).

We simply put the country names and all their prefixes to the sorted set.

```javascript
require('dotenv').config()
var Redis = require("ioredis");

var countries = [
    {"name": "Afghanistan", "code": "AF"},
    {"name": "Åland Islands", "code": "AX"},
    {"name": "Albania", "code": "AL"},
    {"name": "Algeria", "code": "DZ"},
    ...
]
var client = new Redis(process.env.REDIS_URL);

for (const country of countries) {
    let term = country.name.toUpperCase();
    let terms = [];

    for (let i = 1; i < term.length; i++) {
        terms.push(0);
        terms.push(term.substring(0, i));
    }
    terms.push(0);
    terms.push(term + "*");
    (async () => {
        await client.zadd("terms", ...terms)
    })();
}
```

### Step 5: Deploy Your Function

Edit `serverless.yml` as below and replace your Redis URL:

```yaml
service: auto-complete-api
# add this if you set REDIS_URL in .env
useDotenv: true
frameworkVersion: "2"

provider:
  name: aws
  runtime: nodejs14.x
  lambdaHashingVersion: 20201221
  environment:
    REDIS_URL: REPLACE_YOUR_REDIS_URL

functions:
  query:
    handler: handler.query
    events:
      - httpApi:
          path: /query
          method: get
          cors: true
```

In the project folder run:

```
serverless deploy
```

Now you can run your function with:

```shell
serverless invoke -f query -d '{ "queryStringParameters": {"term":"ca"}}'
```

It should give the following output:

```json
{
  "statusCode": 200,
  "headers": {
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Credentials": true
  },
  "body": "{\"message\":\"Query:ca\",\"result\":[\"CAMBODIA\",\"CAMEROON\",\"CANADA\",\"CAPE VERDE\",\"CAYMAN ISLANDS\"]}"
}
```

You can also test your function using AWS console. In your AWS Lambda section,
click on your function. Scroll down to the code sections and click on the `Test`
button on the top right. Use `{ "queryStringParameters": {"term":"ar"}}` as your
event data.

### Step 6: Run Your Function Locally

In your project folder run:

```shell
serverless invoke local -f query -d '{ "queryStringParameters": {"term":"ca"}}'
```

It should give the following output:

```json
{
  "statusCode": 200,
  "headers": {
    "Access-Control-Allow-Origin": "*",
    "Access-Control-Allow-Credentials": true
  },
  "body": "{\"message\":\"Query:ca\",\"result\":[\"CAMBODIA\",\"CAMEROON\",\"CANADA\",\"CAPE VERDE\",\"CAYMAN ISLANDS\"]}"
}
```


# Build Stateful Applications with AWS App Runner and Serverless Redis

This tutorial shows how to create a serverless and stateful application using AWS App Runner and Redis

AWS App Runner is a container service where AWS runs and scales your container
in a serverless way. The container storage is ephemeral so you should keep the
state in an external data store. In this tutorial we will build a simple
application which will keep the state on Redis and deploy the application to AWS
App Runner.

### The Stack

*   Serverless compute: AWS App Runner (Node.js)
*   Serverless data store: Redis via Upstash
*   Deployment source: github repo

### Project Setup

Create a directory for your project:

```
mkdir app_runner_example

cd app_runner_example
```

Create a node project and install dependencies:

```
npm init

npm install ioredis
```

Create a Redis DB from [Upstash](https://console.upstash.com). In the database
details page, copy the connection code (Node tab).

### The Code

In your node project folder, create server.js and copy the below code:

```javascript
var Redis = require("ioredis");
const http = require("http");

if (typeof client === "undefined") {
  var client = new Redis(process.env.REDIS_URL);
}

const requestListener = async function (req, res) {
  if (req.url !== "/favicon.ico") {
    let count = await client.incr("counter");
    res.writeHead(200);
    res.end("Page view:" + count);
  }
};

const server = http.createServer(requestListener);
server.listen(8080);
```

<Snippet file="redis/ioredisnote.mdx" />

As you see, the code simple increment a counter on Redis and returns the
response as the page view count.

### Deployment

You have two options to deploy your code to the App Runner. You can either share
your Github repo with AWS or register your docker image to ECR. In this
tutorial, we will share
[our Github repo](https://github.com/upstash/app_runner_example) with App
Runner.

Create a github repo for your project and push your code. In AWS console open
the App Runner service. Click on `Create Service` button. Select
`Source code repository` option and add your repository by connecting your
Github and AWS accounts.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/examples/apprunner/source.png" width="800" />
</Frame>

In the next page, choose `Nodejs 12` as your runtime, `npm install` as your
build command, `node server` as your start command and `8080` as your port.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/examples/apprunner/build.png" width="800" />
</Frame>

The next page configures your App Runner service. Set a name for your service.
Set your Redis URL that you copied from Upstash console as `REDIS_URL`
environment variable. Your Redis URL should be something like this:
`rediss://:d34baef614b6fsdeb01b25@us1-lasting-panther-33618.upstash.io:33618`
You can leave other settings as default.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/examples/apprunner/config.png" width="800" />
</Frame>

Click on `Create and Deploy` at the next page. Your service will be ready in a
few minutes. Click on the default domain, you should see the page with a view
counter as [here](https://xmzuanrpf3.us-east-1.awsapprunner.com/).

### App Runner vs AWS Lambda

*   AWS Lambda runs functions, App Runner runs applications. So with App Runner
    you do not need to split your application to functions.
*   App Runner is a more portable solution. You can move your application from App
    Runner to any other container service.
*   AWS Lambda price scales to zero, App Runner's does not. With App Runner you
    need to pay for an at least one instance unless you pause the system.

App Runner is great alternative when you need more control on your serverless
runtime and application. Check out
[this video](https://www.youtube.com/watch?v=x_1X_4j16A4) to learn more about
App Runner.


# Session Management on Google Cloud Run with Serverless Redis

This tutorial shows how to manage user sessions on Google Cloud Run using Serverless Redis.

Developers are moving their apps to serverless architectures and one of the most
common questions is
[how to store user sessions](https://stackoverflow.com/questions/57711095/are-users-sessions-on-google-cloud-run-apps-directed-to-the-same-instance).
You need to keep your state and session data in an external data store because
serverless environments are stateless by design. Unfortunately most of the
databases are not serverless friendly. They do not support per-request pricing
or they require heavy and persistent connections. These also explain the
motivations why we built Upstash. Upstash is a serverless Redis database with
per-request pricing, durable storage.

In this article I will write a basic web application which will run on Google
Cloud Run and keep the user sessions in Upstash Redis. Google Cloud Run provides
Serverless Container service which is also stateless. Cloud Run is more powerful
than serverless functions (AWS Lambda, Cloud Functions) as you can run your own
container. But you can not guarantee that the same container instance will
process the requests of the same user. So you need to keep the user session in
an external storage. Redis is the most popular choice to keep the session data
thanks to its speed and simplicity. Upstash gives you the serverless Redis
database which fits perfectly to your serverless stack.

If you want to store your session data manually on Redis, check
[here](https://upstash.com/docs/redis/tutorials/using_google_cloud_functions). But in
this article I will use [Express session](https://github.com/expressjs/session)
middleware which can work with Redis for user session management.

Here is the [live demo.](https://cloud-run-sessions-dr7fcdmn3a-uc.a.run.app)

Here is the
[source code](https://github.com/upstash/examples/tree/master/examples/cloud-run-sessions)

## The Stack

Serverless processing: Google Cloud Run

Serverless data: Upstash

Web framework: Express

## Project Setup

Create a directory for your project:

```
mkdir cloud-run-sessions

cd cloud-run-sessions
```

Create a node project and install dependencies:

```
npm init

npm install express redis connect-redis express-session
```

Create a Redis DB from [Upstash](https://console.upstash.com). In the database
details page, click the Connect button, copy the connection code (Node.js
node-redis).

If you do not have it already, install Google Cloud SDK as described
[here.](https://cloud.google.com/sdk/docs/install) Set the project and enable
Google Run and Build services:

```
gcloud config set project cloud-run-sessions

gcloud services enable run.googleapis.com

gcloud services enable cloudbuild.googleapis.com
```

## The Code

Create index.js and update as below:

```javascript
var express = require("express");
var parseurl = require("parseurl");
var session = require("express-session");
const redis = require("redis");

var RedisStore = require("connect-redis")(session);
var client = redis.createClient({
  // REPLACE HERE
});

var app = express();

app.use(
  session({
    store: new RedisStore({ client: client }),
    secret: "forest squirrel",
    resave: false,
    saveUninitialized: true,
  })
);

app.use(function (req, res, next) {
  if (!req.session.views) {
    req.session.views = {};
  }

  // get the url pathname
  var pathname = parseurl(req).pathname;

  // count the views
  req.session.views[pathname] = (req.session.views[pathname] || 0) + 1;
  next();
});

app.get("/", function (req, res, next) {
  res.send("you viewed this page " + req.session.views["/"] + " times");
});

app.get("/foo", function (req, res, next) {
  res.send("you viewed this page " + req.session.views["/foo"] + " times");
});

app.get("/bar", function (req, res, next) {
  res.send("you viewed this page " + req.session.views["/bar"] + " times");
});

app.listen(8080, function () {
  console.log("Example app listening on port 8080!");
});
```

Run the app: `node index.js`

Check [http://localhost:3000/foo](http://localhost:3000/foo) in different
browsers to validate it keeps the session.

Add the start script to your `package.json`:

```json
"scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "start": "node index"
  }
```

## Build

Create a Docker file (Dockerfile) in the project folder as below:

```
# Use the official lightweight Node.js 12 image.
# https://hub.docker.com/_/node
FROM node:12-slim

# Create and change to the app directory.
WORKDIR /usr/src/app

# Copy application dependency manifests to the container image.
# A wildcard is used to ensure both package.json AND package-lock.json are copied.
# Copying this separately prevents re-running npm install on every code change.
COPY package*.json ./

# Install dependencies.
RUN npm install

# Copy local code to the container image.
COPY . ./

# Run the web service on container startup.
CMD [ "npm", "start" ]
```

Build your container image:

```
gcloud builds submit --tag gcr.io/cloud-run-sessions/main
```

List your container images: `gcloud container images list`

Run the container locally:

```
gcloud auth configure-docker

docker run -d -p 8080:8080 gcr.io/cloud-run-sessions/main:v0.1
```

In case you have an issue on docker run, check
[here](https://cloud.google.com/container-registry/docs/troubleshooting).

## Deploy

Run:

```
gcloud run deploy cloud-run-sessions \

  --image gcr.io/cloud-run-sessions/main:v0.1 \

  --platform managed \

  --region us-central1 \

  --allow-unauthenticated
```

This command should give you
[the URL of your application](https://cloud-run-sessions-dr7fcdmn3a-uc.a.run.app)
as below:

```
Deploying container to Cloud Run service [cloud-run-sessions] in project [cloud-run-sessions] region [us-central1]

  ✓ Deploying... Done.

  ✓ Creating Revision...

  ✓ Routing traffic...

  ✓ Setting IAM Policy...

Done.

Service [cloud-run-sessions] revision [cloud-run-sessions-00006-dun] has been deployed and is serving 100 percent of traffic.

Service URL: https://cloud-run-sessions-dr7fcdmn3a-uc.a.run.app
```

## Cloud Run vs Cloud Functions

I have developed two small prototypes with both. Here my impression:

*   Simplicity: Cloud functions are simpler to deploy as it does not require any
    container building step.
*   Portability: Cloud Run leverages your container, so anytime you can move your
    application to any containerized system. This is a plus for Cloud Run.
*   Cloud Run looks more powerful as it runs your own container with more
    configuration options. It also allows running longer tasks (can be extended to
    60 minutes)
*   Cloud Run looks more testable as you can run the container locally. Cloud
    Functions require a simulated environment.

Personally, I see Cloud Functions as a pure serverless solution where Cloud Run
is a hybrid solution. I would choose Cloud functions for simple, self contained
tasks or event driven solutions. If my use case is more complex with
portability/testability requirements, then I would choose Cloud Run.


# Use Redis in Cloudflare Workers



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/cloudflare-workers-with-typescript" horizontal>
  You can find the project source code on GitHub.
</Card>

This tutorial showcases using Redis with REST API in Cloudflare Workers. We will
write a sample edge function (Cloudflare Workers) which will show a custom
greeting depending on the location of the client. We will load the greeting
message from Redis so you can update it without touching the code.

### Why Upstash?

*   Cloudflare Workers does not allow TCP connections. Upstash provides REST API
    on top of the Redis database.
*   Upstash is a serverless offering with per-request pricing which fits for edge
    and serverless functions.
*   Upstash Global database provides low latency all over the world.

### Prerequisites

1.  Install the Cloudflare Wrangler CLI with `npm install wrangler --save-dev`

### Project Setup

Create a Cloudflare Worker with the following options:

```shell
➜  tutorials > ✗ npx wrangler init
╭ Create an application with Cloudflare Step 1 of 3
│
├ In which directory do you want to create your application?
│ dir ./greetings-cloudflare
│
├ What would you like to start with?
│ category Hello World example
│
├ Which template would you like to use?
│ type Hello World Worker
│
├ Which language do you want to use?
│ lang TypeScript
│
├ Copying template files
│ files copied to project directory
│
├ Updating name in `package.json`
│ updated `package.json`
│
├ Installing dependencies
│ installed via `npm install`
│
╰ Application created

╭ Configuring your application for Cloudflare Step 2 of 3
│
├ Installing @cloudflare/workers-types
│ installed via npm
│
├ Adding latest types to `tsconfig.json`
│ added @cloudflare/workers-types/2023-07-01
│
├ Retrieving current workerd compatibility date
│ compatibility date 2024-10-22
│
├ Do you want to use git for version control?
│ no git
│
╰ Application configured
```

Install Upstash Redis:

```shell
cd greetings-cloudflare
npm install @upstash/redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `wrangler.toml` file.

```toml wrangler.toml
# existing config

[vars]
UPSTASH_REDIS_REST_URL = <YOUR_URL>
UPSTASH_REDIS_REST_TOKEN = <YOUR_TOKEN>
```

Using CLI Tab in the Upstash Console, add some greetings to your database:

![CLI Tab](https://mintlify.s3-us-west-1.amazonaws.com/upstash/redis/tutorials/img/examples/tutorial-cloudflare-workers.png)

### Greetings Function Setup

Update `src/index.ts`:

```typescript src/index.ts
import { Redis } from '@upstash/redis/cloudflare';

type RedisEnv = {
	UPSTASH_REDIS_REST_URL: string;
	UPSTASH_REDIS_REST_TOKEN: string;
};

export default {
	async fetch(request: Request, env: RedisEnv) {
		const redis = Redis.fromEnv(env);

		const country = request.headers.get('cf-ipcountry');
		if (country) {
			const greeting = await redis.get<string>(country);
			if (greeting) {
				return new Response(greeting);
			}
		}

		return new Response('Hello!');
	},
};
```

The code tries to find out the user's location checking the "cf-ipcountry"
header. Then it loads the corresponding greeting for that location using the Redis
REST API.

### Run Locally

Run the following command to start your dev session:

```shell
npx wrangler dev
```

Visit [localhost:8787](http://localhost:8787)

### Build and Deploy

Build and deploy your app to Cloudflare:

```shell
npx wrangler deploy
```

Visit the output url.


# Backendless Coin Price List with GraphQL API, Serverless Redis and Next.JS



In this tutorial, we will develop a simple coin price list using GraphQL API of
Upstash. You can call the application `backendless` because we will access the
database directly from the client (javascript). See the
[code](https://github.com/upstash/examples/tree/master/examples/coin-price-list).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/coin-price-list/coin-price-list.png" width="800" />
</Frame>

## Motivation

We want to give a use case where you can use the GraphQL API without any backend
code. The use case is publicly available read only data for web applications
where you need low latency. The data is updated frequently by another backend
application, you want your users to see the last updated data. Examples:
Leaderboards, news list, blog list, product list, top N items in the homepages.

### `1` Project Setup:

Create a Next application: `npx create-next-app`.

Install Apollo GraphQL client: `npm i @apollo/client`

### `2` Database Setup

If you do not have one, create a database following this
[guide](../overall/getstarted). Connect your database via Redis CLI and run:

```shell
rpush coins '{ "name" : "Bitcoin", "price": 56819, "image": "https://s2.coinmarketcap.com/static/img/coins/64x64/1.png"}' '{ "name" : "Ethereum", "price": 2130, "image": "https://s2.coinmarketcap.com/static/img/coins/64x64/1027.png"}' '{ "name" : "Cardano", "price": 1.2, "image": "https://s2.coinmarketcap.com/static/img/coins/64x64/2010.png"}' '{ "name" : "Polkadot", "price": 35.96, "image": "https://s2.coinmarketcap.com/static/img/coins/64x64/6636.png"}' '{ "name" : "Stellar", "price": 0.506, "image": "https://s2.coinmarketcap.com/static/img/coins/64x64/512.png"}'
```

### `3` Code

In the Upstash console, copy the read only access key in your API configuration
page (GraphQL Explorer > Configure API). In the `_app.js` create the Apollo
client and replace the your access key as below:

<Warning>
  You need to use Read Only Access Key, because the key will be accessible
  publicly.
</Warning>

```javascript
import "../styles/globals.css";
import {
  ApolloClient,
  ApolloProvider,
  createHttpLink,
  InMemoryCache,
} from "@apollo/client";

const link = createHttpLink({
  uri: "https://graphql-us-east-1.upstash.io/",
  headers: {
    Authorization: "Bearer YOUR_ACCESS_TOKEN",
  },
});
const client = new ApolloClient({
  uri: "https://graphql-us-east-1.upstash.io/",
  cache: new InMemoryCache(),
  link,
});

function MyApp({ Component, pageProps }) {
  return (
    <ApolloProvider client={client}>
      <Component {...pageProps} />{" "}
    </ApolloProvider>
  );
}

export default MyApp;
```

Edit `index.js` as below:

```javascript
import Head from "next/head";
import styles from "../styles/Home.module.css";
import { gql, useQuery } from "@apollo/client";
import React from "react";

const GET_COIN_LIST = gql`
  query {
    redisLRange(key: "coins", start: 0, stop: 6)
  }
`;

export default function Home() {
  let coins = [];
  const { loading, error, data } = useQuery(GET_COIN_LIST);

  if (!loading && !error) {
    for (let x of data.redisLRange) {
      let dd = JSON.parse(x);
      coins.push(dd);
    }
  }

  return (
    <div className={styles.container}>
      <Head>
        <title>Create Next App</title>
        <link rel="icon" href="/favicon.ico" />
      </Head>

      <main className={styles.main}>
        <h3 className={styles.title}>Coin Price List</h3>

        <div className={styles.grid}>
          <table className={styles.coins}>
            <tbody>
              {!loading ? (
                coins.map((item, ind) => (
                  <tr key={ind}>
                    <td>
                      <img src={item.image} width="25" />
                    </td>
                    <td>{item.name}</td>
                    <td className={styles.price}>${item.price}</td>
                  </tr>
                ))
              ) : (
                <tr>
                  <td>
                    <img src="/loader.gif" />
                  </td>
                </tr>
              )}
            </tbody>
          </table>
        </div>
      </main>

      <footer className={styles.footer}>
        <p className={styles.description}>
          <a href="https://docs.upstash.com"> Click for the tutorial </a>
        </p>
      </footer>
    </div>
  );
}
```

### `4` Run

Run your application locally: `npm run dev`

### `5` Live!

Go to [http://localhost:3000/](http://localhost:3000/) 🎉


# Build a Leaderboard API At Edge using Cloudflare Workers and Redis

This tutorial shows how to build a Leaderboard API At Edge using Cloudflare Workers and Redis.

With edge functions, it is possible to run your backend at the closest location
to your users. Cloudflare Workers and Fastly Compute\@Edge runs your function at
the closest location to your user using their CDN infrastructure.

In this article we will implement a very common web use case at Edge. We will
implement a leaderboard API without any backend servers, containers or even
serverless functions. We will just use edge functions. Leaderboard will have the
following APIs:

*   addScore: Adds a score with the player's name. This will write the score to
    the Upstash Redis directly from the Edge functions.
*   getLeaderBoard: Returns the list of score-player pairs. This call will first
    check the Edge cache. If the leaderboard does not exist at the Edge Cache then
    it will fetch it from the Upstash Redis.

<Info>Edge caching is deprecated. Please use global database instead.</Info>

## Project Setup

In this tutorial, we will use Cloudflare Workers and Upstash. You can create a
free database from [Upstash Console](https://console.upstash.com). Then create a
Workers project using
[Wrangler](https://developers.cloudflare.com/workers/get-started/guide).

Install wrangler: `npm install -g @cloudflare/wrangler`

Authenticate: `wrangler login` or `wrangler config`

Then create a project: `wrangler generate edge-leaderboard`

Open `wrangler.toml`. Run `wrangler whoami` and copy/paste your account id to
your wrangler.toml.

Find your REST token from database details page in the
[Upstash Console](https://console.upstash.com). Copy/paste your token to your
wrangler toml as below:

```
name = "edge-leaderboard"
type = "javascript"

account_id = "REPLACE_YOUR_ACCOUNT_ID"
workers_dev = true
route = ""
zone_id = ""

[vars]
TOKEN = "REPLACE_YOUR_UPSTASH_REST_TOKEN"
```

## The Code

The only file we need is the Workers Edge function. Update the index.js as
below:

```javascript
addEventListener("fetch", (event) => {
  event.respondWith(handleRequest(event.request));
});

async function handleRequest(request) {
  if (request.method === "GET") {
    return getLeaderboard();
  } else if (request.method === "POST") {
    return addScore(request);
  } else {
    return new Response("Invalid Request!");
  }
}

async function getLeaderboard() {
  let url =
    "https://us1-full-bug-31874.upstash.io/zrevrange/scores/0/1000/WITHSCORES/?_token=" +
    TOKEN;
  let res = await fetch(new Request(url), {
    cf: {
      cacheTtl: 10,
      cacheEverything: true,
      cacheKey: url,
    },
  });
  return res;
}

async function addScore(request) {
  const { searchParams } = new URL(request.url);
  let player = searchParams.get("player");
  let score = searchParams.get("score");
  let url =
    "https://us1-full-bug-31874.upstash.io/zadd/scores/" +
    score +
    "/" +
    player +
    "?_token=" +
    TOKEN;
  let res = await fetch(url);
  return new Response(await res.text());
}
```

We route the request to two methods: if it is a GET, we return the leaderboard.
If it is a POST, we read the query parameters and add a new score.

In the getLeaderboard() method, you will see we pass a cache configuration to
the fetch() method. It caches the result of the request at the Edge for 10
seconds.

## Test The API

In your project folder run `wrangler dev`. It will give you a local URL. You can
test your API with curl:

Add new scores:

```shell
curl -X POST http://127.0.0.1:8787\?player\=messi\&score\=13

curl -X POST http://127.0.0.1:8787\?player\=ronaldo\&score\=17

curl -X POST http://127.0.0.1:8787\?player\=benzema\&score\=18
```

Get the leaderboard:

```shell
curl -w '\n Latency: %{time_total}s\n' http://127.0.0.1:8787
```

Call the “curl -w '\n Total: %{time_total}s\n'
[http://127.0.0.1:8787](http://127.0.0.1:8787)” multiple times. You will see the
latency becomes very small with the next calls as the cached result comes from
the edge.

If you wait more than 10 seconds then you will see the latency becomes higher as
the cache is evicted and the function fetches the leaderboard from the Upstash
Redis again.

## Deploy The API

First change the type in the wrangler.toml to `webpack`

```
name = "edge-leaderboard"
type = "webpack"
```

Then, run `wrangler publish`. Wrangler will output the URL. If you want to
deploy to a custom domain see
[here](https://developers.cloudflare.com/workers/get-started/guide#optional-configure-for-deploying-to-a-registered-domain).


# Express Session with Serverless Redis

This tutorial shows how to use Upstash as the session storage of your Express application.

This tutorial shows how to use Serverless Redis as your session storage for your
Express Applications.

See the
[code](https://github.com/upstash/examples/tree/main/examples/express-session-with-redis)

### Step-1: Create Project

Create a folder for your project and run: `npm init`

### Step-2: Install Redis and Express

In your project folder run:
`npm install express redis connect-redis express-session`

### Step-3: Create a Redis (Upstash) Database For Free

Create a database as described [here](../overall/getstarted).

### Step-4: index.js

In Upstash console, click the `Connect` button, copy the connection code
(Node.js node-redis). Create index.js file as below and replace the Redis
connection part.

```javascript
var express = require("express");
var parseurl = require("parseurl");
var session = require("express-session");
const redis = require("redis");

var RedisStore = require("connect-redis")(session);
var client = redis.createClient({
  // REPLACE HERE
});

var app = express();

app.use(
  session({
    store: new RedisStore({ client: client }),
    secret: "forest squirrel",
    resave: false,
    saveUninitialized: true,
  })
);

app.use(function (req, res, next) {
  if (!req.session.views) {
    req.session.views = {};
  }

  // get the url pathname
  var pathname = parseurl(req).pathname;

  // count the views
  req.session.views[pathname] = (req.session.views[pathname] || 0) + 1;
  next();
});

app.get("/foo", function (req, res, next) {
  res.send("you viewed this page " + req.session.views["/foo"] + " times");
});

app.get("/bar", function (req, res, next) {
  res.send("you viewed this page " + req.session.views["/bar"] + " times");
});

app.listen(3000, function () {
  console.log("Example app listening on port 3000!");
});
```

### Step-5: Run the app

`node index.js`

### Step-6: Check your work

Open [http://localhost:3000/bar](http://localhost:3000/bar) and [http://localhost:3000/foo](http://localhost:3000/foo) in different
browsers. Check if the view-count is incrementing as expected.

### FAQ:

**There is a default session storage of express-session. Why do I need Redis?**

*Default session store loses the session data when the process crashes.
Moreover, it does not scale. You can not utilize multiple web servers to serve
your sessions.*

**Why Upstash?**

*You can use any Redis offering or self hosted one. But Upstash's serverless
approach with per-request-pricing will help you to minimize your cost with zero
maintenance.*

**How to configure the session storage?**

*See [here](https://github.com/expressjs/session#readme)*


# Serverless Golang API with Redis



This tutorial shows how to build a serverless API with Golang and Redis. The API
will simply count the page views and show it in JSON format.

### The Stack

*   Serverless compute: AWS Lambda (Golang)
*   Serverless data store: Redis via Upstash
*   Deployment tool: AWS SAM

### Prerequisites:

*   An AWS account for AWS Lambda functions.
*   Install AWS SAM CLI tool as described here to create and deploy the project.
*   An Upstash account for serverless Redis.

### Step 1: Init the Project

Run the sam init and then

*   Select AWS Quick Start Templates
*   Select 4 - go1.x
*   Enter your project name: go-redis-example
*   Select 1 - Hello World Example SAM will generate your project in a new folder.

### Step 2: Install a Redis Client

Our only dependency is redis client. Install go-redis via
`go get github.com/go-redis/redis/v8`

### Step 3: Create a Redis Database

Create a Redis database from Upstash console. Free tier should be enough. It is
pretty straight forward but if you need help, check
[getting started](../overall/getstarted) guide. In the database details page,
click the Connect button. You will need the endpoint and password in the next
step.

### Step 4: The function Code

Edit the hello-world>main.go as below:

```go
package main
import (
    "context"
    "encoding/json"
    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
    "github.com/go-redis/redis/v8"
    "strconv"
)
var ctx = context.Background()

type MyResponse struct {
    Count string `json:"count:"`
}

var rdb = redis.NewClient(&redis.Options{
    Addr: "YOUR_REDIS_ENDPOINT",
    Password: "YOUR_REDIS_PASSWORD",
    DB: 0,
})

func handler(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
    count, err := rdb.Incr(ctx, "count").Result()
    if err != nil {
        panic(err)
    }
    response := &MyResponse{
        Count: strconv.FormatInt(count, 10),
    }
    body, err := json.Marshal(response)
    return events.APIGatewayProxyResponse{
        Headers: map[string]string{"Content-Type": "application/json"},
        Body: string(body),
        StatusCode: 200,
        }, nil
}

func main() {
    lambda.Start(handler)
}
```

Replace the "YOUR\_REDIS\_ENDPOINT" and "YOUR\_REDIS\_PASSWORD" with your database's
endpoint and password which you created in the Step 3. The code simply
increments a counter in Redis database and returns its value in json format.

### Step 5: Deployment

Now we are ready to deploy our API. First build it via `sam build`. Then run the
command `sam local start-api`. You can check your API locally on
[http://127.0.0.1:3000/hello](http://127.0.0.1:3000/hello)

<br />

<br /> If it is working, you can deploy your app to AWS by running `sam deploy --guided`.
Enter a stack name and pick your region. After confirming changes, the deployment
should begin. The command will output API Gateway endpoint URL, check the API in
your browser. You can also check your deployment on your AWS console. You will see
your function has been created.

<br />

<br /> Click on your function, you will see the code is uploaded and API Gateway
is configured.

### Notes

*   Check the template.yaml file. You can add new functions and APIGateway
    endpoints editing this file.
*   It is a good practice to keep your Redis endpoint and password as environment
    variable.
*   You can use [serverless framework](https://www.serverless.com/) instead of AWS
    SAM to deploy your function.


# Build a Serverless Histogram API with Redis

This tutorial shows how to build a histogram API with Redis.

While developing
[the latency benchmark for the serverless databases (DynamoDB, FaunaDB, Upstash)](https://blog.upstash.com/latency-comparison),
I wished there was an API where I will record the latency numbers and get the
histogram back. In this tutorial, I will build such an API where you can record
your latency values from any application. It will be a REST API with following
methods:

*   record: Records numeric values into the histogram.
*   get: Returns the histogram object.

### Motivation

I will show how easy to develop a generic API using AWS Lambda and Serverless
Redis.

See [code](https://github.com/upstash/examples/tree/master/examples/histogram-api).

### `1` Create a Redis (Upstash) Database

Create a database as [getting started](../overall/getstarted)

### `2` Serverless Project Setup

If you do not have it already install serverless framework via:
`npm install -g serverless`

In any folder run `serverless` as below:

```text
>> serverless

Serverless: No project detected. Do you want to create a new one? Yes
Serverless: What do you want to make? AWS Node.js
Serverless: What do you want to call this project? histogram-api

Project successfully created in 'histogram-api' folder.

You can monitor, troubleshoot, and test your new service with a free Serverless account.

Serverless: Would you like to enable this? No
You can run the “serverless” command again if you change your mind later.
```

<Note>
  See [Using AWS SAM](/redis/tutorials/using_aws_sam), if you prefer AWS SAM
  over Serverless Framework.
</Note>

Inside the project folder create a node project with the command:

```
npm init
```

Then install the redis client and histogram library with:

```
npm install ioredis

npm install hdr-histogram-js
```

Update the `serverless.yml` as below. Copy your Redis URL from console and
replace below:

```yaml
service: histogram-api
frameworkVersion: "2"

provider:
  name: aws
  runtime: nodejs12.x
  lambdaHashingVersion: 20201221
  environment:
    REDIS_URL: REPLACE_YOUR_URL_HERE

functions:
  record:
    handler: handler.record
    events:
      - httpApi:
          path: /record
          method: post
          cors: true
  get:
    handler: handler.get
    events:
      - httpApi:
          path: /get
          method: get
          cors: true
```

<Snippet file="redis/ioredisnote.mdx" />

### `3` Code

Edit handler.js as below.

```javascript
const hdr = require("hdr-histogram-js");
const Redis = require("ioredis");
if (typeof client === "undefined") {
  var client = new Redis(fixUrl(process.env.REDIS_URL));
}
const headers = {
  "Access-Control-Allow-Origin": "*",
  "Access-Control-Allow-Credentials": true,
};
const SIZE = 10000;

module.exports.get = async (event) => {
  if (!event.queryStringParameters || !event.queryStringParameters.name) {
    return {
      statusCode: 400,
      headers: headers,
      body: JSON.stringify({
        message: "Invalid parameters. Name is needed.",
      }),
    };
  }
  const name = event.queryStringParameters.name;
  const data = await client.lrange(name, 0, SIZE);
  const histogram = hdr.build();
  data.forEach((item) => {
    histogram.recordValue(item);
  });

  return {
    statusCode: 200,
    body: JSON.stringify({
      histogram: histogram,
    }),
  };
};

module.exports.record = async (event) => {
  let body = JSON.parse(event.body);
  if (!body || !body.name || !body.values) {
    return {
      statusCode: 400,
      headers: headers,
      body: JSON.stringify({
        message: "Invalid parameters. Name and values are needed.",
      }),
    };
  }
  const name = body.name;
  const values = body.values;
  await client.lpush(name, values);
  return {
    statusCode: 200,
    body: JSON.stringify({
      message: "Success",
      name: name,
    }),
  };
};

function fixUrl(url) {
  if (!url) {
    return "";
  }
  if (url.startsWith("redis://") && !url.startsWith("redis://:")) {
    return url.replace("redis://", "redis://:");
  }
  if (url.startsWith("rediss://") && !url.startsWith("rediss://:")) {
    return url.replace("rediss://", "rediss://:");
  }
  return url;
}
```

We have two serverless functions above. `get` takes `name` as parameter and
loads a list from Redis. Then builds a histogram using the values in the list.

The `record` function takes `name` and `values` as parameters. It adds the
`values` to the Redis List with name `name`.

The `get` function calculates the histogram over the latest 10000 latency
records. Update the SIZE parameter to change this number.

The `fixUrl` is a helper method which corrects the Redis url format.

### `4` Deploy and Try the API

Deploy your functions with:

```bash
serverless deploy
```

The command will deploy two functions and output two endpoints. Try the
endpoints with setting parameters as below:

Record latency numbers to `perf-test-1`:

```shell
curl --header "Content-Type: application/json" -d "{\"name\":\"perf-test-1\", \"values\": [90,80,34,97,93,45,49,57,99,12]}" https://v7xx4aa2ib.execute-api.us-east-1.amazonaws.com/record
```

Get the histogram for `perf-test-1`:

```shell
curl https://v7xx4aa2ib.execute-api.us-east-1.amazonaws.com/get?name=perf-test-1
```

### Batching

It can be costly to call a remote function each time for latency calculation. In
your application, you should keep an array or queue as a buffer for the latency
numbers, then submit them in batches to the API when the array reaches the batch
size. Something like below:

```javascript
let records = [];
let batchSize = 1000;
function recordLatency(value) {
  records.push(value);
  if (records.length >= batchSize) {
    // the below submits the records to the API then empties the records array.
    submitToAPI(records);
  }
}
```


# Job Processing and Event Queue with Serverless Redis

This tutorial shows how to use Upstash Redis for job/task processing.

### Motivation

Serverless functions are great for many tasks with their dynamic scaling and
flexible pricing models. But when you have a task which is composed of long
running complex steps, it is not feasible to run it in a single serverless
function. A simple solution is simply to offload complicated tasks from the
serverless function. You can process those asynchronously in your preferred
environment, this can be other serverless functions, serverless containers or
traditional server based processes too. To offload your tasks, you need a
reliable event queue. In this article we will use Upstash Redis for this
purpose.

### Scenario

You are developing a `New Employee Registration` form for your company. Saving
employee records to the database is the easy part. Here possible things to do:

*   Create accounts (email, slack etc).
*   Send email to the employee.
*   Send email to the hiring manager and others.
*   Create a JIRA ticket for the IT department so they will set up the employee’s
    computer.

This list can be longer for bigger companies.

*   You want the form to be responsive. You do want a new employee to wait for
    minutes after clicking submit.
*   The above steps are subject to change. You do not want to update your code
    whenever a new procedure is added.

Decoupling the side procedures will solve the above issues. When a new employee
is registered, you can push a new event to the related task queue; then another
process will consume the task.

Let’s build the sample application:

### Project Setup

The project will consist of two modules:

*   Producer will be a serverless function which will receive input parameters
    required to register a new employee. It will also produce events for the task
    queue.
*   Consumer will be a worker application which will continuously consume the task
    queue.

(See
[the source code](https://github.com/upstash/examples/tree/main/examples/task-queue))

### Tech Stack

*   AWS Lambda for Serverless computing
*   [Upstash](https://upstash.com) as Serverless Redis
*   [Bull](https://github.com/OptimalBits/bull) as task queue implementation
*   [Serverless framework](https://www.serverless.com/) for project deployment

### Upstash Database

You can create a free Redis database from [Upstash](https://docs.upstash.com/).

After creating a database, copy the endpoint, port and password as you will need
in the next steps.

### Producer Code

Our producer will be the serverless function which will get the request
parameters and produce the task for the queue. In the real world this code
should do things like saving to the database but I will not implement this for
the sake of simplicity.

1- Create a Serverless project by `serverless` command.

```shell
➜  serverless

Serverless: No project detected. Do you want to create a new one? Yes

Serverless: What do you want to make? AWS Node.js

Serverless: What do you want to call this project? producer

Project successfully created in 'producer' folder.

You can monitor, troubleshoot, and test your new service with a free Serverless account.

Serverless: Would you like to enable this? No

You can run the “serverless” command again if you change your mind later.
```

2- Install [bull](https://github.com/OptimalBits/bull):

`npm install bull`

3- Function code:

```javascript
var Queue = require("bull");

var settings = {
  stalledInterval: 300000, // How often check for stalled jobs (use 0 for never checking).
  guardInterval: 5000, // Poll interval for delayed jobs and added jobs.
  drainDelay: 300, // A timeout for when the queue is in drained state (empty waiting for jobs).
};

module.exports.hello = async (event) => {
  var taskQueue = new Queue(
    "employee registration",
    {
      redis: {
        port: 32016,
        host: "us1-upward-ant-32016.upstash.io",
        password: "ake4ff120d6b4216df220736be7eab087",
        tls: {},
      },
    },
    settings
  );
  await taskQueue.add({ event: event });

  // TODO save the employee record to a database
  return { message: "New employee event enqueued! 34", event };
};
```

Note1: Do not forget to replace your own Redis endpoint, port and password.
Remove the TLS part if you disabled TLS.

Note2: We give extra parameters (settings) to the event queue (Bull), so it will
not exploit Upstash quotas. Update the interval parameters depending on your
tolerance to event latency.

### Consumer Code

We will write a basic Node application to consume the events. Create a new
directory and run `npm init` and `npm install bull`. Then create index.js as
below:

```javascript
var Queue = require("bull");

var settings = {
  stalledInterval: 300000, // How often check for stalled jobs (use 0 for never checking).
  guardInterval: 5000, // Poll interval for delayed jobs and added jobs.
  drainDelay: 300, // A timeout for when the queue is in drained state (empty waiting for jobs).
};

var taskQueue = new Queue(
  "employee registration",
  {
    redis: {
      port: 32016,
      host: "us1-upward-ant-32016.upstash.io",
      password: "ake4ff120d6b4216df220736be7eab087",
      tls: {},
    },
  },
  settings
);

taskQueue
  .process(function (job, done) {
    console.log(job.data);
    // TODO process the new employee event
    done();
  })
  .catch((err) => {
    console.log(err);
  });
```

Note1: Do not forget to replace your own Redis endpoint, port and password.
Remove the TLS part if you disabled TLS.

Note2: We give extra parameters (settings) to the event queue (Bull), so it will
not exploit Upstash quotas. Update the interval parameters depending on your
tolerance to event latency.

### Test the Application

First run the consumer application with

`node index`

To test the producer code, run:

```shell
serverless invoke local -f hello -d "{name:'Bill Gates', email:'bill@upstash.com', position:'Developer', date:'20210620'}"
```

You will see producer will log as below:

![alt\_text](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/examples/producer.png "image_tooltip")

And consumer will log as below:

![alt\_text](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/examples/consumer.png "image_tooltip")


# Next.js with Redis



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/nextjs-app-router" horizontal>
  You can find the project source code on GitHub.
</Card>

<Info>
  This tutorial uses Next.js App Router. If you want to use Pages Router, check out our [Pages Router tutorial](/redis/quickstarts/nextjs-pages-router).
</Info>

This tutorial uses Redis as state store for a Next.js application. We simply add
a counter that pulls the data from Redis.

### Project Setup

Let's create a new Next.js application with App Router and install `@upstash/redis` package.

```shell
npx create-next-app@latest
cd my-app
npm install @upstash/redis
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Home Page Setup

Update `/app/page.tsx`:

```tsx /app/page.tsx
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv();

export default async function Home() {
  const count = await redis.incr("counter");
  return (
    <div className="flex h-screen w-screen items-center justify-center">
      <h1 className="text-4xl font-bold">Counter: {count}</h1>
    </div>
  )
}
```

### Run & Deploy

Run the app locally with `npm run dev`, check `http://localhost:3000/`

Deploy your app with `vercel`

<Info>
  You can also integrate your Vercel projects with Upstash using Vercel
  Integration module. Check [this article](../howto/vercelintegration).
</Info>


# Building a Serverless Notification API for Your Web Application with Redis

This tutorial shows how to create a Serverless Notification API for Your Web Application with Redis.

Notifications and announcements help you communicate with your web site
visitors. It is not feasible to update your code and redeploy your website each
time you want to show a new message. It may also be too much investment to set
up a backend and maintain it to just serve these notifications. In this article,
we will build a website which will load the notification message directly from
the Redis database without a backend.

### Backendless? How is that possible?

Yes, we will not use any backend service, even a serverless function. We will
access Redis from the client side directly. This is possible with the read only
REST API provided by Upstash.

### Requirements

*   The page will display a notification if the user has not already seen the
    notification before.
*   The page will only show the latest notification.

Check out
[the code here](https://github.com/upstash/examples/tree/master/examples/serverless-notification-api).

### Project Setup

I will create a React application but you can use any other web framework. It
will simply call the Redis REST API and show the message as a notification.

Create the app:

```shell
npx create-react-app serverless-notification-api
```

Install a toast component to show the notification:

```shell
npm install --save react-toastify
```

Create a free database from [Upstash](https://console.upstash.com/) and copy the
REST url and read only token. You should switch the Read-Only Token setting. In
the database details page, click on the `Read-Only Token` switch.

![alt\_text](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/examples/restapi.png "Rest Api")

### Implementation

The logic is simple. We will keep the notifications in a Redis Sorted Set. We
will keep a version (integer) in the local storage. We will use the versions as
scores in the sorted set. Each notification message will have a version (score)
and the higher score means the newer message. At each page load, we will query
the Redis sorted set to load the messages which have higher scores than the
locally stored version. After loading a notification message I will set my local
version equal to the latest notification’s version. This will prevent showing
the same notification to the same users more than once. Here the implementation:

```javascript
import logo from "./logo.svg";
import "./App.css";
import { toast, ToastContainer } from "react-toastify";
import "react-toastify/dist/ReactToastify.css";
import { useEffect } from "react";

function App() {
  useEffect(() => {
    async function fetchData() {
      try {
        let version = localStorage.getItem("notification-version");
        version = version ? version : 0;
        const response = await fetch(
          "REPLACE_UPSTASH_REDIS_REST_URL/zrevrangebyscore/messages/+inf/" +
            version +
            "/WITHSCORES/LIMIT/0/1",
          {
            headers: {
              Authorization: "Bearer REPLACE_UPSTASH_REDIS_REST_TOKEN",
            },
          }
        );
        const res = await response.json();
        const v = parseInt(res.result[1]);
        if (v) {
          localStorage.setItem("notification-version", v + 1);
        }
        toast(res.result[0]);
      } catch (e) {
        console.error(e);
      }
    }
    fetchData();
  });

  return (
    <div className="App">
      <header className="App-header">
        <img src={logo} className="App-logo" alt="logo" />
        <p>
          Edit <code>src/App.js</code> and save to reload.
        </p>
        <a
          className="App-link"
          href="https://reactjs.org"
          target="_blank"
          rel="noopener noreferrer"
        >
          Learn React
        </a>
      </header>
      <ToastContainer />
    </div>
  );
}

export default App;
```

### How to Add New Notification Messages

You can simply add new messages to the Redis sorted set with a highest score so
it will be displayed to the user with page loads. For our application the name
of the sorted set is `messages`.

![alt\_text](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/examples/notif/cli.png "cli")

You can also remove a message using the [ZREM](https://redis.io/commands/zrem)
command.

### Conclusion

You do not need a backend to access Upstash Redis thanks to the REST API. You
can expose the token with your client side application, as the token only allows
read-only access. This helps developers to build applications without backend
for many use cases where the data is already available publicly.


# Nuxt with Redis

This tutorial shows how to use Upstash inside your Nuxt application.

This tutorial uses Redis as state store for a Nuxt application. In it, we will build an application
which simply increments a counter and saves & fetches the last increment time.

See [code](https://github.com/upstash/examples/tree/master/examples/nuxt-with-redis) and
[demo](https://nuxt-with-redis.vercel.app)

### `1` Create Nuxt.js Project

Run this in terminal

```bash
npx nuxi@latest init nuxtjs-with-redis
```

Go to the new directory `nuxtjs-with-redis` and install `@upstash/redis`:

```
npm install @upstash/redis
```

### `2` Create a Upstash Redis database

Next, you will need an Upstash Redis database. You can follow
[our guide for creating a new database](https://upstash.com/docs/redis/overall/getstarted).

### `3` Set up environment variables

Copy the `.env.example` file in this directory to `.env`

```bash
cp .env.example .env
```

Then, set the following environment variables:

```
UPSTASH_REDIS_REST_URL=""
UPSTASH_REDIS_REST_TOKEN=""
```

You can get the values of these env variables on the page of your Redis database.

### `4` Define the endpoint

Next, we will define the endpoint which will call Redis:

```javascript title="server/api/increment.ts"
import { defineEventHandler } from "h3";
import { Redis } from "@upstash/redis";

// Initialize Redis
const redis = new Redis({
  url: process.env.UPSTASH_REDIS_REST_URL || "",
  token: process.env.UPSTASH_REDIS_REST_TOKEN || ""
});

export default defineEventHandler(async () => {
  const identifier = "api_call_counter";

  try {
    // Increment the API call counter and get the updated value
    const count = await redis.incr(identifier);

    // Optionally, you can also retrieve other information like the last time it was called
    const lastCalled = await redis.get("last_called");
    const lastCalledAt = lastCalled || "Never";

    // Store the current timestamp as the last called time
    await redis.set("last_called", new Date().toISOString());

    // Return the count and last called time
    return {
      success: true,
      count: count,
      lastCalled: lastCalledAt,
    };
  } catch (error) {
    console.error("Redis error:", error);
    return {
      success: false,
      message: "Error interacting with Redis",
    };
  }
});
```

### `5` Run

Finally, we can run the application and call our endpoint:

```bash
npm run dev
```

If you are using [our example app](https://github.com/upstash/examples/tree/master/examples/nuxt-with-redis),
you can simply click the `Increment` button to run the endpoint we defined.

Otherwise, you can simply make a curl request:

```
curl http://localhost:3000/api/increment
```

When you make the request, you should see something like this:

```
{
  "success": true,
  "count": 166,
  "lastCalled": "2024-10-10T07:04:42.381Z"
}
```

### Notes:

*   For best performance the application should run in the same region with the
    Redis database's region.


# Redis as a Cache for Your FastAPI App



### Introduction

In this tutorial, we’ll learn how to use Redis to add caching to a FastAPI application. By caching API responses in Redis, we can reduce database queries, improve response times, and ensure that frequently requested data is delivered quickly.

We’ll create a simple FastAPI app that fetches weather data from an external API. The app will store the results in Redis, so the next time someone requests the same data, it can be returned from the cache instead of making a new API request. Let’s get started!

### Environment Setup

First, install FastAPI, the Upstash Redis client, and an ASGI server:

```shell
pip install fastapi upstash-redis uvicorn[standard]
```

### Database Setup

Create a Redis database using the [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli), and export the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment:

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

We'll also need to generate a `WEATHER_API_KEY` from [Weather API Website](https://www.weatherapi.com) for free and we will export it.

```shell
export WEATHER_API_KEY=<YOUR_KEY>
```

You can also use `python-dotenv` to load environment variables from your `.env` file.

### Application Setup

In this example, we will build an API that fetches weather data and caches it in Redis.

Create `main.py`:

```python main.py
from fastapi import FastAPI
from upstash_redis import Redis
import requests
import os

app = FastAPI()

# Connect to Redis using environment variables
redis = Redis.from_env()

# Mock API endpoint for weather data
WEATHER_API_URL = "https://api.weatherapi.com/v1/current.json"
API_KEY = os.getenv("WEATHER_API_KEY")

@app.get("/weather/{city}")
def get_weather(city: str):
    cache_key = f"weather:{city}"
    
    # Check if the data exists in cache
    cached_data = redis.get(cache_key)
    if cached_data:
        return {"source": "cache", "data": cached_data}
    
    # Fetch data from external API
    response = requests.get(f"{WEATHER_API_URL}?key={API_KEY}&q={city}")
    weather_data = response.json()
    
    # Store the data in Redis cache with a 10-minute expiration
    redis.setex(cache_key, 600, weather_data)
    
    return {"source": "api", "data": weather_data}
```

### Running the Application

Run the FastAPI app with Uvicorn:

```shell
uvicorn main:app --reload
```

To test the application you can visit `http://127.0.0.1:8000/weather/istanbul` in your browser or use curl to get the weather data for Istanbul. The first request will fetch the data from the weather API and cache it, and subsequent requests will return the cached data until the cache expires after 10 minutes.

To monitor your data in Redis, you can use the [Upstash Console](https://console.upstash.com) and check out the Data Browser tab.

### Code Breakdown

1.  **Redis Setup**: We use `Redis.from_env()` to initialize the Redis connection using the environment variables. Redis will store the weather data with city names as cache keys.

2.  **Cache Lookup**: When a request is made to the `/weather/{city}` endpoint, we check if the weather data is already cached by looking up the `weather:{city}` key in Redis. If the data is found in cache, it's returned immediately.

3.  **Fetching External Data**: If the data is not in cache, the app sends a request to the external weather API to fetch the latest data. The response is then cached using `redis.setex()`, which stores the data with a 10-minute expiration.

4.  **Cache Expiration**: We use a 10-minute TTL (time-to-live) for the cached weather data to ensure it's periodically refreshed. After the TTL expires, the next request will fetch fresh data from the external API and store it in cache again.


# Multithreaded Web Scraping with Redis Caching



In this tutorial, we’ll build a multithreaded web scraper in Python that leverages Redis for caching responses to minimize redundant HTTP requests. The scraper will be capable of handling groups of URLs across multiple threads while caching responses to reduce load and improve performance.

### Database Setup

Create a Redis database using the [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli), and add `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your '.env' file:

```plaintext
UPSTASH_REDIS_REST_URL=your_upstash_redis_url
UPSTASH_REDIS_REST_TOKEN=your_upstash_redis_token
```

This file will be used to load environment variables.

### Installation

First, install the necessary libraries using the following command:

```bash
pip install threading requests upstash-redis python-dotenv
```

### Code Explanation

We’ll create a multithreaded web scraper that performs HTTP requests on a set of grouped URLs. Each thread will check if the response for a URL is cached in Redis. If the URL has been previously requested, it will retrieve the cached response; otherwise, it will perform a fresh HTTP request, cache the result, and store it for future requests.

### Code

Here’s the complete code:

```python
import threading
import requests
from upstash_redis import Redis
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

# Initialize Redis client
redis = Redis.from_env()

# Group URLs by thread, with one or two overlapping URLs across groups
urls_to_scrape_groups = [
    [
        'https://httpbin.org/delay/1',
        'https://httpbin.org/delay/4',
        'https://httpbin.org/delay/2',
        'https://httpbin.org/delay/5',
        'https://httpbin.org/delay/3',
    ],
    [
        'https://httpbin.org/delay/5',  # Overlapping URL
        'https://httpbin.org/delay/6',
        'https://httpbin.org/delay/7',
        'https://httpbin.org/delay/2',  # Overlapping URL
        'https://httpbin.org/delay/8',
    ],
    [
        'https://httpbin.org/delay/3',  # Overlapping URL
        'https://httpbin.org/delay/9',
        'https://httpbin.org/delay/10',
        'https://httpbin.org/delay/4',  # Overlapping URL
        'https://httpbin.org/delay/11',
    ],
]

class Scraper(threading.Thread):
    def __init__(self, urls):
        threading.Thread.__init__(self)
        self.urls = urls
        self.results = {}

    def run(self):
        for url in self.urls:
            cache_key = f"url:{url}"
            
            # Attempt to retrieve cached response
            cached_response = redis.get(cache_key)
            
            if cached_response:
                print(f"[CACHE HIT] {self.name} - URL: {url}")
                self.results[url] = cached_response
                continue  # Skip to the next URL if cache is found
            
            # If no cache, perform the HTTP request
            print(f"[FETCHING] {self.name} - URL: {url}")
            response = requests.get(url)
            if response.status_code == 200:
                self.results[url] = response.text
                # Store the response in Redis cache
                redis.set(cache_key, response.text)
            else:
                print(f"[ERROR] {self.name} - Failed to retrieve {url}")
                self.results[url] = None

def main():
    threads = []
    for urls in urls_to_scrape_groups:
        scraper = Scraper(urls)
        threads.append(scraper)
        scraper.start()

    # Wait for all threads to complete
    for scraper in threads:
        scraper.join()

    print("\nScraping results:")
    for scraper in threads:
        for url, result in scraper.results.items():
            print(f"Thread {scraper.name} - URL: {url} - Response Length: {len(result) if result else 'Failed'}")

if __name__ == "__main__":
    main()
```

### Explanation

1.  **Threaded Scraper Class**: The `Scraper` class is a subclass of `threading.Thread`. Each thread takes a list of URLs and iterates over them to retrieve or fetch their responses.

2.  **Redis Caching**:
    *   Before making an HTTP request, the scraper checks if the response is already in the Redis cache.
    *   If a cached response is found, it uses that response instead of making a new request, marked with `[CACHE HIT]` in the logs.
    *   If no cached response exists, it fetches the content from the URL, caches the result in Redis, and proceeds.

3.  **Overlapping URLs**:
    *   Some URLs are intentionally included in multiple groups to demonstrate the cache functionality across threads. Once a URL’s response is cached by one thread, another thread retrieving the same URL will pull it from the cache instead of re-fetching.

4.  **Main Function**:
    *   The `main` function initiates and starts multiple `Scraper` threads, each handling a group of URLs.
    *   It waits for all threads to complete before printing the results.

### Running the Code

Once everything is set up, run the script using:

```bash
python your_script_name.py
```

### Sample Output

You will see output similar to this:

```
[FETCHING] Thread-1 - URL: https://httpbin.org/delay/1
[FETCHING] Thread-1 - URL: https://httpbin.org/delay/4
[CACHE HIT] Thread-2 - URL: https://httpbin.org/delay/5
[FETCHING] Thread-3 - URL: https://httpbin.org/delay/3
...
```

### Benefits of Using Redis Cache

Using Redis as a cache reduces the number of duplicate requests, particularly for overlapping URLs. It allows for quick retrieval of previously fetched responses, enhancing performance and reducing load.


# Rate Limiting for Your FastAPI App



### Introduction

In this tutorial, we’ll learn how to add rate limiting to a FastAPI application using Upstash Redis. Rate limiting is essential for controlling API usage and with Upstash Redis, you can easily implement rate limiting to protect your API resources.

We’ll set up a simple FastAPI app and apply rate limiting to its endpoints. With Upstash Redis, we’ll configure a fixed window rate limiter that allows a specific number of requests per given time period.

### Environment Setup

First, install FastAPI, the Upstash Redis client, the Upstash rate limiting package, and an ASGI server:

```shell
pip install fastapi upstash-redis upstash-ratelimit uvicorn[standard]
```

### Database Setup

Create a Redis database using the [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli), and export the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment:

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

You can also use `python-dotenv` to load environment variables from your `.env` file.

### Application Setup

In this example, we will build an API endpoint that is rate-limited to a certain number of requests per time window. If the limit is exceeded (e.g., by making more than 10 requests in 10 seconds), the API will return an HTTP 429 error with the message "Rate limit exceeded. Please try again later."

Create `main.py`:

```python main.py
from fastapi import FastAPI, HTTPException
from upstash_ratelimit import Ratelimit, FixedWindow
from upstash_redis import Redis
from dotenv import load_dotenv
import requests

# Load environment variables from .env file
load_dotenv()

# Initialize the FastAPI app
app = FastAPI()

# Initialize Redis client
redis = Redis.from_env()

# Create a rate limiter that allows 10 requests per 10 seconds
ratelimit = Ratelimit(
    redis=redis,
    limiter=FixedWindow(max_requests=10, window=10),  # 10 requests per 10 seconds
    prefix="@upstash/ratelimit"
)

@app.get("/expensive_calculation")
def expensive_calculation():
    identifier = "api"  # Common identifier for rate limiting all users equally
    response = ratelimit.limit(identifier)

    if not response.allowed:
        raise HTTPException(status_code=429, detail="Rate limit exceeded. Please try again later.")
    
    # Placeholder for a resource-intensive operation
    result = do_expensive_calculation()
    return {"message": "Here is your result", "result": result}

# Simulated function for an expensive calculation
def do_expensive_calculation():
    return "Expensive calculation result"

# Test function to check rate limiting
def test_rate_limiting():
    url = "http://127.0.0.1:8000/expensive_calculation"
    success_count = 0
    fail_count = 0

    # Attempt 15 requests in quick succession
    for i in range(15):
        response = requests.get(url)
        
        if response.status_code == 200:
            success_count += 1
            print(f"Request {i+1}: Success - {response.json()['message']}")
        elif response.status_code == 429:
            fail_count += 1
            print(f"Request {i+1}: Failed - Rate limit exceeded")

        # Small delay to avoid flooding

    print("\nTest Summary:")
    print(f"Total Successful Requests: {success_count}")
    print(f"Total Failed Requests due to Rate Limit: {fail_count}")

if __name__ == "__main__":
    # Run the FastAPI app in a separate thread or terminal with:
    # uvicorn main:app --reload

    # To test rate limiting after the server is running
    test_rate_limiting()
```

### Running the Application

Run the FastAPI app with Uvicorn:

```shell
uvicorn main:app --reload
```

Run the test function to check the rate limiting:

```shell
python main.py
```

### Testing Rate Limiting

Here's the output you should see when running the test function:

```
Request 1: Success - Here is your result
Request 2: Success - Here is your result
Request 3: Success - Here is your result
Request 4: Success - Here is your result
Request 5: Success - Here is your result
Request 6: Success - Here is your result
Request 7: Success - Here is your result
Request 8: Success - Here is your result
Request 9: Success - Here is your result
Request 10: Success - Here is your result
Request 11: Failed - Rate limit exceeded
Request 12: Failed - Rate limit exceeded
Request 13: Failed - Rate limit exceeded
Request 14: Failed - Rate limit exceeded
Request 15: Failed - Rate limit exceeded

Test Summary:
Total Successful Requests: 10
Total Failed Requests due to Rate Limit: 5
```

### Code Breakdown

1.  **Redis and Rate Limiter Setup**:
    *   We initialize a `Redis` client with `Redis.from_env()` using environment variables for configuration.
    *   We create a rate limiter using `Ratelimit` with a `FixedWindow` limiter that allows 10 requests per 10 seconds. The `prefix` option is set to organize the Redis keys used by the rate limiter.

2.  **Rate Limiting the Endpoint**:
    *   For the `/expensive_calculation` endpoint, the rate limiter is applied by calling `ratelimit.limit(identifier)`.
    *   The `identifier` variable uniquely identifies this rate limit. You could use user-specific identifiers (like user IDs) to implement per-user limits.
    *   If the request exceeds the allowed limit, an HTTP 429 error is returned.

3.  **Expensive Calculation Simulation**:
    *   The `do_expensive_calculation` function simulates a resource-intensive operation. In real scenarios, this could represent database queries, file processing, or other time-consuming tasks.

### Benefits of Rate Limiting with Redis

Using Redis for rate limiting helps control API usage across multiple instances of your app, making it highly scalable. Redis’s in-memory storage provides fast access to rate-limiting data, ensuring minimal performance impact on your API.


# Building a URL Shortener with Redis



### Introduction

In this tutorial, we’ll build a simple URL shortener using Redis and Python. The short URL service will generate a random short code for each URL, store it in Redis, and allow users to retrieve the original URL using the short code. We’ll also implement an expiration time for each shortened URL, making it expire after a specified period.

### Environment Setup

First, install the necessary dependencies, including Upstash Redis and `python-dotenv` for environment variables:

```shell
pip install upstash-redis
```

### Database Setup

Create a Redis database using the [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli), and export the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REST_TOKEN` to your environment:

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

You can also use `python-dotenv` to load environment variables from a `.env` file:

```text .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Application Setup

In this example, we will build a URL shortener where each short URL will be stored in Redis with an expiration time.

Create `url_shortener.py`:

```py url_shortener.py
import string
import random
from upstash_redis import Redis
from dotenv import load_dotenv
import os

# Load environment variables
load_dotenv()

# Redis connection
redis = Redis.from_env()

# Characters to generate the short URL from
CHARS = string.ascii_letters + string.digits
BASE_URL = "https://short.url/"

# Function to generate a random string for the short URL
def generate_short_code(length=6):
    return ''.join(random.choices(CHARS, k=length))

# Function to shorten the URL with an expiration time
def shorten_url(url, expiration=3600):
    # Generate a random short code
    short_code = generate_short_code()
    # Save the short code in Redis
    redis.set(short_code, url, ex=expiration)
    return BASE_URL + short_code

# Function to get the original URL from the short URL
def get_original_url(short_code):
    return redis.get(short_code)

# Example usage
if __name__ == "__main__":
    original_url = "https://example.com/my-very-long-url"

    # Shorten the URL
    short_url = shorten_url(original_url, expiration=600)
    print(f"Shortened URL: {short_url}")

    # Get the original URL
    original_url = get_original_url(short_url.split("/")[-1])

    if original_url:
        print(f"Original URL: {original_url}")
    else:
        print("Short URL expired or not found")
```

### Running the Application

Simply run the Python script:

```shell
python url_shortener.py
```

This script will shorten a long URL, store the mapping in Redis, and print the shortened URL. It will then attempt to retrieve the original URL using the short code.

Here is an example output:

```shell
Shortened URL: https://short.url/0lSLFI
Original URL: https://example.com/my-very-long-url
```

To monitor your data in Redis, you can use the [Upstash Console](https://console.upstash.com) and check out the Data Browser tab.

### How to Use the URL Shortener for Web Applications

1.  Extract the short code from the shortened URL (e.g., `0lSLFI`).

2.  Look up the original URL in Redis using that code.

3.  Redirect the user to the original URL.

### Code Breakdown

1.  **Random Short Code Generation**: The `generate_short_code` function creates a random string of characters (letters and digits) that will serve as the short code for the URL.

2.  **Storing in Redis**: The `shorten_url` function takes the original URL and stores it in Redis using the randomly generated short code as the key. The `ex` parameter sets an expiration time (in seconds) for how long the shortened URL will be valid.

3.  **Retrieving the Original URL**: The `get_original_url` function takes the short code and looks it up in Redis to retrieve the original URL. If the short code doesn't exist (due to expiration or other reasons), it returns `None`.

4.  **Expiration Handling**: If the short code has expired, Redis automatically removes the entry, and the script will print a message indicating that the URL has expired or cannot be found.


# Serverless Python API with Redis



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/aws-cdk-python" horizontal>
  You can find the project source code on GitHub.
</Card>

This tutorial shows how to build a serverless API for Page View Counter with
Python and Redis. The API will the count page views and show it in JSON format.

### Prerequisites

*   Complete all steps in [Getting started with the AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html)

### Project Setup

Create and navigate to a directory named `counter-cdk`. CDK CLI uses this directory name to name things in your CDK code, so if you decide to use a different name, don't forget to make the appropriate changes when applying this tutorial.

```shell
mkdir counter-cdk && cd counter-cdk
```

Initialize a new CDK project.

```shell
cdk init app --language typescript
```

### Counter Function Setup

Create a folder named `api` under `lib`

```shell
mkdir lib/api
```

Create `/lib/api/requirements.txt`

```txt /lib/api/requirements.txt
upstash-redis
```

Create `/lib/api/index.py`

```py /lib/api/index.py
from upstash_redis import Redis

redis = Redis.from_env()

def handler(event, context):
    count = redis.incr('counter')
    return {
        'statusCode': 200,
        'body': f'Counter: {count}'
    }
```

### Counter Stack Setup

Update `/lib/counter-cdk-stack.ts`

```ts /lib/counter-cdk-stack.ts
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as path from 'path';

export class CounterCdkStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const counterFunction = new lambda.Function(this, 'CounterFunction', {
      code: lambda.Code.fromAsset(path.join(__dirname, 'api'), {
        bundling: {
          image: lambda.Runtime.PYTHON_3_9.bundlingImage,
          command: [
            'bash', '-c',
            'pip install -r requirements.txt -t /asset-output && cp -au . /asset-output'
          ],
        },
      }),
      runtime: lambda.Runtime.PYTHON_3_9,
      handler: 'index.handler',
      environment: {
        UPSTASH_REDIS_REST_URL: process.env.UPSTASH_REDIS_REST_URL || '',
        UPSTASH_REDIS_REST_TOKEN: process.env.UPSTASH_REDIS_REST_TOKEN || '',
      },
    });

    const counterFunctionUrl = counterFunction.addFunctionUrl({
      authType: lambda.FunctionUrlAuthType.NONE,
    });

    new cdk.CfnOutput(this, "counterFunctionUrlOutput", {
      value: counterFunctionUrl.url,
    })
  }
}
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and export `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` to your environment.

```shell
export UPSTASH_REDIS_REST_URL=<YOUR_URL>
export UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Deploy

Run in the top folder:

```shell
cdk synth
cdk bootstrap
cdk deploy
```

Visit the output url.


# AWS Lambda Rate Limiting with Serverless Redis



In this tutorial, we will deploy a AWS Lambda function ratelimited based on the client's IP address using `@upstash/ratelimit` and Serverless Framework.

### Prerequisites

1.  Install the Serverless Framework with `npm i serverless -g`

### Project Setup

Create a Serverless Framework application with the following options:

```shell
➜  tutorials > ✗ serverless
Serverless ϟ Framework

Welcome to Serverless Framework V.4

Create a new project by selecting a Template to generate scaffolding for a specific use-case.

✔ Select A Template: · AWS / Node.js / HTTP API

✔ Name Your Project: · ratelimit-serverless

✔ Template Downloaded

✔ Create Or Select An Existing App: · Create A New App

✔ Name Your New App: · ratelimit-serverless

Your new Service "ratelimit-serverless" is ready. Here are next steps:

• Open Service Directory: cd ratelimit-serverless
• Install Dependencies: npm install (or use another package manager)
• Deploy Your Service: serverless deploy
```

```shell
cd ratelimit-serverless
```

Create `package.json` with `@upstash/ratelimit` as a dependency:

```json package.json
{
    "dependencies": {
      "@upstash/ratelimit": "latest",
      "@upstash/redis": "latest"
    }
  }
```

Install the dependencies:

```shell
npm install
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Ratelimited Function Setup

Update `handler.js`:

```js handler.js
const { Ratelimit } = require("@upstash/ratelimit");
const { Redis } = require("@upstash/redis");

const ratelimit = new Ratelimit({
  redis: Redis.fromEnv(),
  limiter: Ratelimit.slidingWindow(10, "10 s"),
  prefix: "@upstash/ratelimit",
  analytics: true,
});

exports.ratelimit = async (event) => {
  const identifier = event.requestContext.http.sourceIP;
  const { success, limit, remaining, pending } = await ratelimit.limit(
    identifier
  );
  const response = {
    success: success,
    limit: limit,
    remaining: remaining,
  };

  // pending is a promise for handling the analytics submission
  await pending;

  if (!success) {
    return {
      statusCode: 429,
      body: JSON.stringify(response),
    };
  }
  return {
    statusCode: 200,
    body: JSON.stringify(response),
  };
};
```

Update `serverless.yml` to pass the environment variables:

```yaml serverless.yml
service: ratelimit-serverless

provider:
  name: aws
  runtime: nodejs20.x
  environment:
    UPSTASH_REDIS_REST_URL: ${env:UPSTASH_REDIS_REST_URL}
    UPSTASH_REDIS_REST_TOKEN: ${env:UPSTASH_REDIS_REST_TOKEN}

functions:
  ratelimit:
    handler: handler.ratelimit
    events:
      - httpApi:
          path: /
          method: get
```

### Developing

Run the following command to start your dev session.

```shell
serverless dev
```

### Deployment

Run the following command to deploy your service.

```shell
serverless deploy
```

Visit the output url.


# Serverless Redisson

This tutorial shows how to use Upstash with Redisson client.

This tutorial shows how to use Upstash with Redisson client.

See [code](https://github.com/upstash/examples/tree/master/examples/redisson)

### `1` Create a Maven Project

Create a maven project and copy the following dependency to your pom.xml

```xml
<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson</artifactId>
    <version>3.15.4</version>
</dependency>
```

### `2` Create a Redis (Upstash) Database

Create a database as [getting started](../overall/getstarted)

### `3` Code

Create your java file and replace Redis password and URL below. If you enabled
TLS, the endpoint should start with `rediss`

```typescript
public class Main {

    public static void main(String[] args) {
        Config config = new Config();
        config.useSingleServer().setPassword("YOUR_PASSWORD")
                // use "rediss://" for SSL connection
                .setAddress("YOUR_ENDPOINT");
        RedissonClient redisson = Redisson.create(config);
        RMap<String, String> map = redisson.getMap("map");
        map.put("foo", "bar");
        System.out.println(map.get("foo"));
    }
}
```


# Roadmap Voting App with Serverless Redis

This is a single page application powered by upstash and next.js.

<Info>
  We have developed an advanced version of Roadmap Voting App where users should
  log in to request or vote for a new feature. See it live version in [Upstash
  Roadmap](https://roadmap.upstash.com). See [the blog
  post](https://blog.upstash.com/roadmap-application) to learn about it. The
  below example allows users to request features anonymously.
</Info>

In this tutorial we will write a single page application which uses Redis as
state store in a Next.js application.

The example is a basic roadmap voting application where users enter and vote for
feature requests. You can check the complete application in
[Upstash Roadmap page](https://roadmap.upstash.com)

### Deploy Yourself

You can check the source code of the complete application
[here](https://github.com/upstash/serverless-examples/tree/master/roadmap-voting-app).
Thanks to [Upstash\&Vercel integration](https://vercel.com/integrations/upstash),
you can deploy the application yourself with zero cost/code by clicking below:
[![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/git/external?repository-url=https%3A%2F%2Fgithub.com%2Fupstash%2Fserverless-tutorials%2Ftree%2Fmaster%2Froadmap-voting-app\&env=LOGO\&envDescription=Enter%20URL%20for%20your%20project%2Fcompany%20logo\&envLink=https%3A%2F%2Fdocs.upstash.com%2Fdocs%2Ftutorials%2Froadmap_voting_app\&project-name=roadmap-voting\&repo-name=roadmap-voting\&demo-title=Roadmap%20Voting\&demo-description=Roadmap%20Voting%20Page%20for%20Your%20Project\&demo-url=https%3A%2F%2Froadmap.upstash.com\&integration-ids=oac_V3R1GIpkoJorr6fqyiwdhl17)

### Create Next.js Project

We will use Next.js as web framework. So let's create a next.js app and install
the redis client first.

`npx create-next-app nextjs-with-redis`

`npm install ioredis`

### index.js

Our application will be a single page. We will list the features with their
order of votes. There will be 3 actions available for the page user:

*   The user will suggest a new feature.
*   The user will vote up an existing feature.
*   The user will enter their email to be notified of a release of any feature.

The below are the parts that handles all those. If you want to check the full
page see
[here](https://github.com/upstash/serverless-tutorials/blob/master/roadmap-voting-app/pages/index.js)

```javascript
import Head from 'next/head'
import { ToastContainer, toast } from 'react-toastify';
import * as React from "react";

class Home extends React.Component {
    ...
    refreshData() {
        fetch("api/list")
            .then(res => res.json())
            .then(
                (result) => {
                    this.setState({
                        isLoaded: true,
                        items: result.body
                    });
                    this.inputNewFeature.current.value = "";
                },
                (error) => {
                    this.setState({
                        isLoaded: true,
                        error
                    });
                }
            )
    }

    vote(event, title) {
        const requestOptions = {
            method: 'POST',
            headers: {'Content-Type': 'application/json'},
            body: JSON.stringify({"title": title})
        };
        console.log(requestOptions);
        fetch('api/vote', requestOptions)
            .then(response => response.json()).then(data => {
                console.log(data)
                if(data.error) {
                    toast.error(data.error, {hideProgressBar: true, autoClose: 3000});
                } else {
                    this.refreshData()
                }
        })
    }

    handleNewFeature(event) {
        const requestOptions = {
            method: 'POST',
            headers: {'Content-Type': 'application/json'},
            body: JSON.stringify({"title": this.inputNewFeature.current.value})
        };
        fetch('api/create', requestOptions)
            .then(response => response.json()).then(data => {
            if(data.error) {
                toast.error(data.error, {hideProgressBar: true, autoClose: 5000});
            } else {
                toast.info("Your feature has been added to the list.", {hideProgressBar: true, autoClose: 3000});
                this.refreshData()
            }
        });
        event.preventDefault();
    }

    handleNewEmail(event) {
        const requestOptions = {
            method: 'POST',
            headers: {'Content-Type': 'application/json'},
            body: JSON.stringify({"email": this.inputEmail.current.value})
        };
        console.log(requestOptions);
        fetch('api/addemail', requestOptions)
            .then(response => response.json()).then(data => {
            if(data.error) {
                toast.error(data.error, {hideProgressBar: true, autoClose: 3000});
            } else {
                toast.info("Your email has been added to the list.", {hideProgressBar: true, autoClose: 3000});
                this.refreshData()
            }
        });
        event.preventDefault();
    }
}
export default Home;
```

### APIs

With Next.js, you can write server-side APIs within your project. We will have 4
apis:

*   list features
*   vote a feature
*   add a new feature
*   add email

Now let's examine these API implementations:

#### list.js

The list API connects to the Redis and fetches feature requests ordered by their
scores (votes) from the Sorted Set `roadmap`.

<Snippet file="redis/ioredisnote.mdx" />

```javascript
import { fixUrl } from "./utils";
import Redis from "ioredis";

module.exports = async (req, res) => {
  let redis = new Redis(fixUrl(process.env.REDIS_URL));
  let n = await redis.zrevrange("roadmap", 0, 100, "WITHSCORES");
  let result = [];
  for (let i = 0; i < n.length - 1; i += 2) {
    let item = {};
    item["title"] = n[i];
    item["score"] = n[i + 1];
    result.push(item);
  }

  redis.quit();

  res.json({
    body: result,
  });
};
```

#### create.js

This API connects to the Redis server and add a new element to the sorted set
(roadmap) . We use "NX" flag together with ZADD, so a user will not be able to
overwrite an existing feature request with the same title.

```javascript
import Redis from "ioredis";
import { fixUrl } from "./utils";

module.exports = async (req, res) => {
  let redis = new Redis(fixUrl(process.env.REDIS_URL));
  const body = req.body;
  const title = body["title"];
  if (!title) {
    redis.quit();
    res.json({
      error: "Feature can not be empty",
    });
  } else if (title.length < 70) {
    await redis.zadd("roadmap", "NX", 1, title);
    redis.quit();
    res.json({
      body: "success",
    });
  } else {
    redis.quit();
    res.json({
      error: "Max 70 characters please.",
    });
  }
};
```

#### vote.js

This API updates (increments) the score of the selected feature request. It also
keeps the IP addresses of the user to prevent multiple votes on the same feature
request.

```javascript
import Redis from "ioredis";
import { fixUrl } from "./utils";

module.exports = async (req, res) => {
  let redis = new Redis(fixUrl(process.env.REDIS_URL));
  const body = req.body;
  const title = body["title"];
  let ip = req.headers["x-forwarded-for"] || req.headers["Remote_Addr"] || "NA";
  let c = ip === "NA" ? 1 : await redis.sadd("s:" + title, ip);
  if (c === 0) {
    redis.quit();
    res.json({
      error: "You can not vote an item multiple times",
    });
  } else {
    let v = await redis.zincrby("roadmap", 1, title);
    redis.quit();
    res.json({
      body: v,
    });
  }
};
```

#### addemail.js

This API simply adds the user's email to the Redis Set. As the Set already
ensures the uniqueness, we only need to check if the input is a valid email.

```javascript
import Redis from "ioredis";
import { fixUrl } from "./utils";

module.exports = async (req, res) => {
  let redis = new Redis(fixUrl(process.env.REDIS_URL));

  const body = req.body;
  const email = body["email"];

  redis.on("error", function (err) {
    throw err;
  });

  if (email && validateEmail(email)) {
    await redis.sadd("emails", email);
    redis.quit();
    res.json({
      body: "success",
    });
  } else {
    redis.quit();
    res.json({
      error: "Invalid email",
    });
  }
};

function validateEmail(email) {
  const re =
    /^(([^<>()\[\]\\.,;:\s@"]+(\.[^<>()\[\]\\.,;:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/;
  return re.test(String(email).toLowerCase());
}
```

#### css and utils

[index.css](https://github.com/upstash/serverless-tutorials/blob/master/roadmap-voting-app/pages/index.css)
helps page to look good and
[utils.js](https://github.com/upstash/serverless-tutorials/blob/master/roadmap-voting-app/pages/api/utils.js)
fixes common mistakes on Redis URL.

### Notes:

*   If you deploy this application with Vercel; Vercel runs AWS Lambda functions
    to back the API implementations. For best performance choose the the same
    region for both Vercel functions and Upstash cluster.
*   You can access your database details via
    [Upstash Console](https://console.upstash.com)


# Serverless API with Java and Redis



In this tutorial, we will build a stateful serverless API using Java and Redis
on AWS Lambda. The API will simply count the page views and return it as HTTP
response.

### Prerequisites

1.  Install the Serverless Framework installed with an AWS account set up.

```shell
npm i serverless@3.39.0 -g
```

2.  Install JDK and not Java JRE. Set your JAVA\_HOME.
3.  Install Apache Maven.
4.  Create a free Serverless Redis database from
    [Upstash](https://console.upstash.com) as described
    [here](https://upstash.com/docs/redis/overall/getstarted).

### Project Setup

Create the project:

```shell
serverless create --template aws-java-maven --name counter-api -p aws-java-counter-api
```

```shell
cd aws-java-counter-api
```

Add `jedis` as dependency to the `pom.xml`:

```xml pom.xml
...
    <dependency>
        <groupId>redis.clients</groupId>
        <artifactId>jedis</artifactId>
        <version>3.6.0</version>
    </dependency>
...
```

Update `serverless.yml` as below:

```yaml serverless.yml
service: counter-api

frameworkVersion: '3'

provider:
  name: aws
  runtime: java17

package:
  artifact: target/hello-dev.jar

functions:
 hello:
   handler: com.serverless.Handler
   events:
     - httpApi:
         path: /hello
         method: get
```

### Counter Function Setup

Update `src/main/java/com/serverless/Handler.java` as below:

```java src/main/java/com/serverless/Handler.java
package com.serverless;

import java.util.Map;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import redis.clients.jedis.Jedis;

public class Handler implements RequestHandler<Map<String, Object>, ApiGatewayResponse> {
	@Override
	public ApiGatewayResponse handleRequest(Map<String, Object> input, Context context) {
		Jedis jedis = new Jedis("lasting-roughy-29092.upstash.io", 6379, true);
		jedis.auth("********");
		Long value = jedis.incr("counter");
		jedis.close();
		String message = "Hello World, Count:" + value;
		return ApiGatewayResponse.builder()
				.setStatusCode(200)
				.setObjectBody(message)
				.build();
	}
}
```

In the above code, you need to replace your Redis endpoint and password. You can copy Jedis connection code from the [Upstash Redis Console](https://console.upstash.com/redis) -> Your Database -> Connect to your database -> Java.

### Build and Deploy

Build your project:

```shell
mvn clean install
```

Deploy to AWS:

```shell
serverless deploy
```

Visit the output URL to see the counter in action.


# Using AWS SAM



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/aws-sam" horizontal>
  You can find the project source code on GitHub.
</Card>

This tutorial implements a serverless application and deploy it to AWS Lambda
using AWS SAM.

### Prerequisites

1.  [Complete AWS SAM Prerequisites](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/prerequisites.html)
2.  [Install the AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html)

### Project Setup

Create a SAM application with the following options:

```shell
➜  tutorials > ✗ sam init
Which template source would you like to use?
	1 - AWS Quick Start Templates
	2 - Custom Template Location
Choice: 1

Choose an AWS Quick Start application template
	1 - Hello World Example
	2 - Data processing
	3 - Hello World Example with Powertools for AWS Lambda
	4 - Multi-step workflow
	5 - Scheduled task
	6 - Standalone function
	7 - Serverless API
	8 - Infrastructure event management
	9 - Lambda Response Streaming
	10 - Serverless Connector Hello World Example
	11 - Multi-step workflow with Connectors
	12 - GraphQLApi Hello World Example
	13 - Full Stack
	14 - Lambda EFS example
	15 - DynamoDB Example
	16 - Machine Learning
Template: 1

Use the most popular runtime and package type? (Python and zip) [y/N]: y

Would you like to enable X-Ray tracing on the function(s) in your application?  [y/N]: N

Would you like to enable monitoring using CloudWatch Application Insights?
For more info, please view https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch-application-insights.html [y/N]: N

Would you like to set Structured Logging in JSON format on your Lambda functions?  [y/N]: N
```

```shell
cd sam-app
```

### Counter Function Setup

Update `/hello_world/requirements.txt` to include `upstash-redis`:

```txt /hello_world/requirements.txt
requests
upstash-redis
```

Update `/hello_world/app.py`:

```python /hello_world/app.py
from upstash_redis import Redis

redis = Redis.from_env()

def lambda_handler(event, context):
    count = redis.incr('counter')
    return {
        'statusCode': 200,
        'body': f'Counter: {count}'
    }
```

Update `/template.yaml` to pass Upstash Redis environment variables:

```yaml /template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sam-app

  Sample SAM Template for sam-app

Globals:
  Function:
    Timeout: 3
    MemorySize: 128

Parameters:
  UpstashRedisRestURL:
    Type: String
  UpstashRedisRestToken:
    Type: String

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
        - x86_64
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get
      Environment:
        Variables:
          UPSTASH_REDIS_REST_URL: !Ref UpstashRedisRestURL
          UPSTASH_REDIS_REST_TOKEN: !Ref UpstashRedisRestToken

Outputs:
  HelloWorldApi:
    Description: "API Gateway endpoint URL for Prod stage for Hello World function"
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn

```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli). Copy `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` for the next steps.

### Build

```shell
sam build
```

### Deploy

Enter your database related environment variables when prompted.

```shell
sam deploy --guided
```

Visit the HelloWorld API Gateway URL to see the response.


# Serverless Redis on Google Cloud Functions



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/google-cloud-functions" horizontal>
  You can find the project source code on GitHub.
</Card>

Google Cloud Functions is the second most popular serverless execution platform.
Similar to AWS Lambda it is stateless, namely you need to access external
resources to read or write your applications state. In this post, we will
introduce Redis as a database for your Google Cloud functions.

This tutorial shows how to build a serverless API with Redis on Google Cloud
Functions. The API will simply count the views and show it in JSON format.

### Prerequisites

1.  [Create a Google Cloud Project.](https://cloud.google.com/resource-manager/docs/creating-managing-projects)
2.  [Enable billing for your project.](https://cloud.google.com/billing/docs/how-to/verify-billing-enabled#console)
3.  Enable Cloud Functions, Cloud Build, Artifact Registry, Cloud Run, Logging, and Pub/Sub APIs.

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli). Copy `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` for the next steps.

### Counter Function Setup & Deploy

1.  Go to [Cloud Functions](https://console.cloud.google.com/functions/list) in Google Cloud Console.
2.  Click **Create Function**.
3.  Setup **Basics and Trigger** Configuration like below:
    ![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/redis-gcloud/basics.png)
4.  Using your `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN`, setup **Runtime environment variables** under **Runtime, build, connections and privacy settings** like below.
    ![](https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/redis-gcloud/environment.png)
5.  Click **Next**.
6.  Set **Entry point** to `counter`.
7.  Update `index.js`

```js index.js
const { Redis } = require("@upstash/redis");
const functions = require('@google-cloud/functions-framework');

const redis = new Redis({
  url: process.env.UPSTASH_REDIS_REST_URL,
  token: process.env.UPSTASH_REDIS_REST_TOKEN
});

functions.http('counter', async (req, res) => {
  const count = await redis.incr("counter");
  res.send("Counter:" + count);
});
```

8.  Update `package.json` to include `@upstash/redis`.

```json package.json
{
  "dependencies": {
    "@google-cloud/functions-framework": "^3.0.0",
    "@upstash/redis": "^1.31.6"
  }
}
```

9.  Click **Deploy**.
10. Visit the given URL.


# Using Serverless Framework



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/redis-js/tree/main/examples/serverless-framework/counter" horizontal>
  You can find the project source code on GitHub.
</Card>

This tutorial implements a serverless application and deploys it to AWS Lambda
using Serverless Framework

### Prerequisites

1.  Install the Serverless Framework with `npm i serverless -g`

### Project Setup

Create a Serverless Framework application with the following options:

```shell
➜  tutorials > ✗ serverless
Serverless ϟ Framework

Welcome to Serverless Framework V.4

Create a new project by selecting a Template to generate scaffolding for a specific use-case.

✔ Select A Template: · AWS / Node.js / HTTP API

✔ Name Your Project: · counter-serverless

✔ Template Downloaded

✔ Create Or Select An Existing App: · Create A New App

✔ Name Your New App: · counter-serverless

Your new Service "counter-serverless" is ready. Here are next steps:

• Open Service Directory: cd counter-serverless
• Install Dependencies: npm install (or use another package manager)
• Deploy Your Service: serverless deploy
```

```shell
cd counter-serverless
```

Create `package.json` with `@upstash/redis` as a dependency:

```json package.json
{
    "dependencies": {
      "@upstash/redis": "latest"
    }
  }
```

Install the dependencies:

```shell
npm install
```

### Database Setup

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Counter Function Setup

Update `handler.js`:

```js handler.js
const { Redis } = require('@upstash/redis');

const redis = Redis.fromEnv();

exports.counter = async (event) => {
    const count = await redis.incr("counter");
    return {
        statusCode: 200,
        body: JSON.stringify('Counter: ' + count),
    };
};
```

Update `serverless.yml` to pass the environment variables:

```yaml serverless.yml
service: counter-serverless

provider:
  name: aws
  runtime: nodejs20.x
  environment:
    UPSTASH_REDIS_REST_URL: ${env:UPSTASH_REDIS_REST_URL}
    UPSTASH_REDIS_REST_TOKEN: ${env:UPSTASH_REDIS_REST_TOKEN}

functions:
  counter:
    handler: handler.counter
    events:
      - httpApi:
          path: /
          method: get
```

### Developing

Run the following command to start your dev session.

```shell
serverless dev
```

### Deployment

Run the following command to deploy your service.

```shell
serverless deploy
```

Visit the output url.


# Delete Vectors

DELETE https://{endpoint}/delete/{namespace}
Deletes the vectors with the given ids.

You can delete one or more vectors by providing the vector ids
to be deleted as a string or array of strings.

<Tip>
  Vectors will be deleted from the default namespace by default.
  You can use a different namespace by specifying it in the request path.
</Tip>

## Request

<ParamField body="ids" type="string[]" required>
  A vector id or array of vector ids to delete.
  <Info>This is not a field, you should use the array or the string as the request body itself.</Info>
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

<ResponseField name="deleted" type="number">
  The number of the successfully deleted vectors.
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/delete \
    -X DELETE \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '[ "id-0", "id-1" ]'
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/delete/ns \
    -X DELETE \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '"id-0"'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": {
          "deleted": 2
      }
  }
  ```
</ResponseExample>


# Delete Namespace

DELETE https://{endpoint}/delete-namespace/{namespace}
Deletes a namespace of an index.

<Note>
  The default namespace, which is the empty string `""`, cannot be deleted.
</Note>

## Request

This endpoint doesn't require any additional data.

## Path

<ParamField path="namespace" type="string" required>
  The namespace to delete.
</ParamField>

## Response

<ResponseField name="result" type="string">
  `"Success"` string.
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/delete-namespace/ns \
    -X DELETE \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": "Success"
  }
  ```

  ```json 404 Not Found
  {
      "error": "Namespace ns for the index $NAME does not exist",
      "status": 404
  }
  ```
</ResponseExample>


# Fetch Vectors

GET https://{endpoint}/fetch/{namespace}
Fetches the vectors with the provided ids.

You can fetch vector values or metadata of one or more by providing
their vector ids.

<Tip>
  Vectors will be fetched from the default namespace by default.
  You can use a different namespace by specifying it in the request path.
</Tip>

## Request

<ParamField body="ids" type="string[]" required>
  Array of vector ids to fetch.
</ParamField>

<ParamField body="includeMetadata" type="boolean" default="false">
  Whether to include the metadata of the vectors in the response, if any.
  It is recommended to set this to `true` to easily identify vectors.
</ParamField>

<ParamField body="includeVectors" type="boolean" default="false">
  Whether to include the vector values in the response.
  It is recommended to set this to `false` as the vector values can be
  quite big, and not needed most of the time.
</ParamField>

<ParamField body="includeData" type="boolean" default="false">
  Whether to include the data of the vectors in the response, if any.
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

<ResponseField name="Vectors" type="Object[]">
  Array of vectors in the same order they provided in the ids array.
  Array elements can be `null` if no such vector exists with the provided id.

  <Expandable defaultOpen="true">
    <ResponseField name="id" type="string" required>
      The id of the vector.
    </ResponseField>

    <ResponseField name="vector" type="number[]">
      The vector value.
    </ResponseField>

    <ResponseField name="metadata" type="Object">
      The metadata of the vector, if any.
    </ResponseField>

    <ResponseField name="data" type="string">
      The unstructured data of the vector, if any.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/fetch \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "ids": ["id-0"], "includeMetadata": true }'
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/fetch/ns \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "ids": ["id-0", "id-1"], "includeMetadata": true }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": [
          {
              "id": "id-0",
              "metadata": {
                  "link": "upstash.com"
              }
          },
          {
              "id": "id-1"
          }
      ]
  }
  ```
</ResponseExample>


# Fetch Random Vector

GET https://{endpoint}/random/{namespace}
Fetches a random vector.

## Request

This endpoint doesn't require any additional data.

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

The response will be `null` if the namespace is empty.

<ResponseField name="id" type="string" required>
  The id of the vector.
</ResponseField>

<ResponseField name="vector" type="number[]" required>
  The vector value.
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/random \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/random/ns \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": {
          "id": "id-0",
          "vector": [0.1, 0.2]
      }
  }
  ```
</ResponseExample>


# Index Info

GET https://{endpoint}/info
Returns some information about the index.

Info will be updated eventually, so it might take some time to see the effect of changes in this endpoint.

## Request

This request doesn't require any additional data.

## Response

<ResponseField name="vectorCount" type="number" required>
  The number of vectors in the index, that are ready to use.
  This is the total number of vectors across all namespaces.
</ResponseField>

<ResponseField name="pendingVectorCount" type="number" required>
  The number of vectors in the index, that are still processing
  and not ready to use.
  This is the total number of pending vectors across all namespaces.
</ResponseField>

<ResponseField name="indexSize" type="number" required>
  The total size of the index, in **bytes**.
</ResponseField>

<ResponseField name="dimension" type="number" required>
  Dimension of the vectors.
</ResponseField>

<ResponseField name="similarityFunction" type="string" required>
  Name of the similarity function used in indexing and queries.
</ResponseField>

<ResponseField name="namespaces" type="object" required>
  Map of namespace names to namespace .

  <Note>Every index has at least one namespace called default namespace, whose name is the empty string `""`.</Note>

  <Expandable defaultOpen="true">
    <ResponseField name="vectorCount" type="number" required>
      The number of vectors in the namespace, that are ready to use.
    </ResponseField>

    <ResponseField name="pendingVectorCount" type="number" required>
      The number of vectors in the namespace, that are still processing
      and not ready to use.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/info \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": {
          "vectorCount": 7,
          "pendingVectorCount": 0,
          "indexSize": 43501,
          "dimension": 1536,
          "similarityFunction": "COSINE",
          "namespaces": {
              "": {
                  "vectorCount": 6,
                  "pendingVectorCount": 0
              },
              "ns": {
                  "vectorCount": 1,
                  "pendingVectorCount": 0
              }
          }
      }
  }
  ```
</ResponseExample>


# List Namespaces

GET https://{endpoint}/list-namespaces
Lists the names of the namespaces of an index.

## Request

This endpoint doesn't require any additional data.

## Response

<ResponseField name="namespaces" type="string[]" required>
  Array of namespace names.

  <Note>Every index has at least one namespace called default namespace, whose name is the empty string `""`.</Note>
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/list-namespaces \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": ["", "ns0", "ns1"]
  }
  ```
</ResponseExample>


# Query Vectors

POST https://{endpoint}/query/{namespace}
Queries the approximate nearest neighbors of a vector.

<Tip>
  Query will run against the default namespace by default.
  You can use a different namespace by specifying it in the request path.
</Tip>

## Request

It is also possible to send a batch query request by providing an array
of fields below.

<ParamField body="vector" name="vector" type="number[]" required>
  The query vector
  <Note>The query vector should have the same dimensions as your index.</Note>
</ParamField>

<ParamField body="topK" type="number" default="10">
  The total number of the vectors that you want to receive as a query
  result. The response will be sorted based on the distance metric score,
  and at most `topK` many vectors will be returned.
</ParamField>

<ParamField body="includeMetadata" type="boolean" default="false">
  Whether to include the metadata of the vectors in the response, if any.
  It is recommended to set this to `true` to easily identify vectors.
</ParamField>

<ParamField body="includeVectors" type="boolean" default="false">
  Whether to include the vector values in the response.
  It is recommended to set this to `false` as the vector values can be
  quite big, and not needed most of the time.
</ParamField>

<ParamField body="includeData" type="boolean" default="false">
  Whether to include the data of the vectors in the response, if any.
</ParamField>

<ParamField body="filter" type="string" default="">
  [Metadata filter](/vector/features/filtering) to apply.
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

If the request was an array of a single element, or a JSON object,
an object with the following fields is returned.

If the request was an array of more than one items, an array of
objects below is returned, one for each query item.

<Note>
  The score is normalized to always be between 0 and 1.
  The closer the score is to 1, the more similar the vector is to the query vector.
  This does not depend on the distance metric you use.
</Note>

<ResponseField name="Scores" type="Object[]">
  <Expandable defaultOpen="true">
    <ResponseField name="id" type="string" required>
      The id of the vector.
    </ResponseField>

    <ResponseField name="score" type="number" required>
      The similarity score of the vector, calculated based on the distance metric of your index.
    </ResponseField>

    <ResponseField name="vector" type="number[]">
      The vector value.
    </ResponseField>

    <ResponseField name="metadata" type="Object">
      The metadata of the vector, if any.
    </ResponseField>

    <ResponseField name="data" type="string">
      The unstructured data of the vector, if any.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/query \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "vector": [0.1, 0.2], "topK": 2, "includeMetadata": true }'
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/query/ns \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "vector": [0.1, 0.2], "topK": 2, "includeMetadata": true }'
  ```

  ```sh curl (Batch Query)
  curl "$UPSTASH_VECTOR_REST_URL/query" \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '[
          {
            "vector": [0.1, 0.2],
            "topK": 2,
            "includeMetadata": true
          },
          {
            "vector": [0.2, 0.3],
            "topK": 3
          }
        ]'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": [
          {
              "id": "id-0",
              "score": 1.0,
              "metadata": {
                  "link": "upstash.com"
              }
          },
          {
              "id": "id-1",
              "score": 0.99996454
          }
      ]
  }
  ```
</ResponseExample>


# Query Data

POST https://{endpoint}/query-data/{namespace}
Queries the approximate nearest neighbors of a raw text data after embedding it.

<Warning>
  To use this endpoint, the index must be created with an [embedding model](/vector/features/embeddingmodels).
</Warning>

<Tip>
  Query will run against the default namespace by default.
  You can use a different namespace by specifying it in the request path.
</Tip>

## Request

It is also possible to send a batch query request by providing an array
of fields below.

<ParamField body="data" name="data" type="string" required>
  The raw text data to embed and query.
</ParamField>

<ParamField body="topK" type="number" required default="10">
  The total number of the vectors that you want to receive as a query
  result. The response will be sorted based on the distance metric score,
  and at most `topK` many vectors will be returned.
</ParamField>

<ParamField body="includeMetadata" type="boolean" default="false">
  Whether to include the metadata of the vectors in the response, if any.
  It is recommended to set this to `true` to easily identify vectors.
</ParamField>

<ParamField body="includeVectors" type="boolean" default="false">
  Whether to include the vector values in the response.
  It is recommended to set this to `false` as the vector values can be
  quite big, and not needed most of the time.
</ParamField>

<ParamField body="includeData" type="boolean" default="false">
  Whether to include the data of the vectors in the response.
  When set to true, data will contain the raw text data used
  while upserting.
</ParamField>

<ParamField body="filter" type="string" default="">
  [Metadata filter](/vector/features/filtering) to apply.
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

If the request was an array of a single element, or a JSON object,
an object with the following fields is returned.

If the request was an array of more than one items, an array of
objects below is returned, one for each query item.

<Note>
  The score is normalized to always be between 0 and 1.
  The closer the score is to 1, the more similar the vector is to the query vector.
  This does not depend on the distance metric you use.
</Note>

<ResponseField name="Scores" type="Object[]">
  <Expandable defaultOpen="true">
    <ResponseField name="id" type="string" required>
      The id of the vector.
    </ResponseField>

    <ResponseField name="score" type="number" required>
      The similarity score of the vector, calculated based on the distance metric of your index.
    </ResponseField>

    <ResponseField name="vector" type="number[]">
      The vector value.
    </ResponseField>

    <ResponseField name="metadata" type="Object">
      The metadata of the vector, if any.
    </ResponseField>

    <ResponseField name="data" type="string">
      The textual data of the vector before embedding it.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/query-data \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "data": "What is Upstash?", "topK": 2, "includeMetadata": true }'
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/query-data/ns \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "data": "What is Upstash?", "topK": 2, "includeMetadata": true }'
  ```

  ```sh curl (Batch Query)
  curl $UPSTASH_VECTOR_REST_URL/query-data \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '[
          {
            "data": "What is Upstash?",
            "topK": 2,
            "includeMetadata": true
          },
          {
            "data": "What is Upstash Vector?",
            "topK": 3
          }
        ]'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": [
          {
              "id": "id-0",
              "score": 1.0,
              "metadata": {
                  "link": "upstash.com"
              }
          },
          {
              "id": "id-1",
              "score": 0.99996454
          }
      ]
  }
  ```

  ```json 422 Unprocessable Entity
  {
      "error": "Embedding data for this index is not allowed. The index must be created with an embedding model to use it.",
      "status": 422
  }
  ```
</ResponseExample>


# Range Vectors

GET https://{endpoint}/range/{namespace}
Ranges over vectors starting(inclusive) from a cursor until the end of the vectors in the or given limit.

<Tip>
  By default vectors from the default namespace will be iterated.
  You can use a different namespace by specifying it in the request path.
</Tip>

## Request

<ParamField body="cursor" type="string" required>
  The offset to the last retrieved vector. Should be set to `"0"` in the initial
  range.
</ParamField>

<ParamField body="limit" type="number" required>
  The number of maximum vectors that you want in the response of range. (page
  size)
</ParamField>

<ParamField body="includeMetadata" type="boolean" default="false">
  Whether to include the metadata of the vectors in the response, if any.
  It is recommended to set this to `true` to easily identify vectors.
</ParamField>

<ParamField body="includeVectors" type="boolean" default="false">
  Whether to include the vector values in the response.
  It is recommended to set this to `false` as the vector values can be
  quite big, and not needed most of the time.
</ParamField>

<ParamField body="includeData" type="boolean" default="false">
  Whether to include the data of the vectors in the response, if any.
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

<ResponseField name="nextCursor" type="string" required>
  The offset for the next range. You should place this in the `cursor` field for
  the next range. It will be equal to empty string if there are no other vectors to range.
</ResponseField>

<ResponseField name="vectors" type="Object[]" required>
  <Expandable defaultOpen="true">
    <ResponseField name="id" type="string" required>
      The id of the vector.
    </ResponseField>

    <ResponseField name="vector" type="number[]" required>
      The vector value.
    </ResponseField>

    <ResponseField name="metadata" type="Object">
      The metadata of the vector, if any.
    </ResponseField>

    <ResponseField name="data" type="string">
      The unstructured data of the vector, if any.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/range \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "cursor": "0", "limit": 2, "includeMetadata": true }'
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/range/ns \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "cursor": "0", "limit": 2, "includeMetadata": true }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": {
          "nextCursor": "2",
          "vectors": [
              {
                  "id": "id-0",
                  "metadata": {
                      "link": "upstash.com"
                  }
              },
              {
                  "id": "id-1"
              }
          ]
      }
  }
  ```
</ResponseExample>


# Reset Namespace(s)

DELETE https://{endpoint}/reset/{namespace}
Resets one or all namespaces of an index to its initial state by deleting all the vectors.

The namespace will be completely empty after `/reset` is called, but will not be deleted.

<Tip>
  Reset operation will be performed against the default namespace by default.
  You can use a different namespace by specifying it in the request path.
</Tip>

## Request

This request doesn't require any additional data.

## Query

<ParamField query="all" type="string">
  When given, resets all namespaces of an index.
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

<ResponseField name="result" type="string">
  `"Success"` string.
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/reset \
    -X DELETE \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/reset/ns \
    -X DELETE \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
  ```

  ```sh curl (All Namespaces)
  curl $UPSTASH_VECTOR_REST_URL/reset?all \
    -X DELETE \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": "Success"
  }
  ```
</ResponseExample>


# Resume

POST https://{endpoint}/resumable-query-next
Resumes a previously started query to fetch additional results.

## Request

<ParamField body="uuid" type="string" required>
  The unique identifier returned from the start resumable query request.
</ParamField>

<ParamField body="additionalK" type="number" required>
  The number of additional results to fetch.
</ParamField>

## Response

<ResponseField name="Scores" type="Object[]">
  <Expandable defaultOpen="true">
    <ResponseField name="id" type="string" required>
      The id of the vector.
    </ResponseField>

    <ResponseField name="score" type="number" required>
      The similarity score of the vector, calculated based on the distance
      metric of your index.
    </ResponseField>

    <ResponseField name="vector" type="number[]">
      The vector value.
    </ResponseField>

    <ResponseField name="metadata" type="Object">
      The metadata of the vector, if any.
    </ResponseField>

    <ResponseField name="data" type="string">
      The unstructured data of the vector, if any.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/resumable-query-next \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{
      "uuid": "550e8400-e29b-41d4-a716-446655440000",
      "additionalK": 2
    }'
  ```
</RequestExample>


# Start with Data

POST https://{endpoint}/resumable-query-data/{namespace}
Perform queries using text data that can be resumed to fetch additional results.

## Request

<ParamField body="data" type="string" required>
  The text data to be embedded and used for querying.
</ParamField>

<ParamField body="topK" type="number" default="10">
  The total number of the vectors that you want to receive as a query result.
  The response will be sorted based on the distance metric score, and at most
  `topK` many vectors will be returned.
</ParamField>

<ParamField body="includeMetadata" type="boolean" default="false">
  Whether to include the metadata of the vectors in the response, if any. It is
  recommended to set this to `true` to easily identify vectors.
</ParamField>

<ParamField body="includeVectors" type="boolean" default="false">
  Whether to include the vector values in the response. It is recommended to set
  this to `false` as the vector values can be quite big, and not needed most of
  the time.
</ParamField>

<ParamField body="includeData" type="boolean" default="false">
  Whether to include the data of the vectors in the response, if any.
</ParamField>

<ParamField body="filter" type="string" default="">
  [Metadata filter](/vector/features/filtering) to apply.
</ParamField>

<ParamField body="maxIdle" type="number">
  Maximum idle time for the resumable query in seconds.
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use. When no namespace is specified, the default namespace
  will be used.
</ParamField>

## Response

Same as Resumable Query.

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/resumable-query-data \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{
      "data": "Hello world",
      "topK": 2,
      "includeMetadata": true,
      "maxIdle": 3600
    }'
  ```
</RequestExample>


# Start with Vector

POST https://{endpoint}/resumable-query/{namespace}
Perform queries that can be resumed to fetch additional results.

<Tip>
  Resumable queries allow you to fetch results in batches, which is useful for
  large result sets or when you want to implement pagination.
</Tip>

## Request

<ParamField body="vector" type="number[]" required>
  The query vector
  <Note>The query vector should have the same dimensions as your index.</Note>
</ParamField>

<ParamField body="topK" type="number" default="10">
  The total number of the vectors that you want to receive as a query result.
  The response will be sorted based on the distance metric score, and at most
  `topK` many vectors will be returned.
</ParamField>

<ParamField body="includeMetadata" type="boolean" default="false">
  Whether to include the metadata of the vectors in the response, if any. It is
  recommended to set this to `true` to easily identify vectors.
</ParamField>

<ParamField body="includeVectors" type="boolean" default="false">
  Whether to include the vector values in the response. It is recommended to set
  this to `false` as the vector values can be quite big, and not needed most of
  the time.
</ParamField>

<ParamField body="includeData" type="boolean" default="false">
  Whether to include the data of the vectors in the response, if any.
</ParamField>

<ParamField body="filter" type="string" default="">
  [Metadata filter](/vector/features/filtering) to apply.
</ParamField>

<ParamField body="maxIdle" type="number">
  Maximum idle time for the resumable query in seconds.
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use. When no namespace is specified, the default namespace
  will be used.
</ParamField>

## Response

<ResponseField name="uuid" type="string" required>
  A unique identifier for the resumable query.
</ResponseField>

<ResponseField name="scores" type="Object[]">
  <Expandable defaultOpen="true">
    <ResponseField name="id" type="string" required>
      The id of the vector.
    </ResponseField>

    <ResponseField name="score" type="number" required>
      The similarity score of the vector, calculated based on the distance
      metric of your index.
    </ResponseField>

    <ResponseField name="vector" type="number[]">
      The vector value.
    </ResponseField>

    <ResponseField name="metadata" type="Object">
      The metadata of the vector, if any.
    </ResponseField>

    <ResponseField name="data" type="string">
      The unstructured data of the vector, if any.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/resumable-query \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{
      "vector": [0.1, 0.2],
      "topK": 2,
      "includeMetadata": true,
      "maxIdle": 3600
    }'
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/resumable-query/ns \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{
      "vector": [0.1, 0.2],
      "topK": 2,
      "includeMetadata": true,
      "maxIdle": 3600
    }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "uuid": "550e8400-e29b-41d4-a716-446655440000",
    "scores": [
      {
        "id": "id-0",
        "score": 1.0,
        "metadata": {
          "link": "upstash.com"
        }
      },
      {
        "id": "id-1",
        "score": 0.99996454
      }
    ]
  }
  ```
</ResponseExample>


# Stop Resumable Query

POST https://{endpoint}/resumable-query-end
Ends a resumable query and releases associated resources.

## Request

<ParamField body="uuid" type="string" required>
  The unique identifier of the resumable query to end.
</ParamField>

## Response

<ResponseField name="result" type="string">
  A success message indicating the query was ended.
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/resumable-query-end \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{
      "uuid": "550e8400-e29b-41d4-a716-446655440000"
    }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
    "result": "Success"
  }
  ```
</ResponseExample>


# Update Vector

POST https://{endpoint}/update/{namespace}
Updates a vector, data or metadata.

<Tip>
  The vector will be updated int the default namespace by default.
  You can use a different namespace by specifying it in the request path.
</Tip>

## Request

You can update a vector value, data, or metadata; or any combination
of those.

<ParamField body="id" type="string" required>
  The id of the vector.
</ParamField>

<ParamField body="vector" type="number[]">
  The vector value to update to.
  <Note>The vector should have the same dimensions as your index.</Note>
</ParamField>

<ParamField body="data" type="string">
  The raw text data to update to.
  <Note>If the index is created with an [embedding model](/vector/features/embeddingmodels)
  this will embed the data into a vector and will also update the vector, along with data.</Note>
</ParamField>

<ParamField body="metadata" type="Object">
  The metadata to update to.
</ParamField>

<ParamField body="metadataUpdateMode" type="string">
  Whether to overwrite the whole metadata while updating
  it, or patch the metadata (insert new fields or update or delete existing fields)
  according to the `RFC 7396 JSON Merge Patch` algorithm.

  `OVERWRITE` for overwrite, `PATCH` for patch.
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

<ResponseField name="updated" type="number">
  `1` if any vector is updated, `0` otherwise.
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/update \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "id": "id-1", "metadata": { "link": "upstash.com" } }'
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/update/ns \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "id": "id-2", "vector": [0.1, 0.2] }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": {
          "updated": 1
      }
  }
  ```
</ResponseExample>


# Upsert Vectors

POST https://{endpoint}/upsert/{namespace}
Upserts (inserts or updates) the vector.

<Tip>
  The vector will be upserted into the default namespace by default.
  You can use a different namespace by specifying it in the request path.
</Tip>

## Request

You can either upsert a single vector, or multiple vectors in an array.

<ParamField body="id" type="string" required>
  The id of the vector.
</ParamField>

<ParamField body="vector" type="number[]" required>
  The vector value.
  <Note>The vector should have the same dimensions as your index.</Note>
</ParamField>

<ParamField body="metadata" type="Object">
  The metadata of the vector. This makes identifying vectors
  on retrieval easier and can be used to with filters on queries.
</ParamField>

<ParamField body="data" type="string">
  The data of the vector. This is an unstructured raw text
  data, which can be anything associated with this vector.
</ParamField>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

<ResponseField name="result" type="string">
  `"Success"` string.
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/upsert \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '[ 
      { "id": "id-0", "vector": [0.1, 0.2], "metadata": { "link": "upstash.com" } }, 
      { "id": "id-1", "vector": [0.2, 0.3] }
    ]'
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/upsert/ns \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "id": "id-2", "vector": [0.1, 0.2], "metadata": { "link": "upstash.com" } }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": "Success"
  }
  ```

  ```json 422 Unprocessable Entity
  {
      "error": "Invalid vector dimension: 2, expected: 256",
      "status": 422
  }
  ```
</ResponseExample>


# Upsert Data

POST https://{endpoint}/upsert-data/{namespace}
Upserts (inserts or updates) the raw text data after embedding it.

<Warning>
  To use this endpoint, the index must be created with an [embedding model](/vector/features/embeddingmodels).
</Warning>

<Tip>
  Vector embedding of the raw text data will be upserted into the
  default namespace by default.
  You can use a different namespace by specifying it in the request path.
</Tip>

## Request

You can either upsert a single data, or multiple data in an array.

<ParamField body="id" type="string" required>
  The id of the vector.
</ParamField>

<ParamField body="data" type="string" required>
  The raw text data to embed and upsert.
</ParamField>

<ParamField body="metadata" type="Object">
  The metadata of the vector. This makes identifying vectors
  on retrieval easier and can be used to with filters on queries.
</ParamField>

<Note>
  Data field of the vector will be automatically set to the
  raw text data, so that you can access it later, during
  queries.
</Note>

## Path

<ParamField path="namespace" type="string" default="">
  The namespace to use.
  When no namespace is specified, the default namespace will be used.
</ParamField>

## Response

<ResponseField name="result" type="string">
  `"Success"` string.
</ResponseField>

<RequestExample>
  ```sh curl
  curl $UPSTASH_VECTOR_REST_URL/upsert-data \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '[ 
      { "id": "id-0", "data": "Upstash is a serverless data platform.", "metadata": { "link": "upstash.com" } }, 
      { "id": "id-1", "data": "Upstash Vector is a serverless vector database." }
    ]'
  ```

  ```sh curl (Namespace)
  curl $UPSTASH_VECTOR_REST_URL/upsert-data/ns \
    -X POST \
    -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
    -d '{ "id": "id-2", "data": "Upstash is a serverless data platform.", "metadata": { "link": "upstash.com" } }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  {
      "result": "Success"
  }
  ```

  ```json 422 Unprocessable Entity
  {
      "error": "Embedding data for this index is not allowed. The index must be created with an embedding model to use it.",
      "status": 422
  }
  ```
</ResponseExample>


# Getting Started



If you do not have a vector database already, follow [these steps](https://upstash.com/docs/vector/overall/getstarted) to create one.

In the database details section of the Upstash Console, scroll down to `Connect` section and select the `cURL` tab. You can simply copy the curl expression and run on your terminal.

```shell
curl $UPSTASH_VECTOR_REST_URL/upsert \
  -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
  -d '{"id": "id-0", "vector": [0.87, 0.99]}'
```

## Response

REST API returns a JSON response. When command execution is successful, response JSON will have a single result field and its value will contain the Redis response.

Example:

```json
{ "result": "Success" }
```

When command execution is not successful, response JSON will have a single error field and its value will contain the error message.

Example:

```json
{
  "error": "Unauthorized: Invalid auth token",
  "status": 401
}
```

#### HTTP Response Codes

| Status Code              | Description                                                                                         |
| ------------------------ | --------------------------------------------------------------------------------------------------- |
| `200 OK`                 | When request is accepted and successfully executed.                                                 |
| `400 Bad Request`        | When there's a syntax error, an invalid/unsupported command is sent or command execution fails.     |
| `401 Unauthorized`       | When authentication fails; auth token is missing or invalid.                                        |
| `405 Method Not Allowed` | When an unsupported HTTP method is used. Only `HEAD`, `GET`, `POST`, and `PUT` methods are allowed. |

***


# Examples



<iframe src="https://vector-example-dashboard.vercel.app" width="100%" height="1000px" style={{ border: "0" }} />


# Algorithm



## Approximate Nearest Neighbor Search

The primary functionality of the vector store is straightforward: identifying the most similar vectors to a given vector.
While the concept is simple, translating it into a practical product poses significant challenges.

A simple and basic approach to searching in a vector database is to perform an exhaustive search by comparing a query vector to every other vector stored in the database one by one. However, this consumes too many resources and results in very high latencies, making it not very practical. To address this problem, Approximate Nearest Neighbor (`ANN`) algorithms are used. `ANN` search approximates the true nearest neighbor, which means it might not find the absolute closest point, but it will find one that's close enough, with a **low-latency** and by consuming **fewer resources**.
In the literature, the comparison of the results of `ANNS` with exhaustive search is called the recall rate.
The higher the recall rate the better the results.

Several `ANNS` algorithms, such as `HNSW`\[1], `NSG`\[2], and `DiskANN`\[3], are available for use,
each with its distinct characteristics. One of the difficult problems in ANN algorithms is that indexing and querying vectors may require storing the whole data in memory. When the dataset is huge, then memory requirements for indexing may exceed available memory. `DiskANN` algorithm tries to solve this problem by using disk as the main storage for indexes and for performing queries directly on disk.`DiskANN` paper acknowledges that, if you try to store your vectors
in disk and use `HNSW` or `NSG`, you may end up with again very high latencies. `DiskANN` is focused
on serving queries from disk with **low-latency** and **good recall rate**.
And this helps Upstash Vector to be **cost-effective**, therefore cheaper compared to alternatives.

Even though `DiskANN` has its advantages, it also requires more work to be practical.
Main problem is that, you can't insert/update existing index without reindexing all the vectors.
For this problem, there is another improved paper `FreshDiskANN`\[4]. `FreshDiskANN` improves `DiskANN` via introducing
a temporary index for up-to-date data in memory. Queries are served from both the temporary (up-to-date) index
and also from the disk. And these temporary indexes are merged to the disk from time-to-time behind the scene.

Upstash Vector is based on `DiskANN` and `FreshDiskANN` with more improvements based on our
tests and observations.

### References

1.  Malkov, Y. A., Yashunin, D. A. (2016). *Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs*. CoRR, abs/1603.09320 (2016). \[[https://arxiv.org/abs/1603.09320](https://arxiv.org/abs/1603.09320)]
2.  Fu, C., Xiang, C., Wang, C., Cai, D. (2019). *Fast Approximate Nearest Neighbor Search with Navigating Spreading-Out Graphs*. Proceedings of the VLDB, 12(5), 461–474. doi: 10.14778/3303753.3303754. \[[https://www.vldb.org/pvldb/vol12/p461-fu.pdf](https://www.vldb.org/pvldb/vol12/p461-fu.pdf)]
3.  Subramanya, S. J., Devvrit, Kadekodi, R., Krishaswamy, R., Simhadri, H. V. (2019). *DiskANN: Fast Accurate Billion-Point Nearest Neighbor Search on a Single Node*. In Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS '19), Article No.: 1233, Pages 13766–13776. \[[https://dl.acm.org/doi/abs/10.5555/3454287.3455520](https://dl.acm.org/doi/abs/10.5555/3454287.3455520)]
4.  Singh, A., Subramanya, S. J., Krishnaswamy, R., Simhadri, H. V. (2021). *FreshDiskANN: A Fast and Accurate Graph-Based ANN Index for Streaming Similarity Search*. CoRR abs/2105.09613 (2021). \[[https://arxiv.org/abs/2105.09613](https://arxiv.org/abs/2105.09613)]


# Embedding Models



To store text in a vector database, it must first be converted into a vector,
also known as an embedding. Typically, this vectorization is done by a third
party.

By selecting an embedding model when you create your Upstash Vector database,
you can now upsert and query raw string data when using your database instead of
converting your text to a vector first. The vectorization is done automatically
by your selected model.

## Upstash Embedding Models - Video Guide

Let's look at how Upstash embeddings work, how the models we offer compare, and
which model is best for your use case.

<iframe width="560" height="315" src="https://www.youtube.com/embed/aImBIYwn5Ew?rel=0&disablekb=1" title="YouTube video player" frameBorder="0" allow="accelerometer; fullscreen; clipboard-write; encrypted-media; gyroscope" allowFullScreen />

## Models

Upstash Vector comes with a variety of embedding models that score well in the
[MTEB](https://huggingface.co/spaces/mteb/leaderboard) leaderboard, a benchmark
for measuring the performance of embedding models. They support use cases such
as classification, clustering, or retrieval.

You can choose the following general purpose models:

| Name                                                                                                    | Dimension | Sequence Length | MTEB  |
| ------------------------------------------------------------------------------------------------------- | --------- | --------------- | ----- |
| [mixedbread-ai/mxbai-embed-large-v1](https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1)         | 1024      | 512             | 64.68 |
| [WhereIsAI/UAE-Large-V1](https://huggingface.co/WhereIsAI/UAE-Large-V1)                                 | 1024      | 512             | 64.64 |
| [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5)                                 | 1024      | 512             | 64.23 |
| [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5)                                   | 768       | 512             | 63.55 |
| [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5)                                 | 384       | 512             | 62.17 |
| [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | 384       | 256             | 56.26 |
| [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3)                                                       | 1024      | 8192            | \*    |
| [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased)                   | 768       | 512             | 38.33 |

<Note>
  The sequence length is not a hard limit. Models truncate the input
  appropriately when given a raw text data that would result in more tokens than
  the given sequence length. However, we recommend using appropriate models and
  not exceeding their sequence length to have more accurate results.
</Note>

<Note>
  MTEB score for the `BAAI/bge-m3` is not fully measured.
</Note>

## Using a Model

To start using embedding models, create the index with a model of your choice.

<Frame style={{ width: '600px' }}>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/create_index_with_model.png" />
</Frame>

Then, you can start upserting and querying raw text data without any extra
setup.

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.upsert(
        [("id-0", "Upstash is a serverless data platform.", {"field": "value"})],
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.upsert({
      id: "id-0",
      data: "Upstash is a serverless data platform.",
      metadata: {
        field: "value",
      },
    })
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	index.UpsertData(vector.UpsertData{
    		Id:       "id-0",
    		Data:     "Upstash is a serverless data platform.",
    		Metadata: map[string]any{"field": "value"},
    	})
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/upsert-data \
      -X POST \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{"id": "1", "data": "Upstash is a serverless data platform.", "metadata": {"field": "value"}}'
    ```
  </Tab>
</Tabs>

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.query(
        data="What is Upstash?",
        top_k=1,
        include_metadata=True,
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.query({
      data: "What is Upstash?",
      topK: 1,
      includeMetadata: true,
    })
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	index.QueryData(vector.QueryData{
    		Data:            "What is Upstash?",
    		TopK:            1,
    		IncludeMetadata: true,
    	})
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/query-data \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{"data": "What is Upstash?", "topK": 1, "includeMetadata": "true"}'
    ```
  </Tab>
</Tabs>


# Metadata Filtering



You can further limit the vector similarity search by providing a filter based on a specific metadata criteria.

Queries with metadata filters only return vectors which have metadata matching with the filter.

Upstash Vector allows you to filter keys which have the following value types:

*   string
*   number
*   boolean
*   object
*   array

Filtering is implemented as a combination of in and post-filtering. Every query is assigned a filtering budget,
determining the number of candidate vectors that can be compared against the filter during query execution. If this
budget is exceeded, the system fallbacks into post-filtering. Therefore, with highly selective filters, fewer
than `topK` vectors may be returned.

## Filter Syntax

A filter has a syntax that resembles SQL, which consists of operators on object keys and boolean operators
to combine them.

Assuming you have a metadata like below:

```json
{
    "city": "Istanbul",
    "country": "Turkey",
    "is_capital": false,
    "population": 15460000,
    "geography": {
        "continent": "Asia",
        "coordinates": {
            "latitude": 41.0082,
            "longitude": 28.9784
        }
    },
    "economy": {
        "currency": "TRY",
        "major_industries": [
            "Tourism",
            "Textiles",
            "Finance"
        ]
    }
}
```

Then, you can query similar vectors with a filter like below:

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
      url="UPSTASH_VECTOR_REST_URL",
      token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.query(
      vector=[0.9215, 0.3897],
      filter="population >= 1000000 AND geography.continent = 'Asia'",
      top_k=5,
      include_metadata=True
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.query({
      vector: [0.9215, 0.3897],
      filter: "population >= 1000000 AND geography.continent = 'Asia'",
      topK: 5,
      includeMetadata: true,
    });
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	index.Query(vector.Query{
    		Vector:          []float32{0.9215, 0.3897},
    		Filter:          `population >= 1000000 AND geography.continent = 'Asia'`,
    		TopK:            5,
    		IncludeMetadata: true,
    	})
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/query \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{
       "vector":[0.9215,0.3897],
       "topK" : 5,
       "filter": "population >= 1000000 AND geography.continent = \"Asia\"",
       "includeMetadata": true
    }'
    ```
  </Tab>
</Tabs>

### Operators

#### Equals (=)

The `equals` operator filters keys whose value is equal to given literal.

It is applicable to *string*, *number*, and *boolean* values.

```SQL
country = 'Turkey' AND population = 15460000 AND is_capital = false
```

#### Not Equals (!=)

The `not equals` operator filters keys whose value is not equal to given literal.

It is applicable to *string*, *number*, and *boolean* values.

```SQL
country != 'Germany' AND population != 12500000 AND is_capital != true
```

#### Less Than (\<)

The `less than` operator filters keys whose value is less than the given literal.

It is applicable to *number* values.

```SQL
population < 20000000 OR geography.coordinates.longitude < 30.0
```

#### Less Than or Equals (\<=)

The `less than or equals` operator filters keys whose value is less than or equal to the given literal.

It is applicable to *number* values.

```SQL
population <= 20000000 OR geography.coordinates.longitude <= 30.0
```

#### Greater Than (>)

The `greater than` operator filters keys whose value is greater than the given literal.

It is applicable to *number* values.

```SQL
population > 10000000 OR geography.coordinates.latitude > 39.5
```

#### Greater Than or Equals (>=)

The `greater than or equals` operator filters keys whose value is greater than or equal to the given literal.

It is applicable to *number* values.

```SQL
population >= 10000000 OR geography.coordinates.latitude >= 39.5
```

#### Glob

The `glob` operator filters keys whose value matches the given UNIX glob pattern.

It is applicable to *string* values.

It is a case sensitive operator.

The glob operator supports the following wildcards:

*   `*` matches zero or more characters.
*   `?` matches exactly one character.
*   `[]` matches one character from the list
    *   `[abc]` matches either `a`, `b`, or `c`.
    *   `[a-z]` matches one of the range of characters from `a` to `z`.
    *   `[^abc]` matches any one character other than `a`, `b`, or `c`.
    *   `[^a-z]` matches any one character other than `a` to `z`.

For example, the filter below would only match with city names whose second character is `s` or `z`,
and ends with anything other than `m` to `z`.

```SQL
city GLOB '?[sz]*[^m-z]'
```

#### Not Glob

The `not glob` operator filters keys whose value does not match the given UNIX glob pattern.

It is applicable to *string* values.

It has the same properties with the glob operator.

For example, the filter below would only match with city names whose first character is anything other than `A`.

```SQL
city NOT GLOB 'A*'
```

#### In

The `in` operator filters keys whose value is equal to any of the given literals.

It is applicable to *string*, *number*, and *boolean* values.

```SQL
country IN ('Germany', 'Turkey', 'France')
```

Semantically, it is equivalent to equals operator applied to all of the given literals with `OR` boolean operator in between:

```SQL
country = 'Germany' OR country = 'Turkey' OR country = 'France'
```

#### Not In

The `not in` operator filters keys whose value is not equal to any of the given literals.

It is applicable to *string*, *number*, and *boolean* values.

```SQL
economy.currency NOT IN ('USD', 'EUR')
```

Semantically, it is equivalent to not equals operator applied to all of the given literals with `AND` boolean operator in between:

```SQL
economy.currency != 'USD' AND economy.currency != 'EUR'
```

#### Contains

The `contains` operator filters keys whose value contains the given literal.

It is applicable to `array` values.

```SQL
economy.major_industries CONTAINS 'Tourism'
```

#### Not Contains

The `not contains` operator filters keys whose value does not contain the given literal.

It is applicable to `array` values.

```SQL
economy.major_industries NOT CONTAINS 'Steel Production'
```

#### Has Field

The `has field` operator filters keys which having the given JSON field.

```SQL
HAS FIELD geography.coordinates
```

#### Has Not Field

The `has not field` operator filters keys which not having the given JSON field.

```SQL
HAS NOT FIELD geography.coordinates.longitude
```

### Boolean Operators

Operators above can be combined with `AND` and `OR` boolean operators to form
compound filters.

```SQL
country = 'Turkey' AND population > 10000000
```

Boolean operators can be grouped with parentheses to have higher precendence.

```SQL
country = 'Turkey' AND (population > 10000000 OR is_capital = false)
```

When no parentheses are provided in ambigous filters, `AND` will have higher
precendence than `OR`. So, the filter

```SQL
country = 'Turkey' AND population > 10000000 OR is_capital = false
```

would be equivalent to

```SQL
(country = 'Turkey' AND population > 10000000) OR is_capital = false
```

### Filtering Nested Objects

It is possible to filter nested object keys by referencing them with the `.` accessor.

Nested objects can be at arbitrary depths, so more than one `.` accessor can be used
in the same identifier.

```SQL
economy.currency != 'USD' AND geography.coordinates.latitude >= 35.0
```

### Filtering Array Elements

Apart from the `CONTAINS` and `NOT CONTAINS` operators, individual array elements can also
be filtered by referencing them with the `[]` accessor by their indexes.

Indexing is zero based.

```SQL
economy.major_industries[0] = 'Tourism'
```

Also, it is possible to index from the back using the `#` character with negative values.
`#` can be thought as the number of elements in the array, so `[#-1]` would reference the
last character.

```SQL
economy.major_industries[#-1] = 'Finance'
```

### Miscellaneous

*   Identifiers (the left side of the operators) should be of the form `[a-zA-Z_][a-zA-Z_0-9.[\]#-]*`. In simpler terms, they should
    start with characters from the English alphabet or `_`, and can continue with same characters plus numbers and other accessors
    like `.`, `[0]`, or `[#-1]`.
*   The string literals (strings in the right side of the operators) can be either single or double quoted.
*   Boolean literals are represented as `1` or `0`.
*   The operators, boolean operators, and boolean literals are case insensitive.


# Metadata and Data



## Metadata

Metadata feature allows you to store context with your vectors to make a connection.
There can be a couple of uses of this:

1.  You can put the source of the vector in the metadata to use in your application from the query response.
2.  You can put some metadata to further filter the results upon the query.

You can set metadata with your vector as follows:

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.upsert(
        [("id-0", [0.9215, 0.3897]), {"url": "https://imgur.com/z9AVZLb"}],
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.upsert({
      "id": "id-0",
      vector: [0.9215, 0.3897],
      metadata: {
        url: "https://imgur.com/z9AVZLb",
      },
    })
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	index.Upsert(vector.Upsert{
    		Id:       "id-0",
    		Vector:   []float32{0.9215, 0.3897},
    		Metadata: map[string]any{"url": "https://imgur.com/z9AVZLb"},
    	})
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/upsert \
      -X POST \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{
       "id":"id-0",
       "vector":[0.9215,0.3897],
       "metadata":{
          "url":"https://imgur.com/z9AVZLb"
       }
    }'
    ```
  </Tab>
</Tabs>

When you do a query or fetch, you can opt-in to retrieve the metadata as follows:

*   **Query Example**

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.query(
        [0.9215, 0.3897],
        top_k=5,
        include_metadata=True,
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.query({
      vector: [0.9215, 0.3897],
      topK: 5,
      includeMetadata: true,
    })
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	index.Query(vector.Query{
    		Vector:          []float32{0.9215, 0.3897},
    		TopK:            5,
    		IncludeMetadata: true,
    	})
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/query \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{
       "vector":[0.9215,0.3897],
       "topK" : 5,
       "includeMetadata": true
    }'
    ```
  </Tab>
</Tabs>

```json
{
  "result": [
    {
      "id": "id-0",
      "score": 1,
      "metadata": {
        "url": "https://imgur.com/z9AVZLb"
      }
    },
    {
      "id": "id-3",
      "score": 0.99961007,
      "metadata": {
        "url": "https://imgur.com/zfOPmnI"
      }
    }
  ]
}
```

Also, you can filter the results further by providing a metadata filter:

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.query(
        [0.9215, 0.3897],
        top_k=5,
        include_metadata=True,
        filter="url GLOB '*imgur.com*'",
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.query({
      vector: [0.9215, 0.3897],
      topK: 5,
      includeMetadata: true,
      filter: "url GLOB '*imgur.com*'",
    })
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	index.Query(vector.Query{
    		Vector:          []float32{0.9215, 0.3897},
    		TopK:            5,
    		IncludeMetadata: true,
    		Filter:          "url GLOB '*imgur.com*'",
    	})
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/query \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{
       "vector":[0.9215,0.3897],
       "topK" : 5,
       "includeMetadata": true,
       "filter": "url GLOB \"*imgur.com*\""
    }'
    ```
  </Tab>
</Tabs>

See [Metadata Filtering documentation](https://upstash.com/docs/vector/features/filtering) for more details.

*   **Range Example**

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.range(
        cursor="0",
        limit=3,
        include_metadata=True,
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.range({
      cursor: "0",
      limit: 3,
      includeMetadata: true,
    })
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	index.Range(vector.Range{
    		Cursor:          "0",
    		Limit:           3,
    		IncludeMetadata: true,
    	})
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/range \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{ "cursor" : "0",  "limit" : 3, "includeMetadata": true}'
    ```
  </Tab>
</Tabs>

```json
{
  "result": {
    "nextCursor": "4",
    "vectors": [
      { "id": "id-0", "metadata": { "url": "https://imgur.com/z9AVZLb" } },
      { "id": "id-1", "metadata": { "url": "https://imgur.com/a2nCEIt" } },
      { "id": "id-2", "metadata": { "url": "https://imgur.com/zfOPmnI" } }
    ]
  }
}
```

## Data

Data is another kind of information you can store per vector
to attribute some context to it. Compared to metadata, it is not
structured, and it can only be fetched in queries, not used
to further filter them.

It is especially useful when you upsert raw text data, so that you
would have access to the textual form of vector along with the
embedded vector values.

It can save you from storing contextual information per vector
in a separate database.

You can set both the metadata and data, or only one of them
while upserting your vectors as follows:

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.upsert(
        [
            {
                "id": "id-0",
                "vector": [0.9215, 0.3897],
                "metadata": {"url": "https://imgur.com/z9AVZLb"},
                "data": "data-0",
            },
            {
                "id": "id-1",
                "vector": [0.3897, 0.9215],
                "data": "data-1",
            },
        ],
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.upsert([
      {
        id: "id-0",
        vector: [0.9215, 0.3897],
        metadata: {"url": "https://imgur.com/z9AVZLb"},
        data: "data-0",
      },
      {
        id: "id-1",
        vector: [0.3897, 0.9215],
        data: "data-1",
      },
    ])
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/upsert \
      -X POST \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '[
            {
                "id": "id-0",
                "vector": [0.9215, 0.3897],
                "metadata": {"url": "https://imgur.com/z9AVZLb"},
                "data": "data-0"
            },
            {
                "id": "id-1",
                "vector": [0.3897, 0.9215],
                "data": "data-1"
            }
        ]'
    ```
  </Tab>
</Tabs>

When a raw text data is upserted, the data will be set to
the raw text data automatically:

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.upsert(
        [
            {
                "id": "id-2",
                "data": "Upstash is a serverless data platform.",
            },
        ],
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.upsert([
      {
        id: "id-2",
        data: "Upstash is a serverless data platform.",
      }
    ])
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/upsert-data \
      -X POST \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{
            "id": "id-0",
            "data": "Upstash is a serverless data platform."
          }'
    ```
  </Tab>
</Tabs>

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    result = index.query(
        data="What is Upstash?",
        include_data=True,
    )

    for res in result:
        print(f"{res.id}: {res.data}")
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    const result = await index.query({
      data: "What is Upstash?",
      includeData: true,
      topK: 3
    })

    for (const vector of result) {
      console.log(`${vector.id}: ${vector.data}`)
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/query-data \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{
            "data": "What is Upstash?",
            "includeData": true,
          }'
    ```
  </Tab>
</Tabs>

Similar to metadata, the data field can be requested in queries, range
iterator, and fetch requests, by setting the `includeData` to `true` as
shown above.


# Namespaces



Upstash Vector allows you to partition a single index into multiple isolated namespaces. Each namespace acts as a self-contained subset of the index, and read and write requests are limited to one namespace.

Each vector index has at least one default namespace and optionally many more.

If no namespace is specified, the operations will use the default namespace, which has the name `""` (empty string).

<Note>
  Indexes created before the namespaces feature can still be used as they are,
  without modification. All operations are assumed to use the default namespace.
</Note>

## Using a Namespace

Namespaces are created implicitly when an upsert operation is performed, so there is no specific endpoint for creating a namespace.

For example, the code snippet below will create the namespace `ns` if it does not already exist, upsert and query the vector only on that namespace.

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.upsert(
        [("id-0", [0.9215, 0.3897])],
        namespace="ns",
    )

    index.query(
        [0.9215, 0.3897],
        top_k=5,
        namespace="ns",
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    const namespace = index.namespace("ns")

    await namespace.upsert({
      id: "id-0",
      vector: [0.9215, 0.3897],
    })

    await namespace.query({
      vector: [0.9215, 0.3897],
      topK: 5,
    })
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	namespace := index.Namespace("ns")

    	namespace.Upsert(vector.Upsert{
    		Id:     "id-0",
    		Vector: []float32{0.9215, 0.3897},
    	})

    	namespace.Query(vector.Query{
    		Vector: []float32{0.9215, 0.3897},
    		TopK:   5,
    	})
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/upsert/ns \
      -X POST \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{"id":"id-0", "vector":[0.9215,0.3897]}'

    curl $UPSTASH_VECTOR_REST_URL/query/ns \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{"vector":[0.9215,0.3897], "topK" : 5}'
    ```
  </Tab>
</Tabs>

Under the hood, when using a namespace, your requests are sent to `<vector-url>/<command>/namespace-name` to only be executed only against that namespace.

## Deleting a Namespace

Namespaces can be deleted by using the
[Delete Namespace](../api/endpoints/delete-namespace) API.

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.delete_namespace("ns")
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.deleteNamespace("ns")
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	namespace := index.Namespace("ns")

    	namespace.DeleteNamespace()
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/delete-namespace/ns \
      -X DELETE \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
    ```
  </Tab>
</Tabs>

## Listing Namespaces

All active namespaces can be listed by using the
[List Namespaces](../api/endpoints/list-namespaces) API.

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(
        url="UPSTASH_VECTOR_REST_URL",
        token="UPSTASH_VECTOR_REST_TOKEN",
    )

    index.list_namespaces()
    ```
  </Tab>

  <Tab title="JavaScript">
    ```js
    import { Index } from "@upstash/vector"

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.listNamespaces()
    ```
  </Tab>

  <Tab title="Go">
    ```go
    package main

    import (
    	"github.com/upstash/vector-go"
    )

    func main() {
    	index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

    	index.ListNamespaces()
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/list-namespaces \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN"
    ```
  </Tab>
</Tabs>


# Vector Similarity Functions



When creating a vector index in Upstash Vector, you have the flexibility to choose from different vector similarity functions.
Each function yields distinct query results, catering to specific use cases. Here are the three supported similarity functions:

<Note>
  The score returned from query requests is a normalized value between 0 and 1,
  where 1 indicates the highest similarity and 0 the lowest regardless of the
  similarity function used.
</Note>

#### Cosine Similarity

Cosine similarity measures the cosine of the angle between two vectors. It is particularly useful when the magnitude of the vectors is not essential, and the focus is on the orientation.

**Use Cases:**

*   **Natural Language Processing (NLP):** Ideal for comparing document embeddings or word vectors, as it captures semantic similarity irrespective of vector magnitude.
*   **Recommendation Systems:** Effective in recommending items based on user preferences or content similarities.

**Score calculation:**
` (1 + cosine_similarity(v1, v2)) / 2;`

<Frame>
  <img className="block h-32 dark:hidden" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/cosine_similarity.svg" />

  <img className="hidden h-32 dark:block" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/cosine_similarity_dark.svg" />
</Frame>

#### Euclidean Distance

Euclidean distance calculates the straight-line distance between two vectors in a multi-dimensional space. It is well-suited for scenarios where the magnitude of vectors is crucial, providing a measure of their spatial separation.

**Use Cases:**

*   **Computer Vision:** Useful in image processing tasks, such as image recognition or object detection, where the spatial arrangement of features is significant.
*   **Anomaly Detection:** Valuable for detecting anomalies in datasets, as it considers both the direction and magnitude of differences between vectors.

**Score calculation:**
`1 / (1 + squared_distance(v1, v2))`

<Frame>
  <img className="block h-32 dark:hidden" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/squared_distance.svg" />

  <img className="hidden h-32 dark:block" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/squared_distance_dark.svg" />
</Frame>

#### Dot Product

The dot product measures the similarity by multiplying the corresponding components of two vectors and summing the results. It provides a measure of alignment between vectors.
Note that to use dot product, the vectors needs to be normalized to be of unit length.

**Use Cases:**

*   **Machine Learning Models:** Commonly used in machine learning for tasks like sentiment analysis or classification, where feature alignment is critical.
*   **Collaborative Filtering:** Effective in collaborative filtering scenarios, such as recommending items based on user behavior or preferences.

**Score calculation:**
` (1 + dot_product(v1, v2)) / 2`

<Frame>
  <img className="block h-32 dark:hidden" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/dot_product.svg" />

  <img className="hidden h-32 dark:block" src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/dot_product_dark.svg" />
</Frame>


# FAQ



**Account and Usage:**

*   **What is the maximum number of indexes I can create in my account?**

    You can create up to 10 indexes per account. For additional needs, please send us an email to  [support@upstash.com](mailto:support@upstash.com)

*   **What is the maximum dimension I can set for my indexes?**

    For the free tier, the maximum dimension can be 1536, for fixed and pay as you go tiers, this can be 3072, and for the pro tier, this can go up to 5376

*   **How do you calculate the storage to be charged?**

    Storage is a sum of vector storage and metadata storage. Vector storage depends on the number of dimensions and the number of vectors in that index. Each dimension is estimated to be 4 bytes, resulting in vector storage being calculated as vector count \* dimension count \* 4 bytes. For metadata storage, it is calculated based on the vector count in an index multiplied by the metadata size, which can be up to 48 Kbytes.

*   **What happens when I exceed my Metadata Storage limit?**

    Metadata Storage has a soft limit, and you will receive an email reminder to optimize your index for optimal performance if you approach this limit. The storage cost remains at \$0.25 per GB. For enhanced performance, we recommend upgrading to Pro tier that meets your requirements, which includes incremental metadata storage

*   **What happens when I exceed bandwidth limit of 200GB?**

    When you exceed the 200GB monthly bandwidth limit, you will be charged at \$0.03 per GB for the additional usage. The initial 200GB is provided free of charge.

*   **If I upload my dataset for pay as you go tier and don’t have any read/write operation during the month, how much will I pay?**

    You will only be charged for the storage of your index and metadata at a rate of \$0.25 per GB. There are no hourly charges for having an index created on Upstash

*   **How do you count Updates and Queries for billing purposes?**

    We charge \$0.40 per 100,000 operations, which includes ranges, queries, fetches, upserts and deletes, all counted equally. Ranges, Queries and Fetches are all categorized as query type of operation, whereas Upserts and Deletes are counted as updates. If upsert and delete operations involve multiple vectors at a time, the billable request count will be equal to number of vectors upserted or deleted at one request.

*   **If an update/query return or insert multiple vectors at a time, will each vector be counted as separate operations or will that be counted as one operation for billing purposes?**

    For the query type of operations, irrespective of the number of vectors involved in it, we count the operation as one and bill accordingly. For example, a fetch operation can return 10 vectors, and this will be counted only as 1 operation for billing purposes. However for update operations (upsert and delete), number of vectors involved in the operations is reflected as billable request count. For example a batch upsert of 1000 vectors is counted as 1000 billable request.

*   **What is the maximum number of vectors I can have in an index?**

    The maximum number of vectors in an index depends on the dimension size. Every plan has a max vector\*dimension limit. For pay as you go tier, this limit is set at 2 billion. For instance, under this plan a user can have an index with a size of 5 million vectors and a vector dimension of 400. For details about the limits of each plan, check our pricing page.

**Functionality:**

*   **Do you support hybrid search?**

    We don’t support hybrid search at the moment. Although this is in our roadmap we still haven’t finalized the approach to help our users eliminate the need to have a secondary database for better search result. As we define our approach on implementing hybrid search and gather more user feedback, we will share this through our blog posts

*   **Can I filter by metadata?**

    Yes, see [Metadata Filtering](../features/filtering).

*   **Do you support replication?**

    Replication is not supported in the current release, but we are actively working on including it in our upcoming release.

*   **If I do not specify a UUID during adding vectors, will Upstash Vector create one automatically?**

    No, ID field is required and cannot be an empty string while inserting vectors to your index

**Data Management:**

*   **How can I upload a large dataset quickly?**

    You can perform batch inserts to upload datasets more efficiently. You don't need a separate batch specific operation to insert multiple vectors. The 'upsert' operation accepts either a single vector or an array of vectors. We recommend using an array size of up to 1000 for efficient batch inserts. For more details, please refer to the documentation

*   **How can I remove a metadata field from a vector?**

    You need to run Upsert function on the same vector ID to remove existing metadata fields from a vector. Check our  documentation on using `Upsert`,

**Infrastructure and Availability:**

*   **Which cloud provider is Upstash Vector hosted on?**

    Upstash is currently offered on AWS. We have plans to expand it to other cloud providers, please reach out [support@upstash.com](mailto:support@upstash.com) to raise this request as this will help us prioritize accordingly.

*   **In what regions is Upstash available?**

    Upstash Vector is currently available in two regions; AWS us-east1 and AWS eu-west-1. We are in the process of expanding to more regions where other Upstash products are currently offered.

*   **Should I be on AWS to be able to use Upstash Vector?**

    No. Your client can be anywhere but the clients in AWS regions will give you better performance.

**Feature Request and Upgrade:**

*   **How can I request a feature during this beta release?**

    Please reach out to [support@upstash.com](mailto:support@upstash.com). You can also raise your feature request through our chatbot, located at the bottom of Upstash page. The chatbot will guide you through the process of submitting your feature request.

*   **How to upgrade from free tier to pay-as-you-go tier?**

    To upgrade from the free tier to the pay-as-you-go tier, simply enter your credit card information. This will allow you to exceed the free tier limits, you will start to be charged only after you exceed free tier limits.


# Changelog



<Update label="September 2024">
  *   Introduced resumable query/search functionality, allowing to initiate and continue queries across multiple api calls.
      *   [Typescript SDK](../sdks/ts/commands/resumable-query)
      *   [Python SDK](../sdks/py/example_calls/resumable-query)
  *   Added embedding latency charts to the Upstash Console vector usage page.
</Update>

<Update label="July 2024">
  Added new [`HAS FIELD`](../features/filtering#has-field) and [`HAS NOT FIELD`](../features/filtering#has-not-field) metadata filtering operators.
</Update>

<Update label="June 2024">
  *   Implemented a new feature to store raw data in text format alongside metadata. See [Metadata and Data](../features/metadata#data).
  *   Added an API for updating vector, data, or metadata. It's also possible to update the metadata without overwriting all.
      See [Update Vector](../api/endpoints/update) API.
  *   Added reset all namespaces API. See [Reset Namespace](../api/endpoints/reset) API.
  *   Improved query APIs to send a batch of queries in a single request. See [Query](../api/endpoints/query) API.
</Update>

<Update label="May 2024">
  *   Introduced [namespaces feature](https://upstash.com/docs/vector/features/namespaces).
  *   Added GCP US-Central1 region.
</Update>

<Update label="April 2024">
  Added option to upsert and query raw data using [Upstash embedding service](https://upstash.com/docs/vector/features/embeddingmodels).
</Update>

<Update label="February 2024">
  Implemented metadata filtering with SQL-like syntax. See [Metadata Filtering](https://upstash.com/docs/vector/features/filtering).
</Update>

<Update label="January 2024">
  [Initial release!](https://upstash.com/blog/introducing-vector-database)
</Update>


# Compare



// todo melek


# Getting Started



<Check>
  **Prerequisite**

  You need an Upstash account before creating a vector, create one
  [here](https://console.upstash.com).
</Check>

## Create an Index

Once you logged in, you can create a Vector Index by clicking on the `Create Index` button in the Vector tab.

<Frame style={{width: '600px'}}>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/getstarted/create_index.png" />
</Frame>

**Name:** Type a name for your index.

**Region:** Choose the region for your index. For optimal performance, select the region closest to your applications. We plan to support additional regions and cloud providers. Feel free to send your requests to [support@upstash.com](mailto:support@upstash.com)

**Dimensions:** Select the dimensions and distance metric depending on your model.

<Frame style={{width: '600px'}}>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/getstarted/select_plan.png" />
</Frame>

**Free:** The free plan is suitable for small projects. It has a limit of 10,000 queries and 10,000 updates daily.

**Pay as You Go:** Pay as you go plan is a flexible plan with per-request-pricing. It is suitable for projects with unpredictable traffic.

**Fixed:** Fixed plan is suitable for projects with predictable traffic. It has a fixed monthly price with 1M query and 1M updates daily.

**Pro:** Pro plan is suitable for projects with high traffic and storage needs. It has a fixed monthly price with extra security and isolation features.

**Enterprise:** If you plan to have over a billion vectors then Enterprise plan is for you. It has a fixed monthly price with extra security and isolation features. Contact us at [sales@upstash.com](mailto:sales@upstash.com) for more information.

## Insert Index

You can access data in your index using REST API or our SDKs. You can copy the sample code from the `Connect` section in the console.

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(url="UPSTASH_VECTOR_REST_URL", token="UPSTASH_VECTOR_REST_TOKEN")

    index.upsert(
      vectors=[
        ("1", [0.6, 0.8], {"field": "value"}),
      ]
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```ts
    import { Index } from "@upstash/vector";

    const index = new Index({
        url: "UPSTASH_VECTOR_REST_URL",
        token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.upsert({ id: "1", vector: [0.6, 0.8], metadata: {field: "value"} })
    ```
  </Tab>

  <Tab title="Go">
    ```go
    import "github.com/upstash/vector-go"

    func main() {
      index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

      index.Upsert(vector.Upsert{
    	  Id:       "1",
    	  Vector:   []float32{0.6, 0.8},
    	  Metadata: map[string]any{"field": "value"},
      })
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell
    curl $UPSTASH_VECTOR_REST_URL/upsert \
      -X POST \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{"id": "1", "vector": [0.6, 0.8], "metadata": {"field": "value"}}'
    ```
  </Tab>
</Tabs>

## Query Index

You can perform a similarity search by providing a query vector as a parameter. The dimension of the query vector must match the dimension of your index. Also, you can query by metadata filtering.

<Note>
  Upstash is eventually consistent, so there may be a delay before the newly inserted or updated vectors are ready for querying.
</Note>

<Tabs>
  <Tab title="Python">
    ```python
    from upstash_vector import Index

    index = Index(url="UPSTASH_VECTOR_REST_URL", token="UPSTASH_VECTOR_REST_TOKEN")

    index.query(
        vector=[0.6, 0.8],
        top_k=3,
        include_metadata=True,
    )
    ```
  </Tab>

  <Tab title="JavaScript">
    ```ts
    import { Index } from "@upstash/vector";

    const index = new Index({
      url: "UPSTASH_VECTOR_REST_URL",
      token: "UPSTASH_VECTOR_REST_TOKEN",
    })

    await index.query({ vector: [0.6, 0.8], topK: 3, includeMetadata: true })
    ```
  </Tab>

  <Tab title="Go">
    ```go
    import "github.com/upstash/vector-go"

    func main() {
      index := vector.NewIndex("UPSTASH_VECTOR_REST_URL", "UPSTASH_VECTOR_REST_TOKEN")

      index.Query(vector.Query{
    	  Vector:          []float32{0.6, 0.8},
    	  TopK:            3,
    	  IncludeMetadata: true,
      })
    }
    ```
  </Tab>

  <Tab title="curl">
    ```shell

    curl $UPSTASH_VECTOR_REST_URL/query \
      -H "Authorization: Bearer $UPSTASH_VECTOR_REST_TOKEN" \
      -d '{"vector": [0.6, 0.8], "topK": 3, "includeMetadata": "true"}'
    ```
  </Tab>
</Tabs>

## Charts and Query Browser

<Frame style={{width: '600px'}}>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/vector/getstarted/charts.png" />
</Frame>

In Upstash console, you can see the charts of your index and query your index with a simple UI. There are following charts:

*   **Daily Requests:** The number of queries and updates to your index in the last 5 days.
*   **Throughput:** The number of queries and updates to your index in the selected time period.
*   **Latency:** The mean and P99 latency of queries and updates to your index in the selected time period.
*   **Vector Count:** The number of vectors in your index in the selected time period.
*   **Data Size:** The size of your index in the selected time period.


# Pricing & Limits



Please check our [pricing page](https://upstash.com/pricing/vector) for the most up-to-date information on pricing and limits.


# What is Upstash Vector?



Upstash Vector is a serverless vector database designed for working with vector embeddings.

In the domain of databases, a vector database is essential for managing numeric representations of
objects (images, sounds, text, etc.) in a multi-dimensional space.
These databases are focused on efficiently handling vectors for storage, retrieval, and, most importantly, querying based on similarity.
They are instrumental in integrating personalized data into AI applications
enabling the AI system to provide tailored answers derived from your own dataset rather than generic responses,
thereby enhancing the relevance and specificity of the generated insights.

# Core Features:

*   **Serverless Architecture and Pricing:**
    Upstash Vector operates on a serverless model, abstracting away hosting and management complexities. Engineers are billed based on API calls,
    ensuring a transparent and cost-effective pricing structure aligned with our general philosophy and practices across all products.
    See [pricing](https://upstash.com/pricing/vector) for more details.

*   **Low-Cost High-Performance Queries:**
    Improved upon to the DiskANN\[1], Upstash Vector enables engineers to execute queries with high recall rates and low latencies compared to exhaustive search methods.
    DiskANN's efficiency significantly enhances the performance of vector queries, making it a powerful tool for data retrieval and analysis.
    See [Approximate Nearest Neighbor Search](../features/algorithm) to get more insight on the tech behind the Upstash Vector.

*   **Vector Similarity Functions:**
    Powered by DiskANN, Upstash Vector supports different similarity functions, including Euclidean distance, Cosine similarity, and Dot Product.
    See [Vector Similarity](../features/similarityfunctions) for more details.

*   **REST API and SDKs:**
    The product is equipped with a REST API and SDKs (Python and TypeScript) for seamless integration into engineering workflows. Explore the technical details of these interfaces to incorporate Upstash Vector into your codebase effortlessly.

*   **Metadata Support:**
    Upstash Vector allows users to attach metadata to vectors, enhancing data context.
    This feature contributes to referencing the original content upon retrieval.
    See [Metadata documentation](../features/metadata) for more details.

*   **Metadata Filtering:**
    Upstash Vector allows users to filter out the similarity search results based on specific metadata criteria.
    See [Metadata Filtering documentation](../features/filtering) for more details.

### References

1.  Subramanya, S. J., Devvrit, Kadekodi, R., Krishaswamy, R., Simhadri, H. V. (2019). *DiskANN: Fast Accurate Billion-Point Nearest Neighbor Search on a Single Node*. In Proceedings of the 33rd International Conference on Neural Information Processing Systems (NeurIPS '19), Article No.: 1233, Pages 13766–13776. \[[https://dl.acm.org/doi/abs/10.5555/3454287.3455520](https://dl.acm.org/doi/abs/10.5555/3454287.3455520)]


# Go SDK





# Semantic Cache JS





# Semantic Cache Python





# Delete



## Method

The `delete` method allows you to remove vectors from the index based on their identifiers. It returns the following field in response:

*   `deleted`: An integer indicating how many vectors were deleted with the command.

## Delete Example

```python
from upstash_vector import Index

index = Index.from_env()

# Specify the identifiers of vectors to be deleted
ids_to_delete = ["id1", "id2", "id3"]

# Delete the specified vectors
delete_result = index.delete(ids=ids_to_delete)

# Display the number of vectors deleted
print("Number of Vectors Deleted:", delete_result.deleted)
```

Alternatively, you can delete a singular vector:

```python
index.delete("id-4")
```

Also, you can specify a namespace to operate on. When no namespace
is provided, the default namespace will be used.

```python
index.delete("id-4", namespace="ns")
```


# Fetch



## Method

The `fetch` method allows you to retrieve vectors from the index based on their identifiers. It takes the following input parameters:

*   `ids`: A string or a list of strings representing the identifiers of the vectors to be fetched.
*   `include_vectors`: A boolean flag indicating whether to include vectors in the fetch results.
*   `include_metadata`: A boolean flag indicating whether to include metadata in the fetch results.
*   `include_data`: A boolean flag indicating whether to include data in the fetch results.

As a response, following field is returned:

*   `vectors`: A list containing information for each fetched vector, including `id`, `vector`, `metadata`, and `data`.

## Fetch Example

```python
from upstash_vector import Index

index = Index.from_env()

# Specify the identifiers of vectors to be fetched
ids_to_fetch = ["id1", "id2", "id3"]

# Fetch the specified vectors with vectors and metadata included
fetch_result = index.fetch(
    ids=ids_to_fetch,
    include_vectors=True,
    include_metadata=True,
    include_data=True,
)

# Display the fetched vectors
for vector_info in fetch_result:
    print("ID:", vector_info.id)
    print("Vector:", vector_info.vector)
    print("Metadata:", vector_info.metadata)
    print("Data:", vector_info.data)
```

Alternatively, you can fetch a singular vector:

```python
index.fetch("id-4")
```

Also, you can specify a namespace to operate on. When no namespace
is provided, the default namespace will be used.

```python
index.fetch("id-4", namespace="ns")
```


# Info



## Method

The `info` method provides statistical information about the index, returning the following fields:

*   `vector_count`: The total number of vectors in the index.
*   `pending_vector_count`: The number of vectors that are currently pending (not yet fully processed).
*   `index_size`: The size of the index in bytes.
*   `dimension`: How many dimensions the index has
*   `similarity_function`: Similarity function chosen for the index
*   `namespaces`: Map of namespace names of the index to their statistics.

## Info Example

```python
from upstash_vector import Index

index = Index.from_env()

from upstash_vector import Index

index = Index.from_env()

# Get statistical information about the index
info_result = index.info()

# Display the info result
print("Vector Count:", info_result.vector_count)
print("Pending Vector Count:", info_result.pending_vector_count)
print("Index Size:", info_result.index_size)
print("Dimension:", info_result.dimension)
print("Similarity Function:", info_result.similarity_function)

for ns, ns_info in info_result.namespaces.items():
    print("Namespace:", ns, "Vector Count:", ns_info.vector_count)
    print("Namespace:", ns, "Pending Vector Count:", ns_info.pending_vector_count)
```


# Query



## Method

To retrieve vectors from the index based on specific criteria, you can use the `query` method, which accepts the following parameters:

*   `vector`: The reference vector for similarity comparison.
*   `include_metadata`: A boolean flag indicating whether to include metadata in the query results.
*   `include_vector`: A boolean flag indicating whether to include vectors in the query results.
*   `include_data`: A boolean flag indicating whether to include data in the query results.
*   `top_k`: The number of top matching vectors to retrieve.
*   `filter`: Metadata filtering of the vector is used to query your data based on the filters and narrow down the query results.

As response, the object has the following fields:

*   `id`: The identifier associated with the matching vector.
*   `metadata`: Additional information or attributes linked to the matching vector.
*   `score`: A measure of similarity indicating how closely the vector matches the query vector. The score is normalized to the range \[0, 1], where 1 indicates a perfect match.
*   `vector`: The vector itself (included only if `include_vector` is set to `True`).
*   `data`: Additional unstructured information linked to the matching vector.

<Tip>If you wanna learn more about filtering check: [Metadata Filtering](https://upstash.com/docs/vector/features/filtering)</Tip>

## Query Example

```python
import random

from upstash_vector import Index

index = Index.from_env()

# Generate a random vector for similarity comparison
dimension = 128  # Adjust based on your index's dimension
query_vector = [random.random() for _ in range(dimension)]

# Execute the query
query_result = index.query(
    vector=query_vector,
    include_metadata=True,
    include_data=True,
    include_vectors=False,
    top_k=5,
    filter="genre = 'fantasy' and title = 'Lord of the Rings'",
)

# Print the query result
for result in query_result:
    print("Score:", result.score)
    print("ID:", result.id)
    print("Vector:", result.vector)
    print("Metadata:", result.metadata)
    print("Data:", result.data)
```

## Batch Query Method

It is also possible to perform a batch of queries in a single
call to eliminate round trips to server.

## Batch Query Example

```python
import random

from upstash_vector import Index

index = Index.from_env()

# Generate a random vector for similarity comparison
dimension = 128  # Adjust based on your index's dimension
query_vectors = [[random.random() for _ in range(dimension)] for _ in range(2)]

# Execute the query
query_results = index.query_many(
    queries=[
        {
            "vector": query_vectors[0],
            "include_metadata": True,
            "include_data": True,
            "include_vectors": False,
            "top_k": 5,
            "filter": "genre = 'fantasy' and title = 'Lord of the Rings'",
        },
        {
            "vector": query_vectors[1],
            "include_metadata": False,
            "include_data": False,
            "include_vectors": True,
            "top_k": 3,
            "filter": "genre = 'drama'",
        },
    ]
)

for i, query_result in enumerate(query_results):
    print(f"Query-{i} result:")

    # Print the query result
    for result in query_result:
        print("Score:", result.score)
        print("ID:", result.id)
        print("Vector:", result.vector)
        print("Metadata:", result.metadata)
        print("Data:", result.data)
```

Also, you can specify a namespace to operate on. When no namespace
is provided, the default namespace will be used.

```python
index.query(..., namespace="ns")
```


# Range



## Method

The `range` method allows you to retrieve vectors from the index within a specified range. The function accepts the following parameters:

*   `cursor`: A cursor to start the range query.
*   `limit`: The maximum number of vectors to retrieve in a single query.
*   `include_vectors`: A boolean flag indicating whether to include vectors in the range results.
*   `include_metadata`: A boolean flag indicating whether to include metadata in the range results.
*   `include_data`: A boolean flag indicating whether to include data in the range results.

As response, the object has the following fields:

*   `next_cursor`: A cursor indicating the position to start the next range query. If `""`, there are no more results.
*   `vectors`: A list containing information for each vector, including `id`, `vector`, and `metadata`.

## Range Example

```python
from upstash_vector import Index

index = Index.from_env()

# Execute the range query
range_result = index.range(
    cursor="",
    limit=10,
    include_vectors=False,
    include_metadata=True,
    include_data=True,
)

# Print the range result
print("Next Cursor:", range_result.next_cursor)

for vector_info in range_result.vectors:
    print("ID:", vector_info.id)
    print("Vector:", vector_info.vector)
    print("Metadata:", vector_info.metadata)
    print("Data:", vector_info.data)
```

## Scanning Whole Index

For scanning the entire index, you can use a similar loop as shown below:

```python
res = index.range(cursor="", limit=5)
print(res.vectors)

while res.next_cursor != "":
    res = index.range(cursor=res.next_cursor, limit=10)
    print(res.vectors)
```

Also, you can specify a namespace to operate on. When no namespace
is provided, the default namespace will be used.

```python
index.range(..., namespace="ns")
```


# Reset



## Method

The `reset` method allows you to clear all vectors and metadata from a particular
namespace or all namespaces of an index.

## Reset Example

Resets the default namespace.

```python
from upstash_vector import Index
index = Index.from_env()

index.reset()
```

## Reset Namespace Example

Resets the given namespace.

```python
from upstash_vector import Index
index = Index.from_env()

index.reset(namespace="ns")
```

## Reset All Namespaces

Resets all the namespaces of an index.

```python
from upstash_vector import Index
index = Index.from_env()

index.reset(all=True)
```


# Resumable Query



The resumable query feature allows you to perform queries that can be resumed to fetch additional results. This is particularly useful for large result sets or when implementing pagination.

## Synchronous Usage

### Creating a Resumable Query

To create a resumable query, use the `resumable_query` method of the `Index` class:

```python
query = index.resumable_query(
    vector=[0.1, 0.2],  # or use 'data' parameter for text-based queries
    top_k=2,
    include_metadata=True,
    include_vectors=True,
    namespace="your_namespace"
)
```

Parameters:

*   vector: A list of floats representing the query vector.
*   data: A string for text-based queries (mutually exclusive with vector).
*   top\_k: The number of results to fetch initially.
*   include\_metadata: Whether to include metadata in the results.
*   include\_vectors: Whether to include vector values in the results.
*   max-idle: The maximum idle time for the query in seconds.
*   namespace: The namespace to query (optional).

### Starting the Query

To start the query and get initial results:

```python
initial_results = query.start()
```

The initial\_results will be a list of result objects, each having properties like
id and metadata (if included).

### Fetching More Results

To fetch additional results:

```python
next_results = query.fetch_next(number_of_results)
```

This method returns a list of additional results. If no more results are available,
it returns an empty list.

### Stopping the Query

When you're done with the query, stop it to release resources:

```python
stop_result = query.stop()
assert stop_result == 'Success'
```

### Asynchronous Usage

For asynchronous operations, use the AsyncIndex class:

#### Creating an Async Resumable Query

```python
query = await async_index.resumable_query(
  vector=[0.1, 0.2],
  top_k=2,
  include_metadata=True,
  include_vectors=True,
  namespace='your_namespace'
)
```

#### Starting the Async Query

```python
initial_results = await query.async_start()
```

#### Fetching More Results Asynchronously

```python
next_results = await query.async_fetch_next(number_of_results)
```

#### Stopping the Async Query

```python
stop_result = await query.async_stop()
assert stop_result == 'Success'
```

#### Error Handling

After stopping a query, attempting to fetch more results or stop it again will raise a `ClientError`:

```python
with pytest.raises(ClientError):
  query.fetch_next(1)
  query.async_fetch_next(1)

for async with pytest.raises(ClientError):
  query.stop() # or await query.async_stop() for async
```

Example: Fetching All Results Here's an example of how to fetch all results using
a resumable query:

```python
query = index.resumable_query(
  vector=[0.1, 0.2],
  top_k=2,
  include_metadata=True
)
results = query.start()
while True:
  next_batch = query.fetch_next(2)
  if not next_batch:
    break
  results.extend(next_batch)
query.stop()
```

This pattern allows you to efficiently retrieve large result sets without loading
everything into memory at once.

<Note>
  Remember to always stop your query when you're done to release server-side
  resources.
</Note>

### Example

```python
from upstash_vector import Index

# Create an index instance
index = Index()

# Upsert vectors into the index
index.upsert(
    vectors=[
        ("id1", [0.1, 0.2], {"field": "value1"}),
        ("id2", [0.3, 0.4], {"field": "value2"}),
        ("id3", [0.5, 0.6], {"field": "value3"}),
    ],
    namespace="example-namespace"
)

# Start a resumable query
query = index.resumable_query(
    vector=[0.1, 0.2],
    top_k=2,
    include_metadata=True,
    include_vectors=True,
    namespace="example-namespace"
)

# Fetch initial results
results = query.start()

# Access result data
for result in results:
    print(f"ID: {result.id}, Metadata: {result.metadata}")

# Stop the query when done
query.stop()
```


# Update



## Methods

The `update` method enables you to update the `vector`, `metadata`, or `data`
of a vector.

## Update Example

```python
from upstash_vector import Index

index = Index.from_env()

updated = index.update(
    id="id1",
    metadata={"new": "metadata"},
    data="new-data",
)

print(updated)
```

## Patch Metadata Example

It is also possible to patch metadata (update or delete existing fields or
set new fields) according the [JSON Merge Patch](https://datatracker.ietf.org/doc/html/rfc7386) algorithm.

```python
from upstash_vector import Index
from upstash_vector.types import MetadataUpdateMode

index = Index.from_env()

updated = index.update(
    id="id2",
    metadata={
        "existing-field": "new-value",
        "existing-field-to-delete": None,
        "new-field": "new-value",
    },
    metadata_update_mode=MetadataUpdateMode.PATCH,
)

print(updated)
```

Also, you can specify a namespace to operate on. When no namespace
is provided, the default namespace will be used.

```python
index.update(..., namespace="ns")
```


# Upsert



## Methods

The `upsert` method enables you to insert or update vectors in the index.
You can perform upsert operations in three ways: using a vector object, a tuple, or a dictionary.

### Upsert Via Vector Object

```python
import random

from upstash_vector import Index, Vector

index = Index.from_env()

dimension = 128  # Adjust based on your index's dimension
upsert_amount = 100

vectors = [
    Vector(
        id=f"generated-id-{i}",
        vector=[random.random() for _ in range(dimension)],
        metadata={"some_field": f"some_value-{i}"},
        data=f"some-unstructured-data-{i}",
    )
    for i in range(upsert_amount)
]

index.upsert(vectors=vectors)
```

### Upsert Via Tuple

```python
import random

from upstash_vector import Index

index = Index.from_env()

dimension = 128  # Adjust based on your index's dimension
upsert_amount = 100

vectors = [
    (
        f"generated-id-{i}",
        [random.random() for _ in range(dimension)],
        {"some_field": f"some_value-{i}"},
        f"some-unstructured-data-{i}",
    )
    for i in range(upsert_amount)
]

index.upsert(vectors=vectors)
```

### Upsert Via Dictionary

```python
import random

from upstash_vector import Index

index = Index.from_env()

dimension = 128  # Adjust based on your index's dimension
upsert_amount = 100

vectors = [
    {
        "id": f"generated-id-{i}",
        "vector": [random.random() for _ in range(dimension)],
        "metadata": {"some_field": f"some_value-{i}"},
        "data": f"some-unstructured-data-{i}",
    }
    for i in range(upsert_amount)
]

index.upsert(vectors=vectors)
```

Also, you can specify a namespace to operate on. When no namespace
is provided, the default namespace will be used.

```python
index.upsert(..., namespace="ns")
```


# Features



# Retry Mechanism

The `upstash-vector` SDK incorporates a reliable retry mechanism to manage network or API issues.
In the event of a failed request, the SDK automatically attempts up to three retries,
with each attempt spaced one second apart.

For customization of the retry behavior, you have the flexibility to set the `retries` and
`retry_interval` (in seconds) parameters according to your specific requirements.
For instance:

```python
from upstash_vector import Index

# Try 5 times with a 2-second interval between retries
index = Index.from_env(retries=5, retry_interval=2.0)
```


# Getting Started



The `upstash-vector` SDK is a lightweight, HTTP-based Upstash Vector client designed for Python. It seamlessly operates in both serverless and serverful environments, ensuring optimal compatibility across various connection setups.

This SDK simplifies interaction with Upstash Vector through the [Upstash Vector API](https://docs.upstash.com/vector/api/get-started).

It is designed to work with Python versions 3.8 and above.

Explore the source code, contribute, and stay informed through our [GitHub Repository](https://github.com/upstash/vector-py).

## Install

To begin using `upstash-vector`, you can install it via PyPI using the following command:

```bash
pip install upstash-vector
```

## Usage

Before using upstash-vector, you'll need to set up a vector database on [Upstash](https://console.upstash.com/). Once created, grab your URL and TOKEN from the Upstash console.

To initialize the index client:

```python
from upstash_vector import Index
index = Index(url="UPSTASH_VECTOR_REST_URL", token="UPSTASH_VECTOR_REST_TOKEN")
```

Alternatively, you can automatically load the credentials from the environment:

```python
from upstash_vector import Index
index = Index.from_env()
```

For serverless environments that allow it, it's recommended to initialize the client outside the request handler to be reused while your function is still "hot."

Here's an example of how you can use the SDK in your Python application:

```python
import random
from upstash_vector import Index

# Initialize the index client using environment variables
index = Index.from_env()

def main():
    # Define the dimension based on the index configuration
    dimension = 128
    # Generate a random vector for upsert
    vector_to_upsert = [random.random() for _ in range(dimension)]
    # Additional metadata associated with the vector
    metadata = {"text": "example test for metadata"}

    # Upsert the vector into the index
    index.upsert(vectors=[
        ("id-for-vector", vector_to_upsert, metadata)
    ])

```

The example above demonstrates how to upsert a vector with metadata using the SDK into the Upstash Vector database.

## More SDK Features

For additional functionalities and usage examples, check out the [Commands](/vector/sdks/py/example_calls) section in the documentation.


# Methods



RAGChat exposes several useful methods:

| Method            | Description                                                           |
| ----------------- | --------------------------------------------------------------------- |
| `ragChat.chat`    | automatically retrieves relevant knowledge to include in LLM response |
| `ragChat.context` | access or modify your Vector DB                                       |
| `ragChat.history` | access or modify your chat history                                    |

## `chat`

The chat method enables conversations with a language model (LLM) using your vector database as a knowledge base and chat history.

### Key Features:

*   Get LLM answers based on your knowledge base
*   Custom message history length
*   Supports streaming for a better UX
*   Customizable hooks for fine-grained control over the chat process
*   Optional pure LLM chat mode without RAG

### Important Considerations:

*   Adjust `similarityThreshold` to fine-tune accuracy of results
*   Balance information: Too much context can lead to inaccuracies
*   Use `historyLength` and `topK` options to manage the amount of data fed to the LLM

### Request

<ParamField path="input" type="string" required>
  Question to ask your RAG chat
</ParamField>

<ParamField path="options" type="ChatOptions">
  <Expandable title="properties">
    <ParamField path="historyLength" type="number">
      Number of previous conversation messages included in the request to LLM
    </ParamField>

    {" "}

    <ParamField path="historyTTL" type="number">
      Configuration to retain chat history. After the specified time, the history is automatically
      deleted
    </ParamField>

    {" "}

    <ParamField path="similarityThreshold" type="number">
      Value from 0-1 to determine when a piece of information from your knowledge base is considered
      relevant to the current query (default: 0.5)
    </ParamField>

    {" "}

    <ParamField path="topK" type="number">
      Number of data points included in request to LLM
    </ParamField>

    {" "}

    <ParamField path="ratelimitDetails" type="function">
      Details of the rate limit applied (if any)
    </ParamField>

    {" "}

    <ParamField path="onChunk" type="function">
      Hook to modify or get data and details of each chunk. Can be used to alter streamed content.
    </ParamField>

    {" "}

    <ParamField path="onContextFetched" type="function">
      Hook to access (and modify) the retrieved context
    </ParamField>

    {" "}

    <ParamField path="onChatHistoryFetched" type="function">
      Hook to access (and modify) the conversation history
    </ParamField>

    {" "}

    <ParamField path="disableRAG" type="boolean">
      Allows you to disable the inclusion of the knowledge base with query and just chat to the LLM
      directly
    </ParamField>

    {" "}

    <ParamField path="streaming" type="boolean">
      Stream AI-responses in real time as they are generated
    </ParamField>

    {" "}

    <ParamField path="sessionId" type="string">
      Chat session ID of the user interacting with the application
    </ParamField>

    {" "}

    <ParamField path="namespace" type="string">
      Limit included context elements to a specific namespace (i.e. results for a specific user)
    </ParamField>

    {" "}

    <ParamField path="metadata" type="UpstashDict">
      Metadata about your chat message. This could be used to store anything in the chat history. By
      default, the RAG Chat SDK uses this to persist the used model name in the history.
    </ParamField>

    {" "}

    <ParamField path="ratelimitSessionId" type="string">
      Rate limit session ID of the user interacting with the application
    </ParamField>

    <ParamField path="promptFn" type="CustomPrompt">
      Provide a custom LLM prompt with access to the user question, chat history and knowledge base items
    </ParamField>
  </Expandable>
</ParamField>

### Response

<ResponseField name="response" type="ChatReturnType">
  <Expandable>
    <ResponseField name="output" type="ReadableStream<string> | string">
      LLM output as a readable stream or a string
    </ResponseField>

    <ResponseField name="isStream" type="boolean">
      Indicates if the output is a stream (mostly for internal use)
    </ResponseField>

    <ResponseField name="metadata" type="Record<string, unknown>">
      Metadata attached to item in knowledge base (if any)
    </ResponseField>
  </Expandable>
</ResponseField>

### Usage Tips:

*   Adjust `historyLength` based on the complexity of your conversations
*   Use `topK` to control the number of relevant context items retrieved
*   Enable `streaming` for a better UX
*   Use the `onContextFetched` and `onChatHistoryFetched` hooks to inspect and modify data before it's sent to the LLM
*   Set `disableRAG` to true when building custom pipelines or for simple LLM interactions

### Example:

```javascript
const chatResponse = await ragChat.chat({
  input: "What are the key features of our latest product?",
  options: {
    historyLength: 5,
    topK: 3,
    streaming: true,
    onContextFetched: (context) => {
      console.log("Retrieved context:", context)
      return context
    },
    metadata: { source: "product_inquiry" },
  },
})

if (chatResponse.isStream) {
  for await (const chunk of chatResponse.output) {
    console.log(chunk)
  }
} else {
  console.log(chatResponse.output)
}
```

## `context`

Context describes the items retrieved from your vector database considered relevant for the current query. Add, delete or update data from your knowledge base:

| Method                | Description                                    |
| --------------------- | ---------------------------------------------- |
| `add`                 | add various data types into a vector database. |
| `addMany`             | add multiple pieces of data at once.           |
| `delete`              | remove data from your knowledge base           |
| `deleteEntireContext` | clears your entire knowledge base              |

### `add`

The `add` method adds various types of context to your application. It supports multiple data formats, automatically handles text splitting, and simplifies adding and managing context.

#### Supported Data Types:

*   Plain text
*   Embeddings
*   PDF files
*   CSV files
*   Text files
*   HTML content (from files or URLs)

#### Key Features:

*   Automatic text splitting for PDF, CSV, and HTML input
*   Flexible configuration options for each data type
*   Support for both file and URL sources for HTML content

#### Request

<ParamField path="payload" type="AddContextPayload">
  <Expandable>
    <ParamField path="type" type="string" required>
      Available data types: **"text"**, **"embedding"**, **"pdf"**, **"csv"**, **"text-file"**,
      **"html"**.
    </ParamField>

    <ParamField path="data" type="string | number[]">
      The data to add. This is required for types **"text"** and **"embedding"**.
    </ParamField>

    <ParamField path="fileSource" type="FilePath | Blob">
      The source of the file. This is required for types **"pdf"**, **"csv"**, **"text-file"**, and
      "html" when the source is a file.
    </ParamField>

    <ParamField path="source" type="URL">
      The URL source. This is required for type **"html"** when the source is a URL.
    </ParamField>

    <ParamField path="options" type="AddContextOptions">
      Additional options for adding context.
    </ParamField>

    <ParamField path="config" type="Partial<RecursiveCharacterTextSplitterParams>">
      Configuration for text splitting. This is optional for types **"pdf"**, **"text-file"**, and
      **"html"**.
    </ParamField>

    <ParamField path="pdfConfig" type="{ parsedItemSeparator?: string; splitPages?: boolean }">
      Configuration specific to PDF parsing.
    </ParamField>

    <ParamField path="csvConfig" type="{ column?: string; separator?: string }">
      Configuration specific to CSV parsing.
    </ParamField>

    <ParamField path="htmlConfig" type="WebBaseLoaderParams">
      Configuration specific to HTML parsing.
    </ParamField>

    <ParamField path="id" type="string | number">
      Optional ID for the context.
    </ParamField>
  </Expandable>
</ParamField>

#### Response

<ResponseField name="response" type="SaveOperationResult">
  <Expandable>
    <ResponseField name="success" type="boolean">
      Indicates if the operation was successful.
    </ResponseField>

    <Expandable condition="success === true">
      <ResponseField name="ids" type="string[]">
        The IDs of the saved items.
      </ResponseField>
    </Expandable>

    <Expandable condition="success === false">
      <ResponseField name="error" type="string">
        The error message if the operation failed.
      </ResponseField>
    </Expandable>
  </Expandable>
</ResponseField>

#### Usage Tips:

*   For large documents, consider using the PDF or text file options to use automatic text splitting.
*   When adding CSV data, use the `csvConfig` to specify which column to use and adjust the delimiter if necessary.
*   For web content, you can either specify an HTML file or a URL using the appropriate configuration.
*   Use the `options` parameter to fine-tune how the context is added and processed.

#### Example:

```javascript
const result = await context.add({
  type: "pdf",
  fileSource: "./document.pdf",
  pdfConfig: { splitPages: true },
  options: { metadata: { source: "company_report" } },
})

if (result.success) {
  console.log(`Added ${result.ids.length} context items`)
} else {
  console.error(`Failed to add context: ${result.error}`)
}
```

### `addMany`

The `addMany` method is an efficient way to add multiple contexts to your application in a single operation.
It extends the capabilities of the add method to handle batch processing, making it ideal for large-scale data ingestion.

### `delete`

The `delete` method allows you to remove one or more context items from your vector database. It provides an easy way to manage your knowledge base by deleting specific items or groups of items.

#### Key Features:

*   Supports deletion of a single item or multiple items in a single operation
*   Optional namespace specification for targeted deletion

#### Request

<ParamField path="payload" type="DeleteContextPayload">
  <Expandable>
    <ParamField path="id" type="string | string[]" required>
      The ID(s) of the context items to be deleted.
    </ParamField>

    <ParamField path="namespace" type="string">
      Optional namespace to specify which index to delete from. If not specified, the default
      namespace will be deleted.
    </ParamField>
  </Expandable>
</ParamField>

#### Response

The method doesn't return a value, but it's asynchronous, so you should wait for it to finish.

#### Usage Tips:

*   Use a single ID string to delete a single item, or an array of ID strings for batch deletions
*   Specify a namespace when working with multiple indices to ensure you're deleting from the correct one
*   Always wait for the delete operation to complete before proceeding with other operations.

#### Example:

```javascript
// Deleting a single item
await context.delete({
  id: "item_123",
  namespace: "product_catalog",
})

// Deleting multiple items
await context.delete({
  id: ["item_456", "item_789", "item_101"],
  namespace: "customer_data",
})
```

### `deleteEntireContext`

The `deleteEntireContext` method removes all knowledge base items, either for a single namespace or for all namespaces.

#### Key Features:

*   Clears your entire knowledge base
*   Optional: Specify a namespace for targeted deletion

#### Request

<ParamField path="options" type="ResetOptions">
  <Expandable>
    <ParamField path="namespace" type="string">
      Optional namespace to specify which index to reset. If not specified, the entire vector
      database will be reset.
    </ParamField>
  </Expandable>
</ParamField>

#### Response

The method doesn't return a value, but it's asynchronous, so you should wait for it to complete.

#### Usage Tips:

*   Use this method with caution, as it will permanently delete all context items.
*   Specify a namespace if you only want to reset a specific part of your database.

#### Example:

```javascript
// Resetting the entire vector database
await context.deleteEntireContext()
console.log("Entire context deleted successfully")

// Resetting a specific namespace
await context.deleteEntireContext({
  namespace: "temporary_data",
})
```

<Warning>
  This operation is irreversible. Make sure you really want to delete all context items before you
  before calling this method. It's recommended that you only use this method when you're absolutely
  sure you want to delete all data.
</Warning>

## `history`

The `history` feature allows you to manage your chat history. It provides methods to add, retrieve, and delete messages, giving you full control over the conversation data stored in your application.

<Info>
  In typical use, you won't need to manipulate your chat history manually. However, having the
  ability to directly access and modify messages can be useful in certain scenarios.
</Info>

### Key Features:

*   Add individual messages to the chat history
*   Retrieve messages from the chat history
*   Delete specific messages or entire conversations
*   Support for multiple chat sessions
*   Optional time-to-live (TTL) for automatic session expiry

### `addMessage`

The `addMessage` method allows you to add a new message to the chat history for a specific session.

#### Request

<ParamField path="payload" type="HistoryAddMessage">
  <Expandable>
    <ParamField path="message" type="Omit<UpstashMessage, 'id'>" required>
      The message object to be added to the chat history. It includes the following fields:

      <Expandable>
        <ParamField path="role" type="'assistant' | 'user'" required>
          Specifies whether the message is from the assistant or the user.
        </ParamField>

        <ParamField path="content" type="string" required>
          The message content.
        </ParamField>

        <ParamField path="metadata" type="TMetadata">
          Optional metadata associated with the message.
        </ParamField>

        <ParamField path="usage_metadata" type="object">
          Optional metadata about token usage:

          <Expandable>
            <ParamField path="input_tokens" type="number">
              Number of input tokens used.
            </ParamField>

            <ParamField path="output_tokens" type="number">
              Number of output tokens used.
            </ParamField>

            <ParamField path="total_tokens" type="number">
              Total number of tokens used.
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>

      Note: The 'id' field is omitted as it's handled internally.
    </ParamField>

    <ParamField path="sessionId" type="string">
      Optional identifier for the chat session. If not provided, a default session ID will be used.
    </ParamField>

    <ParamField path="sessionTTL" type="number">
      Optional time-to-live (in seconds) for the chat session. If provided, the session will
      automatically expire after this duration.
    </ParamField>
  </Expandable>
</ParamField>

#### Response

The method doesn't return a value, but it's asynchronous, so you should wait for it to complete.

#### Usage Tips:

*   Ensure the `role` is set to either "assistant" or "user"
*   Use the `usage_metadata` to track token usage if needed for your use case
*   Use unique session IDs to manage multiple concurrent conversations
*   Set a TTL for sessions that should automatically expire

#### Example:

```javascript
await history.addMessage({
  message: {
    role: "user",
    content: "What's the weather like today?",
    metadata: {
      userLocation: "New York",
      timestamp: Date.now(),
    },
    usage_metadata: {
      input_tokens: 6,
      output_tokens: 0,
      total_tokens: 6,
    },
  },
  sessionId: "user_456_weather_chat",
  sessionTTL: 1800, // Session will expire after 30 minutes
})
```

### `deleteMessages`

The `deleteMessages` method allows you to remove all messages associated with a specific chat session.

#### Request

<ParamField path="payload" type="DeleteMessagesPayload">
  <Expandable>
    <ParamField path="sessionId" type="string" required>
      The identifier of the chat session whose messages you want to delete.
    </ParamField>
  </Expandable>
</ParamField>

#### Response

The method doesn't return a value, but it's asynchronous, so you should wait for it to complete.

#### Usage Tips:

*   Use this method to clear the history of a specific chat session.
*   Be careful when using this method, as it will permanently delete all messages in the session.

#### Example:

```javascript
const sessionIdToDelete = "user_789_support_chat"

await history.deleteMessages({ sessionId: sessionIdToDelete })

console.log(`All messages in session ${sessionIdToDelete} have been deleted`)
```

<Warning>
  This operation is irreversible. Once messages are deleted, they cannot be recovered. Use this
  method carefully and implement safeguards in your application to prevent accidental data loss.
</Warning>

### `getMessages`

The `getMessages` method retrieves several messages from a chat session's history. It provides flexibility in selecting the range of messages to fetch and returns them in reverse chronological order (newest first).

#### Request

<ParamField path="options" type="GetMessagesOptions">
  <Expandable>
    <ParamField path="sessionId" type="string" default="DEFAULT_CHAT_SESSION_ID">
      Chat session ID from which to retrieve messages. If not provided, a default
      session ID will be used.
    </ParamField>

    <ParamField path="amount" type="number" default="DEFAULT_HISTORY_LENGTH">
      The amount of messages to retrieve.
    </ParamField>

    <ParamField path="startIndex" type="number" default="0">
      The starting index from which to retrieve messages (i.e. for pagination). Defaults to 0 (most recent message).
    </ParamField>
  </Expandable>
</ParamField>

#### Response

<ResponseField name="messages" type="UpstashMessage[]">
  An array of UpstashMessage objects, where each object contains:

  <Expandable>
    <ResponseField name="role" type="'assistant' | 'user'">
      Indicates whether the message is from the assistant or the user.
    </ResponseField>

    <ResponseField name="content" type="string">
      The actual content of the message.
    </ResponseField>

    <ResponseField name="metadata" type="TMetadata" optional>
      Additional metadata associated with the message.
    </ResponseField>

    <ResponseField name="usage_metadata" type="object" optional>
      Metadata about token usage:

      <Expandable>
        <ResponseField name="input_tokens" type="number">
          Number of input tokens used.
        </ResponseField>

        <ResponseField name="output_tokens" type="number">
          Number of output tokens used.
        </ResponseField>

        <ResponseField name="total_tokens" type="number">
          Total number of tokens used.
        </ResponseField>
      </Expandable>
    </ResponseField>

    <ResponseField name="id" type="string">
      A unique identifier for the message, generated based on its position in the history.
    </ResponseField>
  </Expandable>
</ResponseField>

#### Usage Tips:

*   Use the `sessionId` parameter to retrieve messages from specific conversations
*   Adjust the `amount` parameter to control how many messages you want to fetch
*   Use the `startIndex` parameter to paginate through the chat history or fetch older messages
*   The returned messages are in reverse chronological order (newest first)
*   Each message includes an `id` field, which is generated based on its position in the chat history

#### Example:

```javascript
const messages = await history.getMessages({
  sessionId: "user_123_support_chat",
  amount: 10,
  startIndex: 0,
})

console.log(`Retrieved ${messages.length} messages:`)
messages.forEach((msg) => {
  console.log(`${msg.role}: ${msg.content} (ID: ${msg.id})`)
})
```


# Config Options



### Instance Options

Customize your RAGChat instance with advanced options:

```typescript

export const ragChat = new RAGChat({
  model: openai("gpt-4-turbo"),

  vector: new Index({
    url: "UPSTASH_VECTOR_REST_URL",
    token: "UPSTASH_VECTOR_REST_TOKEN",
  }),

  redis: new Redis({
    url: "UPSTASH_REDIS_REST_URL",
    token: "UPSTASH_REDIS_REST_TOKEN",
  }),

  ratelimit: new Ratelimit({
    redis: Redis.fromEnv(),
    limiter: Ratelimit.slidingWindow(10, "10 s"),
  }),
  
  debug: true,
  /** 
    * Set to `true` if working with web apps and you want to be interactive without stalling users.
    */
  streaming: true,
  /** Chat session ID of the user interacting with the application.
    * @default "upstash-rag-chat-session"
    */
  sessionId: "session_id",
  /**
    * Namespace of the index you wanted to query.
    */
  namespace: "namespace",
  /**
    * Metadata for your chat message. This could be used to store anything in the chat history. 
    * By default RAG Chat SDK uses this to persist used model name in the history
    */
  metadata: { key: "value"},
  
  /** Rate limit session ID of the user interacting with the application.
    * @default "upstash-rag-chat-ratelimit-session"
    */
  ratelimitSessionId: "RatelimitSessionId"

  promptFn: ({context, question, chatHistory}) =>
    `You are an AI assistant with access to an Upstash Vector Store.
      Use the provided context and chat history to answer the question.
      If the answer isn't available, politely inform the user.
      ------
      Chat history:
      ${chatHistory}
      ------
      Context:
      ${context}
      ------
      Question: ${question}
      Answer:`,
});
```

### Chat Options

Here below are the options that you can pass to the `chat` method of the `RAGChat` instance:

```typescript

ragChat.chat(
  "Tell me about machine learning",
  {
    /** Length of the conversation history to include in your LLM query. Increasing this may lead to hallucinations. Retrieves the last N messages.
     * @default 5
     */
    historyLength : 5,
    /** Configuration to retain chat history. After the specified time, the history will be automatically cleared.
     * @default 86_400 // 1 day in seconds
     */
    historyTTL : 86_400,
    /** Configuration to adjust the accuracy of results.
     * @default 0.5
     */
    similarityThreshold : 0.5,
    /** Amount of data points to include in your LLM query.
     * @default 5
     */
    topK : 5,
    /**
     *  Details of applied rate limit.
     */
    ratelimitDetails : (response: Awaited<ReturnType<Ratelimit["limit"]>>) => {},
    /**
     * Hook to modify or get data and details of each chunk. Can be used to alter streamed content.
     */
    onChunk : ({content, inputTokens, chunkTokens, totalTokens, rawContent,}: {
        inputTokens: number;
        chunkTokens: number;
        totalTokens: number;
        content: string;
        rawContent: string;
    }) => {},
    /**
     * Hook to access the retrieved context and modify as you wish.
     */
    onContextFetched : (context: PrepareChatResult) => {
        console.log(context);
        return context
    },
    /**
     * Hook to access the retrieved history and modify as you wish.
     */
    onChatHistoryFetched : (messages: UpstashMessage[]) => {
        console.log(messages);
        return messages;
    };
    /**
     * Allows disabling RAG and use chat as LLM in combination with prompt. This will give you ability to build your own pipelines.
     */
    disableRAG : false
  }
);
```


# Debug



RAGChat provides a powerful debugging feature that allows you to see the inner workings of your RAG applications. By enabling debug mode, you can trace the entire process from user input to final response.

#### Enable Debugging

To activate the debugging feature, simply initialize RAGChat with the `debug` option set to `true`:

```typescript
new RAGChat({ debug: true });
```

#### Debug Output

When debug mode is enabled, RAGChat will log detailed information about each step of the RAG process. Here's a breakdown of the debug output:

1.  **SEND\_PROMPT**: Logs the initial user query.

    ```json
    {
      "timestamp": 1722950191207,
      "logLevel": "INFO",
      "eventType": "SEND_PROMPT",
      "details": {
        "prompt": "Where is the capital of Japan?"
      }
    }
    ```

2.  **RETRIEVE\_CONTEXT**: Shows the relevant context retrieved from the vector store.

    ```json
    {
      "timestamp": 1722950191480,
      "logLevel": "INFO",
      "eventType": "RETRIEVE_CONTEXT",
      "details": {
        "context": [
          {
            "data": "Tokyo is the Capital of Japan.",
            "id": "F5BWpryYkkcKLrp-GznwK"
          }
        ]
      },
      "latency": "171ms"
    }
    ```

3.  **RETRIEVE\_HISTORY**: Displays the chat history retrieved for context.

    ```json
    {
      "timestamp": 1722950191727,
      "logLevel": "INFO",
      "eventType": "RETRIEVE_HISTORY",
      "details": {
        "history": [
          {
            "content": "Where is the capital of Japan?",
            "role": "user",
            "id": "0"
          }
        ]
      },
      "latency": "145ms"
    }
    ```

4.  **FORMAT\_HISTORY**: Shows how the chat history is formatted for the prompt.

    ```json
    {
      "timestamp": 1722950191828,
      "logLevel": "INFO",
      "eventType": "FORMAT_HISTORY",
      "details": {
        "formattedHistory": "USER MESSAGE: Where is the capital of Japan?"
      }
    }
    ```

5.  **FINAL\_PROMPT**: Displays the complete prompt sent to the language model.

    ```json
    {
      "timestamp": 1722950191931,
      "logLevel": "INFO",
      "eventType": "FINAL_PROMPT",
      "details": {
        "prompt": "You are a friendly AI assistant augmented with an Upstash Vector Store.\n  To help you answer the questions, a context and/or chat history will be provided.\n  Answer the question at the end using only the information available in the context or chat history, either one is ok.\n\n  -------------\n  Chat history:\n  USER MESSAGE: Where is the capital of Japan?\n  -------------\n  Context:\n  - Tokyo is the Capital of Japan.\n  -------------\n\n  Question: Where is the capital of Japan?\n  Helpful answer:"
      }
    }
    ```

6.  **LLM\_RESPONSE**: Shows the final response from the language model.
    ```json
    {
      "timestamp": 1722950192593,
      "logLevel": "INFO",
      "eventType": "LLM_RESPONSE",
      "details": {
        "response": "According to the context, Tokyo is the capital of Japan!"
      },
      "latency": "558ms"
    }
    ```


# Features



### Integration and Compatibility

*   Integrates with Next.js, Svelte, Nuxt.js and Solid.js
*   Ingest websites by URL, text files, CVSs, PDFs, and more out of the box
*   Stream AI-generated content in real time
*   Built-in vector store for your knowledge base
*   (Optional) built-in Redis for storing chat messages
*   (Optional) built-in rate limiting

### Flexibility

*   Support for chat sessions
*   Fully customizable prompt
*   Optional pure LLM chat mode without RAG

## Use Cases

*   Build chatbots with domain-specific knowledge
*   Create interactive documentation systems
*   Build AI-powered customer support tools


# Getting Started



Upstash RAG Chat is **TypeScript toolkit for building powerful RAG applications**. With Upstash RAG Chat, you can focus on building a high-quality chatbot without having to deal with complicated AI orchestration tools.

You can find the Github Repository [here](https://github.com/upstash/rag-chat).

### Installation

Install the package using your preferred package manager:

```sh
pnpm add @upstash/rag-chat

bun add @upstash/rag-chat

npm i @upstash/rag-chat
```

### Quick start

1.  Set up your environment variables:

```sh
UPSTASH_VECTOR_REST_URL="XXXXX"
UPSTASH_VECTOR_REST_TOKEN="XXXXX"


# if you use OpenAI compatible models
OPENAI_API_KEY="XXXXX"

# or if you use Upstash hosted models
QSTASH_TOKEN="XXXXX"

# Optional: For Redis-based chat history (default is in-memory)
UPSTASH_REDIS_REST_URL="XXXXX"
UPSTASH_REDIS_REST_TOKEN="XXXXX"
```

2.  Initialize and use RAGChat:

```typescript
import { RAGChat } from "@upstash/rag-chat";

const ragChat = new RAGChat();

const response = await ragChat.chat("Tell me about machine learning");
console.log(response);
```

### Configuring Your Chat Model

RAGChat supports both Upstash-hosted models and all OpenAI and OpenAI-compatible models out of the box:

#### Using OpenAI Models

To use an OpenAI model, first initialize RAGChat:

```typescript
import { RAGChat, openai } from "@upstash/rag-chat";

export const ragChat = new RAGChat({
  model: openai("gpt-4-turbo"),
});
```

And set your OpenAI API key as an environment variable:

```bash
OPENAI_API_KEY=...
```

#### Using Upstash-hosted Open-Source Models

To use an Upstash model, first initialize RAGChat:

```typescript
import { RAGChat, upstash } from "@upstash/rag-chat";

export const ragChat = new RAGChat({
  model: upstash("mistralai/Mistral-7B-Instruct-v0.2"),
});
```

And set your Upstash QStash API key environment variable:

```bash
QSTASH_TOKEN=...
```

#### Using Custom Providers - TogetherAi, Replicate

Initialize RAGChat with custom provider's API key and url:

```typescript
import { RAGChat, custom } from "@upstash/rag-chat";

export const ragChat = new RAGChat({
  model: custom("codellama/CodeLlama-70b-Instruct-hf", {
    apiKey: "TOGETHER_AI_API_KEY",
    baseUrl: "https://api.together.xyz/v1",
  }),
});
```

### Managing Your Knowledge Base

Add various types of data to your RAG application:

#### Adding Text

```typescript
await ragChat.context.add({
  type: "text",
  data: "The speed of light is approximately 299,792,458 meters per second.",
});

//OR

await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

#### Adding PDF Content

```typescript
await ragChat.context.add({
  type: "pdf",
  fileSource: "./data/quantum_computing_basics.pdf",

  // optional 👇: only add this knowledge to a specific namespace
  options: { namespace: "user-123-documents" },
});
```

#### Adding Web Content

```typescript
await ragChat.context.add({
  type: "html",
  source: "https://en.wikipedia.org/wiki/Quantum_computing",

  // optional 👇: custom page parsing settings
  config: { chunkOverlap: 50, chunkSize: 200 },
});
```

#### Removing Content

Remove specific documents:

```typescript
await ragChat.context.delete({ id: "1", namespace: "user-123-documents" });
```

### Managing Chat History

RAGChat provides robust functionality for interacting with and managing chat history. This allows you to maintain context, review past interactions, and customize the conversation flow.

#### Retrieving Chat History

Fetch recent messages from the chat history:

```typescript
const history = await ragChat.history.getMessages({ amount: 10 });
console.log(history); // 👈 Last (up to) 10 messages
```

You can also specify a session ID to retrieve history for a particular conversation:

```typescript
const sessionHistory = await ragChat.history.getMessages({
  amount: 5,
  sessionId: "user-123-session",
});
```

#### Deleting Chat History

Remove chat history for a specific session:

```typescript
ragChat.history.deleteMessages({ sessionId: "user-123-session" });
```

#### Adding Custom Messages

Injecting custom messages into the chat history:

```typescript
// Adding a user message
await ragChat.history.addMessage({
  message: { content: "What's the weather like?", role: "user" },
});

// Adding a system message
await ragChat.history.addMessage({
  message: {
    content: "The AI should provide weather information.",
    role: "system",
  },
});
```


# How To



#### Update prompt

```ts
const result = await ragChat.chat("THIS_IS_USERS_QUESTION", {
  promptFn: ({ question, chat_history }) => {
    return `
      Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question.
      Chat History:
      ${chatHistory}
      Follow Up Input: ${question}
      Standalone question:
    `;
  },
});
```

#### Adjust the context quality

If your chatbot is negatively affected by the provided context, you can adjust the context quality using the `similarityThreshold` option. The similarityThreshold is a number between 0 and 1, with a default value of 0.5. Increasing this value will improve quality, but the chatbot may reject more queries due to lack of relevant data.

```ts
ragChat.chat("Tell me about machine learning", {
  similarityThreshold: 0.5
});
```

#### Disable RAG to bypass context

When RAG disabled, RAG chat skips the vector store and directly sends the question to the LLM model.

```ts
const result = await ragChat.chat("THIS_IS_USERS_QUESTION", {
  disableRAG: true
});
```

#### Enable debug logs

```ts
new RAGChat({ debug: true });
```

#### Access the context

You can access and update the context by using the `onContextFetched` callback. This callback is called after the context is fetched from the vector store before the prompt is sent to the LLM.

```ts
let response = await ragChat.chat(question, {
  onContextFetched: (context) => {
    console.log("Retrieved context:", context)
    return context
  },
});
```

#### Access the chat history

You can access chat history by using the `onChatHistoryFetched` callback. This callback is called after the chat history is fetched from the Redis (or memory) before the prompt is sent to the LLM.

```ts
let response = await ragChat.chat(question, {
  onChatHistoryFetched: (messages) => {
    console.log("Retrieved chat history:", messages)
    return messages
  },
});
```

#### Add metadata into context

You can add metadata while you are inserting a message into the context. The array of metadata fetched from context will be returned with response.

```ts
export const rag = new RAGChat();
await rag.context.add({
  type: "text",
  data: "Mississippi is a state in the United States of America",
  options: { metadata: { source: "wikipedia" }}
});
const response = await rag.chat("Where is Mississippi?");

console.log(response.metadata);
// [ {source: "wikipedia"} ]
```

#### Access the ratelimit details

You can see how much allowance a user has by using the ratelimitDetails hook.

```ts
await ragChat.chat("You shall not pass", {
  streaming: false,
  ratelimitDetails: (response) => {
    remainingLimit = response.remaining;
  },
});
```

#### Access the output during streaming

You can access, read, or modify your streamed data.

```ts
await ragChat.chat("", {
  onChunk({ content, inputTokens, chunkTokens, totalTokens, rawContent }) {
    // Change or read your streamed chunks
    // Examples:
    // - Log the content: console.log(content);
    // - Modify the content: content = content.toUpperCase();
    // - Track token usage: console.log(`Tokens used: ${totalTokens}`);
    // - Analyze raw content: processRawContent(rawContent);
    // - Update UI in real-time: updateStreamingUI(content);
  },
});
```

#### Store metadata in the chat history and access context

```ts
let messages: UpstashMessage[] = []
let context: PrepareChatResult = []

const response = await ragChat.chat(question, {
  onChatHistoryFetched(_messages) {
    messages = _messages
    this.metadata = {
      ...this.metadata,
      usedHistory: JSON.stringify(
        messages.map((message) => {
          delete message.metadata?.usedHistory
          delete message.metadata?.usedContext
          return message
        })
      ),
    }
    return _messages
  },
  onContextFetched(_context) {
    context = _context
    this.metadata = { ...this.metadata, usedContext: context.map((x) => x.data.replace("-", "")) }
    return _context
  },
  streaming: true,
  metadata: {
    modelNameWithProvider: `${LLM_MODELS[llmModel].provider}_${llmModel}`,
  },
})
```


# Anthropic



[Anthropic](https://www.anthropic.com/) is a language model provider. Check out [Anthropic API](https://www.anthropic.com/api) for more information about their models and pricing.

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Anthropic

Create an Anthropic account and get an API key from [Anthropic Console -> Settings -> API keys](https://console.anthropic.com/settings/keys). Set your Anthropic API key as an environment variable:

```bash .env
ANTHROPIC_API_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat with the Anthropic model:

```typescript index.ts
import { RAGChat, anthropic } from "@upstash/rag-chat";
import "dotenv/config";

export const ragChat = new RAGChat({
  model: anthropic("claude-3-5-sonnet-20240620",{apiKey: process.env.ANTHROPIC_API_KEY}),
});
```

Add context to the RAG Chat:

```typescript index.ts
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

Chat with the RAG Chat:

```typescript index.ts
const response = await ragChat.chat("What is the speed of light?");
console.log(response);
```

### Run

Run the project:

```bash
npx tsx index.ts
```


# Custom Models



You can incorporate any 3rd party OpenAI compatible LLM into RAG Chat. We will use [Together AI](https://www.together.ai/) in this tutorial. Check out [Together AI API](https://api.together.ai/models) for more information about their models and pricing.

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Together AI

Create a Together AI account and get an API key from [Together AI API -> Settings -> API KEYS](https://api.together.ai/settings/api-keys). Set your Together AI API key as an environment variable:

```bash .env
TOGETHER_AI_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat with custom model:

```typescript index.ts
import { RAGChat, custom } from "@upstash/rag-chat";
import "dotenv/config";

export const ragChat = new RAGChat({
  model: custom("meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo", {
    apiKey: process.env.TOGETHER_AI_KEY,
    baseUrl: "https://api.together.xyz/v1",
  }),
});
```

Add context to the RAG Chat:

```typescript index.ts
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

Chat with the RAG Chat:

```typescript index.ts
const response = await ragChat.chat("What is the speed of light?");
console.log(response);
```

### Run

Run the project:

```bash
npx tsx index.ts
```


# Groq



[Groq](https://groq.com/) is a language model provider. Check out [Groq Pricing](https://groq.com/pricing/) for more information about their models and pricing.

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Groq

Create a Groq account and get an API key from [Groq Console -> API Keys](https://console.groq.com/keys). Set your Groq API key as an environment variable:

```bash .env
GROQ_AI_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat with the Groq model:

```typescript index.ts
import { RAGChat, groq } from "@upstash/rag-chat";
import "dotenv/config";

export const ragChat = new RAGChat({
  model: groq("llama-3.1-70b-versatile",{apiKey: process.env.GROQ_AI_KEY}),
});
```

Add context to the RAG Chat:

```typescript index.ts
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

Chat with the RAG Chat:

```typescript index.ts
const response = await ragChat.chat("What is the speed of light?");
console.log(response);
```

### Run

Run the project:

```bash
npx tsx index.ts
```


# Helicone



[Helicone](https://www.helicone.ai/) is a powerful observability platform that provides valuable insights into your LLM usage. Check out [Helicone - Pricing](https://www.helicone.ai/pricing) for more information about their product and pricing.

To enable Helicone observability in RAGChat, you simply need to pass your Helicone API key when initializing your model. Here's how to do it for both custom models and OpenAI:

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.env` file.

```shell .env
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Setup Helicone

Create a Helicone account and get an API key from [Helicone -> Settings -> API Keys](https://us.helicone.ai/settings/api-keys). Set your Helicone API key as an environment variable:

```bash .env
HELICONE_API_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat with Helicone analytics:

```typescript index.ts
import { RAGChat, upstash } from "@upstash/rag-chat";
import "dotenv/config";

const ragChat = new RAGChat({
  model: upstash("meta-llama/Meta-Llama-3-8B-Instruct", {
    apiKey: process.env.QSTASH_TOKEN,
    analytics: { name: "helicone", token: process.env.HELICONE_API_KEY! },
  }),
});
```

Add context to the RAG Chat:

```typescript index.ts
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

Chat with the RAG Chat:

```typescript index.ts
const response = await ragChat.chat("What is the speed of light?");
console.log(response);
```

### Run

Run the project:

```bash
npx tsx index.ts
```

Go to the [Helicone Dashboard](https://us.helicone.ai/dashboard) to view your analytics.


# LangSmith



[LangSmith](https://www.langchain.com/langsmith) is a powerful development platform for LLM applications that provides valuable insights, debugging tools, and performance monitoring. Integrating LangSmith with RAGChat can significantly enhance your development workflow and application quality.

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.env` file.

```shell .env
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Setup LangSmith

Create a LangSmith account and get an API key from LangSmith -> Settings -> API Keys. Set your LangSmith API key as an environment variable:

```bash .env
LANGCHAIN_API_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat with LangSmith analytics:

```typescript index.ts
import { RAGChat, upstash } from "@upstash/rag-chat";
import "dotenv/config";

const ragChat = new RAGChat({
  model: upstash("meta-llama/Meta-Llama-3-8B-Instruct", {
    apiKey: process.env.QSTASH_TOKEN,
    analytics: { name: "langsmith", token: process.env.LANGCHAIN_API_KEY! },
  }),
});
```

Add context to the RAG Chat:

```typescript index.ts
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

Chat with the RAG Chat:

```typescript index.ts
const response = await ragChat.chat("What is the speed of light?");
console.log(response);
```

### Run

Run the project:

```bash
npx tsx index.ts
```

Go to the [LangSmith Dashboard](https://smith.langchain.com/) and navigate to Projects to view your analytics.


# Mistral AI



[Mistral AI](https://mistral.ai/) is a language model provider. Check out [Mistral AI Technology](https://mistral.ai/technology/) for more information about their models and pricing.

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Mistral AI

Create a Mistral AI account and get an API key from [Mistral AI Console -> La Plateforme -> API Keys](https://console.mistral.ai/api-keys/). Set your Mistral AI API key as an environment variable:

```bash .env
MISTRAL_AI_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat with the Mistral AI model:

```typescript index.ts
import { RAGChat, mistralai } from "@upstash/rag-chat";
import "dotenv/config";

export const ragChat = new RAGChat({
  model: mistralai("mistral-small-latest",{apiKey: process.env.MISTRAL_AI_KEY}),
});
```

Add context to the RAG Chat:

```typescript index.ts
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

Chat with the RAG Chat:

```typescript index.ts
const response = await ragChat.chat("What is the speed of light?");
console.log(response);
```

### Run

Run the project:

```bash
npx tsx index.ts
```


# Next.js



### Next.js Route Handlers

RAGChat integrates with Next.js route handlers out of the box. See [our example project for a complete view](https://github.com/upstash/rag-chat/tree/master/examples/nextjs/chat-to-website).

Here's how to use it:

#### Basic usage

```typescript
import { ragChat } from "@/utils/rag-chat";
import { NextResponse } from "next/server";

export const POST = async (req: Request) => {
  // 👇 user message
  const { message } = await req.json();
  const { output } = await ragChat.chat(message);
  return NextResponse.json({ output });
};
```

#### Streaming responses

To stream the response from a route handler:

```typescript
import { ragChat } from "@/utils/rag-chat";

export const POST = async (req: Request) => {
  const { message } = await req.json();
  const { output } = await ragChat.chat(message, { streaming: true });
  return new Response(output);
};
```

On the frontend, you can read the streamed data like this:

```typescript
"use client"

export const ChatComponent = () => {
  const [response, setResponse] = useState('');

  async function fetchStream() {
    const response = await fetch("/api/chat", {
      method: "POST",
      body: JSON.stringify({ message: "Your question here" }),
    });

    if (!response.body) {
      console.error("No response body");
      return;
    }

    const reader = response.body.getReader();
    const decoder = new TextDecoder();

    while (true) {
      const { done, value } = await reader.read();
      if (done) break;

      const chunk = decoder.decode(value, { stream: true });
      setResponse(prev => prev + chunk);
    }
  }

  useEffect(() => {
    fetchStream();
  }, []);

  return <div>{response}</div>;
}
```

### Next.js Server Actions

RAGChat supports Next.js server actions natively. See [our example project on GitHub](https://github.com/upstash/rag-chat/tree/master/examples/nextjs/server-actions).

First, define your server action:

```typescript
"use server";

import { ragChat } from "@/utils/rag-chat";
import { createServerActionStream } from "@upstash/rag-chat/nextjs";

export const serverChat = async (message: string) => {
  const { output } = await ragChat.chat(message, { streaming: true });

  // 👇 adapter to let us stream from server actions
  return createServerActionStream(output);
};
```

Second, use the server action in your client component:

```typescript
"use client";

import { readServerActionStream } from "@upstash/rag-chat/nextjs";

export const ChatComponent = () => {
  const [response, setResponse] = useState('');

  const clientChat = async () => {
    const stream = await serverChat("How are you?");

    for await (const chunk of readServerActionStream(stream)) {
      setResponse(prev => prev + chunk);
    }
  };

  return (
    <div>
      <button onClick={clientChat}>Start Chat</button>
      <div>{response}</div>
    </div>
  );
};
```


# Ollama



You can use Ollama as LLM provider for your local development in RAGChat. To use an Ollama model, first initialize RAGChat with the Ollama model:

```typescript
import { RAGChat, ollama } from "@upstash/rag-chat";
export const ragChat = new RAGChat({
  model: ollama("llama3.1"),
});
```


# OpenRouter



[OpenRouter](https://openrouter.ai/) is a language model provider. Check out [OpenRouter Models](https://openrouter.ai/models) for more information about their models and pricing.

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup OpenRouter

Create an OpenRouter account and get an API key from [OpenRouter -> Settings -> API Keys](https://openrouter.ai/settings/keys). Set your OpenRouter API key as an environment variable:

```bash .env
OPEN_ROUTER_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat with the OpenRouter model:

```typescript index.ts
import { RAGChat, openrouter } from "@upstash/rag-chat";
import "dotenv/config";

export const ragChat = new RAGChat({
  model: openrouter("sao10k/l3-lunaris-8b",{apiKey: process.env.OPEN_ROUTER_KEY}),
});
```

Add context to the RAG Chat:

```typescript index.ts
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

Chat with the RAG Chat:

```typescript index.ts
const response = await ragChat.chat("What is the speed of light?");
console.log(response);
```

### Run

Run the project:

```bash
npx tsx index.ts
```


# OpenAI



[OpenAI](https://openai.com) is a language model provider. Check out [OpenAI API](https://openai.com/api/) for more information about their models and pricing.

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup OpenAI

Create an OpenAI account and get an API key from [OpenAI Platform -> Dashboard -> API keys](https://platform.openai.com/api-keys). Set your OpenAI API key as an environment variable:

```bash .env
OPENAI_API_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat with the OpenAI model:

```typescript index.ts
import { RAGChat, openai } from "@upstash/rag-chat";
import "dotenv/config";

export const ragChat = new RAGChat({
  model: openai("gpt-4-turbo"),
});
```

Add context to the RAG Chat:

```typescript index.ts
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

Chat with the RAG Chat:

```typescript index.ts
const response = await ragChat.chat("What is the speed of light?");
console.log(response);
```

### Run

Run the project:

```bash
npx tsx index.ts
```


# Overview



## Models

Work with the state-of-the-art language models from various providers.

<CardGroup cols={2}>
  <Card title="OpenAI" href="/vector/sdks/rag-chat/integrations/openai" />

  <Card title="Together AI" href="/vector/sdks/rag-chat/integrations/togetherai" />

  <Card title="Anthropic" href="/vector/sdks/rag-chat/integrations/anthropic" />

  <Card title="Mistral AI" href="/vector/sdks/rag-chat/integrations/mistralai" />

  <Card title="OpenRouter" href="/vector/sdks/rag-chat/integrations/open-router" />

  <Card title="Groq" href="/vector/sdks/rag-chat/integrations/groq" />

  <Card title="Ollama" href="/vector/sdks/rag-chat/integrations/ollama" />

  <Card title="Custom Models" href="/vector/sdks/rag-chat/integrations/custom" />
</CardGroup>

## Observability

Monitor and analyze your RAG Chat application with ease.

<CardGroup cols={2}>
  <Card title="Helicone" href="/vector/sdks/rag-chat/integrations/helicone" />

  <Card title="LangSmith" href="/vector/sdks/rag-chat/integrations/langsmith" />
</CardGroup>

## Frameworks

Use RAG Chat with industry-leading frameworks.

<CardGroup cols={2}>
  <Card title="Next.js" href="/vector/sdks/rag-chat/integrations/nextjs" />

  <Card title="Vercel AI SDK" href="/vector/sdks/rag-chat/integrations/vercel-ai" />
</CardGroup>

## Processors

Process any kind of data into context for RAG Chat.

<CardGroup cols={2}>
  <Card title="Unstructured" href="/vector/sdks/rag-chat/integrations/unstructured" />
</CardGroup>


# Together AI



[Together AI](https://www.together.ai/) is a language model provider. Check out [Together AI API](https://api.together.ai/models) for more information about their models and pricing.

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Together AI

Create a Together AI account and get an API key from [Together AI API -> Settings -> API KEYS](https://api.together.ai/settings/api-keys). Set your Together AI API key as an environment variable:

```bash .env
TOGETHER_AI_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat with the Together AI model:

```typescript index.ts
import { RAGChat, togetherai } from "@upstash/rag-chat";
import "dotenv/config";

export const ragChat = new RAGChat({
  model: togetherai("meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo",{apiKey: process.env.TOGETHER_AI_KEY}),
});
```

Add context to the RAG Chat:

```typescript index.ts
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
```

Chat with the RAG Chat:

```typescript index.ts
const response = await ragChat.chat("What is the speed of light?");
console.log(response);
```

### Run

Run the project:

```bash
npx tsx index.ts
```


# Unstructured



[Unstructured](https://unstructured.io/) extracts complex data from difficult-to-use formats like HTML, PDF, CSV, and more. Check out [Unstructured - Product](https://unstructured.io/product) for more information about their product and pricing.

### Install RAG Chat SDK

[Install Bun](https://bun.sh/docs/installation) if you haven't.

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
npm i --save-dev @types/bun  
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.env` file.

```shell .env
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Setup Unstructured

Create an Unstructured account and get an API key from [Unstructed -> API Keys](https://app.unstructured.io/keys). Set your Unstructed API key as an environment variable:

```bash .env
UNSTRUCTURED_IO_KEY=<YOUR_API_KEY>
```

### Setup the Project

Initialize RAGChat:

```typescript index.ts
import { RAGChat, upstash } from "@upstash/rag-chat";
import "dotenv/config";

const ragChat = new RAGChat({
  model: upstash("meta-llama/Meta-Llama-3-8B-Instruct"),
});
```

Fetch a webpage and save it to a file:

```typescript index.ts
const fileSource = "./hackernews.html";
const response = await fetch("https://news.ycombinator.com/");
await Bun.write(fileSource, await response.text());
```

Add context to the RAG Chat with the Unstructured processor:

```typescript index.ts
await ragChat.context.add({
  options: {
    namespace: "unstructured-upstash",
  },
  fileSource,
  processor: {
    name: "unstructured",
    options: { apiKey: process.env.UNSTRUCTURED_IO_KEY },
  },
});
```

Chat with the RAG Chat:

```typescript index.ts
const result = await ragChat.chat("What is the second story on hacker news?", {
  streaming: false,
  namespace: "unstructured-upstash",
});
```

### Run

Run the project:

```bash
bun run index.ts
```


# Vercel AI SDK



RAGChat can be easily integrated with the Vercel AI SDK. See [our example project for a more complete view](https://github.com/upstash/rag-chat/tree/master/examples/nextjs/vercel-ai-sdk).

If you want to use RAGChat & Vercel AI SDK in your project, first set up your route handler:

```typescript
import { aiUseChatAdapter } from "@upstash/rag-chat/nextjs";
import { ragChat } from "@/utils/rag-chat";

export async function POST(req: Request) {
  const { messages } = await req.json();
  const lastMessage = messages[messages.length - 1].content;

  const response = await ragChat.chat(lastMessage, { streaming: true });
  return aiUseChatAdapter(response);
}
```

Second, use the `useChat` hook in your frontend component:

```typescript
"use client"

import { useChat } from "ai/react";

const ChatComponent = () => {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: "/api/chat",
    initialInput: "What year was the construction of the Eiffel Tower completed, and what is its height?",
  });

  return (
    <div>
      <ul>
        {messages.map((m) => (
          <li key={m.id}>{m.content}</li>
        ))}
      </ul>

      <form onSubmit={handleSubmit}>
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask a question..."
        />
        <button type="submit">Send</button>
      </form>
    </div>
  );
};
```

## Using `ai-sdk` models in RAG Chat

If you're already using the ai-sdk library and want to incorporate its models into RAG Chat, you can easily do so with minimal configuration changes. This integration allows you to maintain your existing AI setup while benefiting from RAG Chat's advanced features.

```ts
import { openai } from "@ai-sdk/openai";

const ragChat = new RAGChat({
  model: openai("gpt-3.5-turbo"),
  vector,
  redis: new Redis({
    token: process.env.UPSTASH_REDIS_REST_TOKEN!,
    url: process.env.UPSTASH_REDIS_REST_URL!,
  }),
  sessionId: "ai-sdk-session",
  namespace: "ai-sdk",
});
```


# Cloudflare Workers



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/rag-chat/tree/master/examples/cloudflare-workers" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use Upstash RAG Chat with Cloudflare Workers. You can also explore our [Cloudflare Workers - Hono example](https://github.com/upstash/rag-chat/tree/master/examples/nextjs/cloudflare-workers) for detailed, end-to-end examples and best practices.

### Project Setup

Create a new Cloudflare Worker and install `@upstash/rag-chat` package.

```shell
npm create cloudflare@latest -- my-first-worker
cd my-first-worker
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.dev.vars` file.

```shell .dev.vars
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.dev.vars` file.

```shell .dev.vars
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.dev.vars` file.

```shell .dev.vars
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Setup Cloudflare Worker

<Note>Note that `cache` must be set to `false` in the `Index` constructor for Cloudflare Workers as seen in the example below.</Note>

Update `/src/index.ts`:

```typescript /src/index.ts
import { RAGChat, upstash } from "@upstash/rag-chat";
import { Index } from "@upstash/vector";
import { Redis } from "@upstash/redis/cloudflare";

export interface Env {
  UPSTASH_REDIS_REST_TOKEN: string;
  UPSTASH_REDIS_REST_URL: string;
  UPSTASH_VECTOR_REST_TOKEN: string;
  UPSTASH_VECTOR_REST_URL: string;
  QSTASH_TOKEN: string;
}

export default {
  async fetch(request, env, ctx): Promise<Response> {
    const ragChat = new RAGChat({
      redis: new Redis({
		token: env.UPSTASH_REDIS_REST_TOKEN,
		url: env.UPSTASH_REDIS_REST_URL
	  }),
      vector: new Index({
		token: env.UPSTASH_VECTOR_REST_TOKEN,
		url: env.UPSTASH_VECTOR_REST_URL,
		cache: false
	  }),
      model: upstash("meta-llama/Meta-Llama-3-8B-Instruct", {
		apiKey: env.QSTASH_TOKEN
	  }),
	});

    const response = await ragChat.chat("What is the speed of light?");
    return new Response(response.output);
  },
} satisfies ExportedHandler<Env>;
```

### Run

Run the Cloudflare Worker locally:

```shell
npx wrangler dev
```

Visit `http://localhost:8787`

### Deploy

For deployment, use the `wrangler` CLI to securely set environment variables. Run the following command for each secret:

```shell
npx wrangler secret put SECRET_NAME
```

Replace `SECRET_NAME` with the actual name of each environment variable (e.g., `UPSTASH_REDIS_REST_URL`).

Deploy the Cloudflare Worker:

```shell
npx wrangler deploy
```


# Hono



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/rag-chat/tree/master/examples/cloudflare-workers" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use Upstash RAG Chat with Hono. You can also explore our [Cloudflare Workers - Hono example](https://github.com/upstash/rag-chat/tree/master/examples/nextjs/cloudflare-workers) for detailed, end-to-end examples and best practices.

### Project Setup

Create a new Hono application with `cloudflare-workers` template and install `@upstash/rag-chat` package.

```shell
npm create hono@latest my-app
cd my-app
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.dev.vars` file.

```shell .dev.vars
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.dev.vars` file.

```shell .dev.vars
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.dev.vars` file.

```shell .dev.vars
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Create a Hono App

<Note>Note that we are using the `@upstash/redis/cloudflare` package and setting `cache` to `false` in the `Index` constructor to be able to work in Cloudflare Workers, these are not related to Hono.</Note>

Update `/src/index.ts`:

```typescript /src/index.ts
import { Hono } from 'hono'
import { RAGChat, upstash } from "@upstash/rag-chat";
import { Index } from "@upstash/vector";
import { Redis } from "@upstash/redis/cloudflare";

const app = new Hono<{
  Bindings: {
    UPSTASH_REDIS_REST_URL: string;
    UPSTASH_REDIS_REST_TOKEN: string;
    UPSTASH_VECTOR_REST_URL: string;
    UPSTASH_VECTOR_REST_TOKEN: string;
    QSTASH_TOKEN: string;
  };
}>();

app.get('/', async (c) => {
  const ragChat = new RAGChat({
    redis: new Redis({
      token: c.env.UPSTASH_REDIS_REST_TOKEN,
      url: c.env.UPSTASH_REDIS_REST_URL
    }),
    vector: new Index({
      token: c.env.UPSTASH_VECTOR_REST_TOKEN,
      url: c.env.UPSTASH_VECTOR_REST_URL,
      cache: false
    }),
    model: upstash("meta-llama/Meta-Llama-3-8B-Instruct", {
      apiKey: c.env.QSTASH_TOKEN
    }),
  });

  const response = await ragChat.chat("What is the speed of light?");
  return c.text(response.output)
})

export default app
```

### Run

Run the Hono application locally:

```bash
npm run dev
```

Visit `http://localhost:8787`

### Deploy

For deployment, use the `wrangler` CLI to securely set environment variables. Run the following command for each secret:

```shell
npx wrangler secret put SECRET_NAME
```

Replace `SECRET_NAME` with the actual name of each environment variable (e.g., `UPSTASH_REDIS_REST_URL`).

Deploy the Hono application:

```bash
npm run deploy
```


# Next.js



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/rag-chat/tree/master/examples/nextjs/chat-to-website" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use Upstash RAG Chat with Next.js. You can also explore our [Next.js Chat to Website example](https://github.com/upstash/rag-chat/tree/master/examples/nextjs/chat-to-website) for detailed, end-to-end examples and best practices.

### Project Setup

Create a new Next.js application and install `@upstash/rag-chat` package.

```shell
npx create-next-app@latest
cd my-app
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.env` file.

```shell .env
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Configure Middleware for Cookies

Create a middleware in `/app/middleware.ts` to generate a session ID cookie. This cookie will be used to track the user's chat session.

```typescript middleware.ts
import { NextRequest, NextResponse } from "next/server";

export function middleware(req: NextRequest) {
  const res = NextResponse.next();

  const cookie = req.cookies.get("sessionId");

  if (!cookie) {
    res.cookies.set("sessionId", crypto.randomUUID());
  }

  return res;
}
```

### Create a Chat Streaming API Route

We will create a RAG Chat instance in `/app/lib/rag-chat.ts` to later use in our API route and Home page.

```typescript /app/lib/rag-chat.ts
import { RAGChat, upstash } from "@upstash/rag-chat";
import { Redis } from "@upstash/redis";

const redis = Redis.fromEnv();

export const ragChat = new RAGChat({
  model: upstash("meta-llama/Meta-Llama-3-8B-Instruct"),
  redis: redis,
});
```

Using the RAG Chat instance we created, we can now create a chat streaming API route in `/app/api/chat-stream/route.ts`.

```typescript /app/api/chat-stream/route.ts
import { ragChat } from "@/app/lib/rag-chat";
import { aiUseChatAdapter } from "@upstash/rag-chat/nextjs";
import { NextRequest } from "next/server";

export const POST = async (req: NextRequest) => {
  const { messages, sessionId } = await req.json();

  const lastMessage = messages[messages.length - 1].content;

  const response = await ragChat.chat(lastMessage, {
    streaming: true,
    sessionId,
  });

  return aiUseChatAdapter(response);
};
```

### Create a Chat Component

We can now create a chat component in `/app/components/chat.tsx` using the chat streaming API route we created.

```typescript /app/components/chat.tsx
"use client";

import { useChat, Message } from "ai/react";

export const Chat = ({
  sessionId,
  initialMessages,
}: {
  sessionId: string;
  initialMessages: Message[];
}) => {
  const { messages, handleInputChange, handleSubmit, input, setInput } =
    useChat({
      api: "/api/chat-stream",
      body: { sessionId },
      initialMessages,
    });

  return (
    <div>
      <div className="flex h-[calc(100vh-64px)] flex-col overflow-y-auto items-center">
        {messages.length ? (
          messages.map((message, i) => (
            <div className="w-[60vw] flex flex-row gap-3 p-6" key={i}>
              <div className="min-w-12">
                {message.role === "user" ? "User" : "Bot"}
              </div>
              <p className="text-sm font-normal text-gray-900">
                {message.content}
              </p>
            </div>
          ))
        ) : (
          <div className="text-zinc-500 text-sm pt-6">
            Ask your first question to get started.
          </div>
        )}
      </div>
      <form onSubmit={handleSubmit} className="relative">
        <div className="flex flex-row gap-2 p-3 h-16 items-center">
          <input
            onChange={handleInputChange}
            value={input}
            onKeyDown={(e) => {
              if (e.key === "Enter" && !e.shiftKey) {
                e.preventDefault();
                handleSubmit();
                setInput("");
              }
            }}
            placeholder="Enter your question..."
            className="border rounded-xl w-full py-1 px-3"
          />

          <button type="submit" className="border rounded-xl py-1 px-3">
            Send
          </button>
        </div>
      </form>
    </div>
  );
};
```

### Update Home Page

Update `/app/page.tsx` using the Chat component we created.

```tsx /app/page.tsx
import { Chat } from "./components/chat";
import { cookies } from "next/headers";
import { ragChat } from "@/app/lib/rag-chat";

export default async function Home() {
  const sessionId = cookies().get("sessionId")?.value;
  const initialMessages = await ragChat.history.getMessages({
    amount: 10,
    sessionId,
  });
  return <Chat sessionId={sessionId!} initialMessages={initialMessages} />;
}
```

### Run & Deploy

Run the app locally with `npm run dev`, check `http://localhost:3000/`

Deploy your app with `vercel`


# Next.js Server Actions



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/rag-chat/tree/master/examples/nextjs/server-actions" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use Upstash RAG Chat with Next.js Server Actions. You can also explore our [Next.js Server Actions example](https://github.com/upstash/rag-chat/tree/master/examples/nextjs/server-actions) for detailed, end-to-end examples and best practices.

### Project Setup

Create a new Next.js application and install `@upstash/rag-chat` package.

```shell
npx create-next-app@latest
cd my-app
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.env` file.

```shell .env
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Create a Next.js Server Action

Create `/app/actions.ts`:

```typescript /app/actions.ts
"use server";

import { RAGChat, upstash } from "@upstash/rag-chat";
import { createServerActionStream } from "@upstash/rag-chat/nextjs";

const ragChat = new RAGChat({
  model: upstash("meta-llama/Meta-Llama-3-8B-Instruct"),
});

export const serverChat = async (message: string) => {
  const { output } = await ragChat.chat(message, { streaming: true });

  // 👇 adapter to let us stream from server actions
  return createServerActionStream(output);
};
```

### Create a Chat Component

Create `/app/components/chat.tsx` using the server action:

```typescript /app/components/chat.tsx
"use client";

import { useState } from "react";
import { readServerActionStream } from "@upstash/rag-chat/nextjs";
import { serverChat } from "../actions";

export const Chat = () => {
  const [prompt, setPrompt] = useState('');
  const [response, setResponse] = useState('');

  const clientChat = async () => {
    const stream = await serverChat(prompt);

    for await (const chunk of readServerActionStream(stream)) {
      setResponse(prev => prev + chunk);
    }
  };

  return (
    <div className="flex flex-col items-start p-4">
      <input className="border" placeholder="Enter your prompt" value={prompt} onChange={e => setPrompt(e.target.value)} />  
      <button onClick={clientChat}>Start Chat</button>
      <div>{response}</div>
    </div>
  );
};
```

### Update Home Page

Update `/app/page.tsx` using the Chat component:

```tsx /app/page.tsx
import { Chat } from "./components/chat"

export default function Home() {
  return <Chat />;
}
```

### Run & Deploy

Run the app locally with `npm run dev`, check `http://localhost:3000/`

Deploy your app with `vercel`


# Node.js



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/rag-chat/tree/master/examples/nodejs" horizontal>
  You can find the project source code on GitHub.
</Card>

### Install RAG Chat SDK

Initialize the project and install the required packages:

```bash
npm init es6
npm install dotenv
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.env` file.

```shell .env
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Create a Node.js Server

Create `server.ts`:

```typescript server.ts
import { RAGChat, upstash } from "@upstash/rag-chat";
import dotenv from "dotenv";
import { createServer } from "node:http";

dotenv.config();

const server = createServer(async (_, result) => {
  const ragChat = new RAGChat({
    model: upstash("meta-llama/Meta-Llama-3-8B-Instruct"),
  });

  await ragChat.context.add({
    type: "text",
    data: "Paris, the capital of France, is renowned for its iconic landmark, the Eiffel Tower, which was completed in 1889 and stands at 330 meters tall.",
  });

  // 👇 slight delay to allow for vector indexing
  await sleep(3000);

  const response = await ragChat.chat(
    "What year was the construction of the Eiffel Tower completed, and what is its height?"
  );

  result.writeHead(200, { "Content-Type": "text/plain" });
  result.write(response.output);
  result.end();
});

server.listen(8080, () => {
  console.log("Server listening on http://localhost:8080");
});

function sleep(ms: number) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}
```

### Run the Server

```bash
npx tsx server.ts
```

Visit `http://localhost:8080`


# Nuxt



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/rag-chat/tree/master/examples/nuxt" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use Upstash RAG Chat with Nuxt. You can also explore our [Nuxt example](https://github.com/upstash/rag-chat/tree/master/examples/nuxt) for detailed, end-to-end examples and best practices.

### Project Setup

Create a new Nuxt application and install `@upstash/rag-chat` package.

```shell
npx nuxi@latest init my-app
cd my-app
npm install @upstash/rag-chat
```

### Setup Upstash Redis

Create a Redis database using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.env` file.

```shell .env
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Create a Nuxt Server Handler

Create `/server/api/chat.ts`:

```typescript /server/api/chat.ts
import { RAGChat, upstash } from "@upstash/rag-chat";

const ragChat = new RAGChat({
  model: upstash("meta-llama/Meta-Llama-3-8B-Instruct"),
});

export default defineEventHandler(async (event) => {
    const response = await ragChat.chat("What is the speed of light?");
    return response.output;
  })
```

### Run

Run the Nuxt application:

```bash
npm run dev
```

Visit `http://localhost:3000/api/chat`


# Overview



RAG Chat can be used with industry-leading platforms, quickstart with your favorite:

<CardGroup cols={2}>
  <Card title="Next.js" href="/vector/sdks/rag-chat/quickstarts/nextjs" />

  <Card title="Node.js" href="/vector/sdks/rag-chat/quickstarts/nodejs" />

  <Card title="Cloudflare Workers" href="/vector/sdks/rag-chat/quickstarts/cloudflare-workers" />

  <Card title="Nuxt" href="/vector/sdks/rag-chat/quickstarts/nuxt" />

  <Card title="SvelteKit" href="/vector/sdks/rag-chat/quickstarts/sveltekit" />

  <Card title="Hono" href="/vector/sdks/rag-chat/quickstarts/hono" />
</CardGroup>


# SvelteKit



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/rag-chat/tree/master/examples/nextjs/sveltekit" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use Upstash RAG Chat with SvelteKit. You can also explore our [SvelteKit example](https://github.com/upstash/rag-chat/tree/master/examples/nextjs/sveltekit) for detailed, end-to-end examples and best practices.

### Project Setup

Create a new SvelteKit application and install `@upstash/rag-chat` package.

```shell
npm create svelte@latest my-app
cd my-app
npm install @upstash/rag-chat
```

### Setup Upstash Vector

Create a Vector index using [Upstash Console](https://console.upstash.com) or [Upstash CLI](https://github.com/upstash/cli) and copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` into your `.env` file.

```shell .env
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
```

### Setup QStash LLM

Navigate to [QStash Console](https://console.upstash.com/qstash) and copy the `QSTASH_TOKEN` into your `.env` file.

```shell .env
QSTASH_TOKEN=<YOUR_TOKEN>
```

### Create a SvelteKit Handler

Create `/src/routes/chat/+server.ts`:

```typescript /src/routes/chat/+server.ts
import { RAGChat, upstash } from "@upstash/rag-chat";
import { Index } from "@upstash/vector";
import { env } from "$env/dynamic/private";

const ragChat = new RAGChat({
  vector: new Index({
    token: env.UPSTASH_VECTOR_REST_TOKEN,
    url: env.UPSTASH_VECTOR_REST_URL
  }),
  model: upstash("meta-llama/Meta-Llama-3-8B-Instruct", {
    apiKey: env.QSTASH_TOKEN
  }),
});

export async function GET() {
  const response = await ragChat.chat("What is the speed of light?");
  return new Response(response.output, {
    headers: {
      'Content-Type': 'application/json'
    }
  });
}
```

### Run

Run the SvelteKit application:

```bash
npm run dev
```

Visit `http://localhost:5173/chat`


# Remove



Used to delete vectors with array of IDs

## Arguments

<ResponseField name="IDs" type="string[] | number[]" required>
  The IDs of the vectors you want to delete.
</ResponseField>

<ResponseField name="namespace" type="{ namespace?: string }">
  Namespace to delete from. If not set, default namespace is used.
</ResponseField>

## Response

<ResponseField name="delete" type="number" required>
  The number of the deleted vectors.
</ResponseField>

<RequestExample>
  ```typescript Basic
  const responseDelete = await index.delete(["2", "3"]);
  //{ deleted: 2}
  ```

  ```typescript Namespace
  const responseDelete = await index.delete(["2", "3"], { namespace: "my-namespace" });
  //{ deleted: 2}
  ```
</RequestExample>


# Fetch



Used to retrieve the vector by ID.

## Arguments

<ResponseField name="IDs" type="string[] | number[]" required>
  The IDs of the vectors you want to fetch.
</ResponseField>

<ResponseField name="Options" type="Object">
  <Expandable defaultOpen="true">
    <ResponseField name="includeMetadata" type="boolean">
      Whether to include the metadata of the vectors in the response. Setting
      this `true` would be the best practice, since it will make it easier to
      identify the vectors.
    </ResponseField>

    <ParamField body="includeVectors" type="boolean">
      Whether to include the vector themselves in the response.
    </ParamField>

    <ParamField body="includeData" type="boolean">
      Whether to include [the data field](/vector/features/metadata#data) in the response.
    </ParamField>

    <ResponseField name="namespace" type="string">
      Namespace to fetch from. If not set, default namespace is used.
    </ResponseField>
  </Expandable>
</ResponseField>

## Response

<ResponseField name="FetchResponse" type="Vector[]" required>
  This field is `null` if no vector with the specified id is found.

  <Expandable defaultOpen="true">
    <ResponseField name="id" type="string | number" required>
      The ID of the resulting vector. <br />
    </ResponseField>

    <ResponseField name="vector" type="number[]">
      The vectors (if `includeVectors` is set to true)
    </ResponseField>

    <ResponseField name="metadata" type="Record<string, unknown>">
      The metadata of the vectors (if `includeMetadata` is set to true)
    </ResponseField>

    <ResponseField name="data" type="string">
      The data of the vector (if `includeData` is set to true)
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```typescript Basic
  const fetchResponse = await index.fetch(["2", "3"], {});
  // [{ id: "2" }, { id: "3" }]
  ```

  ```typescript Vector Not Found
  const fetchResponse = await index.fetch(["2", "3"], {});
  // [{ id: "2" }, null]
  ```
</RequestExample>


# Info



Used to retrieve the stats of an index.

## Response

<ResponseField name="vectorCount" type="number" required>
  The total number of vectors in the index, that are ready to use.
</ResponseField>

<ResponseField name="pendingVectorCount" type="number" required>
  The number of vectors in the index, that is still processing and not ready to
  use.
</ResponseField>

<ResponseField name="indexSize" type="number" required>
  The size of the index, in `b`.
</ResponseField>

<ResponseField name="dimension" type="number" required>
  Dimension of the vectors.
</ResponseField>

<ResponseField name="similarityFunction" type="string" required>
  Name of the similarity function used in indexing and queries.
</ResponseField>

<ResponseField name="namespaces" type="Record<string, Object>" required>
  A map of namespaces to their information in the following format

  <Expandable defaultOpen="true">
    <ResponseField name="vectorCount" type="number" required>
      The total number of vectors in the index, that are ready to use.
    </ResponseField>

    <ResponseField name="pendingVectorCount" type="number" required>
      The number of vectors in the index, that is still processing and not ready to
      use.
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```typescript Basic
  const infoResponse = await index.info();
  /*
  { 
    vectorCount: 17,
    pendingVectorCount: 0,
    indexSize: 551158,
    dimension: 1536,
    similarityFunction: "COSINE",
    namespaces: {
      "": { // default namespace
        vectorCount: 10,
        pendingVectorCount: 0,
      },
      "my-namespace": {
        vectorCount: 7,
        pendingVectorCount: 0,
      }
    }
  }
  */
  ```
</RequestExample>


# Query



The query method is designed to retrieve the most similar vectors from the index, using the specific distance metric defined for your index. This method supports a variety of options to configure the query to your needs.

<Note>
  The dimension of the query vector must match the dimension of your index.
</Note>

<Note>
  The score returned from query requests is a normalized value between 0 and 1,
  where 1 indicates the highest similarity and 0 the lowest regardless of the
  similarity function used.
</Note>

## Arguments

<ResponseField name="Payload" type="QueryOptions" required>
  <Expandable defaultOpen="true">
    <ResponseField name="vector | data" type="number[] | string" required>
      <Note>
        There are two ways to use the query method. You can either create the vectors on your own and pass directly the `vector` field. Or you can pass the `data` field and create the embeddings using Upstash Embedding.
      </Note>

      The query data/vector that you want to search for in the index.
    </ResponseField>

    <ResponseField name="topK" type="number" required>
      The total number of the vectors that you want to receive as a query
      result. The response will be sorted based on the distance metric score,
      and `topK` vectors will be returned.
    </ResponseField>

    <ResponseField name="includeMetadata" type="boolean">
      Whether to include the metadata of the vectors in the response. Setting
      this `true` would be the best practice, since it will make it easier to
      identify the vectors.
    </ResponseField>

    <ResponseField name="includeVectors" type="boolean">
      Whether to include the vector themselves in the response.
    </ResponseField>

    <ResponseField name="includeData" type="boolean">
      Whether to include [the data field](/vector/features/metadata#data) in the response.
    </ResponseField>

    <ResponseField name="filter" type="string">
      The metadata filtering of the vector. This is used to query your data based on the filters and narrow down the query results.
      If you wanna learn more about filtering check: [Metadata Filtering](https://upstash.com/docs/vector/features/filtering)
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="Namespace" type="{ namespace?: string }">
  Namespace to query. If not set, default namespace is used.
</ResponseField>

## Response

<ResponseField name="QueryResponse" type="Vector[]" required>
  <Expandable defaultOpen="true">
    <ResponseField name="id" type="string | number" required>
      The ID of the resulting vector.
    </ResponseField>

    <ResponseField name="score" type="number" required>
      The score of the vector data, calculated based on the distance metric of your index.
    </ResponseField>

    <ResponseField name="vector" type="number[]">
      The resulting vector (if `includeVectors` is set to true)
    </ResponseField>

    <ResponseField name="metadata" type="Record<string, unknown>">
      The metadata of the vector (if `includeMetadata` is set to true)
    </ResponseField>

    <ResponseField name="data" type="string">
      The data of the vector (if `includeData` is set to true)
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```typescript Using Vector
  await index.query({
    topK: 2,
    vector: [ ... ],
    includeMetadata: true,
    includeVectors: true
  }, { namespace: "my-namespace" })
  /*
  [
    {
      id: '6345',
      score: 0.85,
      vector: [],
      metadata: {
        sentence: "Upstash is great."
      }
    },
    {
      id: '1233',
      score: 0.75,
      vector: [],
      metadata: undefined
    },
  ]
  */
  ```

  ```typescript Using Data
  const results = await index.query({
    data: "Movie about an adventure of a hobbit in a fantasy world.",
    includeVectors: true,
    includeMetadata: true,
    topK: 1,
    filter: "genre = 'fantasy' and title = 'Lord of the Rings'",
  });
  /*
  [
    {
      id: "1234",
      vector: [0.1, 0.2, 0.3, 0.4, 0.5],
      score: 0.9999999,
      metadata: {
        title: "Lord of The Rings",
        genre: "fantasy",
        category: "classic",
      },
    }
  ]
  */
  ```

  ```typescript Improved Typechecking
  type Metadata = {
    title: string,
    genre: 'sci-fi' | 'fantasy' | 'horror' | 'action'
  }

  const results = await index.query<Metadata>({
    vector: [
      ... // query embedding
    ],
    includeVectors: true,
    topK: 1,
    filter: "genre = 'fantasy' and title = 'Lord of the Rings'"
  })

  if (results[0].metadata) {
    // Since we passed the Metadata type parameter above,
    // we can interact with metadata fields without having to
    // do any typecasting.
    const { title, genre } = results[0].metadata;
    console.log(`The best match in fantasy was ${title}`)
  }
  ```
</RequestExample>


# Range



The range method is used to retrieve vectors in chunks with pagination. This method supports a variety of options to configure the query to your needs.

<Note>
  The dimension of the query vector must match the dimension of your index.
</Note>

## Arguments

<ParamField body="cursor" type="string | number" required>
  The cursor to the last retrieved vector. Should be set to `0` in the initial
  range request.
</ParamField>

<ParamField body="limit" type="number" required>
  The number of maximum vectors wanted in the response of range. (page size)
</ParamField>

<ParamField body="includeMetadata" type="boolean">
  Whether to include the metadata of the vectors in the response. Setting this
  `true` would be the best practice, since it will make it easier to identify
  the vectors.
</ParamField>

<ParamField body="includeVectors" type="boolean">
  Whether to include the vector themselves in the response.
</ParamField>

<ParamField body="includeData" type="boolean">
  Whether to include [the data field](/vector/features/metadata#data) in the response.
</ParamField>

<ParamField name="Namespace" type="{ namespace?: string }">
  Namespace to call range for. If not set, default namespace is used.
</ParamField>

## Response

<ResponseField name="RangeResponse" type="" required>
  <Expandable defaultOpen="true">
    <ResponseField name="nextCursor" type="string | number" required>
      Cursor to use in the next range request.
    </ResponseField>

    <ResponseField name="vectors" type="Vector | Vector[]" required>
      <Expandable defaultOpen="true">
        <ResponseField name="id" type="string | number" required>
          The ID of the vector
        </ResponseField>

        <ResponseField name="vector" type="number[]">
          The vectors (if `includeVectors` is set to true)
        </ResponseField>

        <ResponseField name="metadata" type="Record<string, unknown>">
          The metadata of the vectors (if `includeMetadata` is set to true)
        </ResponseField>

        <ResponseField name="data" type="string">
          The data of the vector (if `includeData` is set to true)
        </ResponseField>
      </Expandable>
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```typescript Basic
  const responseRange = await index.range({
    cursor: 0,
    limit: 2,
    includeMetadata: true,
  }, { namespace: "my-namespace" });
  /*
  {
    nextCursor: '2',
    vectors: [
      { 
        id: '0',
        metadata: {
          keyword: "Vector"
        } 
      },
      { 
        id: '19',
        metadata: {
          keyword: "Redis"
        } 
      }
    ]
  }
  */
  ```

  ```typescript Improved Typechecking
  type Metadata = {
    title: string;
    genre: "sci-fi" | "fantasy" | "horror" | "action";
  };

  const responseRange = await index.range<Metadata>({
    cursor: 0,
    limit: 2,
    includeMetadata: true,
  });

  if (responseRange[0].metadata) {
    // Since we passed the Metadata type parameter above,
    // we can interact with metadata fields without having to
    // do any typecasting.
    const { title, genre } = results[0].metadata;
    console.log(`The best match in fantasy was ${title}`);
  }
  ```
</RequestExample>


# Reset



The `reset` method allows you to clear all vectors and metadata from a particular
namespace or all namespaces of an index.

## Arguments

There are two arguments available. You should only pass one of them:

<ResponseField name="namespace" type="string">
  Specifies a namespace to reset. Leave empty for the default namespace.
</ResponseField>

<ResponseField name="all" type="true | undefined">
  Whether to reset all namespaces. Can only be set to `true`.
</ResponseField>

## Response

`'Success'` if the index is successfully resetted.

<RequestExample>
  ```typescript Basic
  const responseReset = await index.reset();
  // 'Successful'
  ```

  ```typescript All Namespaces
  const responseReset = await index.reset({ all: true });
  // 'Successful'
  ```
</RequestExample>


# Resumable Query



The resumableQuery method allows you to perform queries that can be resumed to fetch additional results. This is particularly useful for large result sets or when implementing pagination.

<Note>
  The dimension of the query vector must match the dimension of your index.
</Note>

<Note>
  The score returned from query requests is a normalized value between 0 and 1,
  where 1 indicates the highest similarity and 0 the lowest regardless of the
  similarity function used.
</Note>

## Arguments

<ResponseField name="Payload" type="ResumableQueryOptions" required>
  <Expandable defaultOpen="true">
    <ResponseField name="vector | data" type="number[] | string" required>
      <Note>
        There are two ways to use the resumableQuery method. You can either create the vectors on your own and pass directly the `vector` field. Or you can pass the `data` field and create the embeddings using Upstash Embedding.
      </Note>

      The query data/vector that you want to search for in the index.
    </ResponseField>

    <ResponseField name="topK" type="number" required>
      The initial number of vectors to retrieve in the query result. The response will be sorted based on the distance metric score.
    </ResponseField>

    <ResponseField name="includeMetadata" type="boolean">
      Whether to include the metadata of the vectors in the response. Setting
      this `true` would be the best practice, since it will make it easier to
      identify the vectors.
    </ResponseField>

    <ResponseField name="includeVectors" type="boolean">
      Whether to include the vector values in the response.
    </ResponseField>

    <ResponseField name="includeData" type="boolean">
      Whether to include [the data field](/vector/features/metadata#data) in the response.
    </ResponseField>

    <ResponseField name="filter" type="string">
      The metadata filtering of the vector. This is used to query your data based on the filters and narrow down the query results.

      If you want to learn more about filtering check: [Metadata Filtering](https://upstash.com/docs/vector/features/filtering)
    </ResponseField>

    <ResponseField name="maxIdle" type="number">
      Maximum idle time for the resumable query in seconds.
    </ResponseField>
  </Expandable>
</ResponseField>

## Response

<ResponseField name="ResumableQueryResponse" type="Object" required>
  <Expandable defaultOpen="true">
    <ResponseField name="result" type="Vector[]" required>
      The initial query results.
    </ResponseField>

    <ResponseField name="fetchNext" type="(additionalK: number) => Promise<Vector[]>" required>
      A function to fetch the next batch of results.
    </ResponseField>

    <ResponseField name="stop" type="() => Promise<string>" required>
      A function to stop the resumable query and release resources.
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="Vector" type="Object">
  <Expandable defaultOpen="true">
    <ResponseField name="id" type="string | number" required>
      The ID of the resulting vector.
    </ResponseField>

    <ResponseField name="score" type="number" required>
      The score of the vector data, calculated based on the distance metric of
      your index.
    </ResponseField>

    <ResponseField name="vector" type="number[]">
      The vector value (if includeVectors is true).
    </ResponseField>

    <ResponseField name="metadata" type="Record<string, unknown>">
      The metadata of the vector. This is used to make it easier to identify the
      vector on queries (if includeMetadata is true).
    </ResponseField>

    <ResponseField name="data" type="string">
      The data field (if includeData is true).
    </ResponseField>
  </Expandable>
</ResponseField>

<RequestExample>
  ```typescript with Vector
  const { result, fetchNext, stop } = await index.resumableQuery({
    maxIdle: 3600,
    topK: 50,
    vector: [0, 1, 2, ..., 383], // 384-dimensional vector
    includeMetadata: true,
    includeVectors: true,
  });

  console.log(result);
  /*
  [
    {
      id: '6345',
      score: 1.00000012,
      vector: [0, 1, 2, ..., 383],
      metadata: {
        sentence: "Upstash is great."
      }
    },
    // ... more results
  ]
  */

  const nextBatch = await fetchNext(5); // Fetch next 5 results
  console.log(nextBatch);

  await stop(); // Stop the resumable query
  ```

  ```typescript with Data
  const { result, fetchNext, stop } = await index.resumableQuery({
    maxIdle: 3600,
    topK: 50,
    data: "lord of the rings"
    includeMetadata: true,
    includeData: true,
  });

  console.log(result);
  /*
  [
    {
      id: '6345',
      score: 1.00000012,
      data: "hobbit",
      metadata: {
        sentence: "Upstash is great."
      }
    },
    // ... more results
  ]
  */

  const nextBatch = await fetchNext(5); // Fetch next 5 results
  console.log(nextBatch);

  await stop(); // Stop the resumable query
  ```

  ```typescript with Metadata Type
  type Metadata = {
    title: string,
    genre: 'sci-fi' | 'fantasy' | 'horror' | 'action'
  }

  const { result, fetchNext, stop } = await index.resumableQuery<Metadata>({
    vector: [
      ... // query embedding
    ],
    includeMetadata: true,
    topK: 1,
    filter: "genre = 'fantasy' and title = 'Lord of the Rings'",
    maxIdle: 3600,
  })

  if (result[0].metadata) {
    // Since we passed the Metadata type parameter above,
    // we can interact with metadata fields without having to
    // do any typecasting.
    const { title, genre } = result[0].metadata;
    console.log(`The best match in fantasy was ${title}`)
  }

  await stop();
  ```
</RequestExample>


# Upsert



Used to add new vectors or update an existing vector.

<Note>
  You can only upsert vectors with same dimension count(size) as your index.
</Note>

## Arguments

There are two ways to use the upsert method. You can either create the vectors on your own and pass them directly.
Or you can pass the data and create the embeddings using Upstash Embedding. The possible payloads are:

<ResponseField name="VectorPayload" type="Vector | Vector[]" required>
  <Expandable>
    <ResponseField name="id" type="string | number" required>
      The ID of the vector
    </ResponseField>

    <ResponseField name="vector" type="number[]" required>
      The vectors to add to the store
    </ResponseField>

    <ResponseField name="metadata" type="Record<string, unknown>">
      Metadata of the vector
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="Namespace" type="{ namespace?: string }">
  Namespace to upsert to. If not set, default namespace is used.
</ResponseField>

**OR**

<ResponseField name="DataPayload" type="Data | Data[]" required>
  <Expandable>
    <ResponseField name="id" type="string | number" required>
      The ID of the vector
    </ResponseField>

    <ResponseField name="data" type="string" required>
      The vectors to add to the store
    </ResponseField>

    <ResponseField name="metadata" type="Record<string, unknown>">
      Metadata of the vector
    </ResponseField>
  </Expandable>
</ResponseField>

<ResponseField name="Namespace" type="{ namespace?: string }">
  Namespace to upsert to. If not set, default namespace is used.
</ResponseField>

## Response

<ResponseField type="str" required>
  `'Success'` on successful operation.
</ResponseField>

<RequestExample>
  ```typescript Single Vector
  await index.upsert({
    id: "1234",
    vector: [0.1, 0.2, 0.3, 0.4, 0.5],
    metadata: {
      title: "Lord of The Rings",
      genre: "drama",
      category: "classic",
    },
  });
  ```

  ```typescript Multiple Vectors
  await index.upsert([
    {
      id: "6789",
      vector: [0.6, 0.7, 0.8, 0.9, 0.9],
    },
    {
      id: "1234",
      vector: [0.1, 0.2, 0.3, 0.4, 0.5],
      metadata: {
        title: "Lord of The Rings",
        genre: "drama",
        category: "classic",
      },
    },
  ]);
  ```

  ```typescript Namespace
  await index.upsert([
    {
      id: "6789",
      vector: [0.6, 0.7, 0.8, 0.9, 0.9],
    },
  ], { namespace: "my-namespace" });
  ```

  ```typescript Update Vector
  await index.upsert({
  	id: "1234",
  	vector: [0.1, 0.2, 0.3, 0.4, 0.5]
  	metadata: {
  		title: "Redis"
  	}
  })

  await index.upsert({
  	id: "1234",
  	metadata: {
  		title: "QStash"
  	}
  })
  ```

  ```typescript Single Data
  await index.upsert({
    id: "1234",
    data: "'The Lord of the Rings' follows Frodo Baggins and his allies on a quest to destroy a powerful ring and save Middle-earth from the dark lord Sauron.",
    metadata: {
      title: "Lord of The Rings",
      genre: "drama",
      category: "classic",
    },
  });
  ```

  ```typescript Multiple Data
  await index.upsert([
    {
      id: "6789",
      data: "'Harry Potter' follows the journey of a young wizard, Harry Potter, as he attends Hogwarts School of Witchcraft and Wizardry, forms deep friendships, and confronts the dark wizard Voldemort, who seeks immortality and domination over the magical world.",
    },
    {
      id: "1234",
      data: "'The Lord of the Rings' follows Frodo Baggins and his allies on a quest to destroy a powerful ring and save Middle-earth from the dark lord Sauron.",
      metadata: {
        title: "Lord of The Rings",
        genre: "drama",
        category: "classic",
      },
    },
  ]);
  ```

  ```typescript Update data
  await index.upsert({
  	id: "1234",
  	data: "Upstash product"
  	metadata: {
  		title: "Redis"
  	}
  })

  await index.upsert({
  	id: "1234",
  	metadata: {
  		title: "QStash"
  	}
  })
  ```
</RequestExample>


# Contributing



## Preparing the environment

This project uses [Bun](https://bun.sh/) for packaging and dependency management. Make sure you have the relevant dependencies.

```commandline
curl -fsSL https://bun.sh/install | bash
```

You will also need a vector database on [Upstash](https://console.upstash.com/).

## Code Formatting

```bash
bun run fmt
```

## Running tests

To run all the tests, make sure you have the relevant environment variables.

```bash
bun run test
```


# Getting Started



`@upstash/vector` is a Typescript SDK for Upstash Vector, enabling easier operations on Vector Store with full type coverage.

Using `@upstash/vector` you can:

*   Upsert a vector with metadata to an index.
*   Fetching the vectors with specified IDs.
*   Querying a vector over pre-defined embeddings.
*   Remove vectors from an index.
*   Access index stats.
*   Reset everything related to an index.

You can find the Github Repository [here](https://github.com/upstash/vector-js).

## Install

<CodeGroup>
  ```shell npm
  npm install @upstash/vector
  ```

  ```shell pnpm
  pnpm add @upstash/vector
  ```
</CodeGroup>

## Usage

### Initializing the client

There are two pieces of configuration required to use the Upstash vector client: a REST token and REST URL. These values can be passed using environment variables or in code through a configuration object. Find your configuration values in the console dashboard at [https://console.upstash.com/](https://console.upstash.com/).

#### Using environment variables

The environment variables used to configure the client are the following. You can follow [this guide](/vector/overall/getstarted) to retrieve credentials.

```bash
UPSTASH_VECTOR_REST_URL="your_rest_url"
UPSTASH_VECTOR_REST_TOKEN="your_rest_token"
```

When these environment variables are set, the client constructor does not require any additional arguments.

```typescript
import { Index } from "@upstash/vector";

const index = new Index();
```

#### Using a configuration object

If you prefer to pass configuration in code, the constructor accepts a config object containing the `url` and `token` values. This
could be useful if your application needs to interact with multiple projects, each with a different configuration.

```typescript
import { Index } from "@upstash/vector";

const index = new Index({
  url: "<UPSTASH_VECTOR_REST_URL>",
  token: "<UPSTASH_VECTOR_REST_TOKEN>",
});
```

## TypeScript Usage

### Index level types

The Vector SDK supports defining your metadata type at the index level for complete type-safety.

```typescript
import { Index } from "@upstash/vector";

type Metadata = { genre: string, year: number };

const index = new Index<Metadata>();
```

Passing a metadata type at the index level will provide strong type safety for the metadata coming back from or required for the following commands:

*   `query`
*   `upsert`
*   `fetch`
*   `range`

### Command level types

In some cases, you might not want to define a metadata type at the index level. In this case, you can either override the index level type definition for a specific command, or pass a metadata type to specific commands instead.

```typescript
import { Index } from "@upstash/vector";

type Metadata = { genre: string, year: number };

const index = new Index();

index.upsert<Metadata>({ id: 1, vector: [...], metadata: { 
  genre: "comedy",
  year: 1990
}});
```

The passing of a strong metadata type is possible for each of the four index operations listed above, we use upsert as an example.


# Overview



# Upstash Vector Node.js Client

This is the official Node.js client for [Upstash](https://upstash.com/), written in TypeScript.

## Installation

```
npm install @upstash/vector
pnpm add @upstash/vector
```

## Usage

### Initializing the client

There are two pieces of configuration required to use the Upstash vector client: an REST token and REST URL. These values can be passed using environment variables or in code through a configuration object. Find your configuration values in the console dashboard at [https://console.upstash.com/](https://console.upstash.com/).

#### Using environment variables

The environment variables used to configure the client are the following:

```bash
UPSTASH_VECTOR_REST_URL="your_rest_url"
UPSTASH_VECTOR_REST_TOKEN="your_rest_token"
```

When these environment variables are set, the client constructor does not require any additional arguments.

```typescript
import { Index } from "@upstash/vector";

const index = new Index();
```

#### Using a configuration object

If you prefer to pass configuration in code, the constructor accepts a config object containing the `url` and `token` values. This
could be useful if your application needs to interact with multiple projects, each with a different configuration.

```typescript
import { Index } from "@upstash/vector";

const index = new Index({
  url: "<UPSTASH_VECTOR_REST_URL>",
  token: "<UPSTASH_VECTOR_REST_TOKEN>",
});
```

## Index operations

Upstash vector indexes support operations for working with vector data using operations such as upsert, query, fetch, delete, stats and reset.

### Accessing an index

To perform data operations on an index, access it using the `index` method.

```typescript
await index.fetch([....], { includeMetadata: true, includeVectors: true });
```

### Accessing an index, with metadata typing

If you are storing metadata alongside your vector values, you can pass a type parameter to `index()` in order to get proper TypeScript typechecking.

```typescript
type Metadata = {
  title: string,
  genre: 'sci-fi' | 'fantasy' | 'horror' | 'action'
}

await index.upsert([{
  id: '1234',
  vector: [
    .... // embedding values
  ],
  metadata: {
    title: 'Lord of The Rings',
    genre: 'drama',
    category: 'classic'
  }
}])

const results = await index.query<Metadata>({
  vector: [
    ... // query embedding
  ],
  includeVectors: true,
  topK: 1,
})

if (results[0].metadata) {
  // Since we passed the Metadata type parameter above,
  // we can interact with metadata fields without having to
  // do any typecasting.
  const { title, genre, category } = results[0].metadata;
  console.log(`The best match in fantasy was ${title}`)
}
```

### Upsert records

Upstash vector expects records inserted into indexes to have the following form:

```typescript
type UpstashRecord = {
  id: number | string;
  vector: number[];
  metadata?: Record<string, unknown>;
};

// Upsert multiple records
await index.upsert([...]);

// Upsert a single record
await index.upsert({...});

```

### Querying

#### Querying with vector values

The query method accepts a large number of options. The dimension of the query vector must match the dimension of your index.

```typescript
type QueryOptions = {
  vector: number[];
  topK: number;
  includeVectors?: boolean;
  includeMetadata?: boolean;
};
```

For example, to query by vector values you would pass the `vector` param in the options configuration. For brevity sake this example query vector is tiny (dimension 2), but in a more realistic use case this query vector would be an embedding outputted by a model. Look at the [Example code](#example-code) to see more realistic examples of how to use `query`.

```typescript
> await index.query({ topK: 3, vector: [ ... ]})
{
  matches: [
    {
      id: '6345',
      score: 1.00000012,
      vector: [],
      metadata: undefined
    },
    {
      id: '1233',
      score: 1.00000012,
      vector: [],
      metadata: undefined
    },
    {
      id: '4142',
      score: 1.00000012,
      vector: [],
      metadata: undefined
    }
  ],
  namespace: ''
}
```

You include options to `includeMetadata: true` or `includeVectors: true` if you need this information. By default these are not returned to keep the response payload small.

### Update a record

You may want to update vector `vector` or `metadata`. Specify the id and the attribute value you want to update.

```typescript
await index.upsert({
  id: "18593",
  metadata: { genre: "romance" },
});
```

### Fetch records by their IDs

```typescript
const fetchResult = await index.fetch(["id-1", "id-2"]);
```

### Delete records

For convenience there are several delete-related options. You can verify the results of a delete operation by trying to `fetch()` a record.

```typescript
// Delete multiple records
await index.delete([...]);

// Delete a single record
await index.delete("id-to-delete");
```

### Stats

To get statistics of your index, you can use the client like so:

```typescript
await index.stats(["id-1", "id-2", "id-3"]);
```

### Reset

To delete everything related with that index:

```typescript
await index.reset();
```

## Contributing


# Create and Deploy RAG Applications with Gradio



In this tutorial, we'll demonstrate how to use Gradio to build an interactive Semantic Search and Question Answering app using Hugging Face embeddings, Upstash Vector, and LangChain. Users can enter a question, and the app will retrieve relevant information and provide an answer.

### Important Note on Python Version

Recent Python versions may cause compatibility issues with `torch`, a dependency for Hugging Face models. Therefore, we recommend using **Python 3.9** to avoid any installation issues.

### Installation and Setup

First, we need to set up our environment and install the necessary libraries. Install the dependencies by running the following command:

```bash
pip install gradio langchain sentence_transformers upstash-vector python-dotenv transformers langchain-community langchain-huggingface
```

Next, create a `.env` file in your project directory with the following content, replacing `your_upstash_url` and `your_upstash_token` with your actual Upstash credentials:

```
UPSTASH_VECTOR_REST_URL=your_upstash_url
UPSTASH_VECTOR_REST_TOKEN=your_upstash_token
```

This configuration file will allow us to load the required environment variables.

### Code

We will load our environment variables, initialize the Hugging Face embeddings model, set up Upstash Vector, and configure a Hugging Face Question Answering model.

```python
# Import libraries
import gradio as gr
from dotenv import load_dotenv
from langchain_huggingface.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores.upstash import UpstashVectorStore
from transformers import pipeline
from langchain.schema import Document

# Load environment variables
load_dotenv()

# Set up embeddings and Upstash Vector store
embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-mpnet-base-v2")
vector_store = UpstashVectorStore(embedding=embeddings)
```

Next, we will create sample documents, embed them using Hugging Face embeddings, and store them in Upstash Vector.

```python
# Sample documents to embed and store
documents = [
    Document(page_content="Global warming is causing sea levels to rise."),
    Document(page_content="AI is transforming many industries."),
    Document(page_content="Renewable energy is vital for sustainable development.")
]
vector_store.add_documents(documents=documents, batch_size=100, embedding_chunk_size=200)
```

When inserting documents, they are first embedded using the `Embeddings` object. Many embedding models, such as the Hugging Face models, support embedding multiple documents at once. This allows for efficient processing by batching documents and embedding them in parallel.

*   The `embedding_chunk_size` parameter controls the number of documents processed in parallel when creating embeddings.

Once the embeddings are created, they are stored in Upstash Vector. To reduce the number of HTTP requests, the vectors are also batched when they are sent to Upstash Vector.

*   The `batch_size` parameter controls the number of vectors included in each HTTP request when sending to Upstash Vector.

<Note type="info">
  In the Upstash Vector free tier, there is a limit of 1000 vectors per batch.
</Note>

Now, we can set up a Question Answering model and the Gradio interface.

```python
# Set up a Hugging Face Question Answering model
qa_pipeline = pipeline("question-answering", model="distilbert-base-cased-distilled-squad")

# Gradio interface function
def answer_question(query):
    # Retrieve relevant documents from Upstash Vector
    results = vector_store.similarity_search(query, k=3)
    
    # Use the most relevant document for QA
    if results:
        context = results[0].page_content
        qa_input = {"question": query, "context": context}
        answer = qa_pipeline(qa_input)["answer"]
        return f"Answer: {answer}\n\nContext: {context}"
    else:
        return "No relevant context found."

# Set up Gradio interface
iface = gr.Interface(
    fn=answer_question,
    inputs="text",
    outputs="text",
    title="RAG Application",
    description="Ask a question, and the app will retrieve relevant information and provide an answer."
)

# Launch the Gradio app
iface.launch()
```

### Running the App

After setting up the code, run your script to start the Gradio app. You will be presented with an interface where you can enter a question. The app will retrieve the most relevant information from the embedded documents and provide an answer based on the content.

### Notes

*   **Deployment**: To create a public link, set `share=True` in `launch()`. This will generate a public URL for your Gradio app. This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to [Hugging Face Spaces](https://huggingface.co/spaces)
*   **Batch Processing**: The `batch_size` and `embedding_chunk_size` parameters allow you to control the efficiency of document processing and storage in Upstash Vector.
*   **Namespaces**: Upstash Vector supports namespaces for organizing different types of documents. You can set a namespace while creating the `UpstashVectorStore` instance.


# Use Hugging Face Embeddings with Upstash Vector



In this tutorial, we'll demonstrate how to use Hugging Face embeddings with Upstash Vector and LangChain to perform a similarity search. We will upload a few sample documents, embed them using Hugging Face, and then perform a search query to find the most semantically similar documents.

### Important Note on Python Version

Recent Python versions may cause compatibility issues with `torch`, a dependency for Hugging Face models. Therefore, we recommend using **Python 3.9** to avoid any installation issues.

### Installation and Setup

First, we need to set up our environment and install the necessary libraries. Install the dependencies by running the following command:

```bash
pip install langchain sentence_transformers upstash-vector python-dotenv langchain-community langchain-huggingface
```

Next, create a `.env` file in your project directory with the following content, replacing `your_upstash_url` and `your_upstash_token` with your actual Upstash credentials:

```
UPSTASH_VECTOR_REST_URL=your_upstash_url
UPSTASH_VECTOR_REST_TOKEN=your_upstash_token
```

This configuration file will allow us to load the required environment variables.

### Code

We will load our environment variables and initialize the Hugging Face embeddings model along with the Upstash Vector store.

```python
# Load environment variables for API keys and Upstash configuration
from dotenv import load_dotenv
import os
load_dotenv()

# Import required libraries
from langchain_huggingface.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores.upstash import UpstashVectorStore

# Initialize Hugging Face embeddings model
embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-mpnet-base-v2")

# Set up Upstash Vector Store (automatically uses the environment variables)
vector_store = UpstashVectorStore(embedding=embeddings)
```

Next, we will create sample documents and embed them using Hugging Face embeddings, then store them in Upstash Vector.

```python
# Import the required Document class from LangChain
from langchain.schema import Document

# Sample documents to embed and store as Document objects
documents = [
    Document(page_content="Global warming is causing sea levels to rise."),
    Document(page_content="Artificial intelligence is transforming many industries."),
    Document(page_content="Renewable energy is vital for sustainable development.")
]

# Embed documents and store in Upstash Vector with batching
vector_store.add_documents(
    documents=documents,
    batch_size=100,               
    embedding_chunk_size=200      
)

print("Documents with embeddings have been stored in Upstash Vector.")
```

When inserting documents, they are first embedded using the `Embeddings` object. Many embedding models, such as the Hugging Face models, support embedding multiple documents at once. This allows for efficient processing by batching documents and embedding them in parallel.

*   The `embedding_chunk_size` parameter controls the number of documents processed in parallel when creating embeddings.

Once the embeddings are created, they are stored in Upstash Vector. To reduce the number of HTTP requests, the vectors are also batched when they are sent to Upstash Vector.

*   The `batch_size` parameter controls the number of vectors included in each HTTP request when sending to Upstash Vector.

<Note type="info">
  In the Upstash Vector free tier, there is a limit of 1000 vectors per batch.
</Note>

Now, we can perform a semantic search.

```python
# Querying Vectors using a text query
query_text = "What are the effects of global warming?"
query_embedding = embeddings.embed_query(query_text)

# Perform similarity search with the query text
result_text_query = vector_store.similarity_search(
    query=query_text,
    k=5  # Number of top results to return
)

print("Results for text-based similarity search:")
for res in result_text_query:
    print(res.page_content)
```

Here's the output of our text-based similarity search:

```
Results for text-based similarity search:
Global warming is causing sea levels to rise.
Renewable energy is vital for sustainable development.
Artificial intelligence is transforming many industries.
```

Alternatively, you can perform a similarity search using the vector directly.

```python
# Querying Vectors using a vector directly
result_vector_query = vector_store.similarity_search_by_vector(
    embedding=query_embedding,
    k=5
)

print("Results for vector-based similarity search:")
for res in result_vector_query:
    print(res.page_content)
```

This will output similar results, as it is searching based on the similarity of the embedding.

### Notes

*   You can specify batch sizes and chunk sizes to control the efficiency of document processing and storage in Upstash Vector.
*   Upstash Vector supports namespaces for organizing different types of documents. You can set a namespace while creating the `UpstashVectorStore` instance.

To learn more about LangChain and its integration with Upstash Vector, visit the [LangChain documentation](https://python.langchain.com/docs/integrations/vectorstores/upstash/).


# Implement Semantic Search with LangChain



In this tutorial, we'll demonstrate how to use Upstash Vector with LangChain to perform a similarity search. We will upload a document about global warming and perform a search query to find the most semantically similar documents using embeddings generated automatically by Upstash.

### Installation and Setup

First, we need to create a Vector Index in the [Upstash Console](https://console.upstash.com). Once we have our index, we will copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` and paste them to our `.env` file. To learn more about index creation, you can check out [this page](https://docs.upstash.com/vector/overall/getstarted).

Add the following content to your `.env` file (replace with your actual URL and token):

```
UPSTASH_VECTOR_REST_URL=your_upstash_url
UPSTASH_VECTOR_REST_TOKEN=your_upstash_token
```

We now need to install the following libraries via PyPI:

```bash
pip install upstash-vector python-dotenv langchain langchain-community
```

### Code

We will load our environment variables and initialize the index from the environment variables (URL and token).

```python
from dotenv import load_dotenv
from langchain_community.vectorstores.upstash import UpstashVectorStore

load_dotenv()

# Create a vector store instance where embeddings are generated by Upstash
store = UpstashVectorStore(embedding=True)
```

Next, we will upload the `global_warming.txt` document to the Upstash Vector index.

```python
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import CharacterTextSplitter

# Load the document
loader = TextLoader("documents/global_warming.txt")
documents = loader.load()

# Split the document into chunks
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)
```

We will now insert the documents into the Upstash Vector index.

```python
inserted_vectors = store.add_documents(docs)
```

Finally, we will perform a semantic search.

```python
result = store.similarity_search("Technology's role in global warming.", k=5)
print(result)
```

Here's the output:

```
[Document(metadata={'source': 'documents/global_warming.txt'}, page_content='Technology and innovation play a crucial role in advancing sustainable food systems and mitigating the impact of global warming. Precision agriculture, for example, uses data analytics, remote sensing, and GPS technology to optimize the use of resources such as water, fertilizer, and pesticides. By applying inputs more efficiently, precision agriculture reduces waste, lowers GHG emissions, and improves crop yields.\n\nIn the realm of food production, alternative proteins—such as plant-based meats, lab-grown meats, and insect-based proteins—offer promising solutions to reduce the environmental impact of livestock farming. These innovations require fewer resources to produce and generate lower GHG emissions compared to traditional animal agriculture. Additionally, vertical farming and hydroponics allow for the cultivation of crops in controlled environments, using less land and water while reducing the need for chemical inputs.\n\nConclusion'),
 Document(metadata={'source': 'documents/global_warming.txt'}, page_content='Global warming presents a formidable challenge to food production systems, but sustainable food practices offer a viable solution to mitigate its effects. By adopting regenerative agriculture, reducing food waste, and shifting toward plant-based diets, humanity can reduce the environmental impact of agriculture while ensuring food security for future generations. Furthermore, technology and innovation will continue to play a pivotal role in advancing sustainable food systems, allowing for more efficient and eco-friendly food production methods. In the face of climate change, the integration of sustainable food practices is not only an environmental imperative but also a pathway toward a healthier and more resilient future for both people and the planet.'),
 Document(metadata={'source': 'documents/global_warming.txt'}, page_content='Global Warming and Sustainable Foods: An Inextricable Connection'),
 Document(metadata={'source': 'documents/global_warming.txt'}, page_content='One of the key components of sustainable food systems is regenerative agriculture, which focuses on restoring soil health, enhancing biodiversity, and sequestering carbon. Regenerative farming practices include crop rotation, cover cropping, agroforestry, and reduced tillage, all of which contribute to building healthy soils that can absorb and store carbon. By improving soil health, regenerative agriculture also increases the resilience of crops to climate change, as healthy soils retain more water and nutrients, making crops more resistant to droughts and pests.'),
 Document(metadata={'source': 'documents/global_warming.txt'}, page_content='Numerous studies have shown that shifting to plant-based diets can significantly reduce GHG emissions and mitigate the effects of global warming. According to a report by the United Nations Food and Agriculture Organization (FAO), reducing the consumption of animal-based foods could reduce global agricultural emissions by up to 50%. Moreover, plant-based diets are associated with reduced land and water use, as growing crops for direct human consumption is more resource-efficient than growing feed for livestock.\n\nIn addition to their environmental benefits, plant-based diets offer numerous health advantages. They are rich in fiber, vitamins, and minerals, and have been linked to lower risks of chronic diseases such as heart disease, diabetes, and certain cancers. As global populations grow and the demand for food increases, promoting plant-based diets offers a sustainable solution to addressing both environmental and nutritional challenges.\n\nThe Role of Technology and Innovation')]
```

### Notes

*   You can also query with score using `similarity_search_with_score` method.

*   Namespaces can be used to separate different types of documents. You can specify a namespace when creating the `UpstashVectorStore` instance.

```
store = UpstashVectorStore(embedding=True, namespace="my_namespace")
```

*   You can use OpenAI's embeddings by setting `embedding=OpenAIEmbeddings()` in the `UpstashVectorStore` instance.

```
from langchain_openai import OpenAIEmbeddings
store = UpstashVectorStore(embedding=OpenAIEmbeddings())
```

To learn more about LangChain and its integration with Upstash Vector, you can visit the [LangChain documentation](https://python.langchain.com/docs/integrations/vectorstores/upstash/).


# RAG with LlamaIndex



In this tutorial, we’ll demonstrate how to use Upstash Vector with LlamaIndex to perform RAG (Retrieval-Augmented Generation). We will upload a document about global warming and generate responses to our questions based on the contents of the document.

### Installation and Setup

First, we need to create a Vector Index in the [Upstash Console](https://console.upstash.com). Make sure to set the index dimensions to 1536 and the distance metric to Cosine. Once we have our index, we will copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` and paste them into our `.env` file. To learn more about index creation, you can check out our [getting started page](https://docs.upstash.com/vector/overall/getstarted).

Add the following content to your `.env` file (replace with your actual URL, token and API key):

```
UPSTASH_VECTOR_REST_URL=your_upstash_url
UPSTASH_VECTOR_REST_TOKEN=your_upstash_token
OPENAI_API_KEY=your_openai_api_key
```

We now need to install the following libraries via PyPI:

```bash
pip install llama-index upstash-vector llama-index-vector-stores-upstash python-dotenv
```

### Code

We will load our environment variables, initialize the index, and configure it to use the specified dimensions and distance metric.

```python
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
from llama_index.vector_stores.upstash import UpstashVectorStore
from llama_index.core import StorageContext
import openai
import os
from dotenv import load_dotenv

# Load environment variables
load_dotenv()
openai.api_key = os.environ["OPENAI_API_KEY"]

# Setup the Upstash vector store 
upstash_vector_store = UpstashVectorStore(
    url=os.environ["UPSTASH_VECTOR_REST_URL"],
    token=os.environ["UPSTASH_VECTOR_REST_TOKEN"],
)

# Read the document about global warming from the documents directory
documents = SimpleDirectoryReader("./documents/").load_data()

# Initialize the storage context with the Upstash vector store
storage_context = StorageContext.from_defaults(vector_store=upstash_vector_store)

# Create the index from the loaded document with 1536 dimensions and cosine distance
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)
```

Next, we will query the document:

```python
# Initialize the query engine
query_engine = index.as_query_engine()

# Query the document about global warming
res1 = query_engine.query("What is global warming?")
print(res1)

res2 = query_engine.query("How should we modify our diets to reduce our carbon footprint?")
print(res2)
```

### Sample Output

Here is the output of the queries:

```
Global warming refers to the long-term increase in Earth’s average temperature due to the accumulation of greenhouse gases (GHGs) such as carbon dioxide (CO2), methane (CH4), and nitrous oxide (N2O) in the atmosphere. The primary drivers of GHG emissions include the burning of fossil fuels for energy, industrial processes, deforestation, and unsustainable agricultural practices. As these gases trap heat, they create a “greenhouse effect,” leading to rising temperatures, melting polar ice caps, rising sea...

Shifting towards plant-based diets, which emphasize the consumption of vegetables, fruits, legumes, grains, and nuts, can significantly reduce our carbon footprint. Plant-based diets have a much lower environmental footprint compared to animal-based foods, particularly red meat, which is highly resource-intensive and contributes disproportionately to greenhouse gas emissions. Studies have shown that reducing the consumption of animal-based foods and increasing the intake of plant-based foods can help red...
```

### Notes

*   Namespaces can be used to separate different types of documents. You can specify a namespace when creating the `UpstashVectorStore` instance:

```
vector_store = UpstashVectorStore(
    url=your_upstash_url, 
    token=your_upstash_token, 
    namespace=your_namespace,
    )
```

*   To learn more about LlamaIndex and its integration with Upstash Vector, you can visit the [LlamaIndex documentation](https://docs.llamaindex.ai/en/latest).


# Parsing and Querying Documents with LlamaParse



In this tutorial, we’ll learn how to parse a document using LlamaParse and then query it using an LLM with Upstash Vector.

We’ll split this guide into two parts: parsing a document and then querying the parsed document.

## Installation and Setup

To get started, we need to set up our environment. You can install the necessary libraries using the following command in your terminal:

```bash
pip install llama-index upstash-vector llama-index-vector-stores-upstash python-dotenv
```

We also need to create a Vector Index in the [Upstash Console](https://console.upstash.com). Make sure to set the index dimensions to 1536 and the distance metric to Cosine. To learn more about index creation, you can check out our [getting started page](https://docs.upstash.com/vector/overall/getstarted).

Once we have our index, we will copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` and paste them into our `.env` file.

#### Environment Variables

Create a `.env` file in your project directory and add the following content:

```plaintext
UPSTASH_VECTOR_REST_URL=your_upstash_url
UPSTASH_VECTOR_REST_TOKEN=your_upstash_token
OPENAI_API_KEY=your_openai_api_key
LLAMA_CLOUD_API_KEY=your_llama_cloud_api_key
```

To get your `LLAMA_CLOUD_API_KEY`, you can follow the instructions in [the LlamaCloud documentation](https://docs.cloud.llamaindex.ai/llamaparse/getting_started/get_an_api_key).

## Part 1: Parsing a Document

We can now move on to parsing a document. In this example, we’ll parse a file named `global_warming.txt`.

```python
from llama_parse import LlamaParse
from llama_index.core import SimpleDirectoryReader

# Initialize the LlamaParse parser with the desired result format
parser = LlamaParse(result_type="markdown")  # "markdown" and "text" are available

# Parse the document using the parser
file_extractor = {".txt": parser}
documents = SimpleDirectoryReader(input_files=["./documents/global_warming.txt"], file_extractor=file_extractor).load_data()
```

<Note type="info">
  If you are using Jupyter Notebook, you need to allow nested event loops to parse the document.

  You can do this by adding the following code snippet to your file:

  ```python
  import nest_asyncio
  nest_asyncio.apply()
  ```
</Note>

Now that we have our parsed data, we can query it.

## Part 2: Querying the Parsed Document with an LLM

In this part, we’ll use the `UpstashVectorStore` to create an index, and query the content. We’ll use OpenAI as the language model to interpret the data and respond to questions based on the document. You can use other LLMs that are supported by LlamaIndex as well.

```python
from llama_index.core import VectorStoreIndex
from llama_index.vector_stores.upstash import UpstashVectorStore
from llama_index.core import StorageContext
import openai

# Load environment variables for API keys and Upstash configuration
from dotenv import load_dotenv
import os
load_dotenv()

# Set up OpenAI API key
openai.api_key = os.getenv("OPENAI_API_KEY")

# Set up Upstash Vector Store
upstash_vector_store = UpstashVectorStore(
    url=os.getenv("UPSTASH_VECTOR_REST_URL"),
    token=os.getenv("UPSTASH_VECTOR_REST_TOKEN"),
)

# Create a storage context for Upstash Vector and index the parsed document
storage_context = StorageContext.from_defaults(vector_store=upstash_vector_store)
index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)

# Create a query engine for the index and perform a query
query_engine = index.as_query_engine()
query = "What are the main points discussed in the document?"
response = query_engine.query(query)
print(response)
```

Here's the code output:

```plaintext
The main points discussed in the document include the impact of global warming on agriculture 
and food production systems, the importance of adopting sustainable food practices to mitigate 
these effects, the role of agriculture in contributing to global warming through GHG emissions, 
deforestation, and the use of synthetic fertilizers, and the need for sustainable food systems 
to address environmental challenges and ensure food security for future generations.
```

### Conclusion

With the ability to parse and query documents, you can efficiently summarize content, extract essential information, and answer questions based on the document’s details.

To learn more about LlamaIndex and its integration with Upstash Vector, you can visit the [LlamaIndex documentation](https://docs.llamaindex.ai/en/latest).


# Simple Semantic Search



In this tutorial, we'll demonstrate how to use Upstash Vector for semantic search. We will upload several documents and perform a search query to find the most semantically similar documents using embeddings generated automatically by Upstash.

### Installation and Setup

First, we need to create a Vector Index in the [Upstash Console](https://console.upstash.com). Once we have our index, we will copy the `UPSTASH_VECTOR_REST_URL` and `UPSTASH_VECTOR_REST_TOKEN` and paste them to our `.env` file. To learn more about index creation, you can check out [this page](https://docs.upstash.com/vector/overall/getstarted).

Add the following content to your `.env` file (replace with your actual URL and token):

```
UPSTASH_VECTOR_REST_URL=your_upstash_url
UPSTASH_VECTOR_REST_TOKEN=your_upstash_token
```

We now need to install the `upstash-vector` library via PyPI. Additionally, we will install `python-dotenv` to load environment variables from the `.env` file.

```bash
pip install upstash-vector python-dotenv
```

### Code

Create a Python script (e.g., `main.py`) and add the following code to perform semantic search using Upstash Vector:

```python main.py
from upstash_vector import Index
from dotenv import load_dotenv
import time

# Load environment variables from a .env file
load_dotenv()

# Initialize the index from environment variables (URL and token)
index = Index.from_env()

# Example documents to be indexed
documents = [
    {"id": "1", "text": "Python is a popular programming language."},
    {"id": "2", "text": "Machine learning enables computers to learn from data."},
    {"id": "3", "text": "Upstash provides low-latency database solutions."},
    {"id": "4", "text": "Semantic search is a technique for understanding the meaning of queries."},
    {"id": "5", "text": "Cloud computing allows for scalable and flexible resource management."}
]

# Reset the index to remove previous data
index.reset()

# Upsert documents into Upstash (embeddings are generated automatically)
for doc in documents:
    index.upsert(
        vectors=[
            (doc["id"], doc["text"], {"text": doc["text"]})
        ]
    )
    print(f"Document {doc['id']} inserted.")

# Wait for the documents to be indexed
time.sleep(1)

# Search for documents similar to the query
query = "What is Python?"
results = index.query(data=query, top_k=3, include_metadata=True)

# Display search results
print("Search Results:")
for result in results:
    print(f"ID: {result.id}")
    print(f"Score: {result.score:.4f}")
    print(f"Metadata: {result.metadata}")
    print("-" * 40)  # Separator line between results
```

### Running the Code

To run the code, execute the following command in your terminal:

```bash
python main.py
```

Here is an example output for the search query "What is Python?":

```
Document 1 inserted.
Document 2 inserted.
Document 3 inserted.
Document 4 inserted.
Document 5 inserted.
Search Results:
ID: 1
Score: 0.9080
Metadata: {'text': 'Python is a popular programming language.'}
----------------------------------------
ID: 2
Score: 0.7592
Metadata: {'text': 'Machine learning enables computers to learn from data.'}
----------------------------------------
ID: 4
Score: 0.7388
Metadata: {'text': 'Semantic search is a technique for understanding the meaning of queries.'}
----------------------------------------
```

### Code Breakdown

1.  **Environment Setup**: We use `python-dotenv` to load our environment variables and use the `Index.from_env()` method to initialize the index client.

2.  **Document Insertion**: We define a list of documents, each with a unique ID and text content. The `upsert()` function inserts these documents into our index. These documents are automatically converted into embeddings. To learn more about Upstash Embedding Models, you can check out [this page](https://docs.upstash.com/vector/features/embeddingmodels).

3.  **Index Reset**: Before inserting documents, the `reset()` function clears any existing data in the index.

4.  **Search Query**: After inserting the documents, we perform semantic search. The `query()` function returns the `top_k` most similar documents to the query along with their metadata if `include_metadata` is set to `True`.


# Caveats



## Introduction

In this guide, we'll look at best practices and caveats for using Upstash Workflow.

## Core Principles

### Execute business logic in `context.run`

Your workflow endpoint will be called multiple times during a workflow run. Therefore:

*   Place your business logic code inside the `context.run` function for each step
*   Code outside `context.run` only serves to connect steps

Example:

```typescript api/workflow/route.ts
export const { POST } = serve<string>(async (context) => {
  const input = context.requestPayload

  const result = await context.run("step-1", () => {
    return { success: true }
  })

  console.log("This log will appear multiple times")

  await context.run("step-2", () => {
    console.log("This log will appear just once")
    console.log("Step 1 status is:", result.success)
  })
})
```

### Return Results from context.run for Later Use

Always return step results if needed in subsequent steps.

<CodeGroup>
  ```typescript ❌ Incorrect:
  export const { POST } = serve<string>(async (context) => {
    const input = context.requestPayload

    let result

    await context.run("step-1", async () => {
      result = await someWork(input)
    })
    await context.run("step-2", async () => {
      await someOtherWork(result)
    })
  })
  ```

  ```typescript ✅ Correct
  export const { POST } = serve<string>(async (context) => {
    const input = context.requestPayload

    const result = await context.run("step-1", async () => {
      return await someWork(input)
    })

    await context.run("step-2", async () => {
      someOtherWork(result)
    })
  })
  ```
</CodeGroup>

Because your workflow endpoint is called multiple times, `result` will be unitialized when the endpoint is called again to run `step-2`.

If you are curious about why an endpoint is called multiple times, see [how Workflow works](/workflow/how).

## Avoiding Common Pitfalls

### Avoid Non-deterministic Code Outside `context.run`

A workflow endpoint should always produce the same results, even if it's called multiple times. Avoid:

*   Time-dependent code
*   Randomness
*   Non-idempotent functions

Example of what to avoid:

<CodeGroup>
  ```typescript ❌ Time-dependent code
  export const { POST } = serve<string>(async (context) => {
    const input = context.requestPayload

    // 👇 Problem: time-dependent code
    if (Date.now() % 5 == 2) {
      await context.run("step-1", () => {
        // ...
      })
    } else {
      await context.run("step-2", () => {
        // ...
      })
    }
  })
  ```

  ```typescript ❌ Random code
  export const { POST } = serve<string>(async (context) => {
    const input = context.requestPayload

    // 👇 Problem: random code
    if (Math.floor(Math.random() * 10) % 5 == 2) {
      await context.run("step-1", () => {
        // ...
      })
    } else {
      await context.run("step-2", () => {
        // ...
      })
    }
  })
  ```
</CodeGroup>

### Ensure Idempotency in `context.run`

Business logic should be idempotent due to potential retries in distributed systems. In other words, **when a workflow runs twice with the same input, the end result should be the same as if the workflow only ran once**.

In the example below, the `someWork` function must be idempotent:

```typescript api/workflow/route.ts
export const { POST } = serve<string>(async (context) => {
  const input = context.requestPayload

  await context.run("step-1", async () => {
    return someWork(input)
  })
})
```

Imagine that `someWork` executes once and makes a change to a database. However, before the database had a chance to respond with the successful change, the connection is lost. Your Workflow cannot know if the database change was successful or not. The caller has no choice but to retry, which will cause `someWork` to run twice.

If `someWork` is not idempotent, this could lead to unintended consequences. For example duplicated records or corrupted data. Idempotency is crucial to maintaining the integrity and reliability of your workflow.

### Don't Nest Context Methods

Avoid calling `context.call`, `context.sleep`, `context.sleepFor`, or `context.run` within another `context.run`.

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"

export const { POST } = serve<string>(async (context) => {
  const input = context.requestPayload

  await context.run("step-1", async () => {
    await context.sleep(...) // ❌ INCORRECT
    await context.run(...) // ❌ INCORRECT
    await context.call(...) // ❌ INCORRECT
  })
})
```


# Workflow Client



Workflow client allows you to interact with your workflow runs. Currently, it has three basic functionality:

*   [cancel a running workflow run](/workflow/howto/cancel)
*   notify a workflow run [waiting for an event](/workflow/basics/context#context-waitforevent)
*   get workflow runs waiting for some event

We are planning to add more functionality in the future. See [the roadmap](/workflow/roadmap) for more details.

### Trigger Workflow

Using the `trigger` method, you can start a workflow run and get its run id.

```ts
import { Client } from "@upstash/workflow";

const client = new Client({ token: "<QSTASH_TOKEN>" })
const { workflowRunId } = await client.trigger({
  url: "https://workflow-endpoint.com",
  body: "hello there!",         // optional body
  headers: { ... },             // optional headers
  workflowRunId: "my-workflow", // optional workflow run id
  retries: 3                    // optional retries in the initial request
})

console.log(workflowRunId)
// prints wfr_my-workflow
```

If `workflowRunId` parameter isn't passed, a run id will be generated randomly. If `workflowRunId` is passed, `wfr_` prefix will be added to it.

For other alternatives of starting a workflow, see the [documentation on starting a workflow run](/workflow/howto/start).

### Cancel Workflow

To cancel a workflow, simply use the `cancel` method:

```javascript
import { Client } from "@upstash/workflow";

const client = new Client({ token: "<QSTASH_TOKEN>" })
await client.cancel({ workflowRunId: "<WORKFLOW_RUN_ID>" })
```

### Notify Waiting Workflow

To notify a workflow waiting for an event, you can use the `notify` method:

```javascript
import { Client } from "@upstash/workflow";

const client = new Client({ token: "<QSTASH_TOKEN>" })
await client.notify({
  eventId: "my-event-id",
  eventData: "my-data" // data passed to the workflow run
});
```

The data passed in `eventData` will be available to the workflow run in the
[`eventData` field of the `context.waitForEvent` method](/workflow/basics/context#context-waitforevent).

### Get Waiters of Event

To get the list of waiters for some event id, you can use the `getWaiters` method:

```javascript
import { Client } from "@upstash/workflow";

const client = new Client({ token: "<QSTASH_TOKEN>" })
const result = await client.getWaiters({
  eventId: "my-event-id"
})
```

Result will be a list of `Waiter` objects:

<Snippet file="qstash/waiter.mdx" />


# Workflow Context



A workflow's context is a JavaScript object provided by the serve function and is used to define your workflow endpoint. This context object offers utility methods for creating workflow steps, managing delays, and performing timeout-resistant HTTP calls.

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"

export const { POST } = serve(
  // 👇 the workflow context
  async (context) => {
    // ...
  }
)
```

This context object provides utility methods to create workflow steps, wait for certain periods of time or perform timeout-resistant HTTP calls.

Further, the context object provides all request headers, the incoming request payload and current workflow ID.

## Context Object Properties

*   `qstashClient`: QStash client used by the serve method

*   `workflowRunId`: Current workflow run ID

*   `url`: Publically accessible workflow endpoint URL

*   `failureUrl`: URL for workflow failure notifications.

*   `requestPayload`: Incoming request payload

*   `rawInitialPayload`: String version of the initial payload

*   `headers`: Request headers

*   `env`: Environment variables

## Core Workflow Methods

### context.run

Defines and executes a workflow step.

<CodeGroup>
  ```typescript Serial execution
  import { serve } from "@upstash/workflow/nextjs"

  export const { POST } = serve<string>(async (context) => {
    const input = context.requestPayload

    const result1 = await context.run("step-1", async () => {
      return someWork(input)
    })

    await context.run("step-2", async () => {
      someOtherWork(result1)
    })
  })
  ```

  ```typescript Parallel execution
  import { serve } from "@upstash/workflow/nextjs"

  export const { POST } = serve<string>(
    async (context) => {
      const input = context.requestPayload;

      const promise1 = context.run("step-1", async () => {
        return someWork(input);
      });

      const promise2 = context.run("step-2", async () => {
        return someOtherWork(input);
      });

      await Promise.all([promise1, promise2]);
    },
  );
  ```
</CodeGroup>

### context.sleep

Pauses workflow execution for a specified duration.

<Tip>
  Always `await` a `sleep` action to properly pause execution.
</Tip>

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"
import { signIn, sendEmail } from "@/utils/onboarding-utils"

export const { POST } = serve<User>(
  async (context) => {
    const userData = context.requestPayload;

    const user = await context.run("sign-in", async () => {
      const signedInUser = await signIn(userData);
      return signedInUser;
    });

    // 👇 Wait for one day (in seconds)
    await context.sleep("wait-until-welcome-email", "1d");

    await context.run("send-welcome-email", async () => {
      return sendEmail(user.name, user.email);
    });
  },
);
```

### context.sleepUntil

Pauses workflow execution until a specific timestamp.

<Tip>
  Always await a `sleepUntil` action to properly pause execution.
</Tip>

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"
import { signIn, sendEmail } from "@/utils/onboarding-utils"

export const { POST } = serve<User>(async (context) => {
  const userData = context.requestPayload

  const user = await context.run("sign-in", async () => {
    return signIn(userData)
  })

  // 👇 Calculate the date for one week from now
  const oneWeekFromNow = new Date()
  oneWeekFromNow.setDate(oneWeekFromNow.getDate() + 7)

  // 👇 Wait until the calculated date
  await context.sleepUntil("wait-for-one-week", oneWeekFromNow)

  await context.run("send-welcome-email", async () => {
    return sendEmail(user.name, user.email)
  })
})
```

### context.call

Performs an HTTP call as a workflow step, allowing for longer response times.

Can take up to 15 minutes or 2 hours, depending on your [QStash plans](https://upstash.com/pricing/qstash) max HTTP connection timeout.

```javascript
import { serve } from "@upstash/workflow/nextjs"

export const { POST } = serve<{ topic: string }>(async (context) => {
  const request = context.requestPayload

  const {
    status, // response status
    headers, // response headers
    body // response body
  } = await context.call(
    "generate-long-essay", // Step name
    {
      url: "https://api.openai.com/v1/chat/completions", // Endpoint URL
      method: "POST",
      body: { // Request body
        model: "gpt-4o",
        messages: [
          {
            role: "system",
            content:
              "You are a helpful assistant writing really long essays that would cause a normal serverless function to timeout.",
          },
          { role: "user", content: request.topic },
        ],
      },
      headers: { // request headers
        authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
      }
    }
  )
})
```

context.call attempts to parse the response body as JSON. If this is not possible, the body is returned as it is.

In TypeScript, you can declare the expected result type as follows:

```typescript
type ResultType = {
  field1: string,
  field2: number
};

const result = await context.call<ResultType>( ... ); 
```

By default, `context.call` only calls the given URL once. If you wish to add retries, you can do so with the `retries` option:

```ts
await context.call("call", {
  url: ...,
  retries: 3
})
```

<Note>
  The context.call method can make requests to any public API endpoint. However, it cannot:

  *   Make requests to localhost (unless you set up a local tunnel, [here's how](http://localhost:3000/workflow/howto/local-development))
  *   Make requests to internal Upstash QStash endpoints
</Note>

### context.waitForEvent

Stops the workflow run until it is notified externally to continue.

There is also a timeout setting which makes the workflow continue if it's not notified within the time frame.

```javascript
import { serve } from "@upstash/workflow/nextjs"

export const { POST } = serve<{ topic: string }>(async (context) => {
  const request = context.requestPayload

  const {
    eventData, // data passed in notify
    timeout    // boolean denoting whether the step was notified or timed out
  } = await context.waitForEvent(
    "wait for some event",
    "my-event-id",
    "1000s" // 1000 second timeout
  );
})
```

A workflow run waiting for event can be notified in two ways:

*   `context.notify`: [Notify step explained below](/workflow/basics/context#context-notify)
*   `client.notify`: [Notify method of the Workflow Client](/workflow/basics/client#notify-waiting-workflow).

### context.notify

Notifies workflows waiting for an event with some payload.

```javascript
import { serve } from "@upstash/workflow/nextjs"

export const { POST } = serve<{ topic: string }>(async (context) => {
  const payload = context.requestPayload

  const {
    notifyResponse // result of notify, which is a list of notified waiters
  } = await context.notify("notify step", "my-event-Id", payload);
})
```

`notifyResponse` is a list of `NotifyResponse` objects:

```typescript
export type NotifyResponse = {
  waiter: Waiter;
  messageId: string;
  error: string;
};
```

More details about the `Waiter` object:

<Snippet file="qstash/waiter.mdx" />

## Error handling and retries

*   context.run automatically retries on failures. Default is 3 retries with exponential backoff.
*   context.call doesn't retry by default. If you wish to add retry, you can use `retries` option.
*   Future releases will allow more configuration of retry behavior.

## Limitations and Plan-Specific-Features

*   Sleep durations and HTTP timeouts vary based on your QStash plan.
*   See your plan's "Max Delay" and "Max HTTP Connection Timeout" for specific limits.


# Create a workflow



Use the `serve` method to define an endpoint that serves a workflow. It provides a context object that you use to create all of your workflow's steps.

For example, in a Next.js app:

```typescript
import { serve } from "@upstash/workflow/nextjs"

export const { POST } = serve(async (context) => {
  const result = await context.run("step-1", async () => {
    // define a piece of business logic as step 1
  })

  await context.run("step-2", async () => {
    // define another piece of business logic as step 2
  })
})
```

## Platform support

Besides Next.js, the `serve` method supports multiple frameworks and platforms. See [all supported platforms](/workflow/quickstarts/platforms) here.

## Options

### `failureUrl`

Use the `failureUrl` option to specify a URL your workflow will call if it exhausted all retries and fails.

```typescript
export const { POST } = serve<string>(
  async (context) => { ... },
  {
    failureUrl: "https://<YOUR-FAILURE-ENDPOINT>/..."
  }
);
```

If specified, this URL will be called with [the failure callback payload](qstash/features/callbacks#what-is-a-failure-callback), and the error message will be included in the `body` field.

The default value is `undefined`, meaning no failure URL is called.

### `failureFunction`

Use the `failureFunction` to define a function that's executed when your workflow exhausts all its [retries](/qstash/features/retry) and fails.

```typescript
export const { POST } = serve<string>(
  async (context) => { ... },
  {
    failureFunction: async (
      context,      // context during failure
      failStatus,   // failure status
      failResponse, // failure message
      failHeaders   // failure headers
    ) => {
      // handle the failure
    }
  }
);
```

If both `failureUrl` and `failureFunction` are provided, the failure function takes precedence and the value of `failureUrl` is ignored.

Note that if you are using a [custom authorization method](/workflow/howto/security#custom-authorization-method), you should also add authentication to `failureFunction`.

By default, `failureFunction` is `undefined`, meaning that no function is executed on failure.

### `retries`

To specify the number of times QStash will call the workflow endpoint in case of errors, you can use the `verbose` parameter.

The default value is 3.

```typescript
export const { POST } = serve<string>(
  async (context) => { ... },
  {
    retries: 3
  }
);
```

### `verbose`

To gain insights into how the workflow operates, you can enable verbose mode:

```typescript
export const { POST } = serve<string>(
  async (context) => { ... },
  {
    verbose: true
  }
);
```

Each log entry has the following structure:

```
{
  timestamp: number,
  workflowRunId: string,
  logLevel: string,
  eventType: string,
  details: unknown,
}
```

The `eventType` can be:

*   `ENDPOINT_START` each time the workflow endpoint is called
*   `RUN_SINGLE` or `RUN_PARALLEL` when step(s) are executed
*   `SUBMIT_STEP` when a single step is executed
*   `SUBMIT_FIRST_INVOCATION` when a new workflow run starts
*   `SUBMIT_CLEANUP` when a workflow run finishes
*   `SUBMIT_THIRD_PARTY_RESULT` when a third-party call result is received (see `context.call`)

Verbose mode is disabled by default.

### `initialPayloadParser`

When calling the workflow endpoint to start a workflow run, your initial request's payload is expected to be either empty, a string, or JSON.

If your payload differs, you can process it as needed using the `initialPayloadParser` option:

```typescript
type InitialPayload = {
  foo: string
  bar: number
}

// 👇 1: provide initial payload type
export const { POST } = serve<InitialPayload>(
  async (context) => {
    // 👇 3: parsing result is available as requestPayload
    const payload: InitialPayload = context.requestPayload
  },
  {
    // 👇 2: custom parsing for initial payload
    initialPayloadParser: (initialPayload) => {
      const payload: InitialPayload =
        parsePayload(initialPayload)
      return payload
    },
  }
)
```

### `url`

Since a workflow run involves multiple calls to the workflow endpoint, the `serve` method needs to know where your endpoint is hosted.

Typically, your workflow infers this using the `request.url` field.

However, if you use a proxy or a local tunnel for development, you may want to override the URL inferred from `request.url`:

```typescript
export const { POST } = serve<string>(
  async (context) => { ... },
  {
    url: "https://<YOUR-DEPLOYED-APP>.com/api/workflow"
  }
);
```

By default, `url` is `undefined`, and the URL is inferred from `request.url`.

### `baseUrl`

An alternative to the `url` option is the `baseUrl` option. While `url` replaces the entire URL inferred from `request.url`, `baseUrl` only changes the base of the URL:

```typescript
export const { POST } = serve<string>(
  async (context) => {
    ...
  },
  // options:
  {
    baseUrl: "<LOCAL-TUNNEL-PUBLIC-URL>"
  }
);
```

The default value of `baseUrl` is `undefined`.

If you have multiple endpoints and you don't want to set `baseUrl` in every single one, you can set the `UPSTASH_WORKFLOW_URL` environment variable to apply `baseUrl` across your entire application.

Setting this environment variable is especially useful during [local development](/workflow/howto/local-development). In production, `baseUrl` or `UPSTASH_WORKFLOW_URL` are not necessary.

***

## Further options

The following options can be considered convenience methods. They are intended to support edge cases or testing pipelines and are **not required for regular use**.

### `qstashClient`

The `qstashClient` option allows you to pass a QStash Client explicitly. This can be helpful when using multiple QStash clients in the same project with different environment variables.

```typescript
import { Client } from "@upstash/qstash";
import { serve } from "@upstash/workflow/nextjs";

export const { POST } = serve(
  async (context) => { ... },
  {
    qstashClient: new Client({ token: process.env.QSTASH_TOKEN })
  }
);
```

By default, a `qstashClient` is initialized as:

```typescript
new Client({
  baseUrl: process.env.QSTASH_URL!,
  token: process.env.QSTASH_TOKEN!,
})
```

### `receiver`

You can pass a QStash Receiver to verify that **every** request the endpoint receives comes from QStash, preventing anyone from triggering your workflow.

```typescript
import { Receiver } from "@upstash/qstash";
import { serve } from "@upstash/workflow/nextjs";

export const { POST } = serve<string>(
  async (context) => { ... },
  {
    receiver: new Receiver({
      // 👇 grab these variables from your QStash dashboard
      currentSigningKey: process.env.QSTASH_CURRENT_SIGNING_KEY!,
      nextSigningKey: process.env.QSTASH_NEXT_SIGNING_KEY!,
    })
  }
);
```

The default receiver is automatically used if the environment variables `QSTASH_CURRENT_SIGNING_KEY` and `QSTASH_NEXT_SIGNING_KEY` are set. If you want to turn off the Receiver, remove these environment variables or pass `receiver: undefined` in the options. Note that this will skip any verification that requests are coming from QStash and allow anyone to start your workflow.

### `env`

By default, the SDK uses `process.env` to initialize the QStash client and the receiver (if the two environment variables are set). If `process.env` doesn't exist, the SDK won't be able to access the environment variables. In this case, you can either pass `qstashClient` and `receiver` options or use the `env` option.

If you pass `env`, this `env` will be used instead of `process.env`:

```typescript
import { Receiver } from "@upstash/qstash";
import { serve } from "@upstash/workflow/nextjs";

export const { POST } = serve<string>(
  async (context) => {
    // the env option will be available in the env field of the context:
    const env = context.env;
  },
  {
    receiver: new Receiver({
      // 👇 grab these variables from your QStash dashboard
      currentSigningKey: process.env.QSTASH_CURRENT_SIGNING_KEY!,
      nextSigningKey: process.env.QSTASH_NEXT_SIGNING_KEY!,
    })
  }
);
```


# Changelog



<Warning>Qstash changelogs are [here](/qstash/overall/changelog)</Warning>

<Update label="October 2024">
  *   Optimized the console by trimming event bodies, reducing resource usage and enabling efficient querying of events with large payloads.
  *   Began development on a new architecture to deliver faster event processing on the server.
  *   Added [Wait Notify](/workflow/howto/events) feature.
</Update>

<Update label="September 2024">
  *   Bug fixes and internal logging improvements.
</Update>

<Update label="August 2024">
  *   Released [Upstash Workflow](/workflow/getstarted).
</Update>


# AI Generation



## Introduction

This example demonstrates advanced AI data processing using Upstash Workflow. The following example workflow downloads a large dataset, processes it in chunks using OpenAI's GPT-4 model, aggregates the results and generates a report.

## Use Case

Our workflow will:

1.  Receive a request to process a dataset
2.  Download the dataset from a remote source
3.  Process the data in chunks using OpenAI
4.  Aggregate results
5.  Generate and send a final report

## Code Example

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"
import {
  downloadData,
  aggregateResults,
  generateReport,
  sendReport,
  getDatasetUrl,
  splitIntoChunks,
} from "./utils"

type OpenAiResponse = {
  choices: {
    message: {
      role: string,
      content: string
    }
  }[]
}

export const { POST } = serve<{ datasetId: string; userId: string }>(
  async (context) => {
    const request = context.requestPayload

    // Step 1: Download the dataset
    const datasetUrl = await context.run("get-dataset-url", async () => {
      return await getDatasetUrl(request.datasetId)
    })

    // HTTP request with much longer timeout (2hrs)
    const { body: dataset } = await context.call("download-dataset", {
      url: datasetUrl,
      method: "GET"
    })
      

    // Step 2: Process data in chunks using OpenAI
    const chunkSize = 1000
    const chunks = splitIntoChunks(dataset, chunkSize)
    const processedChunks: string[] = []

    for (let i = 0; i < chunks.length; i++) {
      const { body: processedChunk } = await context.call<OpenAiResponse>(
        `process-chunk-${i}`,
        {
          url: "https://api.openai.com/v1/chat/completions",
          headers: { authorization: `Bearer ${process.env.OPENAI_API_KEY}` },
          method: "POST",
          body: {
            model: "gpt-4",
            messages: [
              {
                role: "system",
                content:
                  "You are an AI assistant tasked with analyzing data chunks. Provide a brief summary and key insights for the given data.",
              },
              {
                role: "user",
                content: `Analyze this data chunk: ${JSON.stringify(chunks[i])}`,
              },
            ],
            max_tokens: 150,
          },
        }
      )

      processedChunks.push(processedChunk.choices[0].message.content)

      // Every 10 chunks, we'll aggregate intermediate results
      if (i % 10 === 9 || i === chunks.length - 1) {
        await context.run(`aggregate-results${i}`, async () => {
          await aggregateResults(processedChunks)
          processedChunks.length = 0
        })
      }
    }

    // Step 3: Generate and send data report
    const report = await context.run("generate-report", async () => {
      return await generateReport(request.datasetId)
    })

    await context.run("send-report", async () => {
      await sendReport(report, request.userId)
    })
  }
)
```

## Code Breakdown

### 1. Preparing our data

We start by retrieving the dataset URL and then downloading the dataset:

```typescript api/workflow/route.ts
const datasetUrl = await context.run("get-dataset-url", async () => {
  return await getDatasetUrl(request.datasetId)
})

const { body: dataset } = await context.call("download-dataset", {
  url: datasetUrl,
  method: "GET"
})
```

Note that we use `context.call` for the download, a way to make HTTP requests that run for much longer than your serverless execution limit would normally allow.

### 2. Processing our data

We split the dataset into chunks and process each one using OpenAI's GPT-4 model:

```typescript api/workflow/route.ts
for (let i = 0; i < chunks.length; i++) {
  const { body: processedChunk } = await context.call<OpenAiResponse>(
    `process-chunk-${i}`,
    {
      url: "https://api.openai.com/v1/chat/completions",
      headers: { authorization: `Bearer ${process.env.OPENAI_API_KEY}` },
      method: "POST",
      body: {
        model: "gpt-4",
        messages: [
          {
            role: "system",
            content:
              "You are an AI assistant tasked with analyzing data chunks. Provide a brief summary and key insights for the given data.",
          },
          {
            role: "user",
            content: `Analyze this data chunk: ${JSON.stringify(chunks[i])}`,
          },
        ],
        max_tokens: 150,
      },
    }
  )
```

### 3. Aggregating our data

After processing our data in smaller chunks to avoid any function timeouts, we aggregate results every 10 chunks:

```typescript api/workflow/route.ts
if (i % 10 === 9 || i === chunks.length - 1) {
  await context.run(`aggregate-results${i}`, async () => {
    await aggregateResults(processedChunks)
    processedChunks.length = 0
  })
}
```

### 4. Sending a report

Finally, we generate a report based on the aggregated results and send it to the user:

```typescript api/workflow/route.ts
const report = await context.run("generate-report", async () => {
  return await generateReport(request.datasetId)
})

await context.run("send-report", async () => {
  await sendReport(report, request.userId)
})
```

## Key Features

1.  **Non-blocking HTTP Calls**: We use `context.call` for API requests so they don't consume the endpoint's execution time (great for optimizing serverless cost).

2.  **Long-running tasks**: The dataset download can take up to 2 hours, though is realistically limited by function memory.


# Auth Provider Webhook



This example demonstrates an authentication provider webhook process using Upstash Workflow.
The workflow handles the user creation, trial management, email reminders and notifications.

## Use Case

Our workflow will:

1.  Receive a webhook event from an authentication provider (e.g. Firebase, Auth0, Clerk etc.)
2.  Create a new user in our database
3.  Create a new user in Stripe
4.  Start a trial in Stripe
5.  Send a welcome email
6.  Send a reminder email if the user hasn't solved any questions in the last 7 days
7.  Send a trial warning email if the user hasn't upgraded 2 days before the trial ends
8.  Send a trial ended email if the user hasn't upgraded

## Code Example

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs";
import { WorkflowContext } from '@upstash/qstash/workflow'

/**
 * This can be the payload of the user created webhook event coming from your
 * auth provider (e.g. Firebase, Auth0, Clerk etc.)
 */
type UserCreatedPayload = {
  name: string;
  email: string;
};

export const { POST } = serve<UserCreatedPayload>(async (context) => {
  const { name, email } = context.requestPayload;

  const { userid } = await context.run("sync user", async () => {
    return await createUserInDatabase({ name, email });
  });

  await context.run("create new user in stripe", async () => {
    await createNewUserInStripe(email);
  });

  await context.run("start trial in Stripe", async () => {
    await startTrialInStripe(email);
  });

  await context.run("send welcome email", async () => {
    await sendEmail(
      email,
      "Welcome to our platform!, You have 14 days of free trial."
    );
  });

  await context.sleep("wait", 7 * 24 * 60 * 60);

  // get user stats and send email with them
  const stats = await context.run("get user stats", async () => {
    return await getUserStats(userid);
  });
  await sendProblemSolvedEmail({context, email, stats});

  // wait until there are two days to the end of trial period
  // and check upgrade status
  await context.sleep("wait for trial warning", 5 * 24 * 60 * 60);
  const isUpgraded = await context.run("check upgraded plan", async () => {
    return await checkUpgradedPlan(email);
  });

  // end the workflow if upgraded
  if (isUpgraded) return;

  await context.run("send trial warning email", async () => {
    await sendEmail(
      email,
      "Your trial is about to end in 2 days. Please upgrade your plan to keep using our platform."
    );
  });

  await context.sleep("wait for trial end", 2 * 24 * 60 * 60);

  await context.run("send trial end email", async () => {
    await sendEmail(
      email,
      "Your trial has ended. Please upgrade your plan to keep using our platform."
    );
  });
});

async function sendProblemSolvedEmail({
  context: WorkflowContext<UserCreatedPayload>
  email: string,
  stats: { totalProblemsSolved: number }
}) {
  if (stats.totalProblemsSolved === 0) {
    await context.run("send no answers email", async () => {
      await sendEmail(
        email,
        "Hey, you haven't solved any questions in the last 7 days..."
      );
    });
  } else {
    await context.run("send stats email", async () => {
      await sendEmail(
        email,
        `You have solved ${stats.totalProblemsSolved} problems in the last 7 days. Keep it up!`
      );
    });
  }
}

async function createUserInDatabase({
  name,
  email,
}: {
  name: string;
  email: string;
}) {
  console.log("Creating a user in the database:", name, email);
  return { userid: "12345" };
}

async function createNewUserInStripe(email: string) {
  // Implement logic to create a new user in Stripe
  console.log("Creating a user in Stripe for", email);
}

async function startTrialInStripe(email: string) {
  // Implement logic to start a trial in Stripe
  console.log("Starting a trial of 14 days in Stripe for", email);
}

async function getUserStats(userid: string) {
  // Implement logic to get user stats
  console.log("Getting user stats for", userid);
  return {
    totalProblemsSolved: 10_000,
    mostInterestedTopic: "JavaScript",
  };
}

async function checkUpgradedPlan(email: string) {
  // Implement logic to check if the user has upgraded the plan
  console.log("Checking if the user has upgraded the plan", email);
  return false;
}

async function sendEmail(email: string, content: string) {
  // Implement logic to send an email
  console.log("Sending email to", email, content);
}
```

## Code Breakdown

### 1. Sync User

We start by creating a new user in our database:

```typescript
const { userid } = await context.run("sync user", async () => {
  return await createUserInDatabase({ name, email });
});
```

### 2. Create New User in Stripe

Next, we create a new user in Stripe:

```typescript
await context.run("create new user in stripe", async () => {
  await createNewUserInStripe(email);
});
```

### 3. Start Trial in Stripe

We start a trial in Stripe:

```typescript
await context.run("start trial in Stripe", async () => {
  await startTrialInStripe(email);
});
```

### 4. Send Welcome Email

We send a welcome email to the user:

```typescript
await context.run("send welcome email", async () => {
  await sendEmail(
    email,
    "Welcome to our platform!, You have 14 days of free trial."
  );
});
```

### 5. Send Reminder Email

After 7 days, we check if the user has solved any questions. If not, we send a reminder email:

```typescript
await context.sleep("wait", 7 * 24 * 60 * 60);

const stats = await context.run("get user stats", async () => {
  return await getUserStats(userid);
});
await sendProblemSolvedEmail({context, email, stats});
```

The `sendProblemSolvedEmail` method:

```typescript
async function sendProblemSolvedEmail({
  context: WorkflowContext<UserCreatedPayload>
  email: string,
  stats: { totalProblemsSolved: number }
}) {
  if (stats.totalProblemsSolved === 0) {
    await context.run("send no answers email", async () => {
      await sendEmail(
        email,
        "Hey, you haven't solved any questions in the last 7 days..."
      );
    });
  } else {
    await context.run("send stats email", async () => {
      await sendEmail(
        email,
        `You have solved ${stats.totalProblemsSolved} problems in the last 7 days. Keep it up!`
      );
    });
  }
}
```

### 6. Send Trial Warning Email

If the user hasn't upgraded 2 days before the trial ends, we send a trial warning email:

```typescript
await context.sleep("wait for trial warning", 5 * 24 * 60 * 60);

const isUpgraded = await context.run("check upgraded plan", async () => {
  return await checkUpgradedPlan(email);
});

if (isUpgraded) return;

await context.run("send trial warning email", async () => {
  await sendEmail(
    email,
    "Your trial is about to end in 2 days. Please upgrade your plan to keep using our platform."
  );
});
```

If they upgraded, we end the workflow by returning.

### 7. Send Trial Ended Email

If the user hasn't upgraded after the trial ends, we send a trial ended email:

```typescript
await context.sleep("wait for trial end", 2 * 24 * 60 * 60);

await context.run("send trial end email", async () => {
  await sendEmail(
    email,
    "Your trial has ended. Please upgrade your plan to keep using our platform."
  );
});
```


# Custom Retry Logic



## Key Features

This example demonstrates how to implement custom retry logic when using third-party services in your Upstash Workflow.

We'll use OpenAI as an example for such a third-party service. **Our retry logic uses response status codes and headers to control when to retry, sleep, or store the third-party API response**.

## Code Example

The following code:

1.  Attempts to make an API call up to 10 times.
2.  Dynamically adjusts request delays based on response headers or status.
3.  Stores successful responses asynchronously.

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"
import { convertToSeconds, storeResponse } from "@/lib/utils"

const OPENAI_CONFIG = {
  url: "https://api.openai.com/v1/chat/completions",
  model: "gpt-4",
  maxTokens: 150,
  maxRetries: 10,
  baseDelay: 10, // seconds
}

const createSystemMessage = () => ({
  role: "system",
  content: "You are an AI assistant providing a brief summary and key insights for any given data.",
})

const createUserMessage = (data: string) => ({
  role: "user",
  content: `Analyze this data chunk: ${data}`,
})

export const { POST } = serve<{ userData: string }>(async (workflow) => {
  // 👇 initial data sent along when triggering the workflow
  const { userData } = workflow.requestPayload

  for (let attempt = 0; attempt < 10; attempt++) {
    const response = await workflow.call(`call-openai`, {
      url: OPENAI_CONFIG.url,
      method: "POST",
      headers: {
        authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
      },
      body: {
        model: OPENAI_CONFIG.model,
        messages: [createSystemMessage(), createUserMessage(userData)],
        max_tokens: OPENAI_CONFIG.maxTokens,
      },
    })

    // Success case
    if (response.status < 300) {
      await workflow.run("store-response-in-db", () => storeResponse(response.body))
      return
    }

    // Rate limit case - wait and retry
    if (response.status === 429) {
      const resetTime =
        response.header["x-ratelimit-reset-tokens"]?.[0] ||
        response.header["x-ratelimit-reset-requests"]?.[0] ||
        OPENAI_CONFIG.baseDelay

      // assuming `resetTime` is in seconds
      await workflow.sleep("sleep-until-retry", Number(resetTime))

      continue
    }

    // Any other scenario - pause for 5 seconds to avoid overloading OpenAI API
    await workflow.sleep("pause-to-avoid-spam", 5)
  }
})
```

## Code Breakdown

### 1. Setting up our Workflow

This POST endpoint serves our workflow. We create a loop to attempt the API call (we're about to write) up to 10 times.

```typescript
export const { POST } = serve<{ userData: string }>(async (workflow) => {
  for (let attempt = 0; attempt < 10; attempt++) {
    // TODO: call API in here
  }
})
```

### 2. Making a Third-Party API Call

We use `workflow.call` to send a request to any third-party API, for example OpenAI.

<Note>
  Using `workflow.call` to request data from an API is one of the most powerful Upstash Workflow
  features. Your request can take much longer than any function timeout would normally allow,
  completely bypassing any platform-specific timeout limits.
</Note>

Our request to OpenAI includes an auth header, model parameters, and the data to be processed by the AI. The response from this function call (`response`) is used to determine our retry logic based on its status code and headers.

```typescript
const response = await workflow.call(`call-openai`, {
  url: OPENAI_CONFIG.url,
  method: "POST",
  headers: {
    authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
  },
  body: {
    model: OPENAI_CONFIG.model,
    messages: [createSystemMessage(), createUserMessage(userData)],
    max_tokens: OPENAI_CONFIG.maxTokens,
  },
})
```

### 3. Processing a Successful Response (Status Code \< 300)

If the OpenAI response is successful (status code under 300), we store the response in our database. We create a new workflow task (`workflow.run`) to do this for maximum reliability.

```typescript
if (response.status < 300) {
  await workflow.run("store-response-in-db", () => storeResponse(response.body))
  return
}
```

### 4. Handling Rate Limits (Status Code 429)

If the API response indicates a rate limit error (status code 429), we retrieve our rate limit reset values from the response headers. We calculate the time until the rate limit resets and then pause execution (`workflow.sleep`) for this duration.

```typescript
if (response.status === 429) {
  const resetTime =
    response.header["x-ratelimit-reset-tokens"]?.[0] ||
    response.header["x-ratelimit-reset-requests"]?.[0] ||
    OPENAI_CONFIG.baseDelay

  // assuming `resetTime` is in seconds
  await workflow.sleep("sleep-until-retry", Number(resetTime))

  continue
}
```

### 5. Waiting Before the Next Retry Attempt

To avoid making too many requests in a short period and possibly overloading the OpenAI API, we pause our workflow before the next retry attempt (i.e., 5 seconds), regardless of rate limits.

```typescript
await workflow.sleep("pause-to-avoid-spam", 5)
```


# Customer Onboarding



This example demonstrates a customer onboarding process using Upstash Workflow. The following example workflow registers a new user, sends welcome emails, and periodically checks and responds to the user's activity state.

## Use Case

Our workflow will:

1.  Register a new user to our service
2.  Send them a welcome email
3.  Wait for a certain time
4.  Periodically check the user's state
5.  Send appropriate emails based on the user's activity

## Code Example

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"

type InitialData = {
  email: string
}

export const { POST } = serve<InitialData>(async (context) => {
  const { email } = context.requestPayload

  await context.run("new-signup", async () => {
    await sendEmail("Welcome to the platform", email)
  })

  await context.sleep("wait-for-3-days", 60 * 60 * 24 * 3)

  while (true) {
    const state = await context.run("check-user-state", async () => {
      return await getUserState()
    })

    if (state === "non-active") {
      await context.run("send-email-non-active", async () => {
        await sendEmail("Email to non-active users", email)
      })
    } else if (state === "active") {
      await context.run("send-email-active", async () => {
        await sendEmail("Send newsletter to active users", email)
      })
    }

    await context.sleep("wait-for-1-month", 60 * 60 * 24 * 30)
  }
})

async function sendEmail(message: string, email: string) {
  // Implement email sending logic here
  console.log(`Sending ${message} email to ${email}`)
}

type UserState = "non-active" | "active"

const getUserState = async (): Promise<UserState> => {
  // Implement user state logic here
  return "non-active"
}
```

## Code Breakdown

### 1. New User Signup

We start by sending a newly signed-up user a welcome email:

```typescript api/workflow/route.ts
await context.run("new-signup", async () => {
  await sendEmail("Welcome to the platform", email)
})
```

### 2. Initial Waiting Period

To leave time for the user to interact with our platform, we use `context.sleep` to pause our workflow for 3 days:

```typescript api/workflow/route.ts
await context.sleep("wait-for-3-days", 60 * 60 * 24 * 3)
```

### 3. Periodic State Check

We enter an infinite loop to periodically (every month) check the user's engagement level with our platform and send appropriate emails:

```typescript api/workflow/route.ts
while (true) {
  const state = await context.run("check-user-state", async () => {
    return await getUserState()
  })

  if (state === "non-active") {
    await context.run("send-email-non-active", async () => {
      await sendEmail("Email to non-active users", email)
    })
  } else if (state === "active") {
    await context.run("send-email-active", async () => {
      await sendEmail("Send newsletter to active users", email)
    })
  }

  await context.sleep("wait-for-1-month", 60 * 60 * 24 * 30)
}
```

## Key Features

1.  **Non-blocking sleep**: We use `context.sleep` for pausing the workflow without consuming execution time (great for optimizing serverless cost).

2.  **Long-running task**: This workflow runs indefinitely, checking and responding to a users engagement state every month.


# E-commerce Order Fulfillment



## Introduction

This example demonstrates an automated e-commerce order fulfillment process using Upstash Workflow. The workflow takes an order, verifies the stock, processes the payment, and handles order dispatch and customer notifications.

## Use Case

Our workflow will:

1.  Receive an order request
2.  Verify the availability of the items in stock
3.  Process the payment
4.  Initiate the dispatch of the order
5.  Send confirmation and delivery notifications to the customer

## Code Example

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"
import {
  createOrderId,
  checkStockAvailability,
  processPayment,
  dispatchOrder,
  sendOrderConfirmation,
  sendDispatchNotification,
} from "./utils"

type OrderPayload = {
  userId: string
  items: { productId: string, quantity: number }[]
}

export const { POST } = serve<OrderPayload>(async (context) => {
  const { userId, items } = context.requestPayload;

  // Step 1: Create Order Id
  const orderId = await context.run("create-order-id", async () => {
    return await createOrderId(userId);
  });

  // Step 2: Verify stock availability
  const stockAvailable = await context.run("check-stock", async () => {
    return await checkStockAvailability(items);
  });

  if (!stockAvailable) {
    console.warn("Some items are out of stock");
    return;
  };

  // Step 3: Process payment
  await context.run("process-payment", async () => {
    return await processPayment(orderId)
  })

  // Step 4: Dispatch the order
  await context.run("dispatch-order", async () => {
    return await dispatchOrder(orderId, items)
  })

  // Step 5: Send order confirmation email
  await context.run("send-confirmation", async () => {
    return await sendOrderConfirmation(userId, orderId)
  })

  // Step 6: Send dispatch notification
  await context.run("send-dispatch-notification", async () => {
    return await sendDispatchNotification(userId, orderId)
  })
})
```

## Code Breakdown

### 1. Verifying Stock Availability

We start by creating an order id and verifying if the items in the order are available in stock. If not, we throw an error to halt the process:

```typescript api/workflow/route.ts
const orderId = await context.run("create-order-id", async () => {
  return await createOrderId(userId);
});

const stockAvailable = await context.run("check-stock", async () => {
  return await checkStockAvailability(items)
})

if (!stockAvailable) {
  console.warn("Some items are out of stock")
  return;
}
```

### 2. Processing Payment

Once the stock is verified, the workflow processes the payment for the order:

```typescript api/workflow/route.ts
await context.run("process-payment", async () => {
  return await processPayment(orderId)
})
```

### 3. Dispatching the Order

After payment confirmation, we dispatch the order for delivery:

```typescript api/workflow/route.ts
await context.run("dispatch-order", async () => {
  return await dispatchOrder(orderId, items)
})
```

### 4. Sending Confirmation and Notification Emails

Lastly, we send an order confirmation email to the customer and notify them when the order is dispatched:

```typescript api/workflow/route.ts
await context.run("send-confirmation", async () => {
  return await sendOrderConfirmation(userId, orderId)
})

await context.run("send-dispatch-notification", async () => {
  return await sendDispatchNotification(userId, orderId)
})
```

## Key Features

1.  **Stock Verification**: Ensures items are available in stock before processing the payment, avoiding issues with unavailable products.

2.  **Payment Processing**: Handles payment securely and only proceeds to dispatch if successful.

3.  **Customer Notifications**: Keeps the customer informed at each step of the order process, improving user experience.


# Image Processing



## Introduction

This example demonstrates how to process images using Upstash Workflow. The following workflow will upload an image, resize it into multiple resolutions, apply filters, and store the processed versions for later retrieval.

## Use Case

Our workflow will:

1.  Receive an image upload request
2.  Resize the image into different resolutions
3.  Apply various filters to the resized images
4.  Store the processed images in a cloud storage

## Code Example

```typescript
import { serve } from "@upstash/workflow/nextjs"
import {
  resizeImage,
  applyFilters,
  storeImage,
  getImageUrl,
} from "./utils"

type ImageResult = {
  imageUrl: string
}

export const { POST } = serve<{ imageId: string; userId: string }>(
  async (context) => {
    const { imageId, userId } = context.requestPayload

    // Step 1: Retrieve the uploaded image
    const imageUrl = await context.run("get-image-url", async () => {
      return await getImageUrl(imageId)
    })

    // Step 2: Resize the image to multiple resolutions
    const resolutions = [640, 1280, 1920]

    const resizedImages: { body: ImageResult }[] = await Promise.all(resolutions.map(
      resolution => context.call<ImageResult>(
        `resize-image-${resolution}`,
        {
          // endpoint which returns ImageResult type in response
          url: "https://image-processing-service.com/resize",
          method: "POST",
          body: {
            imageUrl,
            width: resolution,
          }
        }
      )
    ))

    // Step 3: Apply filters to each resized image
    const filters = ["grayscale", "sepia", "contrast"]
    const processedImagePromises: Promise<string>[] = []

    for (const resizedImage of resizedImages) {
      for (const filter of filters) {
        const processedImagePromise = context.call<ImageResult>(
          `apply-filter-${filter}`,
          {
            // endpoint which returns ImageResult type in response
            url: "https://image-processing-service.com/filter",
            method: "POST",
            body: {
              imageUrl: resizedImage.body.imageUrl,
              filter,
            }
          }
        )
        processedImagePromises.push(processedImagePromise)
      }
    }
    const processedImages: { body: ImageResult }[] = await Promise.all(processedImagePromises)

    // Step 4: Store processed images in cloud storage
    const storedImageUrls: string[] = await Promise.all(
      processedImages.map(
        processedImage => context.run(`store-image`, async () => {
          return await storeImage(processedImage.body.imageUrl)
        })
      )
    )
  }
)
```

## Code Breakdown

### 1. Retrieving the Image

We begin by getting the URL of the uploaded image based on its ID:

```typescript
const imageUrl = await context.run("get-image-url", async () => {
  return await getImageUrl(imageId)
})
```

### 2. Resizing the Image

We resize the image into three different resolutions (640, 1280, 1920) using an external image processing service.

We call `context.call` for each resolution and use `Promise.all` to run them parallel:

```typescript
const resolutions = [640, 1280, 1920]

const resizedImages: { body: ImageResult }[] = await Promise.all(resolutions.map(
  resolution => context.call<ImageResult>(
    `resize-image-${resolution}`,
    {
      // endpoint which returns ImageResult type in response
      url: "https://image-processing-service.com/resize",
      method: "POST",
      body: {
        imageUrl,
        width: resolution,
      }
    }
  )
))
```

### 3. Applying Filters

After resizing, we apply filters such as grayscale, sepia, and contrast to the resized images.

Again, we call `context.call` for each filter & image pair. We collect the promises of these requests in an array `processedImagePromise`. Then, we call `Promise.all` again to run them all parallel.

```typescript
const filters = ["grayscale", "sepia", "contrast"]
const processedImagePromises: Promise<string>[] = []

for (const resizedImage of resizedImages) {
  for (const filter of filters) {
    const processedImagePromise = context.call<ImageResult>(
      `apply-filter-${filter}`,
      {
        // endpoint which returns ImageResult type in response
        url: "https://image-processing-service.com/filter",
        method: "POST",
        body: {
          imageUrl: resizedImage.body.imageUrl,
          filter,
        }
      }
    )
    processedImagePromises.push(processedImagePromise)
  }
}
const processedImages: { body: ImageResult }[] = await Promise.all(processedImagePromises)
```

### 4. Storing the Processed Images

We store each processed image in cloud storage and return the URLs of the stored images:

```typescript
const storedImageUrls: string[] = await Promise.all(
  processedImages.map(
    processedImage => context.run(`store-image`, async () => {
      return await storeImage(processedImage.body.imageUrl)
    })
  )
)
```

## Key Features

1.  **Batch Processing**: This workflow handles batch resizing and filtering of images in parallel to minimize processing time.
2.  **Scalability**: Image resizing and filtering are handled through external services, making it easy to scale. Also, the requests to these services are handled by QStash, not by the compute where the workflow is hosted.
3.  **Storage Integration**: The workflow integrates with cloud storage to persist processed images for future access.


# Payment Retries



This example demonstrates a payment retry process using Upstash Workflow.
The following example handles retrying a payment, sending emails, and suspending accounts.

## Use Case

Our workflow will:

1.  Attempt to process a payment
2.  Retry the payment if it fails with a 24-hour delay
3.  If the payment succeeds:
    *   Unsuspend the user's account if it was suspended
    *   Send an invoice email
4.  If the payment fails after 3 retries:
    *   Suspend the user's account

## Code Example

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs";

type ChargeUserPayload = {
  email: string;
};

export const { POST } = serve<ChargeUserPayload>(async (context) => {
  const { email } = context.requestPayload;

  for (let i = 0; i < 3; i++) {
    // attempt to charge the user
    const result = await context.run("charge customer", async () => {
      try {
        return await chargeCustomer(i + 1),
      } catch (e) {
        console.error(e);
        return
      }
    });

    if (!result) {
      // Wait for a day
      await context.sleep("wait for retry", 24 * 60 * 60);
    } else {
      // Unsuspend User
      const isSuspended = await context.run("check suspension", async () => {
        return await checkSuspension(email);
      });
      if (isSuspended) {
        await context.run("unsuspend user", async () => {
          await unsuspendUser(email);
        });
      }

      // send invoice email
      await context.run("send invoice email", async () => {
        await sendEmail(
          email,
          `Payment successfull. Incoice: ${result.invoiceId}, Total cost: $${result.totalCost}`
        );
      });

      // by retuning, we end the workflow run
      return;
    }
  }

  // suspend user if the user isn't suspended
  const isSuspended = await context.run("check suspension", async () => {
    return await checkSuspension(email);
  });

  if (!isSuspended) {
    await context.run("suspend user", async () => {
      await suspendUser(email);
    });

    await context.run("send suspended email", async () => {
      await sendEmail(
        email,
        "Your account has been suspended due to payment failure. Please update your payment method."
      );
    });
  }
});

async function sendEmail(email: string, content: string) {
  // Implement the logic to send an email
  console.log("Sending email to", email, "with content:", content);
}

async function checkSuspension(email: string) {
  // Implement the logic to check if the user is suspended
  console.log("Checking suspension status for", email);
  return true;
}

async function suspendUser(email: string) {
  // Implement the logic to suspend the user
  console.log("Suspending the user", email);
}

async function unsuspendUser(email: string) {
  // Implement the logic to unsuspend the user
  console.log("Unsuspending the user", email);
}

async function chargeCustomer(attempt: number) {
  // Implement the logic to charge the customer
  console.log("Charging the customer");

  if (attempt <= 2) {
    throw new Error("Payment failed");
  }

  return {
    invoiceId: "INV123",
    totalCost: 100,
  } as const;
}
```

## Code Breakdown

### 1. Charge Customer

We attempt to charge the customer:

```typescript
const result = await context.run("charge customer", async () => {
  try {
    return await chargeCustomer(i + 1),
  } catch (e) {
    console.error(e);
    return
  }
});
```

<Note>
  If we haven't put a try-catch block here, the workflow would still have retried the step.
  Hovewer, because we want to run custom logic when the payment fails, we catch the error here.
</Note>

### 2. Retry Payment

We try to charge the customer 3 times with a 24-hour delay between each attempt:

```typescript
for (let i = 0; i < 3; i++) {
  // attempt to charge the customer

  if (!result) {
    // Wait for a day
    await context.sleep("wait for retry", 24 * 60 * 60);  
  } else {
    // Payment succeeded
    // Unsuspend user, send invoice email
    // end the workflow:
    return;
  }
}
```

### 3. If Payment Succeeds

#### 3.1. Unsuspend User

We check if the user is suspended and unsuspend them if they are:

```typescript
const isSuspended = await context.run("check suspension", async () => {
  return await checkSuspension(email);
});
if (isSuspended) {
  await context.run("unsuspend user", async () => {
    await unsuspendUser(email);
  });
}
```

#### 3.2. Send Invoice Email

We send an invoice we got from the payment step to the user:

```typescript
await context.run("send invoice email", async () => {
  await sendEmail(
    email,
    `Payment successfull. Incoice: ${result.invoiceId}, Total cost: $${result.totalCost}`
  );
});
```

<Tip>
  One of the biggest advantages of using Upstash Workflow is that you have access to the result of the previous steps.
  This allows you to pass data between steps without having to store it in a database.
</Tip>

### 4. If Payment Fails After 3 Retries

#### 4.1. Suspend User

If the payment fails after 3 retries, we suspend the user and send them an email to notify them:

```typescript
const isSuspended = await context.run("check suspension", async () => {
  return await checkSuspension(email);
});

if (!isSuspended) {
  await context.run("suspend user", async () => {
    await suspendUser(email);
  });

  await context.run("send suspended email", async () => {
    await sendEmail(
      context.requestPayload.email,
      "Your account has been suspended due to payment failure. Please update your payment method."
    );
  });
}
```


# Waiting for Events



## Introduction

This example demonstrates how to handle order processing using Upstash Workflow. The workflow will wait for an external system to process an order and resume once it is 'notified'. See our documentation [for more details about events](/workflow/howto/events).

## Use Case

Our workflow will be:

1.  Receive an order request.
2.  Send an email to request order processing.
3.  Wait for an external event that indicates the order has been processed.
4.  Handle timeout scenarios if the event is not received in time.
5.  Save the results

## Code Example

```typescript
import { serve } from "@upstash/workflow/nextjs";

export const { POST } = serve(async (context) => {
  const { orderId, userEmail } = context.requestPayload;

  // Step 1: request order processing
  await context.run("request order processing", async () => {
    await requestProcessing(orderId)
  })

  // Step 2: Wait for the order to be processed
  const { eventData, timeout } = await context.waitForEvent(
    "wait for order processing",
    `order-${orderId}`,
    600 // 10 minutes timeout
  );

  if (timeout) {
    // end workflow in case of timeout
    return;
  }

  const processedData = eventData;

  // Step 3: Log the processed order
  await context.run("process-order", async () => {
    console.log(`Order ${orderId} processed:`, processedData);
  });

  // Step 4: Send a confirmation email
  await context.run("send-confirmation-email", async () => {
    await sendEmail(
      userEmail,
      "Your order has been processed!",
      processedData
    );
  });
});
```

Code for notifying the worklfow:

```typescript
import { Client } from "@upstash/workflow";

const client = new Client({ token: "<QSTASH_TOKEN>" })
const orderId = "1324"

await client.notify({
  eventId: `order-${orderId}`,
  eventData: { deliveryTime: "2 days" }
});
```

## Code Breakdown

### 1. Requesting Order Processing

In its first step, we call a method to request processing of the order in the request payload:

```typescript
await context.run("request order processing", async () => {
  await requestProcessing(orderId)
})
```

You can imagine that this step sends an email to the company responsible for the delivery.

### 2. Waiting for the Event

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/events/wait.png" />
</Frame>

Next, the workflow waits for the order to be processed. It uses [the `waitForEvent` method](/workflow/basics/context#context-waitforevent) to pause execution and listen for the event. A timeout of 10 minutes (600 seconds) is applied to avoid indefinite waiting.

```typescript
const { eventData, timeout } = await context.waitForEvent(
  "wait for order processing",
  `order-${orderId}`,
  600 // 10 minutes timeout
);

if (timeout) {
  // end workflow in case of timeout
  return;
}
```

The workflow listens for the event with the ID `order-${orderId}`. When the external service notifies the workflow with this ID, the workflow resumes. If no event is received in time, the timeout ensures the workflow doesn't hang.

### 3. Processing the Order

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/events/resume.png" />
</Frame>

Once the workflow resumes, the order data (`eventData`) is logged. This step could include updates to the database, inventory adjustments, or other backend operations specific to order processing.

```typescript
await context.run("process-order", async () => {
  console.log(`Order ${orderId} processed:`, processedData);
});
```

### 4. Sending a Confirmation Email

Once the order is processed, the workflow sends a confirmation email to the user.

```typescript
await context.run("send-confirmation-email", async () => {
  await sendEmail(
    userEmail,
    "Your order has been processed!",
    processedData
  );
});
```

## External Event Notification

To notify the workflow that the order has been processed, the external system sends a notification to the workflow with relevant data using the [`notify` method from the `Client`](/workflow/basics/client#notify-waiting-workflow). Alternatively, you can use [`context.notify` method](/workflow/basics/context#context-notify).

```typescript
import { Client } from "@upstash/workflow";

const client = new Client({ token: "<QSTASH_TOKEN>" });
const orderId = "1324";

await client.notify({
  eventId: `order-${orderId}`,
  eventData: { deliveryTime: "2 days" }
});
```

## Key Features

1.  **Asynchronous Event Handling**: The workflow waits for an external event (order processing) to occur and only resumes once the event is received.
2.  **Timeout Control**: A timeout mechanism ensures that the workflow doesn't hang if the event isn't received within the expected time.
3.  **Order Processing**: Once notified, the workflow handles order processing and sends a confirmation email to the customer.
4.  **External Event Notifications**: The external system can notify the workflow, providing the necessary data to resume execution and complete the process.


# Getting Started



## Overview

Upstash Workflow lets you write **durable, reliable and performant serverless functions**. Get delivery guarantees, automatic retries on failure, scheduling and more without managing any infrastructure.

<iframe width="720" height="405" src="https://www.youtube.com/embed/5As9tZMN3a8?rel=0&disablekb=1" title="YouTube video player" frameBorder="0" allow="accelerometer; fullscreen; clipboard-write; encrypted-media; gyroscope" allowFullScreen />

## Key Features

*   **Failure Resilience**: If your platform experiences a temporary outage, your workflow can pick up right where it left off. Upstash Workflows ensures that your application will continue, even when your environment is unstable.
*   **Long-Running Executions on Time-Limited Platforms**: Run long-running REST endpoints, such as complex AI models or video processing tools, even on serverless platforms with strict time limits.
*   **Events with Wait/Notify Mechanism**: Create workflows that wait for external events before proceeding. Ideal for use cases like waiting for user input (e.g., clicking a confirmation email) or handling asynchronous notifications from external systems.
*   **Scheduled Jobs**: Run jobs at regular intervals with support for cron expressions. Perfect for recurring tasks like reminders, regular reports, or newsletters. [Set up scheduled jobs here](/workflow/howto/schedule).
*   **Parallel Runs**: Start independent tasks in parallel and wait for them to finish simultaneously, reducing latency. [Read more about parallel runs here](/workflow/howto/parallel-runs).
*   **Long Delays**: Need your code to “sleep” for days, weeks, or even months? Upstash Workflows supports long delays beyond the time limits imposed by serverless platforms.
*   **Delivery Guarantees**: Upstash Workflows guarantees at-least-once delivery. Once a request is accepted, it will run—even if the hosting platform experiences interruptions. In the rare case of failure, the request is logged in a Dead Letter Queue, so you never lose track of critical events.
*   **Cost Reduction for Serverless Environments**: Serverless platforms charge based on execution time. With Upstash Workflows, you can offload time-consuming, I/O-bound tasks from serverless platforms, reducing operational costs. [See cost reduction example here](https://image-gen-with-workflow.vercel.app/).
*   **Rate Limiting (Coming Soon)**: Prevent overwhelming external services by configuring rate limits for your workflows. Control the maximum number of operations per second, and ensure that burst traffic is managed without dropping any calls.
*   **Observability**: Monitor your application's steps with comprehensive insights. Filter events to see which functions succeeded, failed, retried, or stalled. [Learn more about observability here](/workflow/howto/monitor).

## Quickstarts

Workflow supports Next.js, Cloudflare Workers and [many more frameworks](/workflow/quickstarts/platforms).

<CardGroup cols={2}>
  <Card title="Next.js" icon="node-js" href="/workflow/quickstarts/vercel-nextjs">
    Build a Next.js application with QStash Workflow
  </Card>

  <Card title="Cloudflare Workers" icon="cloudflare" href="/workflow/quickstarts/cloudflare-workers">
    Use and deploy Upstash Workflow on Cloudflare Workers
  </Card>
</CardGroup>

## Example Use Cases

Here are some example real world use-cases for Upstash Workflow:

<CardGroup cols={2}>
  <Card title="AI Data Processing" icon="sparkles" href="/workflow/examples/allInOne">
    Download a large dataset without timeouts, process the data in chunks and generate a report.
  </Card>

  <Card title="Waiting For Events" icon="traffic-light" href="/workflow/examples/waitForEvent">
    Control workflow execution with events, log event data and send emails
  </Card>

  <Card title="Authorization Webhook" icon="webhook" href="/workflow/examples/authWebhook">
    Start a workflow from a webhook. Handle user creation, trial management, email reminders and notifications.
  </Card>

  <Card title="Customer Onboarding" icon="user" href="/workflow/examples/customerOnboarding">
    Register a new user, send welcome emails, and periodically check and respond to the user's activity state with emails.
  </Card>

  <Card title="E-Commerce Order Fulfillment" icon="cart-flatbed-boxes" href="/workflow/examples/eCommerceOrderFulfillment">
    Receive an order request, verify the stock, process the payment, and handle order dispatch and customer notifications.
  </Card>

  <Card title="Image Processing" icon="image" href="/workflow/examples/imageProcessing">
    Manage uploading images to the data store. Apply filters and resize the images to different resolutions in parallel.
  </Card>

  <Card title="Retry Payments" icon="credit-card" href="/workflow/examples/paymentRetry">
    Retry payments with a day of delay, send emails, and suspend account if payment fails after the retries.
  </Card>
</CardGroup>

## How it works

Upstash Workflow builds on the principle of steps. Instead of defining a single, complex piece of business logic, workflows contain multiple individual steps.

In case of an error, a failed step is retried individually without needing to re-run any previous steps. Instead of the entire business logic, *each step* can take up your serverless function execution duration, and many more benefits.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/workflow-concept.png" />
</Frame>

## Next.js code example

Let's see a practical implementation of Upstash Workflow using Next.js and customer onboarding as an example. See our [Next.js Quickstart](/workflow/quickstarts/vercel-nextjs) for a complete guide.

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs";
import { sendEmail } from "./emailUtils";

// Type-safety for starting our workflow
interface InitialData {
  userId: string
  email: string
  name: string
}

export const { POST } = serve<InitialData>(async (context) => {
  const { userId, email, name } = context.requestPayload;

  // Step 1: Send welcome email
  await context.run("send-welcome-email", async () => {
    await sendEmail(email, "Welcome to our service!");
  });

  // Step 2: Wait for 3 days (in seconds)
  await context.sleep("sleep-until-follow-up", 60 * 60 * 24 * 3);

  // Step 3: AI-generate personalized follow-up message
  const { body: aiResponse } = await context.call(
    "generate-personalized-message",
    {
      url: "https://api.openai.com/v1/chat/completions",
      method: "POST",
      headers: { ... }
      body: {
        model: "gpt-3.5-turbo",
        messages: [
          { role: "system", content: "You are an assistant creating personalized follow-up messages." },
          { role: "user", content: `Create a short, friendly follow-up message for ${name} who joined our service 3 days ago.` }
        ]
      },
    }
  );

  const personalizedMessage = aiResponse.choices[0].message.content;

  // Step 4: Send personalized follow-up email
  await context.run("send-follow-up-email", async () => {
    await sendEmail(email, personalizedMessage);
  });
});
```

<Tip>
  Any HTTP request using `context.call`, like the AI-generation above, does not
  count towards your function's execution time and does not increase your
  serverless bill. It can also run for up to 2 hours, completely bypassing any
  platform-specific function timeouts.
</Tip>

The above example should give you a rough idea of how a workflow looks in code. For step-by-step instructions on setting up your first workflow with images along the way, see our [Next.js Quickstart](/workflow/quickstarts/vercel-nextjs).

***

Here are more details about what the `context` object does:

*   [context.run](/workflow/basics/context#context-run)
*   [context.sleep](/workflow/basics/context#context-sleep)
*   [context.sleepUntil](/workflow/basics/context#context-sleepuntil)
*   [context.call](/workflow/basics/context#context-call)
*   [context.waitForEvent](/workflow/basics/context#context-waitforevent)
*   [context.notify](/workflow/basics/context#context-notify)

See [caveats](/workflow/basics/caveats) for more complex API usage and best-practices when using Upstash Workflow.

***

Guides on common workflow topics:

*   [Cancel a running workflow](/workflow/howto/cancel)
*   [Wait for External Events](/workflow/howto/events)
*   [Handle failed workflow runs](/workflow/howto/failures)
*   [Monitor active workflows in real-time](/workflow/howto/failures)
*   [Schedule repeated workflow runs](/workflow/howto/schedule)
*   [Secure a workflow endpoint](/workflow/howto/security)
*   [Handle workflow route code changes](/workflow/howto/changes)
*   [Develop your workflows locally](/workflow/howto/local-development)
*   [Pricing](/workflow/pricing)

If you're curious about the behind-the-scenes about how we ensure separate step execution or prevent serverless timeouts, we wrote about it [here](/workflow/how)! :)

Here is our [Upstash Workflow roadmap](/workflow/roadmap) to see what we planned for the future.


# How Workflow works



We created Upstash Workflow to help you ship **reliable code with minimal changes to your existing infrastructure**:

*   No more serverless function timeouts
*   Automatic recovery when a workflow fails mid-execution
*   Automatic retries when your service is temporarily unavailable
*   Easily monitor system activity in real-time

Upstash Workflow is built on top of QStash, our serverless messaging and scheduling solution, to achieve these features. Here's a simplified breakdown of how it works:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/workflow-concept.png" />
</Frame>

1.  Call your workflow endpoint to trigger the workflow

2.  The Workflow SDK sends a request to QStash with an automatically generated workflow run ID, headers, and the initial data as the request body

3.  QStash calls your workflow endpoint with the current step to execute and the results of the previous steps:
    *   Skip successfully executed steps
    *   Assign previous step results to their respective variables
    *   Invoke the next step

***

In serverless environments, an API route is normally limited by a maximum function execution time (i.e. 10 seconds). By using `context.run`, *each step* can now take up to the maximum duration, as each step is a separate HTTP request containing data from previous steps.

As Upstash Workflow is built on top of QStash, the `context` methods map to QStash features:

*   `context.run` -> regular QStash call
*   `context.sleep` and `context.sleepUntil` -> QStash's [delay feature](/qstash/features/delay)
*   `context.call` -> QStash's [callback feature](/qstash/features/callbacks)

Retries, for example, are built into all QStash calls by default and, therefore, apply to every step in your workflow. For step-by-step resumability, QStash keeps a copy of your workflow state until the run is complete.

This architecture makes your serverless functions **durable, reliable, and optimized for performance**, even during runtime errors or temporarily unavailable services.


# Cancel a run



You can cancel a running workflow both programatically and from your Upstash Workflow console.

## Cancelling via console

In your Upstash Workflow console, find the run you'd like to cancel and press the `Cancel Workflow` button on the right side:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/cancel_workflow.png" />
</Frame>

## Cancelling programatically

```javascript
import { Client } from "@upstash/workflow";

const client = new Client({ token: "<QSTASH_TOKEN>" })
await client.cancel({ workflowRunId: "<WORKFLOW_RUN_ID>" })
```

And replace `<WORKFLOW_RUN_ID>` with your actual run ID. You can also use the [Upstash Workflow REST API](/workflow/rest/runs/cancel) to cancel a run programatically.


# Update a workflow



## Understanding workflow continuity

Workflows are composed of multiple steps. When you modify workflow code, it's important to consider how these changes might affect in-progress workflows.

## Potential issues

If your code changes remove or modify existing steps, in-progress workflows may attempt to continue from a point that no longer exists. This can lead to workflow failures, typically resulting in the following error:

```bash
HTTP status 400. Incompatible step name. Expected <STEP_NAME>, got <STEP_NAME>
```

## Safe changes

Modifying a workflow's code is safe when:

*   There are no active workflow runs
*   You're only adding new steps to the end of the workflow

## Guidelines for updating workflows

Consider the following approaches when updating your workflow code:

*   **Accept potential failures:** If you're fine with in-progress workflows failing, you can make any code changes.
*   **Use a different route:** To avoid failures, consider serving the updated workflow under a different route.
*   **Stop traffic before deployment:** If you need to keep the same route, stop all traffic before deploying new code.
*   **Add steps only:** If stopping traffic is not an option, limit your changes to adding new steps at the end of the workflow.

For a deeper understanding of these limitations, see our [how workflows work](/workflow/how) section.


# Waiting for Events



You can use `context.waitForEvent` to pause a workflow until a specific event occurs and resume it with event data when the event is received.

<Tip>
  You can learn more about Workflow events from [our real-world example](/workflow/examples/waitForEvent).
</Tip>

## `context.waitForEvent`

The [`waitForEvent` method](/workflow/basics/context#context-waitforevent) pauses the execution of a workflow and waits for an external event to occur, identified by an event ID. This is particularly useful in asynchronous workflows that rely on external systems to provide data or signals.

```typescript
const { eventData, timeout } = await context.waitForEvent(
  "description of event",
  "event-id",
  timeoutInSeconds
);
```

Third parameter is the timeout. It is the maximum time (in seconds) to wait for the event. If the event doesn't occur within this time, the workflow will proceed, and the `timeout` variable will be `true`.

Maximum timeout value is equal to [the "Max Delay" value of your QStash plan](https://upstash.com/pricing/qstash). It's 7 days for free users, 1 year for pay as you go users and unlimited for pro users.

## `client.notify`

[The `notify` method](/workflow/basics/client#notify-waiting-workflow) is used to notify a workflow that the expected event has occurred. It notifies all workflows waiting for the given `eventId` and provides the `eventData` to the waiting workflows, allowing them to resume execution.

```typescript
import { Client } from "@upstash/workflow";

const client = new Client({ token: "<QSTASH_TOKEN>" });

await client.notify({
  eventId: "event-id",
  eventData: { my: "data" },
});
```

`eventData` provided in `client.notify` will be available in the result of `context.waitForEvent` as it is.

## `context.notify`

You can also notify workflows in another workflow using [the `context.notify` method](/workflow/basics/context#context-notify).

```typescript
const { notifyResponse } = await context.notify(
  "notify step", // notify step name
  "event-Id",    // event id
  { my: "data" } // event data
);
```


# Handle failed runs



This guide shows you how to **gracefully handle failed workflow runs**. This involves best practices on resolving runtime errors, logging and manually retrying runs that have failed multiple times.

## Why a workflow might fail

*   A step in your workflow throws a database error that causes your code to fail at runtime.
*   QStash calls your workflow URL, but the URL is not reachable - for example, because of a temporary outage of your deployment platform.
*   A single step takes longer than your platform's function execution limit.

QStash automatically retries a failed step **three times with exponential backoff** to allow temporary outages to resolve.

<Frame caption="A failed step is automatically retried three times">
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/automatic_retry.png" />
</Frame>

If, even after all retries, your step does not succeed, we'll move the failed run into your [Dead Letter Queue (DLQ)](/qstash/howto/handling-failures#dead-letter-queue). That way, you can always manually retry it again and debug the issue.

<Frame caption="Manually retry from the Dead-Letter-Queue (DLQ)">
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/workflow_dlq.png" />
</Frame>

{/* To take certain action whenever a workflow fails (i.e. cleaning up your database or logging to Sentry), configure a `failureFunction` or a `failureUrl` for your workflow.  

If you want to take an action (a cleanup/log), you can configure either `failureFunction` or a `failureUrl` on the `serve` method of your workflow.
These options allow you to define custom logic or an external endpoint that will be triggered when a failure occurs. */}

## Using a `failureFunction` (recommended)

The `serve` function you use to create a workflow endpoint accepts a `failureFunction` parameter - an easy way to gracefully handle errors (i.e. logging them to Sentry) or your custom handling logic.

```typescript api/workflow/route.ts
export const { POST } = serve<string>(
  async (context) => {
    // Your workflow logic...
  },
  {
    failureFunction: async (context, failStatus, failResponse, failHeaders) => {
      // Handle error, i.e. log to Sentry
    },
  }
)
```

Note: If you use a custom authorization method to secure your workflow endpoint, add authorization to the `failureFunction` too. Otherwise, anyone could invoke your failure function. Read more here: [securing your workflow endpoint](/workflow/howto/security).

## Using a `failureUrl`

The `failureUrl` handles cases where the service hosting your workflow URL is unavailable. In this case, a workflow failure notification is sent to another reachable endpoint.

```typescript
export const { POST } = serve<string>(
  async (context) => {
    // Your workflow logic...
  },
  {
    failureUrl: "https://<YOUR_FAILURE_URL>/workflow-failure",
  }
)
```

## Debugging failed runs

In your DLQ, filter messages via the `Workflow URL` or `Workflow Run ID` to search for a particular failure. We include all request and response headers and bodies to simplify debugging failed runs.

For example, let's debug the following failed run. Judging by the status code `404`, the `Ngrok-Error-Code` header of `ERR_NGROK_3200` and the returned HTML body, we know that the URL our workflow called does not exist.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/workflow_dlq_debug.png" />
</Frame>


# Local development



Upstash Workflow require a publicly accessible API endpoint to function. While we plan to offer a native solution in the future, please use a third-party tunneling service for now.

## Local development with ngrok (recommended)

The easiest way to make a local URL publically available is [ngrok](https://ngrok.com), a free tunneling service.

Create an account on [dashboard.ngrok.com/signup](https://dashboard.ngrok.com/signup) and follow the [setup instructions](https://dashboard.ngrok.com/get-started/setup) to download the ngrok CLI and connect your account. This process takes only a few minutes and is totally free.

For example on MacOS, you can connect your account like this:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/ngrok_mac_setup.png" />
</Frame>

Or on Windows:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/ngrok_windows_setup.png" />
</Frame>

Once you have installed the ngrok CLI, add your ngrok-issued auth token like this:

```bash Terminal
ngrok config add-authtoken <YOUR-AUTH-TOKEN>
```

and replace `<YOUR-AUTH-TOKEN>` with your actual auth token.

## Start the tunnel

Make your local server available publically by running the following command:

```bash
ngrok http <PORT>
```

for example, if your Next.js server is running on port `3000`, the command is:

```bash
ngrok http 3000
```

The output will look something like this:

```plaintext
Session Status                online
Account                       <YOUR-NAME> (Plan: Free)
Version                       3.1.0
Region                        Europe (eu)
Latency                       -
Web Interface                 http://127.0.0.1:4040
Forwarding                    https://e02f-2a02-810d-af40-5284-b139-58cc-89df-b740.eu.ngrok.io -> http://localhost:3000
Connections                   ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00
```

The long URL in the `Forwarding` line serves the same purpose as your localhost URL, the only difference being that it is publically accessible. We need this URL to make our workflow available to QStash for local development, either as the `baseUrl` parameter or the `UPSTASH_WORKFLOW_URL` environment variable (both options provide the same functionality).

Note: The `UPSTASH_WORKFLOW_URL` environment variable is only necessary for local development. In production, the `baseUrl` parameter is automatically set and can be omitted.

<Tip>
  Ensure that the port of your local server matches the one you're using with ngrok. For example, if your server is
  running on port 8080, use `ngrok http 8080`.
</Tip>


# Monitor a run



Logs of the Workflow can be found in the console under the [Workflow tab](https://console.upstash.com/qstash?tab=workflows)

It has two modes that can be changed from to top right corner:

*   **Grouped By Workflow Run Id** : This is the default view where the overall workflow run is summarized. You can see each step, its start date and output and also whether it is succesfully delivered or not.
*   **Flat View** : The flat view is mostly for debugging when a workflow step has failed. It has more details about individual
    events of each step.

We have 4 new event types introduced on top of [existing message events](/qstash/howto/debug-logs). Note that message events are relavent for individual steps.
The new 4 events are to show the state of a workflow run.

*   **RUN\_STARTED**
*   **RUN\_FAILED**
*   **RUN\_CANCELLED**
*   **RUN\_SUCCESS**

Here is an example route implementation with our SDK and its corresponding screenshot from [Workflow tab]() showing its events.

```javascript
import { serve } from "@upstash/workflow/nextjs";
import { retrieveEmail, fetchFromLLm, UserRequest} from "../../../lib/util";


export const { POST } = serve<UserRequest>(
  async (context) => {
    const input = context.requestPayload;

    await context.sleep("sleep", 10);

    const p1 = context.run("retrieveEmail", async () => {
      return retrieveEmail(input.id);
    });

    const p2 = context.run("askllm", async () => {
      return fetchFromLLm(input.question);
    });

    await Promise.all([p1, p2])
  },
);
```

We are calling this endpoint as follows:

```bash
curl -XPOST https://qstash-workflow.vercel.app/api/example -d '{"id": "id_123", "question": "what is the meaning of life?"}'
```

Steps of the related workflow runs look as follows:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/qstash-workflows.png" />
</Frame>

Here is another run where `fetchFromLLm` is throwing an error.
When a particular step is retrying/failed, you can click it to see more details as shown below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/qstash-workflows-step-detail.png" />
</Frame>

While a step is retrying, it is composed of several messages. You can click the `messageId` field above to
see all the events of a particular step to find out why it is retrying or failed.
When clicked, the workflow events will switch to `flat` view with `messageId` filtered as shown below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/qstash-workflows-flat.png" />
</Frame>

In this screen, you can go to one of the `error` events, and see what response it returned to understand
the details of the error as shown below:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/qstash-workflows-error-detail.png" />
</Frame>


# Parallel runs



Just like you can execute multiple JavaScript promises at the same time using `Promise.all`, you can run multiple workflow steps at the same time:

```typescript
const [result1, result2, result3] =
  await Promise.all([
    ctx.run("parallel-step-1", async () => { ... }),
    ctx.run("parallel-step-2", async () => { ... }),
    ctx.run("parallel-step-3", async () => { ... }),
  ])
```

In a complete code example, your workflow could look like this:

```typescript app/api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"
import { checkInventory, brewCoffee, printReceipt } from "@/utils"

export const { POST } = serve(async (ctx) => {
  const [coffeeBeansAvailable, cupsAvailable, milkAvailable] =
    await Promise.all([
      ctx.run("check-coffee-beans", () => checkInventory("coffee-beans")),
      ctx.run("check-cups", () => checkInventory("cups")),
      ctx.run("check-milk", () => checkInventory("milk")),
    ])

  // If all ingedients available, brew coffee
  if (coffeeBeansAvailable && cupsAvailable && milkAvailable) {
    const price = await ctx.run("brew-coffee", async () => {
      return await brewCoffee({ style: "cappuccino" })
    })

    await printReceipt(price)
  }
})
```

After running your workflow, your dashboard shows each step in detail:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/parallel-workflow-runs.png" />
</Frame>


# Schedule repeated runs



You can schedule a workflow to run periodically using a cron definition.

## Scheduling a workflow

For example, let's define a workflow that creates a backup of some important data daily. Our workflow endpoint might look like this:

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"
import { createBackup, uploadBackup } from "./utils"

export const { POST } = serve(
  async (ctx) => {
    const backup = await ctx.run("create-backup", async () => {
      return await createBackup()
    })

    await ctx.run("upload-backup", async () => {
      await uploadBackup(backup)
    })
  },
  {
    failureFunction(context, failStatus, failResponse, failHeader) {
      // immediately get notified for failed backups
      // i.e. send an email, log to Sentry
    },
  }
)
```

To run this endpoint on a schedule, navigate to `Schedules` in your QStash dashboard and click `Create Schedule`:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/create_schedule.png" />
</Frame>

Enter your live endpoint URL, add a CRON expression to define the interval at which your endpoint is called (i.e. every day, every 15 minutes, ...) and click `Schedule`:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/schedule_workflow.png" />
</Frame>

Your workflow will now run repeatedly at the interval you have defined. For more details on CRON expressions, see our [QStash scheduling documentation](/qstash/features/schedules).

## Scheduling a per-user workflow

In order to massively improve the user experience, many applications send weekly summary reports to their users. These could be weekly analytics summaries or SEO statistics to keep users engaged with the platform.

Let's create a user-specific schedule, sending a first report to each user exactly 7 days after they signed up:

```typescript api/sign-up/route.ts
import { signUp } from "@/utils/auth-utils"
import { Client } from "@upstash/qstash"

const client = new Client({ token: process.env.QSTASH_TOKEN! })

export async function POST(request: Request) {
  const userData: UserData = await request.json()

  // Simulate user registration
  const user = await signUp(userData)

  // Calculate the date for the first summary (7 days from now)
  const firstSummaryDate = new Date(Date.now() + 7 * 24 * 60 * 60 * 1000)

  // Create cron expression for weekly summaries starting 7 days from signup
  const cron = `${firstSummaryDate.getMinutes()} ${firstSummaryDate.getHours()} * * ${firstSummaryDate.getDay()}`

  // Schedule weekly account summary
  await client.schedules.create({
    scheduleId: `user-summary-${user.email}`,
    destination: "https://<YOUR_APP_URL>/api/send-weekly-summary",
    body: { userId: user.id },
    cron: cron,
  })

  return NextResponse.json(
    { success: true, message: "User registered and summary scheduled" },
    { status: 201 }
  )
}
```

This code will call our workflow every week, starting exactly seven days after a user signs up. Each call to our workflow will contain the respective user's ID.

Note: when creating a user-specific schedule, pass a unique `scheduleId` to ensure the operation is idempotent. (See [caveats](/workflow/basics/caveats) for more details on why this is important).

Lastly, add the summary-creating and email-sending logic inside of your workflow. For example:

```typescript api/send-weekly-summary/route.ts
import { serve } from "@upstash/workflow/nextjs"
import { getUserData, generateSummary } from "@/utils/user-utils"
import { sendEmail } from "@/utils/email-utils"

// Type-safety for starting our workflow
interface WeeklySummaryData {
  userId: string
}

export const { POST } = serve<WeeklySummaryData>(async (context) => {
  const { userId } = context.requestPayload

  // Step 1: Fetch user data
  const user = await context.run("fetch-user-data", async () => {
    return await getUserData(userId)
  })

  // Step 2: Generate weekly summary
  const summary = await context.run("generate-summary", async () => {
    return await generateSummary(userId)
  })

  // Step 3: Send email with weekly summary
  await context.run("send-summary-email", async () => {
    await sendEmail(user.email, "Your Weekly Summary", summary)
  })
})
```

Just like that, each user will receive an account summary every week, starting one week after signing up.


# Secure an endpoint



To prevent *anyone* from triggering your workflow endpoint, you can use two methods:

*   QStash's [built-in request verification](/qstash/features/security)
*   Custom header and custom authorization mechanism

### Using QStash's built-in request verification (recommended)

Use QStash's built-in request verification to allow only authorized clients to trigger your workflow. Set two environment variables in addition to your QStash API key:

```bash .env
QSTASH_CURRENT_SIGNING_KEY=xxxxxxxxx
QSTASH_NEXT_SIGNING_KEY=xxxxxxxxx
```

And replace `xxxxxxxxx` with your actual keys. Find both of these keys in your QStash dashboard under the **"Signing keys"** tab:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_signing_keys.png" />
</Frame>

This will require **every request** to your workflow endpoint to include a valid signature, including the initial request that triggers a workflow. In other words: all requests need to come either from QStash (which automatically populates the `Upstash-Signature` header) or from a client that manually populates the `Upstash-Signature` header with your signing secret.

We suggest using QStash's publish API to trigger your workflow:

```bash Terminal
curl -XPOST \
    -H 'Authorization: Bearer <YOUR_QSTASH_TOKEN>' \
    -H "Content-type: application/json" \
    -d '{ "initialData": "hello world" }' \
    'https://qstash.upstash.io/v2/publish/https://<your-app-url>/api/workflow'
```

For edge cases that do not support environment variables as outlined above, you can explicitly pass your signing keys to the `serve` function:

```typescript api/workflow/route.ts
import { Receiver } from "@upstash/qstash";
import { serve } from "@upstash/workflow/nextjs";

export const { POST } = serve(async (context) => {
    // Your workflow steps...
  },
  {
    receiver: new Receiver({
      currentSigningKey: process.env.QSTASH_CURRENT_SIGNING_KEY,
      nextSigningKey: process.env.QSTASH_NEXT_SIGNING_KEY
    })
  }
);
```

## Custom Authorization Method

You can use your own authorization mechanism with Upstash Workflow. We ensure that the initial headers and initial request body will be available on every call made to your workflow.

**Important:** When using your own auth mechanism and using a `failureFunction`, make sure to authenticate requests to the failure function as well. Otherwise, any client could trigger the failure function.

```typescript api/workflow/route.ts
import { serve } from "@upstash/workflow/nextjs"

export const { POST } = serve(
  async (context) => {
    const authHeader = context.headers.get("authorization")
    const bearerToken = authHeader?.split(" ")[1]

    if (!isValid(bearerToken)) {
      console.error("Authentication failed.")
      return
    }

    // Your workflow steps..
  },
  {
    failureFunction: async ({ headers }) => {
      const authHeader = context.headers.get("authorization")
      const bearerToken = authHeader?.split(" ")[1]

      if (!isValid(bearerToken)) {
        // ...
      }
    },
  }
)
```


# Start a Run



You’ve defined your workflow, and the final step is to trigger the endpoint!

There are three main ways to start your workflow:

### 1. [Using `client.trigger` (Recommended)](/workflow/basics/client#trigger-workflow)

We recommend using `client.trigger` to start your workflow as it returns the workflow run ID in its result and results in one less QStash publish per workflow.

```ts
// Using the workflow client
import { Client } from "@upstash/workflow";

const client = new Client({ token: "<QSTASH_TOKEN>" });

const { workflowRunId } = await client.trigger({
  url: "https://workflow-endpoint.com",
  body: "hello there!",         // Optional body
  headers: { ... },             // Optional headers
  workflowRunId: "my-workflow", // Optional workflow run ID
  retries: 3                    // Optional retries for the initial request
});
```

### 2. [Publishing a message with QStash](/qstash/howto/publishing)

```ts
// Using the QStash client
import { Client } from "@upstash/qstash";

const client = new Client({ token: "<QSTASH_TOKEN>" });

const { messageId } = await client.publishJSON({
  url: "https://workflow-endpoint.com",
  body: { hello: "there!" },
  headers: { ... },    
  retries: 3           
});
```

### 3. Sending an HTTP Request

<Tabs>
  <Tab title="curl">
    ```bash
    curl -X POST https://workflow-endpoint.com \
         -H "my-header: foo" \
         -b '{"foo": "bar"}'
    ```
  </Tab>

  <Tab title="fetch">
    ```js
    await fetch("https://workflow-endpoint.com", {
      method: "POST",
      body: JSON.stringify({ "foo": "bar" }),
      headers: {
        "my-header": "foo"
      }
    });
    ```
  </Tab>
</Tabs>

### Secured Endpoints

If you’ve [secured your endpoint with signing keys](/workflow/howto/security), only the first two methods (`trigger` and `publish`) will work. Direct calls to the endpoint (e.g., via `curl` or `fetch`) will not be accepted.

### Accessing Payload and Headers

When you call the endpoint, the payload and headers you send will be accessible in the context:

*   The payload is available in the `context.requestPayload` field.
*   The headers are available in the `context.headers` field.

For more details, refer to the documentation on [the Context object](/workflow/basics/context).


# Migrate to the New SDK



In October 2024, we released a new SDK, `@upstash/workflow`, for Upstash Workflow, separating its development from the QStash SDK. Although Upstash Workflow is built on QStash, our goal is to improve the developer experience and support with a dedicated SDK. Development for Upstash Workflow will occur in `@upstash/workflow`, and Workflow-related imports will be removed from `@upstash/qstash` in future releases.

If you started using Upstash Workflow with `@upstash/qstash`, you will need to migrate to `@upstash/workflow`. We have made some backward-incompatible changes, but we aim to make the transition as smooth as possible.
In this guide, we will explain the changes you may need to make for migration.

### Install `@upstash/workflow`

First, we will need to install the new package with:

<Tabs>
  <Tab title="npm">
    ```bash
    npm install @upstash/workflow
    ```
  </Tab>

  <Tab title="pnpm">
    ```bash
    pnpm install @upstash/workflow
    ```
  </Tab>

  <Tab title="bun">
    ```bash
    bun add @upstash/workflow
    ```
  </Tab>
</Tabs>

If you were using `@upstash/qstash` only for workflow, you can uninstall it from your project.

### Serve methods

You will need to change the imports from `@upstash/qstash` to @upstash/workflow:

```ts
// old
import { serve } from "@upstash/qstash/nextjs"

// new 
import { serve } from "@upstash/workflow/nextjs"
```

We have updated what our `serve` methods return. We made this change to make it
easier to extend the API in the future.

For instance, Next.js method changed like this:

```javascript
// old
export const POST = serve(...);

// new
export const { POST } = serve(...);
```

We kept the `serve` method of `Hono` the same. The rest are updated in a similar way.
See [the quickstarts](/workflow/quickstarts/platforms) for the new way `serve`
should be used.

Additionally, `@upstash/workflow/nuxt` import is removed. You should use `@upstash/workflow/h3`
instead. This change was made because `nuxt` uses `h3` under the hood and our `serve` method
for `nuxt` can work with any project using `h3`.

### Updating `context.call`

If you were using [`context.call` method](/workflow/basics/context#context-call) in your workflow, you will need to change
how it's called and what it returns. Here is what the change looks like:

```javascript
// old
const result = await context.call("call step", "<call-url>", "POST", ...)

// new
const {
  status,  // response status
  headers, // response headers
  body     // response body
} = await context.call("call step", {
  url: "<call-url>",
  method: "POST",
  ...
})
```

In the old version, we only returned the response body. Also, if the request
to the url failed, [the workflow run would fail](/workflow/howto/failures).

In the new version, we update how the parameters are passed to the `context.call`.
Additionally, we change the fail behavior: if the request fails, it doesn't make the
workflow fail. Instead, the status and the body is simply returned and workflow
continues as usual.

If you have ongoing workflow runs which call `context.call` during your transition,
`status` and `headers` fields may not be available in these old runs. After your
transition, all workflow runs will have all three fields.

### Renaming Errors

The errors in Workflow were renamed from `QStashWorkflowError` and `QStashWorkflowAbort` to `WorkflowError` and `WorkflowAbort`.


# Pricing



Upstash Workflow is based on QStash and uses a "pay-as-you-go" pricing model. You only incur costs when your app receives traffic, meaning there's no charge when it's not in use. Click [here](https://upstash.com/pricing/qstash) to view the QStash pricing.

A workflow run consists of several QStash messages, with the total cost determined by the number of messages used.

You can track your current message usage and associated costs in the [Overview tab of the console](https://console.upstash.com/qstash?tab=details).

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash/message_cost.png" />
</Frame>

For detailed pricing information based on different plans, visit our [QStash pricing page](https://upstash.com/pricing/qstash).

### Message Usage per Workflow Run

*   [context.run](/workflow/basics/context#context-run), [context.sleep](/workflow/basics/context#context-sleep), [context.sleepUntil](/workflow/basics/context#context-sleepuntil), or [context.waitForEvent](/workflow/basics/context#context-waitforevent) commands generate a single message.
*   The [context.call](/workflow/basics/context#context-call) command generates two messages.
*   If the workflow endpoint or URL in [context.call](/workflow/basics/context#context-call) returns an error or is unreachable, the workflow SDK will retry the call (up to 3 times by default). Each retry counts as a new message.


# Astro



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/workflow-js/tree/main/examples/astro" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use and deploy
Upstash Workflow with Astro. You can also explore our [Astro example](https://github.com/upstash/workflow-js/tree/main/examples/astro)
for a detailed, end-to-end example and best practices.

## Prerequisites

1.  An Upstash QStash API key.
2.  Node.js and npm (another package manager) installed.

If you haven't obtained your QStash API key yet, you can do so by [signing up](https://console.upstash.com/login) for an Upstash account and navigating to your QStash dashboard.

## Step 1: Installation

First, install the Workflow SDK in your Astro project:

<CodeGroup>
  ```bash npm
  npm install @upstash/workflow 
  ```

  ```bash pnpm
  pnpm install @upstash/workflow
  ```

  ```bash bun
  bun add @upstash/workflow
  ```
</CodeGroup>

## Step 2: Configure Environment Variables

Create a `.env` file in your project root and add your QStash token. This key is used to authenticate your application with the QStash service.

```bash Terminal
touch .env
```

Open the environment file and add your QStash token, as well as the environment you're running in:

```bash .env
QSTASH_TOKEN="***"
QSTASH_URL="***"

# if you are working on local environment
UPSTASH_WORKFLOW_URL=""
```

You can learn more about `UPSTASH_WORKFLOW_URL` from [our documentation on local development](https://upstash.com/docs/workflow/howto/local-development).

Replace `***` with your actual QStash token and url found here:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_token.png" />
</Frame>

## Step 3: Create a Workflow Endpoint

A workflow endpoint allows you to define a set of steps that, together, make up a workflow. Each step contains a piece of business logic that is automatically retried on failure, with easy monitoring via our visual workflow dashboard.

To define a workflow endpoint with Astro, navigate into your entrypoint file (usually `src/index.ts`) and add the following code:

<Tabs>
  <Tab title="Minimal example">
    ```typescript src/pages/api/workflow.ts
    import { serve } from "@upstash/workflow/astro";


    export const { POST } = serve<string>(async (context) => {
      const result1 = await context.run("initial-step", () => {
        console.log("initial step ran")
        return "hello world!"
      })

      await context.run("second-step", () => {
        console.log(`second step ran with value ${result1}`)
      })
    }, {
      // env must be passed in astro.
      // for local dev, we need import.meta.env.
      // For deployment, we need process.env:
      env: {
        ...process.env,
        ...import.meta.env
      }
    })

    ```
  </Tab>
</Tabs>

## Step 4: Run the Workflow Endpoint

To start your Astro app locally, run the following command:

```bash Terminal
npm run dev
```

Executing this command prints a local URL to your workflow endpoint. By default, this URL is `http://localhost:3001`.

Upstash Workflow is powered by [QStash](/qstash/overall/getstarted), and QStash needs a publicly accessible URL to run your workflows. You can easily forward your localhost URL to a live URL by following our guide about [setting up your workflow endpoint for local development](/workflow/howto/local-development).

Once you have a live URL, proceeed with either one of the following steps:

### Setting an enviroment variable (recommended)

Set your publically accessible URL (without extensions such as `/workflow`) as an environment variable. This variable is only needed for local development and doesn't need to be set in production:

```bash .env
UPSTASH_WORKFLOW_URL="https://<YOUR_NGROK_OR_TUNNEL_URL>/"
```

### Using the `baseUrl` option

As an alternative to setting an environment variable, you can also use the `baseUrl` option in the `serve` method. This option is only needed for local development and can be omitted in production:

```typescript src/pages/api/workflow.ts
export const { POST } = serve<string>(
  async (context) => { ... }, {
    // just the base URL, no `/api/workflow`
    baseUrl: 'https://<YOUR_NGROK_OR_TUNNEL_URL>/'
  }
)
```

### Triggering the workflow

After setting your live URL as the environment variable or `baseUrl` option, trigger
your workflow by sending a POST request. For each workflow run, a unique workflow
run ID is returned:

```bash Terminal
curl -X POST http://localhost:3000/api/workflow \
    -H "Content-Type: application/json" \
    -d '{"message": "Hello from the workflow!"}'

# result: {"workflowRunId":"wfr_xxxxxx"}
```

See the [documentation on starting a workflow](/workflow/howto/start) for other ways you can start your workflow.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/hono_local_request.png" />
</Frame>

Use this ID to track the workflow run and see its status in your QStash workflow dashboard. All steps are listed with their statuses, headers, and body for a detailed overview of your workflow from start to finish. Click on a step to see its detailed logs.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/dashboard.png" />
</Frame>

## Step 5: Deploying to Production

When deploying your Hono app with Upstash Workflow to production, there are a few key points to keep in mind:

1.  **Environment Variables**: Make sure that all necessary environment variables from your `.env` file are set in your Vercel project settings. For example, your `QSTASH_TOKEN`, and any other configuration variables your workflow might need.

2.  **Remove Local Development Settings**: In your production code, you can remove or conditionally exclude any local development settings. For example, the `baseUrl` option in the `serve` function can be omitted in production.

```typescript src/index.ts
import { serve } from "@upstash/workflow/astro";

export const { POST } = serve<string>(
  async (context) => { ... }, {
    // just the base URL, no `/api/workflow`
    baseUrl: 'https://<YOUR_NGROK_OR_TUNNEL_URL>/'
  }
)
```

3.  **Deployment**: Deploy your Astro app to production as you normally would, for example to Vercel, Heroku, or AWS.

4.  **Verify Workflow Endpoint**: After deployment, verify that your workflow endpoint is accessible by making a POST request to your production URL:

```bash Terminal
curl -X POST <DEPLOYMENT_URL>/api/workflow \
    -H "Content-Type: application/json" \
    -d '{"message": "Hello from the workflow!"}'
```

5.  **Monitor in QStash Dashboard**: Use the QStash dashboard to monitor your production workflows. You can track workflow runs, view step statuses, and access detailed logs.

6.  **Set Up Alerts**: Consider setting up alerts in Sentry or other monitoring tools to be notified of any workflow failures in production.

## Next Steps

1.  Learn how to protect your workflow endpoint from unauthorized access by [securing your workflow endpoint](/workflow/howto/security).

2.  Explore our [Astro example](https://github.com/upstash/workflow-js/tree/main/examples/astro) for a detailed, end-to-end example and best practices.

3.  For setting up and testing your workflows in a local environment, check out our [local development guide](/workflow/howto/local-development).


# Cloudflare Workers



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/workflow-js/tree/main/examples/cloudflare-workers" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use and deploy Upstash Workflow with Cloudflare Workers. You can also explore our [Cloudflare Workers example](https://github.com/upstash/workflow-js/tree/main/examples/cloudflare-workers) or [Hono.js Cloudflare Workers example](https://github.com/upstash/workflow-js/tree/main/examples/cloudflare-workers-hono) for detailed, end-to-end examples and best practices.

## Prerequisites

1.  An Upstash QStash API key.
2.  Node.js and npm (another package manager) installed.

If you haven't obtained your QStash API key yet, you can do so by [signing up](https://console.upstash.com/login) for an Upstash account and navigating to your QStash dashboard.

## Step 1: Installation

First, install the Workflow SDK in your worker project:

<Tabs>
  <Tab title="npm">
    ```bash
    npm install @upstash/workflow
    ```
  </Tab>

  <Tab title="pnpm">
    ```bash
    pnpm install @upstash/workflow
    ```
  </Tab>

  <Tab title="bun">
    ```bash
    bun add @upstash/workflow
    ```
  </Tab>
</Tabs>

## Step 2: Configure Environment Variables

Create a `.dev.vars` file in your project root and add your QStash token. This key is used to authenticate your application with the QStash service.

```bash Terminal
touch .dev.vars
```

Open the environment file and add your QStash token, as well as the environment you're running in:

```bash .dev.vars
QSTASH_TOKEN=xxxxxxxxx
ENVIRONMENT=development
```

Replace `xxxxxxxxx` with your actual QStash token found here:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_token.png" />
</Frame>

## Step 3: Create a Workflow Endpoint

A workflow endpoint allows you to define a set of steps that, together, make up a workflow. Each step contains a piece of business logic that is automatically retried on failure, with easy monitoring via our visual workflow dashboard.

To define a workflow endpoint with Cloudflare Workers, navigate into your workers entrypoint file (usually `src/index.ts`) and add the following code:

```typescript src/index.ts
import { serve } from "@upstash/workflow/cloudflare"

interface Env {
  ENVIRONMENT: "development" | "production"
}

export default serve<{ text: string }>(
  async (context) => {
    const initialPayload = context.requestPayload.text

    const result = await context.run("initial-step", async () => {
        console.log(`Step 1 running with payload: ${initialPayload}`)

        return { text: "initial step ran" }
      }
    )

    await context.run("second-step", async () => {
      console.log(`Step 2 running with result from step 1: ${result.text}`)
    })
  }
)
```

## Step 4: Run the Workflow Endpoint

To start your worker locally, run the following command:

```bash Terminal
npm run wrangler dev
```

Executing this command prints a local URL to your workflow endpoint. By default, this URL is `http://localhost:8787`.

You can verify your correct environment variable setup by checking the wrangler output, which should now have access to your `QSTASH_TOKEN` binding and log your local URL:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/wrangler.png" />
</Frame>

Upstash Workflow is powered by [QStash](/qstash/overall/getstarted), and QStash needs a publicly accessible URL to run your workflows. You can easily forward your localhost URL to a live URL by following our guide about [setting up your workflow endpoint for local development](/workflow/howto/local-development).

Once you have a live URL, proceeed with either one of the following steps:

### Setting an enviroment variable (recommended)

Set your publically accessible URL as an environment variable. This variable is only needed for local development and doesn't need to be set in production:

```bash .dev.vars
UPSTASH_WORKFLOW_URL=https://<YOUR_NGROK_OR_TUNNEL_URL>/
```

### Using the `baseUrl` option

As an alternative to setting an environment variable, you can also use the `baseUrl` option in the `serve` method. This option is only needed for local development and can be omitted in production:

```typescript src/index.ts
import { serve } from "@upstash/workflow/cloudflare"

interface Env {
  ENVIRONMENT: "development" | "production"
}

export default {
  async fetch(req: Request, env: Env, ctx: ExecutionContext) {
    const { fetch: handler } = serve(
      async (context) => { ... },
      {
        // publically accessible base url needed for local development
        baseUrl: env.ENVIRONMENT === "development"
          ? "https://<YOUR_NGROK_OR_TUNNEL_URL>/"
          : undefined,
      }
    );
    return await handler(req, env)
  },
};
```

### Triggering the workflow

After setting your live URL as the environment variable or `baseUrl` option, trigger your workflow by sending a POST request. For each workflow run, a unique workflow run ID is returned:

```bash Terminal
curl -X POST https://<YOUR_NGROK_OR_TUNNEL_URL>/ -D '{"text": "hello world!"}'

# result: {"workflowRunId":"wfr_xxxxxx"}
```

See the [documentation on starting a workflow](/workflow/howto/start) for other ways you can start your workflow.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/workers_local_request.png" />
</Frame>

Use this ID to track the workflow run and see its status in your QStash workflow dashboard. All steps are listed with their statuses, headers, and body for a detailed overview of your workflow from start to finish. Click on a step to see its detailed logs.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/dashboard.png" />
</Frame>

## Step 5: Deploying to Production

When deploying your Cloudflare Worker with Upstash Workflow to production, there are a few key points to keep in mind:

1.  **Environment Variables**: Make sure that all necessary environment variables from your `.dev.vars` file are set in your Cloudflare Worker project settings. For example, your `QSTASH_TOKEN`, `ENVIRONMENT`, and any other configuration variables your workflow might need.

2.  **Remove Local Development Settings**: In your production code, you can remove or conditionally exclude any local development settings. For example, the `baseUrl` option in the `serve` function can be omitted in production:

```typescript src/index.ts
import { serve } from '@upstash/workflow/cloudflare'

interface Env {
  ENVIRONMENT: 'development' | 'production'
}

export default {
  async fetch(req: Request, env: Env, ctx: ExecutionContext) {
    const { fetch: handler } = serve(
      async (context) => { ... },
      {
        // baseUrl is not necessary in production
        baseUrl: env.ENVIRONMENT === 'development'
          ? 'https://<YOUR_NGROK_OR_TUNNEL_URL>/'
          : undefined,
      },
    );
    return await handler(req, env);
  },
}
```

3.  **Deployment**: Deploy your Cloudflare Worker to production as you normally would, for example using the Cloudflare CLI:

    ```bash Terminal
    wrangler deploy
    ```

4.  **Verify Workflow Endpoint**: After deployment, verify that your workflow endpoint is accessible by making a POST request to your production URL:

    ```bash Terminal
    curl -X POST https://<YOUR-PRODUCTION-URL>/ -D '{"text": "hello world!"}'
    ```

5.  **Monitor in QStash Dashboard**: Use the QStash dashboard to monitor your production workflows. You can track workflow runs, view step statuses, and access detailed logs.

6.  **Set Up Alerts**: Consider setting up alerts in Sentry or other monitoring tools to be notified of any workflow failures in production.

## Next Steps

1.  Learn how to protect your workflow endpoint from unauthorized access by [securing your workflow endpoint](/workflow/howto/security).

2.  Explore our [Cloudflare Workers example](https://github.com/upstash/workflow-js/tree/main/examples/cloudflare-workers) for a detailed, end-to-end example and best practices.

3.  For setting up and testing your workflows in a local environment, check out our [local development guide](/workflow/howto/local-development).


# Express.js



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/workflow-js/tree/main/examples/express" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use and deploy
Upstash Workflow with Express.js. You can also explore our [Express.js example](https://github.com/upstash/workflow-js/tree/main/examples/express)
for a detailed, end-to-end example and best practices.

## Prerequisites

1.  An Upstash QStash API key.
2.  Node.js and npm (another package manager) installed.

If you haven't obtained your QStash API key yet, you can do so by [signing up](https://console.upstash.com/login) for an Upstash account and navigating to your QStash dashboard.

## Step 1: Installation

First, install the Workflow SDK in your Express.js project:

<Tabs>
  <Tab title="npm">`bash npm install @upstash/workflow `</Tab>
  <Tab title="pnpm">`bash pnpm install @upstash/workflow `</Tab>
  <Tab title="bun">`bash bun add @upstash/workflow `</Tab>
</Tabs>

## Step 2: Configure Environment Variables

Create a `.env` file in your project root and add your QStash token. This key is used to authenticate your application with the QStash service.

```bash Terminal
touch .env
```

Open the environment file and add your QStash token, as well as the environment you're running in:

```bash .env
QSTASH_TOKEN="***"
QSTASH_URL="***"

# if you are working on local environment
UPSTASH_WORKFLOW_URL=""
```

You can learn more about `UPSTASH_WORKFLOW_URL` from [our documentation on local development](https://upstash.com/docs/workflow/howto/local-development).

Replace `***` with your actual QStash token and url found here:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_token.png" />
</Frame>

## Step 3: Create a Workflow Endpoint

A workflow endpoint allows you to define a set of steps that, together, make up a workflow. Each step contains a piece of business logic that is automatically retried on failure, with easy monitoring via our visual workflow dashboard.

To define a workflow endpoint with Express.js, navigate into your entrypoint file (usually `src/index.ts`) and add the following code:

<Tabs>
  <Tab title="Minimal example">
    ```typescript src/index.ts
    import { serve } from "@upstash/workflow/express";
    import express from 'express';
    import { config } from 'dotenv';

    config();

    const app = express();

    app.use(
      express.json()
    );

    app.post(
      '/workflow',
      serve<{ message: string }>(
        async (context) => {
          const res1 = await context.run("step1", async () => {
            const message = context.requestPayload.message;
            return message;
          })

          await context.run("step2", async () => {
              console.log(res1);
          })
        }
      )
    );

    app.listen(3000, () => {
      console.log('Server running on port 3000');
    });

    ```
  </Tab>
</Tabs>

## Step 4: Run the Workflow Endpoint

To start your Express.js app locally, run the following command:

```bash Terminal
npm run dev
```

Executing this command prints a local URL to your workflow endpoint. By default, this URL is `http://localhost:3000`.

Upstash Workflow is powered by [QStash](/qstash/overall/getstarted), and QStash needs a publicly accessible URL to run your workflows. You can easily forward your localhost URL to a live URL by following our guide about [setting up your workflow endpoint for local development](/workflow/howto/local-development).

Once you have a live URL, proceeed with either one of the following steps:

### Setting an enviroment variable (recommended)

Set your publically accessible URL (without extensions such as `/api/workflow`) as an environment variable. This variable is only needed for local development and doesn't need to be set in production:

```bash .env
UPSTASH_WORKFLOW_URL="https://<YOUR_NGROK_OR_TUNNEL_URL>/"
```

### Using the `baseUrl` option

As an alternative to setting an environment variable, you can also use the `baseUrl` option in the `serve` method. This option is only needed for local development and can be omitted in production:

```typescript src/index.ts
app.post(
  '/workflow',
  serve<{ message: string }>(
    async (context) => { ... }, {
      // just the base URL, no `/workflow`
      baseUrl: 'https://<YOUR_NGROK_OR_TUNNEL_URL>/'
    }
  )
);

```

### Triggering the workflow

<Warning>
  Express.js integration only works with `Content-Type: application/json`
  header. Other payload types are not supported yet.
</Warning>

After setting your live URL as the environment variable or `baseUrl` option, trigger your workflow by sending a POST request. For each workflow run, a unique workflow run ID is returned:

```bash Terminal
curl -X POST http://localhost:3000/workflow \
    -H "Content-Type: application/json" \
    -d '{"message": "Hello from the workflow!"}'

# result: {"workflowRunId":"wfr_xxxxxx"}
```

See the [documentation on starting a workflow](/workflow/howto/start) for other ways you can start your workflow.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/hono_local_request.png" />
</Frame>

Use this ID to track the workflow run and see its status in your QStash workflow dashboard. All steps are listed with their statuses, headers, and body for a detailed overview of your workflow from start to finish. Click on a step to see its detailed logs.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/dashboard.png" />
</Frame>

## Step 5: Deploying to Production

When deploying your Hono app with Upstash Workflow to production, there are a few key points to keep in mind:

1.  **Environment Variables**: Make sure that all necessary environment variables from your `.env` file are set in your Vercel project settings. For example, your `QSTASH_TOKEN`, and any other configuration variables your workflow might need.

2.  **Remove Local Development Settings**: In your production code, you can remove or conditionally exclude any local development settings. For example, the `baseUrl` option in the `serve` function can be omitted in production.

```typescript src/index.ts
import { serve } from "@upstash/workflow/express";
import express from 'express';

const app = express();

app.use(
    express.json()
);

app.use('/workflow',
	serve<{ message: string }>(async (context) => { ... }, {
		baseUrl: 'https://<YOUR_NGROK_OR_TUNNEL_URL>/'
	})
);

app.listen(3000, () => {
    console.log('Server running on port 3000');
});

```

3.  **Deployment**: Deploy your Express.js app to production as you normally would, for example to fly.io, Heroku, or AWS.

4.  **Verify Workflow Endpoint**: After deployment, verify that your workflow endpoint is accessible by making a POST request to your production URL:

```bash Terminal
curl -X POST <DEPLOYMENT_URL>/workflow \
	-H "Content-Type: application/json" \
	-d '{"message": "Hello from the workflow!"}'
```

5.  **Monitor in QStash Dashboard**: Use the QStash dashboard to monitor your production workflows. You can track workflow runs, view step statuses, and access detailed logs.

6.  **Set Up Alerts**: Consider setting up alerts in Sentry or other monitoring tools to be notified of any workflow failures in production.

## Next Steps

1.  Learn how to protect your workflow endpoint from unauthorized access by [securing your workflow endpoint](/workflow/howto/security).

2.  Explore our [Express example](https://github.com/upstash/workflow-js/tree/main/examples/express) for a detailed, end-to-end example and best practices.

3.  For setting up and testing your workflows in a local environment, check out our [local development guide](/workflow/howto/local-development).


# Hono



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/workflow-js/tree/main/examples/hono" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use and deploy Upstash Workflow with Hono. You can also explore our [Hono example](https://github.com/upstash/workflow-js/tree/main/examples/hono) for a detailed, end-to-end example and best practices.

## Prerequisites

1.  An Upstash QStash API key.
2.  Node.js and npm (another package manager) installed.

If you haven't obtained your QStash API key yet, you can do so by [signing up](https://console.upstash.com/login) for an Upstash account and navigating to your QStash dashboard.

## Step 1: Installation

First, install the Workflow SDK in your Hono project:

<Tabs>
  <Tab title="npm">
    ```bash
    npm install @upstash/workflow
    ```
  </Tab>

  <Tab title="pnpm">
    ```bash
    pnpm install @upstash/workflow
    ```
  </Tab>

  <Tab title="bun">
    ```bash
    bun add @upstash/workflow
    ```
  </Tab>
</Tabs>

## Step 2: Configure Environment Variables

Create a `.dev.vars` file in your project root and add your QStash token. This key is used to authenticate your application with the QStash service.

```bash Terminal
touch .dev.vars
```

Open the environment file and add your QStash token, as well as the environment you're running in:

```bash .dev.vars
QSTASH_TOKEN=xxxxxxxxx
ENVIRONMENT=development
```

Replace `xxxxxxxxx` with your actual QStash token found here:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_token.png" />
</Frame>

## Step 3: Create a Workflow Endpoint

A workflow endpoint allows you to define a set of steps that, together, make up a workflow. Each step contains a piece of business logic that is automatically retried on failure, with easy monitoring via our visual workflow dashboard.

To define a workflow endpoint with Hono, navigate into your entrypoint file (usually `src/index.ts`) and add the following code:

<Tabs>
  <Tab title="Minimal example">
    ```typescript src/index.ts
    import { Hono } from "hono"
    import { serve } from "@upstash/workflow/hono"

    const app = new Hono()

    app.post("/workflow",
      serve(async (context) => {
        await context.run("initial-step", () => {
          console.log("initial step ran")
        })

        await context.run("second-step", () => {
          console.log("second step ran")
        })
      })
    )

    export default app
    ```
  </Tab>

  <Tab title="With Hono context">
    ```typescript src/index.ts
    import { Hono } from "hono"
    import { serve, WorkflowBindings } from "@upstash/workflow/hono"
    import { env } from "hono/adapter"

    interface Bindings extends WorkflowBindings {
      ENVIRONMENT: "development" | "production"
    }

    const app = new Hono<{ Bindings: Bindings }>()

    app.post("/workflow", (c) => {
      // 👇 access Honos native context, i.e. getting an env variable
      const { ENVIRONMENT } = env(c)

      // 👇 `unknown` represents your initial payload data type
      const handler = serve<unknown, Bindings>(
        async (context) => { ... }
      )

      return await handler(c)
    })

    export default app
    ```
  </Tab>
</Tabs>

## Step 4: Run the Workflow Endpoint

To start your Hono app locally, run the following command:

```bash Terminal
npm run dev
```

Executing this command prints a local URL to your workflow endpoint. By default, this URL is `http://localhost:8787`.

You can verify your correct environment variable setup by checking the wrangler output, which should now have access to your `QSTASH_TOKEN` binding and log your local URL:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/wrangler.png" />
</Frame>

Upstash Workflow is powered by [QStash](/qstash/overall/getstarted), and QStash needs a publicly accessible URL to run your workflows. You can easily forward your localhost URL to a live URL by following our guide about [setting up your workflow endpoint for local development](/workflow/howto/local-development).

Once you have a live URL, proceeed with either one of the following steps:

### Setting an enviroment variable (recommended)

Set your publically accessible URL (without extensions such as `/workflow`) as an environment variable. This variable is only needed for local development and doesn't need to be set in production:

```bash .dev.vars
UPSTASH_WORKFLOW_URL=https://<YOUR_NGROK_OR_TUNNEL_URL>/
```

### Using the `baseUrl` option

As an alternative to setting an environment variable, you can also use the `baseUrl` option in the `serve` method. This option is only needed for local development and can be omitted in production:

```typescript src/index.ts
import { Hono } from "hono"
import { serve } from "@upstash/workflow/hono"

const app = new Hono()

app.post("/workflow",
  serve(async (context) => { ... }, {
    // just the base URL, no `/workflow`
    baseUrl: 'https://<YOUR_NGROK_OR_TUNNEL_URL>/'
  })
)


export default app
```

### Triggering the workflow

After setting your live URL as the environment variable or `baseUrl` option, trigger your workflow by sending a POST request. For each workflow run, a unique workflow run ID is returned:

```bash Terminal
curl -X POST https://<YOUR_NGROK_OR_TUNNEL_URL>/workflow'

# result: {"workflowRunId":"wfr_xxxxxx"}
```

See the [documentation on starting a workflow](/workflow/howto/start) for other ways you can start your workflow.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/hono_local_request.png" />
</Frame>

Use this ID to track the workflow run and see its status in your QStash workflow dashboard. All steps are listed with their statuses, headers, and body for a detailed overview of your workflow from start to finish. Click on a step to see its detailed logs.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/dashboard.png" />
</Frame>

## Step 5: Deploying to Production

When deploying your Hono app with Upstash Workflow to production, there are a few key points to keep in mind:

1.  **Environment Variables**: Make sure that all necessary environment variables from your `.dev.vars` file are set in your Cloudflare Worker project settings. For example, your `QSTASH_TOKEN`, and any other configuration variables your workflow might need.

2.  **Remove Local Development Settings**: In your production code, you can remove or conditionally exclude any local development settings. For example, the `baseUrl` option in the `serve` function can be omitted in production.

```typescript src/index.ts
import { Hono } from "hono"
import { serve, WorkflowBindings } from "@upstash/workflow/hono"
import { env } from "hono/adapter"

interface Bindings extends WorkflowBindings {
  ENVIRONMENT: "development" | "production"
}

const app = new Hono<{ Bindings: Bindings }>()

app.post("/workflow", (c) => {
  const { ENVIRONMENT } = env(c)

  const handler = serve<unknown, Bindings>(
    async (context) => { ... },
    {
      // baseUrl is not necessary in production
      baseUrl: ENVIRONMENT === "development" 
        ? "https://<YOUR_NGROK_OR_TUNNEL_URL>/" 
        : undefined,
    }
  )

  return await handler(c)
})

export default app

```

3.  **Deployment**: Deploy your Hono app to production as you normally would, for example using the Cloudflare CLI:

    ```bash Terminal
    wrangler deploy
    ```

4.  **Verify Workflow Endpoint**: After deployment, verify that your workflow endpoint is accessible by making a POST request to your production URL:

    ```bash Terminal
    curl -X POST https://<YOUR-PRODUCTION-URL>/workflow
    ```

5.  **Monitor in QStash Dashboard**: Use the QStash dashboard to monitor your production workflows. You can track workflow runs, view step statuses, and access detailed logs.

6.  **Set Up Alerts**: Consider setting up alerts in Sentry or other monitoring tools to be notified of any workflow failures in production.

## Next Steps

1.  Learn how to protect your workflow endpoint from unauthorized access by [securing your workflow endpoint](/workflow/howto/security).

2.  Explore our [Hono example](https://github.com/upstash/workflow-js/tree/main/examples/hono) for a detailed, end-to-end example and best practices.

3.  For setting up and testing your workflows in a local environment, check out our [local development guide](/workflow/howto/local-development).


# Nuxt



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/workflow-js/tree/main/examples/nuxt" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use and deploy Upstash Workflow with Nuxt.js. You can also explore our [Nuxt example](https://github.com/upstash/workflow-js/tree/main/examples/nuxt) for a detailed, end-to-end example and best practices.

## Prerequisites

1.  An Upstash QStash API key.
2.  Node.js and npm (another package manager) installed.

If you haven't obtained your QStash API key yet, you can do so by [signing up](https://console.upstash.com/login) for an Upstash account and navigating to your QStash dashboard.

## Step 1: Installation

First, install the Workflow SDK in your Nuxt project:

<Tabs>
  <Tab title="npm">
    ```bash
    npm install @upstash/workflow
    ```
  </Tab>

  <Tab title="pnpm">
    ```bash
    pnpm install @upstash/workflow
    ```
  </Tab>

  <Tab title="bun">
    ```bash
    bun add @upstash/workflow
    ```
  </Tab>
</Tabs>

## Step 2: Configure Environment Variables

Create a `.env.local` file in your project root and add your QStash token. This token is used to authenticate your application with the QStash service.

```bash Terminal
touch .env.local
```

Open the environment file and add your QStash token:

```bash .env.local
QSTASH_TOKEN=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual QStash token found here:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_token.png" />
</Frame>

## Step 3: Create a Workflow Endpoint

A workflow endpoint allows you to define a set of steps that, together, make up a workflow. Each step contains a piece of business logic that is automatically retried on failure, with easy monitoring via our visual workflow dashboard.

To define a workflow endpoint in a Nuxt.js project, navigate into the Nuxt.js `server/api` directory and create a new file, for example called `workflow.ts`. This file will contain your workflow:

```typescript server/api/workflow.ts
import { serve } from "@upstash/workflow/h3"

const { handler } = serve(
  async (context) => {
    await context.run("initial-step", () => {
      console.log("initial step ran")
    })

    await context.run("second-step", () => {
      console.log("second step ran")
    })
  },
)

export default handler;
```

## Step 4: Run the Workflow Endpoint

Upstash Workflow is powered by [QStash](/qstash/overall/getstarted), and QStash needs a publicly accessible URL to run your workflows. Here's how to [set up your workflow endpoint for local development](/workflow/howto/local-development).

Once you have a live URL, proceeed with either one of the following steps:

### Setting an enviroment variable (recommended)

Set your publically accessible URL as an environment variable. This variable is only needed for local development and doesn't need to be set in production:

```bash .env.local
UPSTASH_WORKFLOW_URL=https://<YOUR_NGROK_OR_TUNNEL_URL>/
```

### Using the `baseUrl` option

As an alternative to setting an environment variable, you can also use the `baseUrl` option in the `serve` method. This option is only needed for local development and can be omitted in production:

```typescript server/api/workflow.ts
import { serve } from "@upstash/workflow/h3"

const { handler } = serve(
  async (context) => { ... },
  {
    // no /api/workflow, just the base URL
    baseUrl: "https://<YOUR_NGROK_OR_TUNNEL_URL>/",
  }
)

export default handler;
```

### Triggering the workflow

After setting your live URL as the environment variable or `baseUrl` option, trigger your workflow by first starting your Nuxt app:

```bash Terminal
npm run dev
```

and then making a POST request to your workflow endpoint. For each workflow run, a unique workflow run ID is returned:

```bash Terminal
curl -X POST https://<YOUR_NGROK_OR_TUNNEL_URL>/api/workflow

# result: {"workflowRunId":"wfr_xxxxxx"}
```

See the [documentation on starting a workflow](/workflow/howto/start) for other ways you can start your workflow.

<Frame>
  {/* also works for nuxt */}

  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/nextjs_local_request.png" />
</Frame>

Use this ID to track the workflow run and see its status in your QStash workflow dashboard. All steps are listed with their statuses, headers, and body for a detailed overview of your workflow from start to finish. Click on a step to see its detailed logs.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/dashboard.png" />
</Frame>

## Step 5: Deploying to Production

When deploying your Nuxt.js application with Upstash Workflow to production, there are a few key points to keep in mind:

1.  **Environment Variables**: Make sure that all necessary environment variables are set in your platforms project settings. For example, your `QSTASH_TOKEN` and any other configuration variables your workflow might need.

2.  **Remove Local Development Settings**: In your production code, you can remove or conditionally exclude any local development settings. For example, the `baseUrl` option in the `serve` function can be omitted in production:

    ```typescript server/api/workflow.ts
    const { handler } = serve(
      async (context) => {
        // Your workflow steps
      },
      {
        // Conditionally set in development, but not in production
        // as the baseUrl is automatically set in production
        baseUrl:
          process.env.NODE_ENV === "development"
            ? "https://<YOUR_NGROK_OR_TUNNEL_URL>/"
            : undefined,
      }
    )
    export default handler;
    ```

3.  **Deployment**: Deploy your Nuxt application to Vercel, Cloudflare Pages or other platforms as you normally would. These platforms will automatically detect and build your Nuxt application.

4.  **Verify Workflow Endpoint**: After deployment, verify that your workflow endpoint is accessible by making a POST request to your production URL:

    ```bash Terminal
    curl -X POST https://<YOUR-PRODUCTION-URL>/api/workflow
    ```

5.  **Monitor in QStash Dashboard**: Use the QStash dashboard to monitor your production workflows. You can track workflow runs, view step statuses, and access detailed logs.

6.  **Set Up Alerts**: Consider setting up alerts in Sentry or other monitoring tools to be notified of any workflow failures in production.

## Next Steps

1.  Learn how to protect your workflow endpoint from unauthorized access by [securing your workflow endpoint](/workflow/howto/security).

2.  Explore our [Nuxt example](https://github.com/upstash/workflow-js/tree/main/examples/nuxt) for a detailed, end-to-end example and best practices.

3.  For setting up and testing your workflows in a local environment, check out our [local development guide](/workflow/howto/local-development).


# Supported Platforms



Upstash Workflow natively supports the following platforms:

*   [Next.js](/workflow/quickstarts/vercel-nextjs)
*   [Cloudflare Workers](/workflow/quickstarts/cloudflare-workers)
*   [Nuxt (H3)](/workflow/quickstarts/nuxt)
*   [Solidjs](/workflow/quickstarts/solidjs)
*   [Svelte](/workflow/quickstarts/svelte)
*   [Hono](/workflow/quickstarts/hono)

## Using Workflow with another platform

If you'd like to use Upstash Workflow for a platform not listed above, you can do so using the base `serve` method:

```javascript
import { serve } from '@upstash/workflow'
```

and adjust this method for your platform. For details, see [our platform-specific method implementations](https://github.com/upstash/workflow-js/tree/main/platforms).


# SolidJS



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/workflow-js/tree/main/examples/solidjs" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use and deploy Upstash Workflows with SolidJS. You can also explore our [SolidJS example](https://github.com/upstash/workflow-js/tree/main/examples/solidjs) for a detailed, end-to-end example and best practices.

## Prerequisites

1.  An Upstash QStash API key.
2.  Node.js and npm (another package manager) installed.

If you haven't obtained your QStash API key yet, you can do so by [signing up](https://console.upstash.com/login) for an Upstash account and navigating to your QStash dashboard.

## Step 1: Installation

First, install the Workflow SDK in your SolidJS project:

<Tabs>
  <Tab title="npm">
    ```bash
    npm install @upstash/workflow
    ```
  </Tab>

  <Tab title="pnpm">
    ```bash
    pnpm install @upstash/workflow
    ```
  </Tab>

  <Tab title="bun">
    ```bash
    bun add @upstash/workflow
    ```
  </Tab>
</Tabs>

## Step 2: Configure Environment Variables

Create a `.env` file in your project root and add your QStash token. This key is used to authenticate your application with the QStash service.

```bash Terminal
touch .env
```

Open the environment file and add your QStash token:

```bash .env
QSTASH_TOKEN=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual QStash token found here:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_token.png" />
</Frame>

## Step 3: Create a Workflow Endpoint

A workflow endpoint allows you to define a set of steps that, together, make up a workflow. Each step contains a piece of business logic that is automatically retried on failure, with easy monitoring via our visual workflow dashboard.

To define a workflow endpoint in a SolidJS project, navigate into the SolidJS `routes/api` directory. Inside this folder, create a `workflow.ts` file that contains your workflow:

```typescript routes/api/workflow.ts
import { serve } from "@upstash/workflow/solidjs"

export const { POST } = serve(async (context) => {
  await context.run("initial-step", () => {
    console.log("initial step ran")
  })

  await context.run("second-step", () => {
    console.log("second step ran")
  })
})
```

## Step 4: Run the Workflow Endpoint

Upstash Workflow is powered by [QStash](/qstash/overall/getstarted), and QStash needs a publicly accessible URL to run your workflows. Here's how to [set up your workflow endpoint for local development](/workflow/howto/local-development).

Once you have a live URL, proceeed with either one of the following steps:

### Setting an enviroment variable (recommended)

Set your publically accessible URL as an environment variable. This variable is only needed for local development and doesn't need to be set in production:

```bash .env
UPSTASH_WORKFLOW_URL=https://<YOUR_NGROK_OR_TUNNEL_URL>/
```

### Using the `baseUrl` option

As an alternative to setting an environment variable, you can also use the `baseUrl` option in the `serve` method. This option is only needed for local development and can be omitted in production:

```typescript routes/api/workflow.ts
import { serve } from "@upstash/workflow/solidjs"

export const { POST } = serve(
  async (context) => { ... },
  {
    // no /api/workflow, just the base URL
    baseUrl: "https://<YOUR_NGROK_OR_TUNNEL_URL>/",
  }
)
```

### Triggering the workflow

After setting your live URL as the environment variable or `baseUrl` option, trigger your workflow by first starting your SolidJS app:

```bash Terminal
npm run dev
```

and then making a POST request to your workflow endpoint. For each workflow run, a unique workflow run ID is returned:

```bash Terminal
curl -X POST https://<YOUR_NGROK_OR_TUNNEL_URL>/api/workflow

# result: {"workflowRunId":"wfr_xxxxxx"}
```

See the [documentation on starting a workflow](/workflow/howto/start) for other ways you can start your workflow.

<Frame>
  {/* also works for solidjs */}

  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/nextjs_local_request.png" />
</Frame>

Use this ID to track the workflow run and see its status in your QStash workflow dashboard. All steps are listed with their statuses, headers, and body for a detailed overview of your workflow from start to finish. Click on a step to see its detailed logs.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/dashboard.png" />
</Frame>

## Step 5: Deploying to Production

When deploying your SolidJS application with Upstash Workflows to production, there are a few key points to keep in mind:

1.  **Environment Variables**: Make sure that all necessary environment variables are set in your platforms project settings. For example, your `QSTASH_TOKEN` and any other configuration variables your workflow might need.

2.  **Remove Local Development Settings**: In your production code, you can remove or conditionally exclude any local development settings. For example, the `baseUrl` option in the `serve` function can be omitted in production:

    ```typescript routes/api/workflow.ts
    export const { POST } = serve(
      async (context) => {
        // Your workflow steps
      },
      {
        // Conditionally set in development, but not in production
        // as the baseUrl is automatically set in production
        baseUrl: process.env.NODE_ENV === "development" ? "https://<YOUR_NGROK_OR_TUNNEL_URL>/" : undefined,
      }
    )
    ```

3.  **Deployment**: Deploy your SolidJS application to Netlify, Vercel or other platforms as you normally would. These platforms will automatically detect and build your SolidJS application.

4.  **Verify Workflow Endpoint**: After deployment, verify that your workflow endpoint is accessible by making a POST request to your production URL:

    ```bash Terminal
    curl -X POST https://<YOUR-PRODUCTION-URL>/api/workflow
    ```

5.  **Monitor in QStash Dashboard**: Use the QStash dashboard to monitor your production workflows. You can track workflow runs, view step statuses, and access detailed logs.

6.  **Set Up Alerts**: Consider setting up alerts in Sentry or other monitoring tools to be notified of any workflow failures in production.

## Next Steps

1.  Learn how to protect your workflow endpoint from unauthorized access by [securing your workflow endpoint](/workflow/howto/security).

2.  Explore our [SolidJS example](https://github.com/upstash/workflow-js/tree/main/examples/solidjs) for a detailed, end-to-end example and best practices.

3.  For setting up and testing your workflows in a local environment, check out our [local development guide](/workflow/howto/local-development).


# SvelteKit



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/workflow-js/tree/main/examples/sveltekit" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use and deploy Upstash Workflow with SvelteKit. You can also explore our [SvelteKit example](https://github.com/upstash/workflow-js/tree/main/examples/sveltekit) for a detailed, end-to-end example and best practices.

## Prerequisites

1.  An Upstash QStash API key.
2.  Node.js and npm (another package manager) installed.

If you haven't obtained your QStash API key yet, you can do so by [signing up](https://console.upstash.com/login) for an Upstash account and navigating to your QStash dashboard.

## Step 1: Installation

First, install the Workflow SDK in your SvelteKit project:

<Tabs>
  <Tab title="npm">
    ```bash
    npm install @upstash/workflow
    ```
  </Tab>

  <Tab title="pnpm">
    ```bash
    pnpm install @upstash/workflow
    ```
  </Tab>

  <Tab title="bun">
    ```bash
    bun add @upstash/workflow
    ```
  </Tab>
</Tabs>

## Step 2: Configure Environment Variables

Create a `.env.local` file in your project root and add your QStash token. This token is used to authenticate your application with the QStash service.

```bash Terminal
touch .env.local
```

Open the environment file and add your QStash token:

```bash .env.local
QSTASH_TOKEN=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual QStash token found here:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_token.png" />
</Frame>

## Step 3: Create a Workflow Endpoint

A workflow endpoint allows you to define a set of steps that, together, make up a workflow. Each step contains a piece of business logic that is automatically retried on failure, with easy monitoring via our visual workflow dashboard.

To define a workflow endpoint, navigate into SvelteKit's `routes/api` directory and create a new folder, for example called `workflow`. Inside this folder, create a `+server.ts` file that contains your workflow:

```typescript routes/api/workflow/+server.ts
import { serve } from "@upstash/workflow/svelte"
import { env } from "$env/dynamic/private"

export const { POST } = serve(
  async (context) => {
    await context.run("initial-step", () => {
      console.log("initial step ran")
    })

    await context.run("second-step", () => {
      console.log("second step ran")
    })
  },
  { env }
)
```

## Step 4: Run the Workflow Endpoint

Upstash Workflow is powered by [QStash](/qstash/overall/getstarted), and QStash needs a publicly accessible URL to run your workflows. Here's how to [set up your workflow endpoint for local development](/workflow/howto/local-development).

Once you have a live URL, proceeed with either one of the following steps:

### Setting an enviroment variable (recommended)

Set your publically accessible base URL (without extensions such as `/api/workflow`) as an environment variable. This variable is only needed for local development and doesn't need to be set in production:

```bash .env.local
UPSTASH_WORKFLOW_URL=https://<YOUR_NGROK_OR_TUNNEL_URL>/
```

### Using the `baseUrl` option

As an alternative to setting an environment variable, you can also use the `baseUrl` option in the `serve` method. This option is only needed for local development and can be omitted in production:

```typescript routes/api/workflow/+server.ts
import { serve } from "@upstash/workflow/svelte";
import { env } from '$env/dynamic/private'

export const { POST } = serve(
  async (context) => { ... }, {
    env,
    // just the base URL, no `/api/workflow`
    baseUrl: "https://<YOUR_NGROK_OR_TUNNEL_URL>/"
  }
)
```

### Triggering the workflow

After setting your live URL as the environment variable or `baseUrl` option, trigger your workflow by first starting your app:

```bash Terminal
npm run dev
```

and then making a POST request to your workflow endpoint. For each workflow run, a unique workflow run ID is returned:

```bash Terminal
curl -X POST https://<YOUR_NGROK_OR_TUNNEL_URL>/api/workflow

# result: {"workflowRunId":"wfr_xxxxxx"}
```

See the [documentation on starting a workflow](/workflow/howto/start) for other ways you can start your workflow.

<Frame>
  {/* also works for sveltekit */}

  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/nextjs_local_request.png" />
</Frame>

Use this ID to track the workflow run and see its status in your QStash workflow dashboard. All steps are listed with their statuses, headers, and body for a detailed overview of your workflow from start to finish. Click on a step to see its detailed logs.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/dashboard.png" />
</Frame>

## Step 5: Deploying to Production

When deploying your SvelteKit application with Upstash Workflow to production, there are a few key points to keep in mind:

1.  **Environment Variables**: Make sure that all necessary environment variables are set in your platforms project settings. For example, your `QSTASH_TOKEN` and any other configuration variables your workflow might need.

2.  **Remove Local Development Settings**: In your production code, you can remove or conditionally exclude any local development settings. For example, the `baseUrl` option in the `serve` function can be omitted in production:

    ```typescript routes/api/workflow/+server.ts
    export const { POST } = serve(
      async (context) => {
        // Your workflow steps
      },
      {
        // Conditionally set in development, but not in production
        // as the baseUrl is automatically set in production
        baseUrl:
          process.env.NODE_ENV === "development"
            ? "https://<YOUR_NGROK_OR_TUNNEL_URL>/"
            : undefined,
      }
    )
    ```

3.  **Deployment**: Deploy your app to Vercel, Netlify or other platforms as you normally would. These platforms will automatically detect and build your SvelteKit application.

4.  **Verify Workflow Endpoint**: After deployment, verify that your workflow endpoint is accessible by making a POST request to your production URL:

    ```bash Terminal
    curl -X POST https://<YOUR-PRODUCTION-URL>/api/workflow
    ```

5.  **Monitor in QStash Dashboard**: Use the QStash dashboard to monitor your production workflows. You can track workflow runs, view step statuses, and access detailed logs.

6.  **Set Up Alerts**: Consider setting up alerts in Sentry or other monitoring tools to be notified of any workflow failures in production.

## Next Steps

1.  Learn how to protect your workflow endpoint from unauthorized access by [securing your workflow endpoint](/workflow/howto/security).

2.  Explore our [SvelteKit example](https://github.com/upstash/workflow-js/tree/main/examples/sveltekit) for a detailed, end-to-end example and best practices.

3.  For setting up and testing your workflows in a local environment, check out our [local development guide](/workflow/howto/local-development).


# Next.js



<Card title="GitHub Repository" icon="github" href="https://github.com/upstash/workflow-js/tree/main/examples/nextjs" horizontal>
  You can find the project source code on GitHub.
</Card>

This guide provides detailed, step-by-step instructions on how to use and deploy Upstash Workflow with Next.js. You can also explore our [Next.js example](https://github.com/upstash/workflow-js/tree/main/examples/nextjs) for a detailed, end-to-end example and best practices.

<iframe width="560" height="315" src="https://www.youtube.com/embed/yOcz5fA3Tsc?rel=0&disablekb=1" title="YouTube video player" frameBorder="0" allow="accelerometer; fullscreen; clipboard-write; encrypted-media; gyroscope" allowFullScreen />

## Prerequisites

1.  An Upstash QStash API key.
2.  Node.js and npm (another package manager) installed.

If you haven't obtained your QStash API key yet, you can do so by [signing up](https://console.upstash.com/login) for an Upstash account and navigating to your QStash dashboard.

## Step 1: Installation

<Note>
  In October 2024, we released a new SDK, `@upstash/workflow`, for Upstash Workflow. If you were using `@upstash/qstash` for Upstash Workflow, you can refer to [the migration guide](/workflow/migration) for the transition.
</Note>

First, install the Workflow SDK in your Next.js project:

<Tabs>
  <Tab title="npm">
    ```bash
    npm install @upstash/workflow
    ```
  </Tab>

  <Tab title="pnpm">
    ```bash
    pnpm install @upstash/workflow
    ```
  </Tab>

  <Tab title="bun">
    ```bash
    bun add @upstash/workflow
    ```
  </Tab>
</Tabs>

## Step 2: Configure Environment Variables

Create a `.env.local` file in your project root and add your QStash token. This token is used to authenticate your application with the QStash service.

```bash Terminal
touch .env.local
```

Open the environment file and add your QStash token:

```bash .env.local
QSTASH_TOKEN=xxxxxxxxx
```

Replace `xxxxxxxxx` with your actual QStash token found here:

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/qstash_token.png" />
</Frame>

## Step 3: Create a Workflow Endpoint

A workflow endpoint allows you to define a set of steps that, together, make up a workflow. Each step contains a piece of business logic that is automatically retried on failure, with easy monitoring via our visual workflow dashboard.

To define a workflow endpoint in a Next.js project, navigate into the Next.js `app/api` directory and create a new folder, for example called `workflow`. Inside this folder, create a `route.ts` file that contains your workflow:

<Tabs>
  <Tab title="Minimal example">
    ```typescript app/api/workflow/route.ts
    import { serve } from "@upstash/workflow/nextjs"

    export const { POST } = serve(
      async (context) => {
        await context.run("initial-step", () => {
          console.log("initial step ran")
        })

        await context.run("second-step", () => {
          console.log("second step ran")
        })
      }
    )
    ```
  </Tab>

  <Tab title="With request object">
    ```typescript app/api/workflow/route.ts
    import { serve } from "@upstash/workflow/nextjs";
    import { NextRequest } from "next/server";

    export const POST = async (request: NextRequest) => {
      // do something with the native request object
      const { POST: handler } = serve(async (context) => {
        // Your workflow steps
      });

      return await handler(request);
    }
    ```
  </Tab>

  <Tab title="Minimal example (Pages Router)">
    ```typescript src/pages/api/workflow.ts
    import { servePagesRouter } from "@upstash/workflow/nextjs";

    const { handler } = servePagesRouter<string>(
      async (context) => {
        await context.run("initial-step", () => {
          console.log("initial step ran")
        })

        await context.run("second-step", () => {
          console.log("second step ran")
        })
      }
    )
    export default handler;
    ```
  </Tab>

  <Tab title="With request object (Pages Router)">
    ```typescript src/pages/api/workflow.ts
    import type { NextApiRequest, NextApiResponse } from "next";
    import { servePagesRouter } from "@upstash/workflow/nextjs";

    export default async function handler(
      req: NextApiRequest,
      res: NextApiResponse
    ) {
      // do something with the native request object
      const { handler } = servePagesRouter(
        async (context) => {
          // Your workflow steps
        }
      )

      await handler(req, res)
    }
    ```
  </Tab>
</Tabs>

## Step 4: Run the Workflow Endpoint

Upstash Workflow is powered by [QStash](/qstash/overall/getstarted), and QStash needs a publicly accessible URL to run your workflows. Here's how to [set up your workflow endpoint for local development](/workflow/howto/local-development).

Once you have a live URL, proceeed with either one of the following steps:

### Setting an enviroment variable (recommended)

Set your publically accessible URL as an environment variable. This variable is only needed for local development and doesn't need to be set in production:

```bash .env.local
UPSTASH_WORKFLOW_URL=https://<YOUR_NGROK_OR_TUNNEL_URL>/
```

### Using the `baseUrl` option

As an alternative to setting an environment variable, you can also use the `baseUrl` option in the `serve` method. This option is only needed for local development and can be omitted in production:

<Tabs>
  <Tab title="App Router">
    ```typescript app/api/workflow/route.ts
    import { serve } from "@upstash/workflow/nextjs"

    export const { POST } = serve(
      async (context) => { ... },
      {
        // no /api/workflow, just the base URL
        baseUrl: "https://<YOUR_NGROK_OR_TUNNEL_URL>/",
      }
    )
    ```
  </Tab>

  <Tab title="Pages Router">
    ```typescript src/pages/api/workflow.ts
    import { servePagesRouter } from "@upstash/workflow/nextjs";

    const { handler } servePagesRouter<string>(
      async (context) => { ... },
      {
        // no /api/workflow, just the base URL
        baseUrl: "https://<YOUR_NGROK_OR_TUNNEL_URL>/",
      }
    )
    export default handler;
    ```
  </Tab>
</Tabs>

### Triggering the workflow

After setting your live URL as the environment variable or `baseUrl` option, trigger your workflow by first starting your Next.js app:

```bash Terminal
npm run dev
```

and then making a POST request to your workflow endpoint. For each workflow run, a unique workflow run ID is returned:

```bash Terminal
curl -X POST https://<YOUR_NGROK_OR_TUNNEL_URL>/api/workflow

# result: {"workflowRunId":"wfr_xxxxxx"}
```

See the [documentation on starting a workflow](/workflow/howto/start) for other ways you can start your workflow.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/nextjs_local_request.png" />
</Frame>

Use this ID to track the workflow run and see its status in your QStash workflow dashboard. All steps are listed with their statuses, headers, and body for a detailed overview of your workflow from start to finish. Click on a step to see its detailed logs.

<Frame>
  <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/qstash-workflow/dashboard.png" />
</Frame>

## Step 5: Deploying to Production

When deploying your Next.js application with Upstash Workflow to production, there are a few key points to keep in mind:

1.  **Environment Variables**: Make sure that all necessary environment variables are set in your Vercel project settings. For example, your `QSTASH_TOKEN` and any other configuration variables your workflow might need.

2.  **Remove Local Development Settings**: In your production code, you can remove or conditionally exclude any local development settings. For example, the `baseUrl` option in the `serve` function can be omitted in production:

    <Tabs>
      <Tab title="App Router">
        ```typescript app/api/workflow/route.ts
        import { serve } from "@upstash/workflow/nextjs";

        export const { POST } = serve(
          async (context) => {
            // Your workflow steps
          },
          {
            // Conditionally set in development, but not in production
            baseUrl:
              process.env.NODE_ENV === "development"
                ? "https://<YOUR_NGROK_OR_TUNNEL_URL>/"
                : undefined,
          }
        )
        ```
      </Tab>

      <Tab title="Pages Router">
        ```typescript src/pages/api/workflow.ts
        import { servePagesRouter } from "@upstash/workflow/nextjs";

        const { handler } = servePagesRouter<string>(
          async (context) => {
            // Your workflow steps
          },
          {
            // Conditionally set in development, but not in production
            baseUrl:
              process.env.NODE_ENV === "development"
                ? "https://<YOUR_NGROK_OR_TUNNEL_URL>/"
                : undefined,
          }
        )
        export default handler;
        ```
      </Tab>
    </Tabs>

3.  **Deployment**: Deploy your Next.js application to Vercel, AWS Amplify or other platforms as you normally would. These platforms will automatically detect and build your Next.js application.

4.  **Verify Workflow Endpoint**: After deployment, verify that your workflow endpoint is accessible by making a POST request to your production URL:

    ```bash Terminal
    curl -X POST https://<YOUR-PRODUCTION-URL>/api/workflow
    ```

5.  **Monitor in QStash Dashboard**: Use the QStash dashboard to monitor your production workflows. You can track workflow runs, view step statuses, and access detailed logs.

6.  **Set Up Alerts**: Consider setting up alerts in Sentry or other monitoring tools to be notified of any workflow failures in production.

## Next Steps

1.  Learn how to protect your workflow endpoint from unauthorized access by [securing your workflow endpoint](/workflow/howto/security).

2.  Explore our [Next.js example](https://github.com/upstash/workflow-js/tree/main/examples/nextjs) for a detailed, end-to-end example and best practices.

3.  For setting up and testing your workflows in a local environment, check out our [local development guide](/workflow/howto/local-development).


# Cancel Workflow

DELETE https://qstash.upstash.io/v2/workflows/runs/{workflowRunId}
Stop a an running workflow

Cancelling a workflow run will remove it from QStash and stop it from being delivered
in the future.

## Request

<ParamField path="workflowRunId" type="string" required>
  The id of the message to cancel.
</ParamField>

## Response

This endpoint returns

*   `200 OK` when successful.
*   `404 NOT FOUND` when a workflow run is not found with the given id.
*   `500 INTERNAL SERVER` when an unexpected error occurs.

<RequestExample>
  ```sh curl
  curl -XDELETE https://qstash.upstash.io/v2/workflows/runs/wfr_TzazoUCuZmFGp2u9cdy5K \
    -H "Authorization: Bearer <token>"
  ```

  ```js Workflow SDK
  import { Client } from "@upstash/workflow";

  const client = new Client({ token: "<QSTASH_TOKEN>" })
  await client.cancel({ workflowRunId: "<WORKFLOW_RUN_ID>" })
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/workflows/runs/wfr_TzazoUCuZmFGp2u9cdy5K', {
    method: 'DELETE',
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.delete(
    'https://qstash.upstash.io/v2/workflows/runs/wfr_TzazoUCuZmFGp2u9cdy5K', 
    headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("DELETE", "https://qstash.upstash.io/v2/workflows/runs/wfr_TzazoUCuZmFGp2u9cdy5K", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```text 200 OK
  OK
  ```
</ResponseExample>


# Notify Workflows

POST https://qstash.upstash.io/v2/notify/{eventId}
Resume workflows waiting for an event

Notify workflows waiting for an event to resume them. See [our documentation to learn more](/workflow/howto/events).

## Request

<ParamField path="eventId" type="string" required>
  Event id to notify
</ParamField>

<ParamField body="body" type="string">
  Event data passed to the notified workflows
</ParamField>

## Response

The response contains a list of notified waiter objects and id of the message sent as a result of the notify request.

```typescript
type NotifyResponse = { waiter: Waiter, messageId: string }[]
```

More information about the `Waiter` object:

<Snippet file="qstash/waiter.mdx" />

If there were no workflows waiting for the event, the result is an empty list.

<RequestExample>
  ```sh curl
  curl -X POST https://qstash.upstash.io/v2/notify/myEvent \
    -H "Authorization: Bearer <token>" \
    -d "Hello World!"
  ```

  ```js Workflow SDK
  import { Client } from "@upstash/workflow";

  const client = new Client({ token: "<QSTASH_TOKEN>" })
  await client.notify({
    eventId: "my-event-id",
    eventData: "Hello World!"
  });
  ```

  ```js Node
  const response = await fetch('https://qstash.upstash.io/v2/notify/myEvent', {
    method: 'POST',
    body: "Hello world!",
    headers: {
      'Authorization': 'Bearer <token>'
    }
  });
  ```

  ```python Python
  import requests

  headers = {
      'Authorization': 'Bearer <token>',
  }

  response = requests.post(
    'https://qstash.upstash.io/v2/notify/myEvent', 
    headers=headers
  )
  ```

  ```go Go
  req, err := http.NewRequest("POST", "https://qstash.upstash.io/v2/notify/myEvent", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("Authorization", "Bearer <token>")
  resp, err := http.DefaultClient.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  defer resp.Body.Close()
  ```
</RequestExample>

<ResponseExample>
  ```json 200 OK
  [
    {
      "waiter": {
        "url": "https://my-workflow.com/path",
        "headers": {
          "Upstash-Workflow-Runid": [
            "wfr_melCHYvPkVhDqIhhk2"
          ],
          "Upstash-Workflow-Url": [
            "https://my-workflow.com/path"
          ]
        },
        "deadline": 1729869206
      },
      "messageId": "msg_26hZCxxbG2TT3AnHEr1w"
    }
  ]
  ```
</ResponseExample>


# Roadmap



*   Enhance the Upstash Workflow SDK to expose and configure all QStash features:
    *   Ordered execution
    *   Rate limit
*   Expand language support with additional SDKs: Python and other languages based on user requests.
*   Local development environment for testing
*   Endpoint for checking the status & progress of a workflow run


# Vercel



# Common Issues and Solutions

### Preview Deployment Protection

**Problem:**
When Deployment Protection setting is enabled on Vercel, it's not possible to trigger and complete a workflow run on a preview deployment.

**Solution:**
Vercel provides a way to bypass this protection by using a bypass secret. To create one, follow these steps:

<Steps>
  <Step title="Settings">Go to **Deployment Protection** section under your Vercel project settings.</Step>

  <Step title="Find related section">
    Click on **Add Secret** under **Protection Bypass for Automation** section.

    <Frame>
      <img src="https://mintlify.s3-us-west-1.amazonaws.com/upstash/img/workflow/troubleshooting/vercel-deployment-protection.png" />
    </Frame>
  </Step>

  <Step title="Generate a bypass token">Don't forget to save it.</Step>
</Steps>

Now that you have a bypass token, you can add it to your workflow URL as a query parameter.

Imagine that you have a Vercel Preview and there is a workflow endpoint at `https://vercel-preview.com/workflow`.
You can call the workflow like this in preview:

```bash
curl -X POST \
  'https://vercel-preview.com/workflow?x-vercel-protection-bypass=<PROTECTION_BYPASS_SECRET>' \
  -d 'Hello world!'
```