Getting Started
Upstash RAG Chat is TypeScript toolkit for building powerful RAG applications. With Upstash RAG Chat, you can focus on building a high-quality chatbot without having to deal with complicated AI orchestration tools.
You can find the Github Repository here.
Installation
Install the package using your preferred package manager:
pnpm add @upstash/rag-chat
bun add @upstash/rag-chat
npm i @upstash/rag-chat
Quick start
- Set up your environment variables:
UPSTASH_VECTOR_REST_URL="XXXXX"
UPSTASH_VECTOR_REST_TOKEN="XXXXX"
# if you use OpenAI compatible models
OPENAI_API_KEY="XXXXX"
# or if you use Upstash hosted models
QSTASH_TOKEN="XXXXX"
# Optional: For Redis-based chat history (default is in-memory)
UPSTASH_REDIS_REST_URL="XXXXX"
UPSTASH_REDIS_REST_TOKEN="XXXXX"
- Initialize and use RAGChat:
import { RAGChat } from "@upstash/rag-chat";
const ragChat = new RAGChat();
const response = await ragChat.chat("Tell me about machine learning");
console.log(response);
Configuring Your Chat Model
RAGChat supports both Upstash-hosted models and all OpenAI and OpenAI-compatible models out of the box:
Using OpenAI Models
To use an OpenAI model, first initialize RAGChat:
import { RAGChat, openai } from "@upstash/rag-chat";
export const ragChat = new RAGChat({
model: openai("gpt-4-turbo"),
});
And set your OpenAI API key as an environment variable:
OPENAI_API_KEY=...
Using Upstash-hosted Open-Source Models
To use an Upstash model, first initialize RAGChat:
import { RAGChat, upstash } from "@upstash/rag-chat";
export const ragChat = new RAGChat({
model: upstash("mistralai/Mistral-7B-Instruct-v0.2"),
});
And set your Upstash QStash API key environment variable:
QSTASH_TOKEN=...
Using Custom Providers - TogetherAi, Replicate
Initialize RAGChat with custom provider’s API key and url:
import { RAGChat, custom } from "@upstash/rag-chat";
export const ragChat = new RAGChat({
model: custom("codellama/CodeLlama-70b-Instruct-hf", {
apiKey: "TOGETHER_AI_API_KEY",
baseUrl: "https://api.together.xyz/v1",
}),
});
Managing Your Knowledge Base
Add various types of data to your RAG application:
Adding Text
await ragChat.context.add({
type: "text",
data: "The speed of light is approximately 299,792,458 meters per second.",
});
//OR
await ragChat.context.add("The speed of light is approximately 299,792,458 meters per second.");
Adding PDF Content
await ragChat.context.add({
type: "pdf",
fileSource: "./data/quantum_computing_basics.pdf",
// optional 👇: only add this knowledge to a specific namespace
options: { namespace: "user-123-documents" },
});
Adding Web Content
await ragChat.context.add({
type: "html",
source: "https://en.wikipedia.org/wiki/Quantum_computing",
// optional 👇: custom page parsing settings
config: { chunkOverlap: 50, chunkSize: 200 },
});
Removing Content
Remove specific documents:
await ragChat.context.delete({ id: "1", namespace: "user-123-documents" });
Managing Chat History
RAGChat provides robust functionality for interacting with and managing chat history. This allows you to maintain context, review past interactions, and customize the conversation flow.
Retrieving Chat History
Fetch recent messages from the chat history:
const history = await ragChat.history.getMessages({ amount: 10 });
console.log(history); // 👈 Last (up to) 10 messages
You can also specify a session ID to retrieve history for a particular conversation:
const sessionHistory = await ragChat.history.getMessages({
amount: 5,
sessionId: "user-123-session",
});
Deleting Chat History
Remove chat history for a specific session:
ragChat.history.deleteMessages({ sessionId: "user-123-session" });
Adding Custom Messages
Injecting custom messages into the chat history:
// Adding a user message
await ragChat.history.addMessage({
message: { content: "What's the weather like?", role: "user" },
});
// Adding a system message
await ragChat.history.addMessage({
message: {
content: "The AI should provide weather information.",
role: "system",
},
});