Unstructured
Unstructured extracts complex data from difficult-to-use formats like HTML, PDF, CSV, and more. Check out Unstructured - Product for more information about their product and pricing.
Install RAG Chat SDK
Install Bun if you haven’t.
Initialize the project and install the required packages:
npm init es6
npm install dotenv
npm install @upstash/rag-chat
npm i --save-dev @types/bun
Setup Upstash Redis
Create a Redis database using Upstash Console or Upstash CLI and copy the UPSTASH_REDIS_REST_URL
and UPSTASH_REDIS_REST_TOKEN
into your .env
file.
UPSTASH_REDIS_REST_URL=<YOUR_URL>
UPSTASH_REDIS_REST_TOKEN=<YOUR_TOKEN>
Setup Upstash Vector
Create a Vector index using Upstash Console or Upstash CLI and copy the UPSTASH_VECTOR_REST_URL
and UPSTASH_VECTOR_REST_TOKEN
into your .env
file.
UPSTASH_VECTOR_REST_URL=<YOUR_URL>
UPSTASH_VECTOR_REST_TOKEN=<YOUR_TOKEN>
Setup QStash LLM
Navigate to QStash Console and copy the QSTASH_TOKEN
into your .env
file.
QSTASH_TOKEN=<YOUR_TOKEN>
Setup Unstructured
Create an Unstructured account and get an API key from Unstructed -> API Keys. Set your Unstructed API key as an environment variable:
UNSTRUCTURED_IO_KEY=<YOUR_API_KEY>
Setup the Project
Initialize RAGChat:
import { RAGChat, upstash } from "@upstash/rag-chat";
import "dotenv/config";
const ragChat = new RAGChat({
model: upstash("meta-llama/Meta-Llama-3-8B-Instruct"),
});
Fetch a webpage and save it to a file:
const fileSource = "./hackernews.html";
const response = await fetch("https://news.ycombinator.com/");
await Bun.write(fileSource, await response.text());
Add context to the RAG Chat with the Unstructured processor:
await ragChat.context.add({
options: {
namespace: "unstructured-upstash",
},
fileSource,
processor: {
name: "unstructured",
options: { apiKey: process.env.UNSTRUCTURED_IO_KEY },
},
});
Chat with the RAG Chat:
const result = await ragChat.chat("What is the second story on hacker news?", {
streaming: false,
namespace: "unstructured-upstash",
});
Run
Run the project:
bun run index.ts
Was this page helpful?