You can incorporate any 3rd party OpenAI compatible LLM into RAG Chat.

import { RAGChat, custom } from "@upstash/rag-chat";

export const ragChat = new RAGChat({
    model: custom("meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo", {
      apiKey: "TOGETHER_AI_KEY",
      baseUrl: "https://api.together.xyz/v1",
    }),
});

Was this page helpful?

GroqOllama