Build a RAG System with Claude & ChatGPT APIs

A developer at Gate of AI built a retrieval-augmented generation (RAG) system using Claude and ChatGPT APIs. The system retrieves relevant documents from a JSON repository and generates contextually rich responses by combining both language models. The tutorial covers environment setup, API integration, and query handling for applications like customer support and research assistants.

https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1lk2k5abtzwax6vsghw1.jpeg 🚀 Technical Briefing: This tutorial is part of our deep-dive series on Agentic Workflows at Gate of AI https://gateofai.com . For the full technical breakdown, interactive code sandbox, and the native Arabic translation, visit the original article here https://gateofai.com/tutorial/build-rag-system-claude-chatgpt-apis/ . <span Tutorial</span <span Intermediate</span <span ⏱ 45 min read</span <span © Gate of AI 2026-06-24</span Learn to build a smart, retrieval-augmented generation system using Claude and ChatGPT APIs to leverage the best of both models for enhanced AI interactions. In this tutorial, we will build a Retrieval-Augmented Generation RAG system that combines the capabilities of Claude and ChatGPT APIs. This system will efficiently fetch relevant data from a document repository and generate contextually rich responses using advanced language models. The final application will allow users to input queries, retrieve pertinent information from a pre-indexed document set, and use AI models to generate comprehensive answers. This integrated approach offers robust performance in applications requiring high accuracy and relevance, such as customer support systems and research assistants. We will begin by setting up our development environment and installing necessary libraries. This involves setting up Node.js and installing the required SDKs for accessing the APIs. npm install openai anthropic dotenv Next, we'll configure our environment variables to securely store our API keys. Create a .env file in your project root and add the following variables: OPENAI API KEY=your openai api key ANTHROPIC API KEY=your anthropic api key First, we need to establish a document repository that our RAG system can query. We'll use a simple JSON file to simulate this repository. Ensure the data is structured for quick access and relevance scoring. js const fs = require 'fs' ; const documents = JSON.parse fs.readFileSync 'documents.json', 'utf8' ; function getRelevantDocuments query { // Simple keyword matching for relevance return documents.filter doc = doc.text.includes query ; } module.exports = { getRelevantDocuments }; This code reads a JSON file containing our documents and filters them based on the query. The getRelevantDocuments function will be used later to fetch relevant documents for any given query. Next, we'll set up the integration with Claude and ChatGPT APIs to process and generate responses. This involves configuring both APIs and establishing a connection to send and receive data. js require 'dotenv' .config ; const { OpenAI } = require 'openai' ; const { Anthropic } = require 'anthropic' ; const openAIClient = new OpenAI { apiKey: process.env.OPENAI API KEY } ; const anthropicClient = new Anthropic { apiKey: process.env.ANTHROPIC API KEY } ; async function generateResponseWithClaude prompt { const response = await anthropicClient.chat.completions.create { model: "claude-3-5-sonnet-20241022", messages: { role: "user", content: prompt } } ; return response.data.choices 0 .message.content; } async function generateResponseWithChatGPT prompt { const response = await openAIClient.chat.completions.create { model: "gpt-4o", messages: { role: "user", content: prompt } } ; return response.data.choices 0 .message.content; } module.exports = { generateResponseWithClaude, generateResponseWithChatGPT }; This code establishes connections to both APIs using the modern SDKs. It defines functions to send a prompt to each service and receive a generated response, which will be used to produce the final answer. We'll now build the core logic to handle user queries. This involves retrieving relevant documents and using our integrated AI functions to generate a cohesive response. js const { getRelevantDocuments } = require './documentRepository' ; const { generateResponseWithClaude, generateResponseWithChatGPT } = require './aiIntegrations' ; async function handleUserQuery query { const relevantDocs = getRelevantDocuments query ; const combinedContext = relevantDocs.map doc = doc.text .join '\n' ; const prompt = Based on these documents:\n${combinedContext}\nAnswer the following question: ${query}; const claudeResponse = await generateResponseWithClaude prompt ; const chatGPTResponse = await generateResponseWithChatGPT prompt ; return { claude: claudeResponse, chatGPT: chatGPTResponse }; } module.exports = { handleUserQuery }; This function combines document retrieval and AI processing to form a complete query handling mechanism. It gathers context from relevant documents and sends it to both Claude and ChatGPT for response generation, allowing you to compare or combine their outputs as needed. ⚠️ Common Mistake: Ensure your environment variables are correctly set up and accessible. Misconfigured keys will lead to authentication errors with the APIs. Once the setup is complete, it's crucial to test your application to ensure everything is functioning as expected. You can create a simple script to simulate user queries and verify the responses. js const { handleUserQuery } = require './queryHandler' ; async = { const query = "How does the RAG system work?"; const responses = await handleUserQuery query ; console.log "Claude's response:", responses.claude ; console.log "ChatGPT's response:", responses.chatGPT ; } ; Running this script should output responses from both Claude and ChatGPT, allowing you to assess their quality and relevance based on the provided document context.