Build a RAG System with Claude & ChatGPT APIs

> 🚀 Technical Briefing: This tutorial is part of our deep-dive series on Agentic Workflows at Gate of AI. For the full technical breakdown, interactive code sandbox, and the native Arabic translation, visit the original article here.

<span>Tutorial</span>
<span>Intermediate</span>
<span>⏱ 45 min read</span>
<span>© Gate of AI 2026-06-24</span>

Learn to build a smart, retrieval-augmented generation system using Claude and ChatGPT APIs to leverage the best of both models for enhanced AI interactions.

In this tutorial, we will build a Retrieval-Augmented Generation (RAG) system that combines the capabilities of Claude and ChatGPT APIs. This system will efficiently fetch relevant data from a document repository and generate contextually rich responses using advanced language models.

The final application will allow users to input queries, retrieve pertinent information from a pre-indexed document set, and use AI models to generate comprehensive answers. This integrated approach offers robust performance in applications requiring high accuracy and relevance, such as customer support systems and research assistants.

We will begin by setting up our development environment and installing necessary libraries. This involves setting up Node.js and installing the required SDKs for accessing the APIs.

npm install openai anthropic dotenv

Next, we'll configure our environment variables to securely store our API keys. Create a .env

file in your project root and add the following variables:

OPENAI_API_KEY=your_openai_api_key
ANTHROPIC_API_KEY=your_anthropic_api_key

First, we need to establish a document repository that our RAG system can query. We'll use a simple JSON file to simulate this repository. Ensure the data is structured for quick access and relevance scoring.

const fs = require('fs');
const documents = JSON.parse(fs.readFileSync('documents.json', 'utf8'));

function getRelevantDocuments(query) {
  // Simple keyword matching for relevance
  return documents.filter(doc => doc.text.includes(query));
}

module.exports = { getRelevantDocuments };

This code reads a JSON file containing our documents and filters them based on the query. The getRelevantDocuments

function will be used later to fetch relevant documents for any given query.

Next, we'll set up the integration with Claude and ChatGPT APIs to process and generate responses. This involves configuring both APIs and establishing a connection to send and receive data.

require('dotenv').config();
const { OpenAI } = require('openai');
const { Anthropic } = require('anthropic');

const openAIClient = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const anthropicClient = new Anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });

async function generateResponseWithClaude(prompt) {
  const response = await anthropicClient.chat.completions.create({
    model: "claude-3-5-sonnet-20241022",
    messages: [{ role: "user", content: prompt }]
  });
  return response.data.choices[0].message.content;
}

async function generateResponseWithChatGPT(prompt) {
  const response = await openAIClient.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: prompt }]
  });
  return response.data.choices[0].message.content;
}

module.exports = { generateResponseWithClaude, generateResponseWithChatGPT };

This code establishes connections to both APIs using the modern SDKs. It defines functions to send a prompt to each service and receive a generated response, which will be used to produce the final answer.

We'll now build the core logic to handle user queries. This involves retrieving relevant documents and using our integrated AI functions to generate a cohesive response.

const { getRelevantDocuments } = require('./documentRepository');
const { generateResponseWithClaude, generateResponseWithChatGPT } = require('./aiIntegrations');

async function handleUserQuery(query) {
  const relevantDocs = getRelevantDocuments(query);
  const combinedContext = relevantDocs.map(doc => doc.text).join('\n');
  const prompt = Based on these documents:\n${combinedContext}\nAnswer the following question: ${query};

const claudeResponse = await generateResponseWithClaude(prompt);
  const chatGPTResponse = await generateResponseWithChatGPT(prompt);

return {
    claude: claudeResponse,
    chatGPT: chatGPTResponse
  };
}

module.exports = { handleUserQuery };

This function combines document retrieval and AI processing to form a complete query handling mechanism. It gathers context from relevant documents and sends it to both Claude and ChatGPT for response generation, allowing you to compare or combine their outputs as needed.

⚠️ Common Mistake: Ensure your environment variables are correctly set up and accessible. Misconfigured keys will lead to authentication errors with the APIs.

Once the setup is complete, it's crucial to test your application to ensure everything is functioning as expected. You can create a simple script to simulate user queries and verify the responses.

const { handleUserQuery } = require('./queryHandler');

(async () => {
  const query = "How does the RAG system work?";
  const responses = await handleUserQuery(query);

console.log("Claude's response:", responses.claude);
  console.log("ChatGPT's response:", responses.chatGPT);
})();

Running this script should output responses from both Claude and ChatGPT, allowing you to assess their quality and relevance based on the provided document context.