I Built RepoPilot: An AI-Powered Tool to Understand Codebases and Generate Tests

A developer built RepoPilot, an AI-powered developer tool that indexes GitHub, GitLab, and Bitbucket repositories to enable codebase-aware chat and automated test generation. The tool combines AST parsing (via TypeScript's compiler ecosystem, Python AST, and a comment-aware Java scan) with dependency graphs to build targeted context for LLM requests, avoiding the inefficiency of passing entire repositories into a prompt. RepoPilot supports context-aware unit and integration test generation as well as scenario-based tests described in natural language.

How a side project evolved into a developer tool combining AST parsing, dependency graphs, repository-aware AI, and automated test generation. As developers, we've all been there. You open an unfamiliar repository. There are hundreds of files, deeply nested dependencies, unfamiliar abstractions, and barely any documentation. Before you can fix a bug or introduce a feature, you need to answer a few questions: AI coding assistants have made parts of this easier. But I wanted to experiment with something more structured. What if an AI assistant could first understand a repository's architecture and dependencies, then use that context to answer questions and generate tests? That's how RepoPilot started. RepoPilot is an AI-powered developer tool designed to help engineers explore unfamiliar codebases, understand their structure, and generate context-aware tests. 🔗 Try RepoPilot: https://repopilotapp.com https://repopilotapp.com Instead of manually copying files into an AI chatbot, you can connect a repository and let RepoPilot build the context required for codebase-aware interactions. Here's what the current version offers: 1. Repository integration Connect GitHub, GitLab, or Bitbucket repositories, including private repositories through supported OAuth integrations. 2. Codebase indexing and dependency analysis RepoPilot analyzes the repository's source code, extracts structural information, and builds relationships between files and code entities. 3. AI-powered codebase chat Ask questions about your repository and receive answers informed by the indexed codebase. 4. Context-aware unit and integration test generation Select source files and generate tests based on their implementation and related dependencies. 5. Scenario-based test generation Describe a business scenario in natural language and let RepoPilot generate relevant test cases using repository context. The goal isn't to replace an engineer's judgment. It's to reduce the repetitive work required to understand a codebase and establish a useful testing foundation. One of the biggest challenges with AI-assisted development is context. Large repositories cannot simply be passed into an LLM prompt in their entirety. Even when a repository fits within a model's context window, providing every file is often inefficient. It increases token consumption, introduces irrelevant information, and makes it harder to focus on the specific implementation being discussed. I wanted RepoPilot to retrieve relevant code based on its actual structural relationships. That led me toward a combination of AST parsing, dependency graphs, and targeted context building . RepoPilot's indexing pipeline uses TypeScript's compiler ecosystem to analyze JavaScript and TypeScript source files, Python AST for Python source files & Comment-aware source scan for Java. Rather than treating code as plain text, the parser extracts structural information such as: This provides a structured representation of the repository. For example, consider: js import { PaymentService } from "./payment.service"; export async function checkout order: Order { const paymentService = new PaymentService ; return paymentService.process order ; } A text-based search might identify the imported class name. An AST-based approach gives us a more reliable way to identify the import relationship and the structure of the function containing it. That information becomes valuable when deciding which related files should be included in an AI request. After parsing, RepoPilot constructs a knowledge graph representing relationships within the codebase. Conceptually: checkout.ts | | imports v payment.service.ts | | imports v payment.gateway.ts Now, if someone requests tests for checkout.ts , we don't necessarily need to provide the entire repository. We can start with the selected file and traverse relevant dependencies. This allows the context-building layer to collect the implementation details most likely to matter. The graph also creates a foundation for future capabilities such as impact analysis and architectural exploration. This was one of the more interesting architectural decisions. I separated the code-analysis pipeline from the AI-generation pipeline. At a high level, the flow looks like this: Git Repository | v Repository Parser | v Dependency Graph | v Context Builder | v Prompt Builder | v LLM Provider | v Generated Tests / AI Responses The context builder is responsible for identifying and collecting relevant code. The prompt builder then transforms that information into task-specific instructions. For example, generating unit tests requires a different prompt structure from answering an architectural question. Keeping these responsibilities separate makes the system easier to extend, debug, and maintain. It also allows the AI provider to evolve independently of the repository-analysis pipeline. One of RepoPilot's primary use cases is test generation. Consider a function that validates an order, checks inventory, and processes payment. A useful test suite should consider more than the happy path. It might include: By supplying the relevant implementation and dependency context, RepoPilot aims to produce more meaningful test suggestions than a generic prompt containing only a function name. The generated tests still require developer review and execution. AI-generated tests are not automatically proof of correctness. But they can significantly reduce the effort required to create an initial testing baseline. I also wanted developers to describe tests in the language of business requirements. For example: Generate tests for a checkout flow where payment succeeds but inventory reservation fails. RepoPilot uses the scenario and available repository context to generate relevant test code. This is particularly useful when developers know the behavior they want to validate but haven't yet identified every implementation detail. It also helps connect business-level requirements with actual source-code behavior. Here's a simplified view of the technologies behind RepoPilot: | Layer | Technology | |---|---| | Frontend | Vue 3, TypeScript | | Backend | Python, FastAPI | | Code analysis | TypeScript Compiler API / ts-morph | | Repository intelligence | Dependency graph and context builder | | AI integration | Provider abstraction for LLM services | | Authentication | Auth0 | | Source integrations | GitHub, GitLab, Bitbucket | | Background processing | Asynchronous jobs and workers | One of my main goals was to keep the architecture modular. I didn't want the repository parser, context builder, and LLM integration to become one large service that would be difficult to maintain. A few lessons stood out during development. 1. Better context matters as much as better prompts. Prompt engineering is useful, but identifying the correct source files and relationships is equally important. 2. Deterministic code analysis and generative AI complement each other. AST parsing provides structural information. LLMs help interpret that information and generate useful outputs. Neither approach needs to replace the other. 3. AI-generated tests require validation. Generated code can contain incorrect assumptions, incomplete mocks, or invalid assertions. Human review and execution remain essential. 4. Security becomes more complicated with private repositories. Repository credentials must be handled carefully, with scoped access, secure storage, and protection against accidental exposure in logs or prompts. 5. Building a product is different from building a prototype. Authentication, quotas, failure handling, deployment, observability, and user onboarding quickly become just as important as the AI functionality. RepoPilot is still evolving. Some areas I'm interested in exploring further include: I'm deliberately keeping production preview execution out of the initial public release while I validate the security, infrastructure, and operational requirements. For now, the focus is on repository intelligence and AI-assisted testing.