I Built an AI Meeting Summarizer with Spring AI — Here's How (and Why)

A developer built an AI meeting summarizer using Java and Spring AI, integrating OpenAI's Whisper for transcription, AssemblyAI for speaker detection, and GPT-4o-mini for generating structured summaries. The tool accepts audio files and outputs meeting titles, overviews, key decisions, and action items.

🚀 The Problem Wait, what did we decide in that meeting? We've all been there. You have a productive meeting, everyone agrees on action items, and by the next day, nobody remembers who was supposed to do what. I wanted to fix this. So I built a tool that: The best part? It works with any audio file — upload an MP3, and get a structured summary in seconds. And I built it entirely with Java and Spring AI. 🛠️ The Tech Stack Component Technology Purpose Framework Spring Boot 3.4.x Application backbone AI Integration Spring AI 1.0.3 Unified AI abstraction Transcription OpenAI Whisper Speech-to-text Speaker Detection AssemblyAI Speaker diarization Summary Generation GPT-4o-mini LLM-powered summarization Why this stack? Spring AI is the killer feature. It provides a clean, Spring-native way to interact with AI models. No boilerplate HTTP clients. No manual JSON parsing. 🧠 How It Works Step 1 : Transcribe with Whisper Spring AI provides OpenAiAudioTranscriptionModel. I inject it directly into my service: @Service public class TranscriptionService { private final OpenAiAudioTranscriptionModel transcriptionModel; @Autowired public TranscriptionService OpenAiAudioTranscriptionModel transcriptionModel { this.transcriptionModel = transcriptionModel; } public String transcribe File audioFile { AudioTranscriptionPrompt prompt = new AudioTranscriptionPrompt new FileSystemResource audioFile , AudioTranscriptionPrompt.builder .withModel "whisper-1" .withResponseFormat TranscriptResponseFormat.JSON .withLanguage "en" .build ; AudioTranscriptionResponse response = transcriptionModel.call prompt ; return response.getResult .getOutput ; } } Step 2 : Detect Speakers with AssemblyAI Spring AI doesn't have built-in speaker diarization, so I integrated AssemblyAI for this. @Service public class SpeakerDetectionService { private String uploadAudio File audioFile throws IOException { // Upload to AssemblyAI // Returns upload url } private String submitTranscription String audioUrl { // Submit with speaker labels: true // Returns transcript ID } private List<SpeakerUtterance getTranscriptWithSpeakers String transcriptId { // Poll until status is "completed" // Returns utterances with speaker labels } } Why this matters: Speaker detection turns a plain transcript into an actionable meeting summary. Without it, you can't assign tasks to specific people. Step 3 : Generate the Summary with GPT This is where the magic happens. I use Spring AI's ChatClient with a custom prompt: @Service public class SummaryService { private final ChatClient chatClient; public String generateSummary String transcript, List<SpeakerUtterance utterances { String prompt = buildPrompt transcript, utterances ; return chatClient.prompt .user prompt .call .content ; } private String buildPrompt String transcript, List<SpeakerUtterance utterances { return """ You are a professional meeting summarizer with 10 years of experience. IMPORTANT: First, generate a concise, professional meeting title max 5 words . Provide your response in this exact format: MEETING TITLE Your concise title here, max 5 words MEETING OVERVIEW One paragraph summary max 50 words KEY DECISIONS - Decision 1 - Decision 2 ACTION ITEMS | Task | Assignee | Deadline | Priority | |------|----------|----------|----------| | Task | Person | Date | HIGH/MEDIUM/LOW | UNRESOLVED ISSUES - Issue 1 NEXT STEPS - Step 1 Rules: 1. ONLY use information from the transcript 2. If a deadline or assignee is missing, write "Not specified" 3. Keep action items to a maximum of 5 4. Be concise and actionable """.formatted transcript, speakerInfo ; } } 💡 Pro tip: Never hardcode your prompt directly in the service. Store it as a classpath resource and load it with @Value. This way you can tweak the output without a code review. 📡 The Controller The REST endpoint ties everything together: @PostMapping "/process" public ResponseEntity<Map<String, Object processMeeting @RequestParam "audio" MultipartFile audioFile { // 1. Save uploaded file to temp location File tempFile = saveToTemp audioFile ; // 2. Transcribe with Whisper String transcript = transcriptionService.transcribe tempFile ; // 3. Detect speakers with AssemblyAI List<SpeakerUtterance utterances = speakerDetectionService.detectSpeakers tempFile ; // 4. Generate summary with GPT String summary = summaryService.generateSummary transcript, utterances ; // 5. Parse tasks from the summary List<Map<String, String tasks = parseTasks summary ; // 6. Return structured JSON return ResponseEntity.ok buildResponse summary, utterances, tasks ; } 📊 What the Output Looks Like Here's a real response from a test meeting: { "meetingTitle": "Student Attendance Concerns", "meetingOverview": "The meeting addressed a decline in student attendance on Fridays...", "keyDecisions": "Host a pancake breakfast to encourage attendance.", "Put up health posters to address seasonal illness." , "tasks": { "description": "Organize pancake breakfast", "assignee": "Not specified", "deadline": "Not specified", "priority": "HIGH" }, { "description": "Create and display health posters", "assignee": "Not specified", "deadline": "Not specified", "priority": "MEDIUM" } , "speakerCount": 3 } 🚀 Try It Yourself The API is live: meeting-summarizer https://meeting-summarizer-pj7h.onrender.com/ 💡 What I Learned 1. Spring AI Eliminates Boilerplate I didn't write a single HTTP client or JSON parser. The ChatClient abstraction handles everything. You can swap the LLM provider by changing one dependency and one properties line. 2. Prompt Engineering Is the Real Skill The output quality depends entirely on the prompt. I iterated on my prompt more than on any code component. A good prompt with system context makes the difference between generic and actionable output. 3. File-Based Processing Has Privacy Benefits Unlike bot-based tools like Fireflies or Otter , my tool processes uploaded files — no one needs to join your meetings. There's growing demand for privacy-first meeting assistants. 4. Speaker Detection Changes Everything Without speaker labels, you just get a wall of text. With them, you get clear ownership of action items. 🔮 What's Next 📚 Resources Spring AI Documentation https://spring.io/projects/spring-ai AssemblyAI Speaker Diarization https://www.assemblyai.com/docs/pre-recorded-audio/label-speakers OpenAI Whisper API https://developers.openai.com/api/docs/guides/speech-to-text GitHub Repository https://github.com/leosarabi/meeting-summarizer 💬 Let's Connect If you're building AI tools with Spring AI, I'd love to hear about your experience. Drop a comment. Thanks for reading 🚀