🚀 The Problem
Wait, what did we decide in that meeting?
We've all been there. You have a productive meeting, everyone agrees on action items, and by the next day, nobody remembers who was supposed to do what.
I wanted to fix this. So I built a tool that:
The best part? It works with any audio file — upload an MP3, and get a structured summary in seconds.
And I built it entirely with Java and Spring AI.
🛠️ The Tech Stack
Component Technology Purpose
Framework Spring Boot 3.4.x Application backbone
AI Integration Spring AI 1.0.3 Unified AI abstraction
Transcription OpenAI Whisper Speech-to-text
Speaker Detection AssemblyAI Speaker diarization
Summary Generation GPT-4o-mini LLM-powered summarization
Why this stack? Spring AI is the killer feature. It provides a clean, Spring-native way to interact with AI models. No boilerplate HTTP clients. No manual JSON parsing.
🧠 How It Works
Step 1: Transcribe with Whisper
Spring AI provides OpenAiAudioTranscriptionModel. I inject it directly into my service:
@Service
public class TranscriptionService {
private final OpenAiAudioTranscriptionModel transcriptionModel;
@Autowired
public TranscriptionService(OpenAiAudioTranscriptionModel transcriptionModel) {
this.transcriptionModel = transcriptionModel;
}
public String transcribe(File audioFile) {
AudioTranscriptionPrompt prompt = new AudioTranscriptionPrompt(
new FileSystemResource(audioFile),
AudioTranscriptionPrompt.builder()
.withModel("whisper-1")
.withResponseFormat(TranscriptResponseFormat.JSON)
.withLanguage("en")
.build()
);
AudioTranscriptionResponse response = transcriptionModel.call(prompt);
return response.getResult().getOutput();
}
}
Step 2: Detect Speakers with AssemblyAI
Spring AI doesn't have built-in speaker diarization, so I integrated AssemblyAI for this.
@Service
public class SpeakerDetectionService {
private String uploadAudio(File audioFile) throws IOException {
// Upload to AssemblyAI
// Returns upload_url
}
private String submitTranscription(String audioUrl) {
// Submit with speaker_labels: true
// Returns transcript ID
}
private List<SpeakerUtterance> getTranscriptWithSpeakers(String transcriptId) {
// Poll until status is "completed"
// Returns utterances with speaker labels
}
}
Why this matters: Speaker detection turns a plain transcript into an actionable meeting summary. Without it, you can't assign tasks to specific people.
Step 3: Generate the Summary with GPT
This is where the magic happens. I use Spring AI's ChatClient with a custom prompt:
@Service
public class SummaryService {
private final ChatClient chatClient;
public String generateSummary(String transcript, List<SpeakerUtterance> utterances) {
String prompt = buildPrompt(transcript, utterances);
return chatClient.prompt()
.user(prompt)
.call()
.content();
}
private String buildPrompt(String transcript, List<SpeakerUtterance> utterances) {
return """
You are a professional meeting summarizer with 10 years of experience.
IMPORTANT: First, generate a concise, professional meeting title (max 5 words).
Provide your response in this exact format:
[Your concise title here, max 5 words]
One paragraph summary (max 50 words)
- Decision 1
- Decision 2
| Task | Assignee | Deadline | Priority |
|------|----------|----------|----------|
| [Task] | [Person] | [Date] | [HIGH/MEDIUM/LOW] |
- Issue 1
- Step 1
Rules:
1. ONLY use information from the transcript
2. If a deadline or assignee is missing, write "Not specified"
3. Keep action items to a maximum of 5
4. Be concise and actionable
""".formatted(transcript, speakerInfo);
}
}
💡 Pro tip: Never hardcode your prompt directly in the service. Store it as a classpath resource and load it with @Value. This way you can tweak the output without a code review.
📡 The Controller
The REST endpoint ties everything together:
@PostMapping("/process")
public ResponseEntity<Map<String, Object>> processMeeting(
@RequestParam("audio") MultipartFile audioFile) {
// 1. Save uploaded file to temp location
File tempFile = saveToTemp(audioFile);
// 2. Transcribe with Whisper
String transcript = transcriptionService.transcribe(tempFile);
// 3. Detect speakers with AssemblyAI
List<SpeakerUtterance> utterances = speakerDetectionService.detectSpeakers(tempFile);
// 4. Generate summary with GPT
String summary = summaryService.generateSummary(transcript, utterances);
// 5. Parse tasks from the summary
List<Map<String, String>> tasks = parseTasks(summary);
// 6. Return structured JSON
return ResponseEntity.ok(buildResponse(summary, utterances, tasks));
}
📊 What the Output Looks Like
Here's a real response from a test meeting:
{
"meetingTitle": "Student Attendance Concerns",
"meetingOverview": "The meeting addressed a decline in student attendance on Fridays...",
"keyDecisions": [
"Host a pancake breakfast to encourage attendance.",
"Put up health posters to address seasonal illness."
],
"tasks": [
{
"description": "Organize pancake breakfast",
"assignee": "Not specified",
"deadline": "Not specified",
"priority": "HIGH"
},
{
"description": "Create and display health posters",
"assignee": "Not specified",
"deadline": "Not specified",
"priority": "MEDIUM"
}
],
"speakerCount": 3
}
🚀 Try It Yourself
The API is live:
💡 What I Learned
1. Spring AI Eliminates Boilerplate
I didn't write a single HTTP client or JSON parser. The ChatClient abstraction handles everything. You can swap the LLM provider by changing one dependency and one properties line.
2. Prompt Engineering Is the Real Skill
The output quality depends entirely on the prompt. I iterated on my prompt more than on any code component. A good prompt with system context makes the difference between generic and actionable output.
3. File-Based Processing Has Privacy Benefits
Unlike bot-based tools (like Fireflies or Otter), my tool processes uploaded files — no one needs to join your meetings. There's growing demand for privacy-first meeting assistants.
4. Speaker Detection Changes Everything
Without speaker labels, you just get a wall of text. With them, you get clear ownership of action items.
🔮 What's Next
📚 Resources
AssemblyAI Speaker Diarization
💬 Let's Connect
If you're building AI tools with Spring AI, I'd love to hear about your experience. Drop a comment.
Thanks for reading! 🚀