🩺 Building RadScan AI: Autonomous Multimodal Radiology Triage on GCP Cloud Run & Vertex AI

RadScan AI, a multimodal radiology triage system built for the All Things Agentic Hackathon, combines a 2.5D Volumetric CNN-BiGRU neural network with Vertex AI Gemini models on Google Cloud Run to detect 12 pathology targets and draft structured reports in under 3 seconds. The system processes 24 parallel depth slices across three planes, using Grad-CAM heatmaps for lesion localization, and scales to zero instances when idle to minimize costs.

Disclaimer:This article was created for the purpose of entering the All Things Agentic Hackathon hosted by Google Cloud and Devpost. Radiologists worldwide face extreme burnout, evaluating 50+ volumetric MRI scans daily. Each scan contains 24 to 40 high-resolution DICOM slices. Manually inspecting every plane, checking ligament continuity across depth slices, and dictating repetitive clinical reports takes 10 to 15 minutes per study . We built RadScan AI to eliminate this workflow bottleneck. By combining GCP Cloud Run L4 GPU scale-to-zero microservices , a 2.5D Volumetric CNN-BiGRU Neural Network trained on 819,100 DICOMs / 530 GB data , and Vertex AI Gemini 3.5 Flash / 1.5 Pro , RadScan AI acts as an autonomous radiology co-pilot that performs 12-target pathology detection, pinpoints lesion coordinates with Grad-CAM visual heatmaps, and drafts structured DICOM reports in under 3 seconds —saving radiologists ~6 minutes per scan. RadScan AI is architected as two decoupled, serverless microservices on Google Cloud Platform : flowchart TD subgraph Client "Next.js 14 Medical Workspace" A 1-Click Sample Buttons / DICOM Upload -- B Multi-Planar Slice Slider B -- C Grad-CAM Heatmap Opacity Layer B -- D Gemini Clinical Report Generator end subgraph Backend "FastAPI Microservice GCP Cloud Run L4 GPU " E POST /api/v1/predict -- F 2.5D Volumetric CNN-BiGRU Engine F -- G Grad-CAM Heatmap Synthesizer E -- H POST /api/v1/report H -- I Google ADK / Vertex AI Gemini 3.5 SDK end Client -- Backend google-cloud-aiplatform and google-genai SDK in native JSON mode response mime type="application/json" .Single 2D MRI slices often mimic tears due to volume averaging artifacts. RadScan AI processes 24 parallel depth slices across Sagittal, Coronal, and Axial planes simultaneously: For a 3D MRI volume stack S = {s1, s2, ..., s24}, spatial CNN features f t = CNN s t are fed into a Bidirectional GRU to track ligament continuity across consecutive depth slices: The explainability weights alpha k^c for target class c at feature map A^k are computed via backpropagated gradients across feature channels: Deploying to GCP Cloud Run ensures our backend scales down to 0 instances when idle, keeping cloud costs at virtually $0/month while serving fast, sub-second inference on demand: Build & Deploy Backend Microservice to Cloud Run gcloud run deploy radscan-ai-backend \ --image gcr.io/YOUR GCP PROJECT ID/radscan-ai-backend:v1 \ --platform managed \ --region us-central1 \ --memory 2Gi \ --cpu 2 \ --allow-unauthenticated Building RadScan AI for the All Things Agentic Hackathon proved that combining high-performance computer vision with structured Gemini 3.5 LLM agents creates production-grade medical automation. AllThingsAgenticHackathon GoogleCloud VertexAI Gemini FastAPI NextJS AIHealthcare