Notes from building a retrieval-augmented generation container for the AMD AI League (Match 3), on an AMD Instinct MI300X.
Match 3 of the AMD AI League sounds simple: build a container that answers questions about a folder of documents. The catch is in the scoring. Each answer only counts if the value is right and the list of cited files is exactly right. No partial credit, no "close enough". Cite one file too many and a correct answer scores zero.
My container scored 10/10 (200/200) on the public sample questions, with exact citation sets, in 1–2 seconds per question. Here is how it works, and the traps that nearly cost me points.
The grader copies a folder of mixed documents into the container and calls one script in two ways:
python3 /app/app.py --index /app/corpus
python3 /app/app.py --corpus /app/corpus --query-id query_01 --query "What is the maximum junction temperature?"
Each query must write /app/output/query_01_output.json:
{"answer": "94", "citations": ["specs/tq40_datasheet_r2.pdf"], "confidence": 0.9}
The corpus is deliberately messy:
Limits: 10 minutes for startup (including indexing), 30 seconds per question, 1–48 GiB of VRAM, a 60 GiB image, and no network during grading.
The grader starts a new process for every question. If you load an 8-billion-parameter model inside app.py, you load it ten times and blow the 30-second budget on every question.
So the container runs two pieces:
server.py: CMD. It loads the models once, keeps the index in memory, and listens on a Unix socket.app.py: citations field scores zero for that question.
Two models, both baked into the image:
On the MI300X: models load in 7.7 s, the sample corpus indexes in 3.6 s, and peak VRAM is about 20 GB.
The challenge spells it out: a corpus walk that crashes on the first unreadable file indexes nothing after it, so which files you lose depends on alphabetical order. That is how a solution passes locally and fails on the graded set.
I went one step further and parse every file in a separate plain-Python worker process. A file that raises, hangs or crashes the parser costs only that file:
PermissionError is caught, skipped and logged._WITHDRAWN) or by phrases like "superseded by", and kept out of the model's context. A detail that matters: the Spreadsheets and CSV rows are indexed one row at a time, with the column headers attached (Part Number: ORR-FAN-2214-B | Description: Fan assembly, field-replaceable | ...). That way a single row is a self-contained, retrievable fact.
Some questions need two files. For example: "The production log shows a thermal throttle incident. Which firmware release fixed the underlying defect?" The log gives an identifier, and the bug database maps that identifier to a fix version. Neither file alone answers it.
Retrieval fuses identifier-aware BM25 (so ORR-1847, E7731 and THERM_ALERT# survive tokenisation) with dense embeddings. Then it takes one extra step: rare identifiers found in the best chunks, which the question didn't mention, pull in the chunks that define them. That puts both links of the chain in front of the model.
The rule from the challenge brief is: cite a file only if removing it would make your answer impossible.
Asking the model nicely is not enough, so the citations go through deterministic checks after the model answers:
To compare values the way the grader does, I first stripped separators and searched for the value as a substring. Then 4.3.2 became 432, which "matched" inside a log line where latency_ms=243 was followed by a date starting 2026. The log looked like a source of the answer, and my citations broke.
The fix was a token-boundary regex. Separators are still optional (Q3 FY27 matches Q3FY27, and v4.3.2 matches 4.3.2), but the value must stand on its own. Small detail, whole question's worth of points.
1. pip can silently replace ROCm torch with a CUDA build. Installing transformers can pull a CUDA torch over the base image's ROCm build. The error then shows up somewhere unrelated. My Dockerfile pins every torch package to the version already in the base image, installs against those constraints, and fails the build if torch.__version__ no longer contains rocm.
2. Test the unreadable file the way the grader does. As root, chmod 000 doesn't stop you reading a file. The grader drops the DAC_OVERRIDE capability, so test with:
docker run --network none --cap-drop DAC_OVERRIDE --device=/dev/kfd --device=/dev/dri ...
A bonus warning about the official self-check. It starts the container without GPU devices. My server couldn't load the model, the client waited out its timeouts, wrote valid empty answers, and every check said PASS. The self-check checks the shape of the output, not the answers. Always score the sample questions yourself on a real GPU run.
| Check | Result |
|---|---|
| Sample questions | 10/10, exact citation sets (200/200) |
| Per question | 1.0–1.9 s (limit 30 s) |
| Startup: model load + index | about 12 s (limit 10 min) |
| Peak VRAM | 20 GB (limit 48 GiB) |
| Image size | 45.8 GiB uncompressed (limit 60 GiB) |
The graded corpus is larger and harder than the sample, so these numbers are a starting point, not a final score.
The code is open source: https://github.com/DataGuy-Eterniti/knight-eterniti (mc3-rag/).
Thanks to @lablab.ai and @amd for the AMD AI League and the MI300X access through AMD Developer Cloud. On to Match 4.
#AMD #ROCm #lablab #RAG #MachineLearning #AMDAILeague