# What does the card say? Text extraction using Amazon Nova 2 Lite > Source: > Published: 2026-08-13 05:41:50+00:00 **No container, no weights, no memory limit to worry about. The only thing I own in this model is a prompt and it did not always go well.** Two models in this pipeline have already had their own article: [card-detect](https://dev.to/aws-builders/is-this-even-a-valid-card-zero-shot-image-classification-model-in-a-lambda-container-58oj) lambda, which answers *is this even an AWS Builder Card?*, and [image-processor](https://dev.to/aws-builders/where-exactly-is-the-card-in-this-photo-image-segmentation-model-inside-a-maxed-out-lambda-51da) lambda, which answers *where exactly is the card in this photo?* Whole pipeline in a nutshell is also described [here](https://dev.to/aws-builders/from-anonymous-photo-to-a-published-page-an-event-driven-ai-image-processing-pipeline-on-aws-2m2n). So what is the current situation? The card is straight and clean by now, but I still don't know what it says. This article describes what happens next. There is another model which is not part of any lambda, but is being called by `image-processor` . That model is `Nova 2 Lite` living in `Amazon Bedrock` and it extracts the card's text. Once extracted, the same lambda write that extracted text to `DynamoDB` as a card's metadata. Both previous articles run the same three questions before putting a model into a lambda. Since this model does not run in the lambda, I don't even care about the answers: `Bedrock` .No container to build, nothing to bake into an image, no `ECR` push, no cold start to measure, no `10240 MB` ceiling to hit. The model is `Amazon Nova 2 Lite` and it doesn't even run in my account. The lambda make an API call towards `Bedrock` , where the text is extracted and later stored in `DynamoDB` as a card's metadata. The only thing I have to worry about here is IAM permissions and writing a good prompt. Reading a card is not just OCR! Here is the actual problem and it is not **just** the text recognition: A player knows what an orange circle means, but model has to be told. Some cards carry all 3 icons, others just 1 or 2 and there are cards with none. AWS offer couple of resources capable of recognizing the text: `Amazon Textract` `Amazon Reckognition` But their inability to understand and interpret the icons made them unusable for this case. This is the only reason I used multimodal model, capable of both tasks. The whole thing is just a `converse()` API call: ``` python def extract_card_fields(png_bytes): # ... (docstring trimmed) ... # Invoke the Bedrock vision model. resp = bedrock.converse( modelId=BEDROCK_MODEL_ID, system=[{"text": EXTRACT_SYSTEM}], messages=[{ "role": "user", "content": [ {"text": EXTRACT_PROMPT}, {"image": {"format": "png", "source": {"bytes": png_bytes}}}, ], }], inferenceConfig={"maxTokens": 1200, "temperature": 0.0}, ) # Parse the model response. text = resp["output"]["message"]["content"][0]["text"] return _parse_json(text) ``` I set up `temperature` to `0.0` because I want it to always return answer it is most confident about. I am expecting always the same answer, no matter how many times the same card goes in. That actually makes sense, since it is reading the text. In order for the model to do anything, it needs its instructions - the prompt. Especially in case of the icons. ``` # ...(beginning and title omitted)... TASK 2: EFFECT The effect is the gameplay area in the middle of the card. The effect area may contain small icons. Not all icon types are always present on s csrd. A card may have: 0 icons, or 1 icon, or 2 icons or all 3 icons. Only use icons that are actually visible on this card. There are exactly 3 possible icon types: ICON TYPE 1: CREDIT Visual appearance: orange circle contains a white number. That number has NO plus sign. Example: orange circle with "1" This icon can be small, so look carefully. Meaning: "Get N credit", if N = 1. ICON TYPE 2: DRAW CARDS Visual appearance: black rounded rectangle contains a white number. the number ALWAYS has a plus sign, like "+1" or "+2" This icon can be small, so look carefully. Meaning: "Draw N card from your Resources Pile" if N = 1. "Draw N cards from your Resources Pile" if N > 1. ICON TYPE 3: CLOUD ADOPTION EFFECT Visual appearance: small white cloud shape with black outline contains a black number. the number ALWAYS has a plus sign, like "+1" or "+2". This icon can be small, so look carefully. Meaning: "Use N cloud adoption effect" if N = 1. "Use N cloud adoption effects" if N > 1. # ...(rest of the prompt omitted)... ``` During the local and live testing, I came into several issues but all I was able to fix with tuning the prompt. The effects were where prompt was loosing to most. Sometimes it was ignoring the icons, other time it was adding them where they weren't, like for this card: I ran it 11 times and 7 times I got it wrong: `Get 1 credit. ` . Draw 1 card from your Resources Pile. Use 1 cloud adoption effect**----> THIS IS NOT ON THE CARD!** The card has **no cloud adoption effect**, but yet it was fabricating it! The solution is easier than you think. After I added section `IMPORTANT ICON RULES` into the prompt, the icon hallucination stopped. ``` IMPORTANT ICON RULES: If a number has NO plus sign and is inside an orange circle, it means credits. If a number has a plus sign and is inside a dark rounded rectangle, it means draw cards. If a number has a plus sign and is inside a white cloud outline, it means cloud adoption effects. A plus sign never means credits. Do not invent missing icons. Do not mention credits unless an orange circle is visible. Do not mention drawing cards unless a dark rounded rectangle is visible. Do not mention cloud adoption effects unless a white cloud icon is visible. ``` Another problem I had was with Credits (a number in orange circle). With initial prompt, it was only interpreting it as: `Get 1 credit` , no matter the number in the circle. The solution was making the prompt into **few-shot example** prompt, adding examples like: ``` "Get N credit", if N = 1. "Draw N card from your Resources Pile" if N = 1. "Draw N cards from your Resources Pile" if N > 1. "Use N cloud adoption effect" if N = 1. "Use N cloud adoption effects" if N > 1 ``` I am expecting a model to read the text on the card, and return 3 key:value pairs: But here's where AWS Builder Cards fights back again - some of them have a "subtitle": Without considering that into the prompt, this was the result of the text extraction: | What model extracted | Full card's title | |---|---| `AWS certified Solutions Architect` | `AWS certified Solutions Architect Associate` | `AWS certified Solutions Architect` | `AWS certified Solutions Architect Professional` | `AWS certified Developer` | `AWS certified Developer Associate` | `AWS certified Sysops Administrator` | `AWS certified Sysops Administrator Associate` | Look at the first two rows. Those are two physically different cards - a **Solutions Architect Associate** and a **Solutions Architect Professional**, but the model returned **the same title** for both. If I had trusted the drafts (without manual approvals), my catalog would hold the same card twice and be missing another one. I had 2 options how to deal with that: Of course I fixed the prompt - the less manual job for me during the approvals, the better! ``` EXTRACT_PROMPT = """Return ONLY valid JSON. The JSON must have exactly these keys: { "title": "", "effect": "", "description": "" } Read the card from top to bottom. TASK 1: TITLE The title is the card name. It can have two parts. Part 1 - the text in the title bar at the top of the card. Always present. Read it in full, exactly as printed. Part 2 - a qualifier printed in its own banner inside the artwork, below the title bar. Only some cards have this. Examples of what it looks like: ASSOCIATE, PROFESSIONAL, FOUNDATIONAL, SPECIALTY. If a banner like that is visible, the title is Part 1 followed by Part 2, written in normal capitalisation: "AWS certified Solutions Architect" + "ASSOCIATE" -> "AWS certified Solutions Architect Associate" If no such banner is visible, the title is Part 1 alone. Do not add a qualifier that is not printed on the card. Examples: "AWS Cloud Practitioner", "David", "AWS certified Solutions Architect Professional". If you cannot read it, use "". # ...(rest of the prompt omitted)... ``` That's just enough for model to understand when the card has a "subtitle". That may seem like not important, but consider text on the the cards contain bold text, links, sepparate lines, etc... If I want final card page to look like the card itself, I have to follow that. Again, this is something that would take me 10 seconds during the manual approval, but why if I can do it with prompt? Few shots example will do the job ``` If you spot text in bold, write it as for makrdown files - that means like this: **this is bold text** Any internet link (URL) you spot, you must write in this format: [link](link). ``` Having implemented all prompt modifications, now I can say in **most** cases, this prompt works 100%. Occasionally there some some minimal hickups, but generally it works perfectly. As you can imagine, I did not write this prompt at once. At least 6 versions of it went live, after I was happy with the outputs. Bedrock returns **title**, **description** and **effect** back to `image-processing` lambda. To create a card's slug markdown file which is performed by lambda `review-editor` in the next steps **LINK TO ARTICLE 5 - TBD**, more values are actually needed. Therefore the `image-processor` lambda actually gathers a lot more values, before sending them to `DynamoDB` . ``` python def write_card_item(card_id, event, year, raw_key, fin_key, fields, uploader=""): # Generate a timestamp for the new record now = datetime.now(timezone.utc).isoformat() # The year is only appended when the title does not already carry it title_draft = fields.get("title", "") year_str = str(year).strip() slug_src = f"{title_draft} {year_str}" if year_str and year_str not in title_draft else title_draft slug_draft = slugify(slug_src) # Store the pending card record in DynamoDB ddb.put_item( TableName=DDB_TABLE, Item={ "cardId": {"S": card_id}, # ---> from S3 key (the uuid, via parse_meta) "slug": {"S": ""}, # ---> empty, for the human "slug_ai_draft": {"S": slug_draft}, # ---> computed locally from Bedrock's title + year "weight": {"S": ""}, # ---> empty, for the human "event": {"S": event or ""}, # ---> from S3 key (via parse_meta) "year": {"S": str(year or "")}, # ---> from S3 key (via parse_meta) "uploader": {"S": uploader or ""}, # ---> from S3 object metadata (typed by the visitor) "category": {"S": "collectibles"}, # ---> hardcoded constant "subcategory": {"S": ""}, # ---> empty, for the human "title": {"S": ""}, # ---> empty, for the human "title_ai_draft": {"S": fields.get("title", "")}, # ---> from Bedrock "effect": {"S": ""}, # ---> empty, for the human "effect_ai_draft": {"S": fields.get("effect", "")}, # ---> from Bedrock "description": {"S": ""}, # ---> empty, for the human "description_ai_draft": {"S": fields.get("description", "")}, # ---> from Bedrock "rawKey": {"S": raw_key}, # ---> from the S3 event (the uploaded object's key) "finishedKey": {"S": fin_key}, # ---> computed locally (images/finished/.png) "status": {"S": "pending"}, # ---> hardcoded constant - the review gate "createdAt": {"S": now}, # ---> computed locally (UTC timestamp) "updatedAt": {"S": now}, # ---> computed locally (same timestamp) }, ) ``` This is what is actually written in the DB. Some values are intentionally left empty and will be filled by next lambda - `review-editor` , while some of the empties have to be manually filled by me. More on that in **ARTICLE 5 - TBD** It is tempting to create a separate lambda function for `Bedrock` and `DynamoDB` calls, which would be completly isolated from `image-processing` lambda. However, I found that as not a good idea, mainly because how the `image-processor` work and what it sends to the `Bedrock` . Lambda **does not send the finished card picture**, from `images/finished` to the text extraction. It sends the ** png bytes**, it stores in its own memory. The split would mean a second function has to download the finished image from `S3` , which brings extra latency, another GET, IAM role, etc...The one argument that would justify splitting, is wide IAM permission current current `image-processor` holds. Having image processing part along with API calls to `Bedrock` and `Dynamo DB` requires permissions for `S3` , `Bedrock` , `DynamoDB` and `sns` in one role. That doesn't go really well with lest privilege concept I am applying where possible in this project, but here I made an exception. Each of the arguments have pros and cons and me personally I was 50:50 on it if to split or keep as one, but I decided to keep it this time. This part of the pipeline is pretty simple, the longest part to test and tune was the prompt. I started on couple of lines, and after endless tests I ended up on almost 100 lines. The result is **extracted text** written in `DynamoDB` as particular cards' metadata. Next step is just manual - me as an admin visually verify the card against the extracted text and approve. Right after that the deployment process starts, which is described in next **ARTICLE 5 - TBD** [From anonymous photo to a published page: An event-driven, AI, image processing pipeline on AWS](https://dev.tolink) [Is this even a valid card? Zero-shot image classification model in a lambda container](https://dev.tolink) [Where exactly is card in this photo? Image segmentation model inside a maxed-out lambda container](https://dev.tolink) Who reviews the reviewer? Building the human step in an AI pipeline -**TBD**