# Extract Anything From Anywhere: Building Document AI Pipeline with GLINER Beyond Zero Shot > Source: > Published: 2026-10-28 16:01:03+00:00 The accuracy of my regex and co-ordinate lookup based named entity recognition system broke suddenly from 94% to 67%. Only then did I realize that the purchase orders that were coming as input to my system were actually of different layouts. The header labels like the buyer name, delivery date, purchase order id etc. were all sitting at different places. The column sequence was shuffled and their names varied too. This was because the purchase order pdf’s were coming from different customers and it had become a headache to keep changing the code based on new layouts. This is when I decided to break-up with my regex based system and try out and build something ‘Intelligent-enough’ and ‘scalable’. If you are someone who has spent ample time writing rule based brittle logics to process documents like identity cards, purchase orders, invoices etc., then you have come to the right place. This might help you break out of the cycle of rebuilding structure-based extraction for every new layout. The ideal system must: · Not care where the information is located inside the source file · Understand the semantic meaning of the labels that need to be extracted · Not bother whether the information is inside a messy ocr string, buried inside a pdf table or listed in the email To achieve this, I built a universal pipeline using GLINER which is a powerful NER (Named Entity Recognition) model. I used the same approach to even extract named entities from identity cards for a different project. So, irrespective of whether I’m extracting the named entities from Purchase Orders using pymupdf or from ID cards using PaddleOCR, the pipeline remained the same. By end of this article you get to know what approach worked over zero-shot and few-shot experiments, how I landed a 93.2% accuracy on recall, and the architectural lessons I learned along the way. When I first encountered GLINER, I was hoping that the zero shot approach might actually work out in my favour. Zero shot is when you give zero additional training examples to the model to identify the entities of your data. Because It was a model which was already pre-trained on large labels and entities, I expected it to magically work on my entities. But during my initial tests I saw that it wasn’t performing well on my data. Then I switched to few-shot learning, where I gave the model a handful of examples to see if that could bridge the gap. And surprisingly the results were much better than the zero shot way. But this was not enough. Because, it was missing out a lot of products, products with names on multiple lines were treated as two products and the quantity and rate were not extracted accurately sometimes. In the world where a mismatch in the products tally or the price mismatch could lead to day-to-day financial errors, I could not use this system. That’s when I decided to fully fine-tune GLINER. Fine-tuning in this case, is not just about feeding additional data to a model. Its about the quality of annotation. Annotating each file of the training dataset manually might consume significant amount of time, but it is the key to achieving high quality results. It was clear that the ‘cleanliness’ of my annotations was directly proportional to the model’s performance. Instead of using llm or some manual approximate annotations, rely on open source annotation tools to annotate each entity with precision and consistency. This helped to prevent the model from learning noise and led to 93.2% recall accuracy across different vendor formats. **1.** **Managing the Context Window** Truncation was the most frustrating issue I encountered. Documents were long, and the models have hard limit on token length. My early experiments were failing because the sliding window was either too small or too large. It took several iterations of tuning the window size and overlap to find the best number where the model could ingest enough data to identify products without hitting its internal memory limits. **2.** **The De-Duplication Layer** When you process long documents using a sliding window with overlap, you inevitably get duplicates. The model might detect the same product in the final window of one chunk and again in the first window of the next. My early outputs were noisy and repetitive, so I built a post-processing layer to address this. This will be specific to the use-case you will be handling. **3.** **The Validation Layer: From Eyeballing to Evidence** Judging the success or failure of the system by merely eyeballing the results can never be trustworthy and convincing. I built an automated validation suite that forces the model to prove its work. By setting up a ground-truth benchmark and measuring Recall systematically, I moved away from guessing. The most rewarding part of this architecture is its modularity. Once I re-used this pipeline of: source files → extraction → annotation → chunking and sliding window → fine-tuning → validation, I realized I had built a reusable document intelligence engine. This same pattern was applied to an entirely different problem: extracting key entities from any ID card as input. The data source changed from pdf to images. So the pre-processor changed from pymupdf to PaddleOCR but the pipeline remained the same. A new instance of model was fine-tuned on the id card data and the pipeline handled the rest. But yes, the pre-processing, postprocessing and chunking & sliding windows needed to be aligned accordingly. It’s not about building one model for all problem, but about perfecting the architectural pattern so that document extraction becomes a repeatable, predictable engineering task. If you’re currently stuck in a cycle of writing brittle scripts for every new document format that crosses your desk, don’t try to make your code more complex. Instead, build a brain that learns. It’s an investment of effort, but once you have the pipeline, you aren’t just automating a process — you’re building the infrastructure for the next document type you’ll encounter tomorrow. Apart from the training loops, sliding windows and the recall metrics the real measure for this use-case wasn’t just the accuracy percentage. It was the hours saved. Before building this system, manually processing, cross referencing, and entering the details from the purchase orders into our workflow took roughly 20-25 minutes of human effort per pdf. But with this pipeline, the entity extraction is completed within 0.5–3 minutes flat, depending on the length of the document and whether you are running inference on GPU or CPU. Of course, because automated entity extraction should never be blindly trusted with financial data, a human still needs to quickly verify the output. But that manual review now takes about 2 minutes per pdf. When you do the math, you aren’t just cutting processing time by over 80%; you are eliminating the cognitive fatigue of manual data entry. The system doesn’t need to be perfect to be a massive win. By letting the AI handle 90%+ of the grunt work instantly, you turn a painful operational bottleneck into a fast, manageable verification queue. *This article was written with the assistance of AI for better articulation. The core ideas, experiments, and technical details are entirely my own* [Extract Anything From Anywhere: Building Document AI Pipeline with GLINER Beyond Zero Shot](https://pub.towardsai.net/extract-anything-from-anywhere-building-document-ai-pipeline-with-gliner-beyond-zero-shot-1d03b46ae381) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.