# Elastic looks to speed up Jina Model Inference by 8x using NVIDIA RTX GPUs > Source: > Published: 2026-10-21 00:00:00+00:00 # Elastic looks to speed up Jina Model Inference by 8x using NVIDIA RTX GPUs Promising early results from moving Jina models on the Jina API to NVIDIA RTX Pro 6000 GPUs: faster embedding, reranking, and OCR for search and agentic AI Giving AI agents the ability to accurately answer questions grounded in content involves running different kinds of models, each handling a different workload, and every step can add latency that users wait through. Jina AI by Elastic provides small, frontier-grade reader, OCR, embedding, and ranking models that preserve meaning from ingestion to retrieval. Elastic offers them as two managed services: the Jina API for direct calls to Jina models and Elastic Inference Service (EIS), which provides pre-configured, GPU-accelerated endpoints natively integrated with Elasticsearch. To keep up with growing agentic AI workloads, Elastic is moving these services from NVIDIA® L40 GPU to [NVIDIA RTX ProTM 6000 Blackwell Server Edition](https://www.nvidia.com/en-gb/data-center/rtx-pro-6000-blackwell-server-edition/) GPUs. Some Jina API models are running on the new GPUs, and customers using them get the speed improvements without having to change anything.  Results so far are promising. For the migrated models, the slowest embedding requests are up to 8x faster, reranking runs more than 4x faster, and OCR converts document pages about 3.7x faster. We are continuing to migrate more models of the Jina API and EIS, so more customers can see the performance improvements as each hosted model is migrated. ## Jina models lead their size class in coverage and accuracy Jina Reader and OCR models convert complex documents and websites, including images, tables, and formulas, into clean text with markdown and HTML for structure and LaTeX for formulas. Jina OCR reads layout, text, tables, and structure together in a single pass, so headings, columns, and tables stay intact. jina-ocr-v1 parses text and scripts in more than 100 languages. The output is ready for embedding, AI training, or use as operational data in applications. It is the [leading under-600M active parameter model](https://www.elastic.co/search-labs/blog/ocr-model-jina-ocr-v1-document-parsing) on the olmOCR-bench benchmark. Jina embedding models convert content and place multilingual and multimodal content in the same embedding space, so one query reaches all the content and results come back on a single confidence scale. Jina omni embedding models handle text, image, audio, and video in more than 100 languages. jina-embeddings-v5-omni-small is the best performing openweight omni model under 2B parameters on MTEB. Jina Rerankers then rescore those results against the original query and put the best matches first. Jina-reranker-v3.5, a 600M parameter model, beats the 4B parameter Qwen3-Reranker-4B model on the STaRK structured data benchmark. ## Small models and demanding workloads Jina’s compact models need less GPU than the larger models they compete with, but model size alone doesn’t determine how much capacity a workload needs. The mix of requests matters just as much. Jina API and EIS are shared services, so large jobs run alongside small jobs on the same GPUs. A single reranking request can carry a long list of candidate documents, while a query behind it might need one quick answer. When the large job occupies the GPU, the small request can end up waiting in a queue, and its latency climbs. When we benchmarked the NVIDIA RTX Pro 6000, supporting mixed workloads was the use case we cared about most. Could small requests stay fast while large jobs ran beside them? ## What Elastic measured We compared production traffic before and after the move and ran controlled tests for cases production can't isolate. Latency percentiles describe the slow end of response times: At the 99th percentile, 99 of every 100 requests finished faster than the value shown. Slow requests are the ones that users notice. | **Workload** | **Test** | **Metric** | **NVIDIA L4 Tensor Core GPU** | **NVIDIA RTXPRO 6000** | **Change** | | Embedding | Production | 99th-percentile latency | 5.86 s | 0.73 s | ~8x | | Reranking | Production with about 40% more average request volume | 90th-percentile latency | 732 ms | 171 ms | 4.3x | | Large reranking job | Controlled test; same software | Completion time | 702 ms (three L4s) | 160 ms (one GPU) | 4.4x | | Small reranking requests alongside large jobs | Mixed-traffic test on a shared GPU pool | Latency | 708 ms | 66 ms | 10.7x | | Document OCR (jina-ocr-v1) | Same set of real document pages | Time to first output token | 0.50 s | 0.20 s | 2.5x | | Document OCR (jina-ocr-v1) | Same set of real document pages | Page conversion speed | 454 pages/hr | 1,671 pages/hr | 3.7x faster | Running small models doesn't, by itself, simplify capacity planning for a shared service. The workload mix matters as much as the model size in deciding how much GPU capacity is needed. On NVIDIA Pro 6000 GPUs, small requests stayed fast while large jobs ran beside them. For our customers, that means a query doesn't slow down because a big job is running on the same service. To learn more about Jina models in Elastic Inference Service, [check out the documentation](https://www.elastic.co/docs/explore-analyze/elastic-inference/eis).  Try out [the Jina API](https://jina.ai/). *The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.* *In this blog post, we may have used or referred to third party generative AI tools, which are owned and operated by their respective owners. Elastic does not have any control over the third party tools and we have no responsibility or liability for their content, operation or use, nor for any loss or damage that may arise from your use of such tools. Please exercise caution when using AI tools with personal, sensitive or confidential information. Any data you submit may be used for AI training or other purposes. There is no guarantee that information you provide will be kept secure or confidential. You should familiarize yourself with the privacy practices and terms of use of any generative AI tools prior to use.*  *Elastic, Elasticsearch, and associated marks are trademarks, logos, or registered trademarks of elasticsearch B.V. in the United States and other countries. All other company and product names are trademarks, logos, or registered trademarks of their respective owners.*