NVIDIA launches model routing infrastructure to direct requests to lowest-cost capable inference model

NVIDIA has launched model routing infrastructure that examines prompts and directs them to the lowest-cost capable inference model, entering the model routing market to help enterprises control inference costs. The technology evaluates performance-price tradeoffs to select the most cost-appropriate model among multiple deployed models across different pricing tiers.

NVIDIA launches model routing infrastructure to direct requests to lowest-cost capable inference model According to CIO, NVIDIA is entering the model routing market with infrastructure that examines prompts and routes them to the most cost-appropriate model based on performance-price tradeoffs. Model routing has emerged as an enterprise technique to control inference costs as organizations deploy multiple models across different pricing tiers. Topics Sources - Press Read article https://www.cio.com/article/4209829/nvidia-moves-into-hot-market-for-model-routers-2.html This intelligence is sourced automatically from public sources across the web and synthesised by the Prefactor AI pipeline. Stories are reviewed before publication.