TL;DR — Key Takeaways
- Red Hat AI 3.5 adds new governance, safety and observability capabilities for managing enterprise AI workloads.
- EvalHub is now generally available, providing a framework for benchmarking and certifying AI application safety.
- The release expands GPU workload prioritization, multi-tenancy, distributed inference and support for third-party Kubernetes environments.
Red Hat today made available an update to its core platform for running artificial intelligence (AI) workloads that provides additional governance, safety and observability capabilities.
Version 3.5 of the Red Hat AI platform makes EvalHub, a framework that Red Hat developed to benchmark the safety of AI applications in a way that can be certified, generally available.
Red Hat has also added observability dashboards to provide insights into metrics used to track inference health, utilization of graphics processing units (GPUs), AI model performance and consumption of AI tokens. Red Hat AI 3.5 adds an ability to prioritize which workloads gain access to a pool of GPU resources.
A Responses application programming interface (API) coupled with an automated retrieval-augmented generation (RAG) capability provides an open source interface for driving multi-agent conversations using the NeMo guardrails that Red Hat has developed in collaboration with NVIDIA.
There is also now a set of pre-built agent templates and starter kits with pre-configured reference implementations for common enterprise patterns, including code review, document processing, and research workflows, that have been added to the Red Hat AI Hub.
Additionally, Red Hat has enhanced multi-tenancy capabilities to completely isolate AI workloads, along with integrations with Red Hat OpenShift hosted control planes deployed on instances of Red Hat OpenShift Virtualization, which encapsulates virtual machines in containers that can be deployed on a Kubernetes cluster.
Red Hat is also making available an ability to offload CPUs while at the same time previewing a storage off capability that makes it possible to handle longer conversations and larger documents without requiring additional IT infrastructure.
Finally, Red Hat has added more than 20 new validated models to its catalog, including Gemma 4 from Google, Nemotron 3 from NVIDIA and Qwen from Alibaba.
Collectively, the management and governance capabilities embedded in Red Hat AI 3.5 make it possible for IT teams to more easily centrally manage the deployment of AI workloads across a distributed computing environment, says Tushar Katarki, senior director for product for Red Hat AI platforms. “Everything will eventually move to a distributed computing environment that will become more federated,” he adds.
The overall goal is to reduce the level of expertise that would otherwise be required to deploy AI workloads across a distributed computing environment, notes Katarki.
At the core of Red Hat AI is Red Hat OpenShift, a curated instance of Kubernetes that Red Hat has extended. In this latest release of Red Hat AI, support for the llm-d distributed inference engine that Red Hat uses to deploy AI workloads has been extended to third-party Kubernetes services provided by Amazon Web Services (AWS), Microsoft and CoreWeave. Red Hat is also previewing a Kubeflow Spark Operator to unify data preparation and model serving on a single platform. There is also a preview of a multimodal capability to serve text, audio, and images using an Omni capability that has been added to vLLM.
It’s not yet clear which platforms will be preferred for deploying AI workloads in production environments, but at the moment, at least, some flavor of Kubernetes is rapidly gaining traction.