Integrating AI Into CTV Search and Discovery

wpnews.pro

#

Integrating AI Into CTV Search and Discovery

The benefits of leveraging AI and spending AI tokens in personalizing the CTV discovery experience and facilitating fruitful content searches are considerable, but so are the costs. Are there ways that OTT services and platforms can manage or minimize those costs, such as only enlisting AI for more complex queries? Dataxis’ Ophelie Boucaud, TiVo’s Chris Ambrozic, and Future Today’s Alok Ranjan discuss token conservation, handling conversational and non-conversational queries, and the “Swiss Army knife approach” in this clip from Streaming Media Connect 2026.

Timing and Token Conservation #

Boucaud kicks off the conversation by saying that one "challenge that pops up when we talk about AI integration is the cost. Obviously, running queries is piling up cost at some point, so there are ways to make sure that you only ask the most complex queries to the most complex models and limit your spending this way." Turning to Ambrozic, she says, "Chris, can you tell us maybe a little bit more about this? You mentioned before that there's a lot around discovery that's very deterministic. Do we actually need to go through AI for this kind of stuff?"

"Straight-up answer," Ambrozic replies, "No, you don't. And that's the great part of doing this right. One of the things that we have is what we call a traffic cop." This feature is designed to recognize when a search request "is not conversational in nature, and doesn't require an LLM and a full dialogue versus something that does. Right now, what we're seeing is about an 85% to 15% split there. So, 85% of the time people are looking for something very specific or they're just doing channel up, channel down. And then 15% of the time they're being conversational in nature, and that's the moment. So if you can get that right, you can manage your token spend, and it becomes about understanding when to use GenAI in a smart way and when not to."

This triage or traffic cop approach to evaluating and directing requests, he continues, "leads into a whole other dimension of understanding," involving "how long people are willing to wait for the results. When I say 'volume down,' I don't want to wait three seconds for an LLM to figure out, 'Oh, he meant turn the volume down.' That's got to go instantaneously in a matter of single to tens of milliseconds. Of course, if I say, 'Tell me about the plot of Top Gun: Maverick,' people have become accustomed now to sitting and waiting because they know that they're exchanging more information in order to get a better result and they're willing to enter into that engagement. So you have the tools with which to control your cost spend combined with response-time spend. All of this can be balanced, but it is an ongoing balance. It requires the tools in place. It requires the rigor to stay on top of it."

"I cannot agree more with Chris," Ranjan chimes in, echoing Ambrozic's emphasis on "token conservation. I think there is a way, if it gets to these complex queries where you need an LLM, even in those contexts, you can use an LLM sometime as an intent parser, just to understand what the intent is, what the user's trying to find. And then you can use that intent to match with a vector database. And in the vector space, things are semantic, things are similar. So that's another way to fall back on if you don't want to spend a lot of tokens. This is a very practical example."

Streaming the Response #

Ranjan also acknowledges the sound UX strategy of recognizing "that people are actually accustomed to waiting right now. That's great."

But instead of counting on or potentially trying a user's patience, he suggests, "what one can do there is stream the response. What I mean by 'streaming the response' is, instead of waiting for the perfect response to come up, show the user what the LLM is thinking. Or if the LLM is producing, let's say, five titles, show the first title. That's something that we have tested it and we saw some great end-user positive response."

"That's nice," Boucaud agrees. "And it also resonates with the way that people are learning how to use AI. We are getting familiar with now having to wait. So it's fine, I guess, if you understand that there's something going on in the background."

AI and the 'Swiss Army Knife' Approach #

Continuing on the theme of applying the right search and discovery technology in the appropriate scenarios and leveraging Gen AI when it's likely to deliver the best return, Boucaud says, "I want to look a bit more at the specific use cases for which AI can really be even more of a game changer. First, maybe we can talk about the longtail content. I would like to start with Alok because you mentioned that you have a very extended catalog. Do you have best practices to share with us in terms of how metadata enrichment, and new kinds of contextual information around the title that goes beyond the more old school taxonomy can help with the discovery of new titles, especially for titles that might have been poorly categorized before when there's less metadata available?"

"You can call it the Swiss [Army] Knife approach. You have to use the right tool for the right problem. I tell my team, 'Don't kill the cockroach with a tank. That's very expensive. So please use the right tool.' We are very ROI-sensitive," he continues, which means that 80 to 85% of the time, Future Today's approach is to rely on "the traditional recommendation and the standard metadata. And in some advanced cases where people are willing to wait for a better result, that's when you bring in the advanced tool. We use mostly open-source tools for that. But in terms of data enhancement, when we get data from 500 different content partners, some of them are bigger studios, and you'll be surprised to hear that some of them don't give us good metadata. We have multiple layers of AI. We look into it multimodal. For each video, and each scene, we look into what the message is."

But often supplied or AI-generated metadata is misleading and human intervention is critical. For example, he continues, "Sometimes there will be one scene of a mother-daughter relationship and they'll [tag it as] 'mother-daughter.' I'll say, 'No, this movie is not focused on a mother-daughter [relationship]. This was just one scene. You can jump to that scene if somebody's looking for a clip in a mobile mode, but in a TV mode, they're not. So you don't want to overburden it. You have to classify and categorize in the right format. So we use AI and we use humans to understand that."

(The next step, he continues, is "to understand the intensity. It is a happy movie? How happy it is on a scale of 1 to 10? Is it inspirational? How inspirational? So we have that kind of detailed information on our catalog. Now again, we look at the audience, people will say things, but they're looking for something else sometimes. And that's also a challenge because they will say they're looking for this, but if you look at their pattern, they prefer something else. So, we mix that implicit behavior they have in the past and the current ask, and then we match that with the content. That's what leads us to the best possible outcome. Again, this is a neverending task. You keep getting better and better. And we have made a lot of progress in last couple of years with the help of AI, and I'm looking forward to doing better."

Join us November 9-11, 2026 for more thought leadership, actionable insights, and lively debate at Streaming Media Connect 2026! Registration is open!

Integrating AI Into CTV Search and Discovery

#

Timing and Token Conservation #

Streaming the Response #

AI and the 'Swiss Army Knife' Approach #

Run your AI side-project on zahid.host