Background & How Agents are a more superior user interface #
Until recently, the concept of AI agents has been so confusing to me. While I use AI agents everyday to build software, run marketing campaigns and everything in between, it has taken me a while internalise their inner workings sufficiently to build my own agent. Every use case I’d come up with would simply scratched out with the thought “I can already do that with X tool”. So I found it hard to build anything because in my mind there were already ways to get the answers I wanted. In a lot of cases, much easier and quicker ways. Things started to make sense when I thought about agents simply as user interfaces; To arrive at that I first needed to remind myself why user interfaces exist and the role they play in transferring information. The goal of a user interface is to answer the various questions and intents that exist in a user’s mind and those questions fall in one of two buckets.
- Direct questions - Straight questions that have straight answers e.g. “How many sales did we make yesterday?”, “How much stock of this SKU do we have left?”. Straight questions that have straight answers.
- Multi-step questions - Questions that require a bit of reasoning and comparison of multiple data points e.g “Compare this months expenses and last months income.”, “How much more do I need to save each week to meet my savings goal given my income?”.
To answer multi-step questions using graphical user interfaces(GUIs) e.g. dashboards, users have to combine data points they see on a screen and based on that, compose and derive answers on their own. That last step is creates mental friction and that’s where AI agents start to make sense because they remove that friction and instead give you straight answers. So instead of shuffling between charts and tables in a dashboard to find out how much you need to save this week to hit your targets, you just tell the agent what you want and it gives you an answer right back. By doing so agents become the more efficient user interface.
The mind frame for building AI agents as a user interface. #
From my learning thus far, agents in their most basic state are just loops over tool equipped LLM calls where the exit condition is determined but the state of LLM response. Building on that, in the same way GUI’s are only as good as the tools(APIs) hooked up to them so are agents. However, the difference when building for the two is that with GUIs you typically need know in advance the questions the user will be asking of the GUI, following which you hook up the appropriate tools that present the answers for each of the earlier presumed questions. In contrast, when building for agents all you need are the foundational tools of a problem space and using those the agent will sufficiently answer most of the questions in that space. With agents you need not know all the questions in advance. What we’re saying when building this loop, this agent, is that “I am handing you these tools with the premise that if you loop over them enough and in the right order, you will be able to answer any question thrown at you”. That thinking is what I found very difficult to arrive at because until now, all of the software I’ve built has had predetermined paths. Everything on the screen I had to preconfigure and pre-map. I had to answer the user’s questions in advance. This new concept of agentic UIs flips that. It requires that I handover agency to the loop. Something I hadn’t done up until now.
What’s next? #
Since I am now a man with a hammer, the obvious next step is to make everything a nail. I shall proceed to try and make an agent out of any and everything I come across. Manually ofcourse. This should help internalise the concepts more before I automate them all away with Claude. First thing I’m building is a personal finance agent. I’ve already started working on this and it’s been the test project for this learningl exercise but I want redo it now as a proper project. It’s already interesting because there are atleast two agents in that project; the chat agent which users interface with and then the data enrichment agent which does just that. I’m pretty excited for it!
What are your mental frames when building agents? Is building them something you’ve also struggled with? What are some gotchas and pointers I should look out for? I’d love to hear your thoughts.