I created Preseason.ai because I was curious about how much power current LLMs have in recommending developer tools to people vibe coding their own web apps.
Vibe coding is obviously in a crazy growth phase, with some companies going from zero to hundreds of millions in revenue in less than a year. And with that, they are also pushing along a set of specific dev tool companies with a similar trajectory.
So I wanted to track this and built an open-source benchmark for it. Every day, Preseason sends prompts to the top open and closed-source LLMs from the main labs and measures which developer tools they recommend.
For example:
- What database do they recommend?
- What authentication provider?
- What UI framework?
- What email provider?
- What search provider?
- LLM Observability
- And so on.
It has been very interesting already.
One example is email.
As you can see below, there is still a lot of competition in this category, with SendGrid and other legacy providers still showing up strongly.
But you can also clearly see Resend as a modern startup that is very well designed, has great documentation, and seems to be growing very fast.
I would not be surprised if a meaningful part of that growth is now coming from LLM recommendations.
Imagine someone starting a new SaaS for a small business right now. They need to send emails to their customers. A lot of the time, the LLM is probably just recommending Resend and sending them a new customer.
That is pretty cool.
Another example is databases.
Another example is search.
Another example is LLM observability.
The project is open source, so please feel free to open a PR if you have a good idea for how to make the measurement more robust. I will keep tracking this as I go and post updates from time to time.