The future of LLM wrappers

Estimated read time: 5 minutes

After way too long of a break, I’m back. Some of you subscribed to what was formerly known as The Prompt Report and may be confused as to why this email is in your inbox. Allow me to explain:

Last summer, I (Alex Albert) joined Anthropic as the first Prompt Engineer and Librarian (yes, that was the official job title). Last month, I switched roles and am now leading Developer Relations. 

As part of this new role, I figured it would be a good idea to get out into the AI world and go chat with people who are “in the arena” so to speak.

The AI industry has struggled with transparency, and people deserve to know what's happening inside the labs. This newsletter is my attempt to capture the industry's vibes and give an insider's perspective on what's going on and how we're feeling. 

With that, The Prompt Report is out and Alex Albert (‘s newsletter/notes/letters? …nothing?) is in. 

Hopefully the new name makes it clear that all views expressed are solely my own and do not express the views or opinions of my employer, and all that disclaimer stuff.

Now, let’s get to it…

This past week I hit the gym with my friend Rahul Sonwalker, founder and CEO of Julius.ai. 

Julius is an AI data analyst that helps you crunch and visualize your data – it's like having a personal Nate Silver three Red Bulls deep on speed dial.

In between sets of bicep curls, we talked shop about the future of knowledge work, running an “LLM-wrapper” company, and building Julius. 

Three things from our convo stuck with me and I’ve been thinking about them all week:

#1 Don’t be afraid to work on a “wrapper”.

Since ChatGPT was released at the end of 2022, there have been a boatload of startups building on top of LLMs. 

Many of these startups have been grouped into a bucket with a dreaded moniker: “LLM wrapper”. 

An "LLM wrapper" is a product that provides a chat interface for an LLM. It's a dreaded term because it implies that the LLM makers will eventually create their own version and put the wrapper companies out of business.

Rahul hears this all the time while working on Julius – and yet, it hasn’t bothered him.

Julius launched the same week OpenAI released their data analyst feature, Code Interpreter. You'd think this would have crushed Julius before it even got off the ground, but Rahul has grown it to over half a million users since then.

So why has Julius been able to succeed even while competing against a goliath? Because of focus on a specific use case and product obsession.

A general empty textbox is intimidating - most people don’t know how to use it effectively. You will find success if you can build a targeted product that just works for a specific subset of people.

The minute details make the killer product experience.

This ties in to my next point…

#2 Too many builders are overcomplicating things right now.

Many startups are making flashy statements about "training their own models." They claim this will yield an LLM so unique it will unlock product-market fit and set them apart from the rest of the startups in the valley.

I’d wager this is overkill 9 times out of 10. 

The worst affliction you could develop right now is trainitis:

I think founders often fall into this trap because of my previous point. They are trying to avoid being categorized as a “wrapper” and fear the association that they believe comes with it. You have to change this mindset if you want to win. There’s too many opportunities right now for this to be the blocker to you building something.

My advice is to just pick one of the frontier LLMs off-the-shelf and redirect all that time you would have spent training a model for marginal % improvement gains into making the product and UX better.

Rahul said something that I thought was really interesting, “If I had 10 Rahuls working full-time on Julius, I still wouldn’t have enough people to address all our pure product opportunities that we see.” 

Take a look around at which startups are getting users and making money right now. Here’s a secret: it’s not the ones who are spending all their time training a new model. 

AI today is like the web in the early days: the Marc Andreessens of the world are defining paradigms that will shape the user experience for generations. Just as Andreessen realized “huh, maybe the internet should have images”, today's builders are creating the foundational blocks for how users will interact with AI for years to come.

Don’t sit this one out because you spent too much time fiddling with the hyperparameters of the latest alpha-falcon-LM-7xb-jumbo model you found on HuggingFace. 

#3 The phrase “A rising tide lifts all boats” has never been more true.

The market for people who would use AI products is massive. In fact, I’d say it’s as large as “everyone who uses the internet.” 

What this means is that we are still early. I talk to people every day that have never heard of AI or don’t really understand how it can help them. There’s a lot of marketing that we need to do to show people all the things it can do. 

This is why Rahul isn’t all that worried about competition in the short term. 

Sure, you don’t want to cede complete market and mindshare to your competitors, but for at least the foreseeable future, marketing that’s done for any AI product increases the number of people who realize just how much AI can do for them. Some of these people will look for AI products to help them analyze their data and Rahul hopes that search points a few of them in Julius’s direction.

In a world where the pie is growing much faster than companies can cut slices out from it, those “few” people represent half a million users in Julius’s case. 

Claim your slice of the pie – it may turn out to be bigger than you would expect. But as you do, remember this:

Even in the fast-moving world of AI, the old rules still apply. Stay focused on solving real problems, obsess over the user experience not the tech, and build without fear of what people may think.

The future of AI belongs to those who don't just watch it unfold but actively shape it. Will you be one of them?

-Alex