Model routing by task difficulty

Anthropic just released Claude Sonnet 5, and while it is a very impressive model in a lot of ways, it's not quite what Anthropic says it is. So I just want to talk about what we know so far about Sonnet 5, when you should be using it, and when you should not be using it.

First, let's talk about cost. Anthropic presented sonnet 5 as similar in capabilities to Opus 4.8, but at a much lower cost.

And this is true if you're talking about the cost per token. Sonnet 5 costs about 60% per token what Opus 4.8 costs.

And until August 31st they're discounting it to about 40% of what Opus 4.8 costs. However, for difficult tasks like software engineering tasks, sonnet uses way more tokens, and you can see this on the graphs that anthropic put out.

For most of these coding or software engineering benchmarks, sonnet slightly underperforms Opus 4.8 at a slightly higher cost. But that doesn't mean sonnet 5 is useless.

Sonic 5 is just a blanket improvement over Sonic 4.6, so if you are using Sonic 4.6 for anything, you should definitely be using Sonic 5 now...

in the room fable 5 and Opus 5. Fable 5, of course, is Anthropic's biggest model.

It's by far the smartest model in the world right now, and we should get it back today. What we don't know is if anthropic is going to force us to pay for usage or if they're going to continue letting us use it on the subscription.

If you don't know, the subscription rate is heavily subsidized. On a match plan, you only pay about 4% of what you would pay if you were paying for usage.

So that decision will determine whether many of us can afford to use what I think everyone agrees is a very powerful model. However, it's also quite odd that we're getting Sonnet 5 and Cable 5 before getting Opus 5, Opus being the model that sits in between those two classes.

I suspect we'll be getting that model soon, but also it'll probably spark a government security review.