There isn't, as I wasn't going for strictness, more like a playful challenge in the vein of Simon's SVG pelican.
Between the two, Opus 4.8 seems more capable. But, I suspect the harness plays a large role here. It's possible the result would be as good if Codex ran 10+ agents and spent an hour on it.
OpenAI and Anthropic usually fast-follow each other, so I wouldn't be surprised if Codex got the same capability in a couple of days (and even an update to the model), then it'll be a better test.
Absolutely! We need new and better benchmarks like this.
I have a question: why not use the maximum available reasoning on each LLM? For example, I see that Opus 4.7 at `max` reasoning but Sonnet 4.6 at `high`. Wouldn't it be a fairer comparison if all were at max?
- Opus 4.7 writes the code
- I make GPT-5.5 in Codex to review it (given context)
- I provide the review back to Opus and ask it to verify the review findings
- Make Opus plan the fixes then execute them
- Ask GPT-5.5 to review the fixes and check if they solve the problems
I whipped up a quick uBO rule to fix that (also makes meta-information lines readable):
thefrontpage.dev##p.newspaper-copy:style(line-height: normal !important; font-size: 1rem !important;)
thefrontpage.dev##p.article-meta:style(font-size: 1rem !important; font-weight: normal !important; letter-spacing: normal !important;)
I agree, but I think it's that small because otherwise, the justified text results in ridiculous spacing.
OP, consider reducing the number of columns from 4 to 3 (at least below very wide viewports), increasing the font size, and then also allowing hyphenation. I think the last will help a lot with the justification problem.
you can click and drag to select part of the audio (and then drag the edges of the selection region that has appeared if you want to adjust it), and then apply the effect. All effects prioritize the current selection first, and if no selection is present then get applied on the entire track.
I'm working on a custom launcher for hooking up Claude Code with various providers (groups env variables in profiles) cause DeepSeek doesn't have vision and sometimes I need browser use with screenshots or Opus reasoning, for other tasks it's fine: https://ccode.kronis.dev/
# After installed (or when run portably with ./ccode)
ccode init-config
ccode edit-config
# Run with default profile
ccode
# Run with named profile
ccode --deepseek
# Set default profile
ccode set-default-profile deepseek
Also turns out that with a local proxy you can get Remote Control working and see the DeepSeek sessions in the desktop app, screenshots on the page. Other than that, I'm happy that it works pretty well and the discount is enough to make me consider going from Anthropic's Max subscription to Pro and using it only where DeepSeek is insufficient. With that proxy I eventually hope to be able to transparently switch models mid-task, if I need Opus for like 5 turns or something.
Overall though I'm not sure exactly how well Claude Code would stack up against OpenCode, since the latter overall feels a bit less hacky with 3rd party models and is even getting niche but nice features like a locally runnable web version: https://opencode.ai/docs/web/
How does the cost compare using the API vs the $20/month plans with other providers?
I did some back of the envelope calculations and it seems like you would pay $5/month using DeepSeek directly or $15-20 with OpenRouter or similar. But would be interested to hear real world usage.
But as usual, there are far cheaper subscriptions with higher limits than Anthropic and OpenAI, that also provide DeepSeek v4 Pro. So you should use those subscriptions first until you max them out, then look at a different subscription.
I don’t even use Claude that much and was hitting limits in the 20$ using sonnet, I’ve deposited 5$ with deepseek and haven’t hit the limit after spending 60million+ tokens. So no way it’s more expensive.
I've been using it pretty extensively over a month and I'm at maybe $7. It thinks for quite a while, but the results have been better than Sonnet for me.
I'm not curious what tasks you tested it for. Im working on coding agent writing code dynamically on request for customers. i'd say code itself very simple and aggressively cached, and patternalized, e.g. we adding lots of hints to the system.
the only real family models that work were claude and openai, surprisingly, for tasks that needs faster speed, gpt 5.4 is very impressive. Deep seek was very average , doing things somewhere in gemini flash 3.0 domain.
I am curious - Is there a way to switch between models depending on the task? Because I believe Deepseek V4 is not multimodal and it will be good to switch back to Claude if vision or other capabilities are required.
I was looking into something similar because I wanted to test a local model for doing basic coding and smart model (deepseek) for planning.
It's basically not possible with claude code, the api endpoint is a single environment variable and whatever models are on that endpoint are what's available.
HOWEVER, if you run a proxy like LiteLLM, you can configure it to send requests to different api endpoints on the back end and expose them as different "models" on the front end, then configure claude code to switch between those virtual models.
Check out the project called superpowers. It can use different models for different agents. I use it witb opencode to have different models for reaearch, planning, execution, testing etc
i've been trying that, in reality every time you try to save it, it's not worth it, the cost of mistake is so high , you can spent 2-3h on just wrong assumption, you lost your time and all the burned tokens.
At this point in the AI wars, it is probably better to have more users of Claude code rather than restrict which LLMs it can connect to. Claude code is probably (currently at least) stickier than the LLM model itself. Getting people into the Claude code ecosystem is worth it.
Later, they can always lock it down more or add Claude LLM only features to it.
I thought so, and then I tried Opencode and Codex and started to appreciate Claude Code a lot more. They've actually done great work with the small details.
I actually have't looked back since trying opencode
The ability to properly see what the agent is doing in tool calls and subagents is really unmatched, CC strips all reasoning and return values, only displaying tool calls, and you're unable to expand a single subagent, it's expand everything and scroll endlessly or show everything collapsed with basically no info at all (read x files, ran x commands)
Just seems like extremely basic features are missing
You can check my profile for which one I like most :) I do think there have been efforts to benchmark different harnesses.
Personally I'm not going to choose one harness or another based on +/- a few percentage points in a benchmark. I'm going to use one the one that I find the most ergonomic, that isn't too bloated, etc. The models are the primary lever, not the harness.
IMHO the ergonomics of their tooling are not great. I'd rather use Codex or even OpenCode.
Configuration alone is very arcane with lacking documentation. Sandboxing/permission system is quite confusing too.
It went the other way, you can't use other harnesses to connect to the cheaper versions of Claude. So clearly they think their current moat is Claude Code use, not the LLM itself.
That's interesting. I thought Claude Code is not as good, therefore people want to use Claude model with other alternatives. This is the other way around.
Which begs the question, regardless of the model, which Claude Code alternative is better? (I keep saying "Claude Code alternative" because I don't know the term... LLM CLI?)
AFAIK the two most popular open source harnesses right now are OpenCode and Pi. They take a pretty different approach, OpenCode includes a lot of features while Pi is very minimal by design and focused on extensibility, to the point where many people are just asking Pi to write a plugin for itself whenever they want it to have a new feature. I personally like Pi's philosophy more and I think its developer justified the choices really well in his blog post:
Oh damn, I haven't noticed because my browser removes the referer header. But I think the image on the block page is a pretty good answer to why he did that.
The image shows Garry Tan, the CEO of Y Combinator. He has lately been on a huge AI psychosis streak, bragging about things like "shipping 37000 lines of code every day" and "using Claude Code so much it burned out his USB-C power connectors". He's in a lobster suit because he's talking about OpenClaw, an AI agent assistant which those same AI psychosis types lean into too much by giving it full read-write access to all their life and then getting surprised when it accidentally deletes all of their emails.
Pi's developer is obviously not anti-AI, and he definitely doesn't hate OpenClaw, since it's based on Pi. But there's a growing number of people who take those things too far, and a lot of them are on HN. You can easily find them in the comments of any AI-related post here. I assume that's the type of people the image is portraying.
My "trick" was to divide things into batches (which can be big with LLMs with larger context sizes) and classify the items in each batch, then take the resulting categories from each batch and feed them into an LLM to group semantically similar categories into groups with a representative category for each group. The representative category can be chosen from the group or created by the LLM. This is an over-simplification of the process but that's the gist of it.
Language support is not mentioned in the repo.
But from the paper, it offers extensive multilingual support (nearly 100 languages) which is good, but I need to test it to see how it compares to Gemini and Mistral OCR.
reply