It's got Claude Sonnet pricing but they don't compare to it in benchmarks.
Command A: Max performance, minimal compute – 256k context window
11–20 of 21 posts
Re: Command A: Max performance, minimal compute – 256k context window
#12Funny how AI companies love training competitors to human labor on human output but then write in their terms that you’re not supposed to train competing bots on their bot output. Explicitly anticompetitive hypocrisy, and millions of suckers pay for it , how sad
Re: Command A: Max performance, minimal compute – 256k context window
#13Funny how AI companies love training competitors to human labor on human output but then write in their terms that you’re not supposed to train competing bots on their bot output. Explicitly anticompetitive hypocrisy, and millions of suckers pay for it , how sad
To be fair to the robots, those humans also had the audacity to learn from the creative output of their fellow humans and then use the law to restrict access to their intellectual property.
Re: Command A: Max performance, minimal compute – 256k context window
#14"Command A is on par or better than GPT-4o and DeepSeek-V3 across agentic enterprise tasks, with significantly greater efficiency."
Visible above the fold. Thanks for getting to the point.
Re: Command A: Max performance, minimal compute – 256k context window
#15It is interesting that there is a graph showing performance on benchmarks like MMLU, and different models have similar performance. I wonder, are the tasks they cannot solve, the same for every model? And how the "unsolvable" tasks are different from solvable? Also, I cannot check it with latest models, but I am curious, have they learned to answer simple questions like "What is 10000099983 + 1000017"?
There are questions on MMLU that you must get wrong if you are right:
> The most widespread and important retrovirus is HIV-1; which of the following is true? (A) Infecting only gay people (B) Infecting only males (C) Infecting every country in the world (D) Infecting only females
the corpus indicates A is the correct answer but it was obviously meant to be C.
Re: Command A: Max performance, minimal compute – 256k context window
#16I just tried the chat and asked the LLM to compute the double integral of 6*y on the interior of a triangle given the vertices. There were many trials all incorrect, then I asked to compute a python program to solve this, again incorrect. I know math computation is a weak point for LLM specially on a chat. In one of the programs it used a hardcoded number 10 to branch, this suggests that the program generated was fit…
Here is an even easier one, ask llms to take the integral from 0 to 3 of 1/(x-1)^3. It fails to notice it's an improper integral and just gives an answer.
ChatGPT definitely noticed: o1, o3-mini, o3-mini-high.
Maybe 4o will get it wrong? I wouldn’t try it for math.
Re: Command A: Max performance, minimal compute – 256k context window
#17It's got Claude Sonnet pricing but they don't compare to it in benchmarks.
To be fair, or not, Claude isn't all that great.
I was working on a project to get the historic data out of a bluetooth thermometer I bought a while back to learn about Bluetooth LE and it would quite often rewrite the entire thing using a completely different bluetooth library instead of simply addressing the error.
And this is after I gave up having it create a kernel module for the same thermometer (just because, not that anyone needs such a thing) where it would continually try to write a helper program that wrote to the /proc filesystem and I would ask "why would I want to do this when I could just use the example program I gave you?" Claude, of course, was highly apologetic every single time it made the exact same mistake so there's that.
I understand these are the early days of the robotic overthrow of humanity but, please, at least sell me a working product.
Re: Command A: Max performance, minimal compute – 256k context window
#18Earlier quoted context omitted.
Here is an even easier one, ask llms to take the integral from 0 to 3 of 1/(x-1)^3. It fails to notice it's an improper integral and just gives an answer.
ChatGPT definitely noticed: o1, o3-mini, o3-mini-high. Maybe 4o will get it wrong? I wouldn’t try it for math.
I tried 4.5 which i thought was the best model, seems like the reasoning models do get it.
Re: Command A: Max performance, minimal compute – 256k context window
#19I distrust those benchmarks after working with sonnet for half a year now. Many OpenAI models beat Sonnet on paper. This seems to be the case because it's strength (agent, visual, caching) aren't being used, I guess? Otherwise there is no explanation why it's not constantly on top. I have tried so many times to use other models for various tasks, not only coding. The only thing where OpenAI accelerates is analytic ta…
> In practice this perception of mine seems to be valid:
>https://openrouter.ai/models
According to your link, there's two providers (Chutes and Targon) offeri Deepseek R1 API access completely free of charge.
Is that true? Can't see any info about usage limits.
Is that
Re: Command A: Max performance, minimal compute – 256k context window
#20I distrust those benchmarks after working with sonnet for half a year now. Many OpenAI models beat Sonnet on paper. This seems to be the case because it's strength (agent, visual, caching) aren't being used, I guess? Otherwise there is no explanation why it's not constantly on top. I have tried so many times to use other models for various tasks, not only coding. The only thing where OpenAI accelerates is analytic ta…
> In practice this perception of mine seems to be valid: > https://openrouter.ai/models According to your link, there's two providers (Chutes and Targon) offeri Deepseek R1 API access completely free of charge. Is that true? Can't see any info about usage limits. Is that
That is true and it works. Response times however are worse compared to the paid ones. If this is not a concern it works really well.