Just tested through openrouter.. gave exactly same task.. the task was to scan existing repo, and generate a single docker-compose file to deploy behind a caddy server, where certain port ranges are already used, the service demands widlcard certificates to be provisioned from outside, and postgre needs to be built-in one... Tested this model, and gpt-5.6-terra-high. Results: this one had few issues. terra: none. The…
DeepSeek V4 Pro 0813
231–240 of 493 posts
Re: DeepSeek V4 Pro 0813
#232Earlier quoted context omitted.
you're thinking of one shot. if they are running an agentic loop then they don't need multiple passes. an agentic loop is multiple passes with tool calls and tools could fail and agent would correct from seeing the failure. a bad model will compound on error and fail, a good model will correct. 1 test is fine to gauge the quality of the model.
An agent doing a task even with multiple back to back calls like normal without an example is zero shot. An agent doing a task with 1 example is one shot. An agent doing a task with a few examples is few shot. I don't think you are correctly using these terms. The multiple back to back LLM calls are done on accumulating context, so if there is a sampling error it could throw the entire session out of whack, because L…
I don't disagree that multiple tests increase confidence, but it's not correct to argue that an agent in a loop harness is equivalent to oneshotting
Re: DeepSeek V4 Pro 0813
#233Just tested through openrouter.. gave exactly same task.. the task was to scan existing repo, and generate a single docker-compose file to deploy behind a caddy server, where certain port ranges are already used, the service demands widlcard certificates to be provisioned from outside, and postgre needs to be built-in one... Tested this model, and gpt-5.6-terra-high. Results: this one had few issues. terra: none. The…
Wait people use terra?
Re: DeepSeek V4 Pro 0813
#234Nice bicycle chain, the little basket with a fish didn't show up in the right place: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
Re: DeepSeek V4 Pro 0813
#235Earlier quoted context omitted.
Depends on project complexity. For one of my more complex projects, I exclusively use sol-high ... nothing below that works correctly. For this however, a comparatively much simpler task, tarra-high works fine.
Right now, sol-xhigh is my favorite model. I feel that Opus 5 is dumber than 4.8. Fable is too expensive to do anything (limit of $50, started a prompt at $25, ended up at $75, is bullshit, but at least it's "free credits"). DeepSeek is okay for random API-based stuff, as it's cheap. Local open models running on a 5090 are hit or miss. I feel that most GGUFs/quants are awful...
Re: DeepSeek V4 Pro 0813
#236Earlier quoted context omitted.
With Pi as a harness I've been using OpenAI models as a worker with an Opus 5 (in Claude Code) planner. I've only had a few issues with Terra High/Medium and absolutely none with Sol Medium+ on a fairly complex Rust project that targets Linux, Mac, Windows and Web, with plenty of nasty FFI, VMs, remotely debugging systems, among some other things within a monorepo. I think the key is to give them a nice assortment of…
Plan with sol-med, implement with luna-high. Rarely a problem.
Re: DeepSeek V4 Pro 0813
#237Earlier quoted context omitted.
I do think this question comes up a lot-- I can understand why. For some well-explained reasons, check out https://darioamodei.com/essay/the-adolescence-of-technology and search for "CCP".
As somebody from neither the US nor China, this argument would be much stronger if the US hadn't started acting like a rogue state - starting wars of aggression and messing up the world's energy supply, actively speeding up climate change, kidnapping leaders of sovereign nations, threatening its allies (!) with invasion, etc etc. The CCP's not great either, sure. But the Americans don't really have a leg to stand on…
Re: DeepSeek V4 Pro 0813
#238Earlier quoted context omitted.
Terra has not been able to do any of the technical tasks I've asked of it correctly. I'm surprised others get use out of it. Anything below Sol high tends to give me mostly unreliable results. I'm using codex as my main harness but maybe it performs better with a different one.
With Pi as a harness I've been using OpenAI models as a worker with an Opus 5 (in Claude Code) planner. I've only had a few issues with Terra High/Medium and absolutely none with Sol Medium+ on a fairly complex Rust project that targets Linux, Mac, Windows and Web, with plenty of nasty FFI, VMs, remotely debugging systems, among some other things within a monorepo. I think the key is to give them a nice assortment of…
Re: DeepSeek V4 Pro 0813
#239Earlier quoted context omitted.
Wait people use terra?
I use mostly Terra. Much better than Opus 5. Much more token mileage.
Re: DeepSeek V4 Pro 0813
#240Earlier quoted context omitted.
I do think this question comes up a lot-- I can understand why. For some well-explained reasons, check out https://darioamodei.com/essay/the-adolescence-of-technology and search for "CCP".
So is your concern more about reducing the risk of an authoritarian China "winning" the AI race? And less about reducing the risk of your data being used against you personally? To me, the risks of an individual helping China to continue to develop their AI by being a customer is pretty marginal compared with the personal risks of my data being used against me.