Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4.6

anthropic.com

Post reply on HN
Print view View original

941–950 of 1001 posts

Re: Claude Opus 4.6

#941

Post by docjay »

insomagent wrote: post #851

I'm not super impressed with the performance, actually. I'm finding that it misunderstands me quite a bit. While it is definitely better at reading big codebases and finding a needle in a haystack, it's nowhere near as good as Opus 4.5 at reading between the lines and figuring out what I really want it to do, even with a pretty well defined issue. It also has a habit of "running wild". If I say "first, verify you und…

You might benefit from a different mental approach to prompting, and models in general. Also, be careful what you wish for because the closer they get to humans the worse they’ll be. You can’t have “far beyond the realm of human capabilities” and “just like Gary” in the same box.

They can chain events together as a sequence, but they don’t have temporal coherence. For those that are born with dimensional privilege “Do X, discuss, then do Y” implies time passing between events, but to a model it’s all a singular event at t=0. The system pressed “3 +” on a calculator and your input presses a number and “=“. If you see the silliness in telling it “BRB” then you’ll see the silliness in foreshadowing ill-defined temporal steps. If it CAN happen in a single response then it very well might happen.

“

Agenda for today at 12pm:

1. Read junk.py

2. Talk about it for 20 minutes

3. Eat lunch for an hour

4. Decide on deleting junk.py

“

12:00 - I just read junk.py.

12:00-12:20 - Oh wow it looks like junk, that’s for sure.

12:20-1:20 - I’m eating lunch now. Yum.

1:20 - I’ve decided to delete it, as you instructed. {delete junk.py}

Because of course, right? What does “talk about it” mean beyond “put some tokens here too”?

If you want it to stop reliably you have to make it output tokens whose next most probable token is EOS (end). Meaning you need it to say what you want, then say something else where the next most probable token after it is .

I’ve tested well over 1,000 prompts on Opus 4.0-4.5 for the exact issue you’re experiencing. The test criteria was having it read a Python file that desperately needs a hero, but without having it immediately volunteer as tribute and run off chasing a squirrel() into the woods.

With thinking enabled the temperature is 1.0, so randomness is maximized, and that makes it easy to find something that always sometimes works unless it doesn’t. “Read X and describe what you see.” - That worked very well with Opus 4.0. Not “tell me what you see”, “explain it”, “describe it”, “then stop”, “then end your response”, or any of hundreds of others. “Describe what you see” worked particularly well at aligning read file->word tokens->EOS… in 176/200 repetitions of the exact same prompt.

What worked 200/200 on all models and all generations? “Read X then halt for further instructions.” The reason that works has nothing to do with the model excitedly waiting for my next utterance, but rather that the typical response tokens for that step are “Awaiting instructions.” and the next most probable token after that is: nothing. EOS.

Re: Claude Opus 4.6

#942

Post by KaiserPro »

simonw wrote: post #589
KaiserPro wrote: post #515

Earlier quoted context omitted.

sorry A lack of context: https://ollama.com/library/gemini-3-pro-preview You can run it on your own infra. Anthropic and openAI are running off nvidia, so are meta(well supposedly they had custom silicon, I'm not sure if its capable of running big models) and mistral. however if google really are running their own inference hardware, then that means the cost is different (developing silicon is not cheap...) as you sa…

You can't run Gemini 3 Pro Preview on your own infrastructure. Ollama sell access to cloud models these days. It's a little weird and confusing.

Ahh fuck, thanks for pointing that out.

I did think its a bit weird that they had open-weighted it

Re: Claude Opus 4.6

#943

Post by gertlex »

matchagaucho wrote: post #938
setgree wrote: post #895

I asked > Can you find an academic article that _looks_ legitimate -- looks like a real journal, by researchers with what look like real academic affiliations, has been cited hundreds or thousands of times -- but is obviously nonsense, e.g. has glaring typos in the abstract, is clearly garbled or nonsensical? It pointed me to a bunch of hoaxes. I clarified: > no, I'm not looking for a hoax, or a deliberate comment on…

Results from a one-shot approach quickly converge on the default “none found” outcome when reasoning isn’t grounded in a paper corpus via proper RAG tooling.

Can you provide more context to your statement? Are you talking about models in general? Or specific recent models? I'm assuming "one-shot approach" is how you classify the parent comment's question (and subsequent refined versions of it).

Re: Claude Opus 4.6

#944

Post by esafak »

setgree wrote: post #895

I asked > Can you find an academic article that _looks_ legitimate -- looks like a real journal, by researchers with what look like real academic affiliations, has been cited hundreds or thousands of times -- but is obviously nonsense, e.g. has glaring typos in the abstract, is clearly garbled or nonsensical? It pointed me to a bunch of hoaxes. I clarified: > no, I'm not looking for a hoax, or a deliberate comment on…

It would have to have been trained on the papers without being aware of retractions for that test to work. Otherwise it will be limited to whatever papers it gets from a search engine query, which likely won't contain any un-retracted illegitimate papers.

Re: Claude Opus 4.6

#945

Post by setgree »

esafak wrote: post #944
setgree wrote: post #895

I asked > Can you find an academic article that _looks_ legitimate -- looks like a real journal, by researchers with what look like real academic affiliations, has been cited hundreds or thousands of times -- but is obviously nonsense, e.g. has glaring typos in the abstract, is clearly garbled or nonsensical? It pointed me to a bunch of hoaxes. I clarified: > no, I'm not looking for a hoax, or a deliberate comment on…

It would have to have been trained on the papers without being aware of retractions for that test to work. Otherwise it will be limited to whatever papers it gets from a search engine query, which likely won't contain any un-retracted illegitimate papers.

I once worked at a crypto company that outsourced some of its marketing work to a content marketing firm. A piece that firm submitted to us contained a link to an "academic" article about global poverty with a totally garbled abstract and absolutely no content whatsoever. I don't know how they found it, because when I search google scholar for a subject, usually the things that come back aren't so blatantly FUBAR. I was hoping Claude could help me find something like that for a point I was making in a blogpost about BS in scientific literature (https://regressiontothemeat.substack.com/p/how-i-read-studie...).

The articles it provided where the AI prompts were left in the text were definitely in the right ballpark, although I do wonder if chatbots mean, going forward, we'll see fewer errors in the "WTF are you even talking about" category which, I must say, were typically funnier and more interesting than just the generic blather of "what a great point. It's not X -- it's Y."

Re: Claude Opus 4.6

#946

Post by codexon »

netdevphoenix wrote: post #839
codexon wrote: post #651

Earlier quoted context omitted.

Even for coding, it seems to still make A LOT of mistakes. https://youtu.be/8brENzmq1pE?t=1544 I feel like everyone is counting chickens before they hatch here with all the doomsday predictions and extrapolating LLM capability into infinity. People that seem to overhype this seem to either be non-technical or are just making landing pages.

Waiting until the moment they get good enough is not a smart thing to do either. If you are a farmer and know it is going to snow, at some point in the next 5 months, you make plans NOW, you don't wait until the temperatures drop and you see the snow falling. Right now, people are waiting for the snowfall before moving their proverbial chickens indoors

Top AI researchers like Yann LeCunn have said that LLMs are a dead end.

It seems to me that LLM performance is plateuing and not improving exponentially anymore. This recent hubbub about rewriting a worse GCC for $20,000 is another example of overhype and regurgitating training data.

You don't know for sure if it is going to "snow" (AI reaches general intelligence) Snow happens frequently, AI reaching general intelligence has never happened. If it ever happens, 99% of jobs are gone and there is really nothing you can do to prepare for this other than maybe buy guns and ammo, and even that might not do anything to robotic soldiers.

People were worried about AI taking their jobs 60 years ago when perceptrons came out, and anyone who avoided a tech career because of that back then would have lost out majorly.

Re: Claude Opus 4.6

#947

Post by sumtechguy »

nerdsniper wrote: post #286
om8 wrote: post #151

Earlier quoted context omitted.

Is there a way to disable it? Sometimes I value agent not having knowledge that it needs to cut corners

90-98% of the time I want the LLM to only have the knowledge I gave it in the prompt. I'm actually kind of scared that I'll wake up one day and the web interface for ChatGPT/Opus/Gemini will pull information from my prior chats.

Had chatgpt reference 3 prior chats a few days ago. So if you are looking for a total reset of context you probably would need to do a small bit of work.

Re: Claude Opus 4.6

#948

Post by terminalshort »

cgh wrote: post #940
terminalshort wrote: post #933

Earlier quoted context omitted.

And there are: https://en.wikipedia.org/wiki/Sokal_affair

> no, I'm not looking for a hoax, or a deliberate comment on the situation. I'm looking for something that drives home the point that a lot of academic papers that look legit are actually meaningless but, as far as we can tell, are sincere The Sokal paper was a hoax so it doesn’t meet the criteria.

The fact that it got published means there is at least one whole journal full of that

Re: Claude Opus 4.6

#949

Post by bazmattaz »

mFixman wrote: post #28

I found that "Agentic Search" is generally useless in most LLMs since sites with useful data tend to block AI models. The answer to "when is it cheaper to buy two singles rather than one return between Cambridge to London?" is available in sites such as BRFares, but no LLM can scrape it so it just makes up a generic useless answer.

My guess is that this is going to be the future for LLMs too. It will get harder or more expensive for AI companies to train their models on the latest information as most sites will block the scrapers or ask for a fee.

There might be a future where you’ll have to pay more for an up to date model vs a legacy (out of date) model

Re: Claude Opus 4.6

#950

Post by jpcompartir »

4.6 is a beast.

Everything in plan mode first + AskUserQuestionTool, review all plans, get it to write its own CLAUDE.md for coding standards and edit where necessary and away you go.

Seems noticeably better than 4.5 at keeping the codebase slim. Obviously it still needs to be kept an eye on, but it's a step up from 4.5.

Post reply on HN