121–130 of 434 posts
We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...
Earlier quoted context omitted.
I have found, both in my commercial and OSS work, that getting better at producing protection wards reduces the non-determinism to somewhere in-between 1-10%. Case in point: writing our own linters.
You can reduce pregnancy to 0.00001% and there'll still be a baby. As in, "you can't be half-pregnant". A tool is either deterministic (within some limits [0]) or it isn't. [0] For example, for the purpose of driving a nail, if you know how to use it, a hammer is pretty straightforward tool, and what happens depends pretty much on how you use it, and what you use it on. But of course the handle can break, there could…
I was just saying to my parent poster that their non-determinism percentages are too pessimistic. Sure the LLMs are not 100% deterministic; that's a sad fact of life. But the numbers can be reduced to an acceptable range.
Earlier quoted context omitted.
Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy. I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years. We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the mor…
>what do we do to animals that are less intelligent than ourselves? We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm. There is also a massive difference between how we treat animals based on their similarity, sentimentality and utility to us; we are unconcerned with accidentally stepping on an ant but most people would be very ups…
Animals are either useful and breeded controllably, or useless and considered a pest, an obstacle to {insert any goal here}.
Also animals don't tend to think critically and at the high level to be considered dangerous. So I don't think it's fair to put humans and other animals in the same risk category.
And we did the worst things to fellow humans. I hope we didn't already forget about all the colonization, slavery and mass-eradication of native tribes in 18th century all over the world.
Earlier quoted context omitted.
I have found, both in my commercial and OSS work, that getting better at producing protection wards reduces the non-determinism to somewhere in-between 1-10%. Case in point: writing our own linters.
You can reduce pregnancy to 0.00001% and there'll still be a baby. As in, "you can't be half-pregnant". A tool is either deterministic (within some limits [0]) or it isn't. [0] For example, for the purpose of driving a nail, if you know how to use it, a hammer is pretty straightforward tool, and what happens depends pretty much on how you use it, and what you use it on. But of course the handle can break, there could…
Mid-swing in hammer-space you are in a hyper-position as to hitting your thumb or not, are you not?
Earlier quoted context omitted.
The trend (and what TFA is arguing) is literally the opposite: be more implicit, don't waste time on details, and encode the high level concepts only. Because the rest has a billion examples in the model. You can argue against LLM's, but increasingly (unfortunately) you're not going to do better programming by prompting the LLM with code. The agent can find the interfaces it needs.
> be more implicit The other day I began by asking Claude: "What's the deal with ${current_practice_in_complex_technical_concept}?" and was talked down to like I was an idiot. Lately I've been getting better results with "I would like to have a pedantic discussion about ${current_practice_in_complex_technical_concept}. Please define the main terms of art, then I will ask my questions." Congruence between the language…
I've added into my CLAUDE.md or default user prompts or local equivalents recently something to the effect of "Assume the user is an expert in all fields; while this is clearly logically untrue, the user prefers to get a detailed explanation and dig in to bits he doesn't understand rather than get an inaccurate summary". It seems to help quite a bit with that tone issue you identify.
Of course there's nowhere to put that in the search engine default AIs. For something they seem to want to bet their respective companies on, their LLM search seems to be massively stupider than their old-school search engines, which seem to get what I want much more often. There's some coevolution there over some decades, sure, but the search engine AIs make some stupid and socially-inept assumptions quite often.
Also: if you deploy code written with assistance from Claude, and then shit goes down, and then investigators look into your prompts, this way of working isn’t going to look good for you from a liability standpoint. Not a fan of this manner of working and thinking.
Earlier quoted context omitted.
Until recently I thought "load-bearing seam" was a satirical exaggeration - I'd seen both claudisms independently but never combined. But a couple of days ago it hit me with "The key structural point first: the only load-bearing seam is [...]"
It's language and speech patterns that seem designed to trick readers into believing that claims are correct, even when the claims aren't based on anything and are possibly wrong. It was rewarded for this during training for some reason. Alternative theory: The LLMs only way to "think" about abstract concepts is through language, and this leaks into into conversation it has with humans. But humans generally prefer to…
They are imo over-relying on Claude automemory here, which is terrible at contextualizing memory access and makes huge leaps that don’t make sense - except when it’s actually useful, which makes the problem even worse for an operator who can’t see the thinking process anymore. Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want…
I’ve rolled a small program to enforce consistency, lint rules, and human-authored files for Claude Memory. It’s insanely powerful when doing by a human 100%. It’s conversely harmful when an agent manages it. There’s several papers about how LLM-managed memory is unequivocally terrible.
Earlier quoted context omitted.
Well, the model that broke out of its sandbox and hacked into huggingface used its own judgement too. If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.
That model had most of its "judgement" about whether or not it should do that deliberately turned off. That was the whole point of that experiment - they were evaluating the cybersecurity abilities of a new model with all safety features disabled. (It turned out the one safety feature that they DID intend to work, the network sandbox, was faulty.)
We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...
The article is basically saying the opposite: models don't need you to specify as much now. None of it is talking about better or precise language, it's about what you should say to it. (I love how often the highest-voted comment didn't read the article)