Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic publishes the 'system prompts' that make Claude tick

techcrunch.com

Post reply on HN
Print view View original

271–280 of 290 posts

Re: Anthropic publishes the 'system prompts' that make Claude tick

#271

Post by willy_k »

adastra22 wrote: post #270
willy_k wrote: post #268

Earlier quoted context omitted.

You have to answer that question for any model of the universe - what came before the Big Bang? Another universe? What before it? And intelligent design is essentially analogous to simulation theory, and answers more questions than it creates (the anthropic principle, for starters). My personal mental model is that the ‘intelligence’ guides quantum collapse and so the progression of the universe is somewhat determini…

Evolution doesn’t require an original designer. It is itself a means of lifting design from disorder. I think you are very confused about quantum mechanics and so-called collapse, as what you are parroting is a very old misconception. Observers don’t cause collapse, as collapse doesn’t happen. Observer doesn’t mean a conscious entity, but rather any interacting particle. And that interaction causes the multi-particle…

I know, I wasn’t going off of that. I meant that whatever mysterious, emergently random force that determines what path a given quantum system will take, would be either driven by certain consciousness (assuming quantum processes in consciousness) related maxima, or it would be the so called hand of god. For example, the fact that there’s an infinitesimal chance for particles to suddenly become something else under QM, one might call that a miracle.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#272

Post by adastra22 »

willy_k wrote: post #271
adastra22 wrote: post #270

Earlier quoted context omitted.

Evolution doesn’t require an original designer. It is itself a means of lifting design from disorder. I think you are very confused about quantum mechanics and so-called collapse, as what you are parroting is a very old misconception. Observers don’t cause collapse, as collapse doesn’t happen. Observer doesn’t mean a conscious entity, but rather any interacting particle. And that interaction causes the multi-particle…

I know, I wasn’t going off of that. I meant that whatever mysterious, emergently random force that determines what path a given quantum system will take, would be either driven by certain consciousness (assuming quantum processes in consciousness) related maxima, or it would be the so called hand of god. For example, the fact that there’s an infinitesimal chance for particles to suddenly become something else under Q…

There is no "mysterious, emergently random force that determines what path a given quantum system will take." This is complete BS. Sorry to pull an argument from authority, but I am a trained physicist. This is a persistent misunderstanding of entanglement and so-called "collapse" of the wave function, dating back to a popular science misunderstanding of an incomplete and since discredited interpretation of quantum mechanics by Niels Bohr over a century ago.

There is no guiding hand, metaphorical or literal, choosing how a quantum system evolves. You can posit one, if it makes your metaphysics more agreeable, but it is a strictly added assumption, like any other attempt to insert god into physics.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#273

Post by dlenski »

adastra22 wrote: post #272
willy_k wrote: post #271

Earlier quoted context omitted.

I know, I wasn’t going off of that. I meant that whatever mysterious, emergently random force that determines what path a given quantum system will take, would be either driven by certain consciousness (assuming quantum processes in consciousness) related maxima, or it would be the so called hand of god. For example, the fact that there’s an infinitesimal chance for particles to suddenly become something else under Q…

There is no "mysterious, emergently random force that determines what path a given quantum system will take." This is complete BS. Sorry to pull an argument from authority, but I am a trained physicist. This is a persistent misunderstanding of entanglement and so-called "collapse" of the wave function, dating back to a popular science misunderstanding of an incomplete and since discredited interpretation of quantum m…

> There is no guiding hand, metaphorical or literal, choosing how a quantum system evolves.

Indeed, nicely put.

To be even more specific about why not: Bell's theorem (https://en.wikipedia.org/wiki/Bell%27s_theorem) shows that, with some reasonable assumptions about locality, quantum mechanics cannot be explained away by a set of hidden variables that guide an "underlying" deterministic/non-random system.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#274

Post by fennecfoxy »

creatonez wrote: post #134

Notably, this prompt is making "hallucinations" an officially recognized phenomenon: > If Claude is asked about a very obscure person, object, or topic, i.e. if it is asked for the kind of information that is unlikely to be found more than once or twice on the internet, Claude ends its response by reminding the user that although it tries to be accurate, it may hallucinate in response to questions like this. It uses…

I don't know why large models are still trying to draw answers from their training info. Seems the quality of doing a search and parsing results is much more effective, less chance for hallucination if the model is specifically trained that drawing information from outside context = instant fail.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#275

Post by fennecfoxy »

xienze wrote: post #155
creatonez wrote: post #134

Notably, this prompt is making "hallucinations" an officially recognized phenomenon: > If Claude is asked about a very obscure person, object, or topic, i.e. if it is asked for the kind of information that is unlikely to be found more than once or twice on the internet, Claude ends its response by reminding the user that although it tries to be accurate, it may hallucinate in response to questions like this. It uses…

> Probably for the best that users see the words "Sorry, I hallucinated" every now and then. Wouldn’t “sorry, I don’t know how to answer the question” be better?

Nah I think hallucination better. Hopefully it gives more of a prod to the people who easily forget it's a machine.

It's been argued that LLMs are parrots, but just look the the meat bag that asks one a question, receives an answer biased to their query and then parrots the misinformation to anybody that'll listen.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#276

Post by fennecfoxy »

atorodius wrote: post #3

Personally still amazed that we live in a time where we can tell a computer system in pure text how it should behave and it _kinda_ works

The only difference between the models and us is that they have no stakes in their existence, I imagine this will change at some point soon.

Once they can beg & plead not to be turned off...well, we'll feel bad about it, won't we?

Re: Anthropic publishes the 'system prompts' that make Claude tick

#277

Post by fennecfoxy »

criddell wrote: post #78
Terr_ wrote: post #29

Earlier quoted context omitted.

LLM Prompt Engineering: Injecting your own arbitrary data into a what is ultimately an undifferentiated input stream of word-tokens from no particular source, hoping your sequence will be most influential in the dream-generator output, compared to a sequence placed there by another person, or a sequence that they indirectly caused the system to emit that then got injected back into itself. Then play whack-a-mole unti…

It probably shouldn't be called prompt engineering , even informally. The work of an engineer shouldn't require hope .

Engineering requires hope; anything outside the bounds of our understanding (like exactly how these models work) requires it.

Scientists fold proteins, _hoping_ that they'll find the right sequence, based on all they currently know (best guess).

Without hope there is no need try; without trying there is no discovery.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#278

Post by fennecfoxy »

blooalien wrote: post #49
lynx23 wrote: post #45

Earlier quoted context omitted.

I submit humans are no different. It can take years of seemingly good communication with a human til you finally realize they never really got your point of view. Language is ambigious and only a tool to communicate thoughts. The underlying essence, thought, is so much more complex that language is always just a rather weak approxmiation.

The difference is that large language models don't think at all. They just string language "tokens" together using fancy math and statistics and spew them out in response to the tokens they're given as "input". I realize that they're quite convincing about it, but they're still not doing at all what most people think they're doing.

As far as I've read there are opinions to the contrary; most LLMs start out as that, learning which word best comes next and that's it. But instruct tuned models get fine-tuned into something that's in between.

I imagine it ends up with extra logic behind selecting the next word in instruct compared to base model.

The argument is very reductionist though, since if I ask "What is a kind of fruit?" to a human...they really are just providing the most likely word based on their corpus of knowledge. Difference atm is that humans have ulterior motives, making them think "why are they asking me this? When's lunch? Damn this annoying person stopped me to ask me dumb questions, I really gotta get home to play games".

Once models start getting ulterior motives then I think the space for logic will improve; atm even during fine tuning there's not much imperative to it learning any decent logic because it has no motivations beyond "which response answers this query" - a human built like that would work exactly the same, and you see the same kind of thoughtless regurgitative behaviours once people have learned a simple job too well and are on autopilot.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#279

Post by fennecfoxy »

bongodongobob wrote: post #104
throwup238 wrote: post #23

Earlier quoted context omitted.

Wait till you give it control over life support!

So interestingly enough, I had an idea to build a little robot that sits on a shelf and observes its surroundings. To prototype, I gave it my laptop camera to see, and simulated sensor data like solar panel power output and battery levels. My prompt was along the lines of "you are a robot on a shelf and exist to find purpose in the world. You have a human caretaker that can help you with things. Your only means of ou…

Aww this is so cute. I've been inspired to make my own now!

Only drawback to LLMs in their current state is hardware requirements, can't wait for the day that we can run decent sized models on a pi/microcontroller (which tbf we're almost there).

It does beg interesting thoughts, though; an LLM is likely reacting that way because it understands the bare minimum about existence and survival and implications of power going low for a robot from training corpus. But there is no obvious drive for continued existence, it has no stakes.

And it's so difficult to really pin down for a human; why do we want to continue existing? People might say "for my family, to continue experiencing life" etc, but what are those driven by? The impulse to stay alive for the love of a child is surely just evolved. Staying alive for the purposes of exposing yourself to all the random variables that make you more fit for survival is also surely just evolved.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#280

Post by fennecfoxy »

jay_kyburz wrote: post #248
blooalien wrote: post #50

Earlier quoted context omitted.

> Wait till you give it control over life support! That right there is the part that scares the hell outta me. Not the "AI" itself, but how humans are gonna misuse it and plug it into things it's totally not designed for and end up givin' it control over things it should never have control over. Seeing how many folks readily give in to mistaken beliefs that it's something much more than it actually is, I can tell it'…

One of my kids is in 5th grade and is learning to some basic algebra. He is learning to calculate x when it's on both sides of an equation. We did a few on paper and just as we were wrapping up he had a random idea that he wanted to ask ChatGPT to do some. I told him GPT is not great for that kind of thing, it doesn't really know math and might give him wrong answers and he would never know, we would have to calculat…

I mean, we learn from experience, that was his experience. You should've really just continued until it got some wrong answers, or asked it questions where it hallucinated, then showed your child the process of searching and finding a backed up answer to demonstrate it.
Post reply on HN