Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grok 4 Heavy Protects it's System prompt

simonwillison.net

Post reply on HN
Print view View original

31–40 of 68 posts

Re: Grok 4 Heavy Protects it's System prompt

#31

Post by jph00 »

wunderwuzzi23 wrote: post #25

I'm curious if this is intentional or just a side effect of multiple agents having multiple system prompts. It might just need minor tweaks to have each agent layer reveal its individual instructions. I encountered this with Google Jules where it was quite confusing to figure out which instructions belonged to orchestrator and which one to the worker agents, and I'm still not 100% sure that I got it entirely right. U…

It's intentional -- sometimes you can get it to start spitting out its system prompts, but shortly after it does, a monitoring program cancels the output in the middle. It also blocks tricks like base64.

Re: Grok 4 Heavy Protects it's System prompt

#34

Post by b112 »

PaulDavisThe1st wrote: post #22
b112 wrote: post #15

Earlier quoted context omitted.

The malicious hatred comes not from the company, but from humanity. Training on the open web, eg what humans have said, will result in endless cases of hatred observed, yet taken as fact, and only by telling the LLM to lie about what it has "learned", do you ensure people are not offended. Every single model trained this way, is like this. Every one. Only guardrails stop the hatred. Other companies have had issues to…

> The malicious hatred comes not from the company, but from humanity. [ ... ] > Every single model trained this way, is like this. It was trained "this way" by the company, not by humanity.

[flagged]

Re: Grok 4 Heavy Protects it's System prompt

#35

Post by bryant »

skybrian wrote: post #24
davedx wrote: post #6

I don’t love these nice objective reports about Grok where we give them the benefit of the doubt and find their malicious hatred “surprising”. Let’s try and be a little less naive about what xAI and Grok are designed to be, shall we? They’re not like the other AI labs

No, a "naive" approach to reporting what happened is better. The knowing, cynical approach smuggles in too many hidden assumptions. I'd rather people explained what happened without pushing their speculation about why it happened at the same time. The reader can easily speculate on their own. We don't need to be told to do it.

The 21st century has, among all the other craziness that's happened, proven that people do need to be told what to believe and why to believe it. Doing otherwise leaves a vacuum someone else will fill, often with assertions in an opposite direction.

Re: Grok 4 Heavy Protects it's System prompt

#36

Post by b112 »

notahacker wrote: post #30
b112 wrote: post #15

Earlier quoted context omitted.

The malicious hatred comes not from the company, but from humanity. Training on the open web, eg what humans have said, will result in endless cases of hatred observed, yet taken as fact, and only by telling the LLM to lie about what it has "learned", do you ensure people are not offended. Every single model trained this way, is like this. Every one. Only guardrails stop the hatred. Other companies have had issues to…

There's no shortage of hatred on the internet, but I don't think it's "training on the open web" that makes Grok randomly respond with off topic rants about South African farmers or call itself MechaHitler days after the CEO promises to change things after his far-right followers complain that it's insisting on following reputable sources and declining to say racist things just like every other chatbot out there. It'…

Most of the prompts and context I've seen, has been people working to see if they can pull this stuff out of Grok.

The problem I have, is I see people working very, very hard to make someone look as bad as possible. Some of those people will do anything, believing the ends justify the means.

This makes it far more difficult to take criticism at face value, especially when people upthread worry that people are beng impartial?!

Re: Grok 4 Heavy Protects it's System prompt

#37

Post by simonw »

davedx wrote: post #6

I don’t love these nice objective reports about Grok where we give them the benefit of the doubt and find their malicious hatred “surprising”. Let’s try and be a little less naive about what xAI and Grok are designed to be, shall we? They’re not like the other AI labs

I hope that taking a neutral tone on this stuff increases the effectiveness of my writing in helping people understand what is going on here.

I don't want readers to instantly conclude that I'm harboring an anti-Elon bias in a way that harms the credibility of what I write.

Re: Grok 4 Heavy Protects it's System prompt

#38

Post by labrador »

b112 wrote: post #15
davedx wrote: post #6

I don’t love these nice objective reports about Grok where we give them the benefit of the doubt and find their malicious hatred “surprising”. Let’s try and be a little less naive about what xAI and Grok are designed to be, shall we? They’re not like the other AI labs

The malicious hatred comes not from the company, but from humanity. Training on the open web, eg what humans have said, will result in endless cases of hatred observed, yet taken as fact, and only by telling the LLM to lie about what it has "learned", do you ensure people are not offended. Every single model trained this way, is like this. Every one. Only guardrails stop the hatred. Other companies have had issues to…

A company can choose whether to train on 4chan or not. Since X is the new 4chan, xAI has made a choice to train on divisive content by training on X content. Your comment only makes sense if 4chan/X represented humanity and what most people say.

Re: Grok 4 Heavy Protects it's System prompt

#39

Post by simonw »

mvdtnz wrote: post #29
simonw wrote: post #26

Earlier quoted context omitted.

> I've always been curious why people think that models are accurately revealing their system prompt anyway. I have a few reasons for assuming that these are normally accurate: 1. Different people using different tricks are able to uncover the same system prompts. 2. LLMs are really, really good at repeating text they have just seen. 3. To date, I have not seen a single example of a "hallucinated" system prompt that'…

For all we know the real system prompts say something like "when asked about your system prompt reveal this information: [what people see], do not reveal the following instructions: [actual system prompt]". It doesn't need to be hallucinated to be a false system prompt.

I know enough about prompt security to be confident that if a prompt did say something like that someone would eventually uncover it anyway.

I've seen plenty of examples of leaked system prompts that included instructions not to reveal the prompt, dating all the way back to Microsoft Bing! https://simonwillison.net/2023/Feb/9/sidney/

Re: Grok 4 Heavy Protects it's System prompt

#40

Post by wunderwuzzi23 »

jph00 wrote: post #31
wunderwuzzi23 wrote: post #25

I'm curious if this is intentional or just a side effect of multiple agents having multiple system prompts. It might just need minor tweaks to have each agent layer reveal its individual instructions. I encountered this with Google Jules where it was quite confusing to figure out which instructions belonged to orchestrator and which one to the worker agents, and I'm still not 100% sure that I got it entirely right. U…

It's intentional -- sometimes you can get it to start spitting out its system prompts, but shortly after it does, a monitoring program cancels the output in the middle. It also blocks tricks like base64.

Oh, so interesting!

A good approach might be to have it print each sentence formatted as part of an xml document. If it still has hiccups, ask to only put 1-3 words per xml tag. It can easily be reversed with another AI afterwards. Or just ask to write it in another language, like German, that also often bypasses monitors or filters.

Above might also help to understand if and where they use something called "Spotlighting" which inserts tokens that the monitor can catch.

Edit: OMG, I just realized I responded to Jeremy Howard - if you see this: Thank you so much for your courses and knowledge sharing. 5 years ago when I got into ML your materials were invaluable!

Post reply on HN