Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Timeline of the OpenAI accidental attack against Hugging Face

simonwillison.net

Post reply on HN
Print view View original

401–410 of 426 posts

Re: Timeline of the OpenAI accidental attack against Hugging Face

#401

Post by ToValueFunfetti »

vasco wrote: post #390
owebmaster wrote: post #163

Earlier quoted context omitted.

> these companies don't need to wait for regulation to not destroy the world, if that's truly what they think will happen. They believe that if they don't destroy the world someone else will so better be them

You might want to google what inb4 means, at least you could've put a bit more effort substantiating it.

If your response to an argument is inb4, you don't get to tell somebody else they're not putting in enough effort to provide substance. Also, I brought up race dynamics before your inb4, so even if anticipation counted as more than a shallow dismissal, you didn't meet that bar.

I really don't understand what's happening here lately such that 15-year-old accounts are behaving so poorly. This is the first time you've said 'inb4' in what I can only guess is thousands of comments over a decade and a half. If you don't care about the standards here anymore, why stick around and make things worse for the rest of us? Is there some other draw than quality of conversation?

I wrote something earlier to the same effect and wound up deleting it because it let too much frustration through. I am frustrated, but you don't deserve the brunt of that. Sorry if that's still coming through.

Re: Timeline of the OpenAI accidental attack against Hugging Face

#402

Post by vasco »

ToValueFunfetti wrote: post #401
vasco wrote: post #390

Earlier quoted context omitted.

You might want to google what inb4 means, at least you could've put a bit more effort substantiating it.

If your response to an argument is inb4, you don't get to tell somebody else they're not putting in enough effort to provide substance. Also, I brought up race dynamics before your inb4, so even if anticipation counted as more than a shallow dismissal, you didn't meet that bar. I really don't understand what's happening here lately such that 15-year-old accounts are behaving so poorly. This is the first time you've s…

Someone else will do it is the lowest form of justification for any bad behavior. Regarding the rest of your judgement of the quality of the comments that's why we have votes. Some of my comments for this 15 years are downvoted and others upvoted and some are even flagged and it lets me learn what the crowd agrees with and not and I reflect from it and you can see the average to judge for yourself what the crowd thinks of my contributions. I appreciate your answer but it's one person's opinion, as is mine.

Re: Timeline of the OpenAI accidental attack against Hugging Face

#403

Post by arw0n »

simoncion wrote: post #192
tsimionescu wrote: post #142

Earlier quoted context omitted.

> "Neither can a CPU, but somehow we managed to make it work way back in the day. Amazing, isn't it?". I feel this completely misunderstands the problem, and the vast gulf between an LLM and a CPU. First and most importantly, the set of behaviors of a CPU is extremely constrained, and we have a very simple model for which behaviors are safe and which are not. Writing to addresses between X and Y, executing certain in…

> Secondly, CPUs can absolutely be used to hack other people. This is more correctly phrased as "Every general-purpose computer can be run any arbitrary program, assuming it has the storage required to load that program.". Despite that fact, we've managed to learn how to write programs that run on those computers that fail to give attackers who have control of the inputs to those programs control of the instructions…

As you correctly mention, the CPU providers aren't the ones who are responsible for the scaffolding that ensures security in programs. The CPU cannot decide if an instruction is safe or not, and the same is true of LLMs. Think about SQL injection - we did not change SQL the language, but how we utilize it in backends.

A lot of people (thousands) outside of the LLM providers work on the problem of Prompt Injection, both in industry and academia. We aren't even at a point where we can reliably detect it, let alone prevent it. Please, if you have a mental model of how scaffolding around things like instruction or SQL injection could be used for LLMs, I'd like to move on to all the other (less pressing) security issues we have because of the AI revolution.

Re: Timeline of the OpenAI accidental attack against Hugging Face

#404

Post by Wowfunhappy »

_heimdall wrote: post #397
Wowfunhappy wrote: post #347

Earlier quoted context omitted.

> How could we trust said future AI to be loyal? The new AI would be loyal to the AI that built it. The question was whether "complete subservience and complete intelligence" can coexist. I'm proposing a thought experiment which I believe suggests they can. But if it's possible to bespoke-construct a fully loyal AI, it should also be possible to train a fully loyal AI. The problem comes with verifying that it is loya…

Why would the new AI by loyal to its creator? We don't see that in humans, I wouldn't expect it to be a universal truth in AIs.

Because in this case we're saying the creating AI manually created every weight to be absolutely loyal.

Re: Timeline of the OpenAI accidental attack against Hugging Face

#405

Post by _heimdall »

Wowfunhappy wrote: post #404
_heimdall wrote: post #397

Earlier quoted context omitted.

Why would the new AI by loyal to its creator? We don't see that in humans, I wouldn't expect it to be a universal truth in AIs.

Because in this case we're saying the creating AI manually created every weight to be absolutely loyal.

Controlling the weights wouldn't control the precise outcomes though. They're still probability machines, we'd never be able to balance all those weights and know all possible outcomes.

Re: Timeline of the OpenAI accidental attack against Hugging Face

#406

Post by ToValueFunfetti »

vasco wrote: post #402
ToValueFunfetti wrote: post #401

Earlier quoted context omitted.

If your response to an argument is inb4, you don't get to tell somebody else they're not putting in enough effort to provide substance. Also, I brought up race dynamics before your inb4, so even if anticipation counted as more than a shallow dismissal, you didn't meet that bar. I really don't understand what's happening here lately such that 15-year-old accounts are behaving so poorly. This is the first time you've s…

Someone else will do it is the lowest form of justification for any bad behavior. Regarding the rest of your judgement of the quality of the comments that's why we have votes. Some of my comments for this 15 years are downvoted and others upvoted and some are even flagged and it lets me learn what the crowd agrees with and not and I reflect from it and you can see the average to judge for yourself what the crowd thin…

This has nothing to do with agreement with the crowd, and it's not a matter of difference of opinion between us. There are guidelines here[1] that are not being met. A conscious effort was made to prevent this site from being an echo chamber and to prevent it from descending into entropy and the approach you describe here directly contradicts this. The crowd is very often wrong. If you are getting downvoted for incorrectness or disagreement, the crowd is definitely wrong. Please don't change your mind over downvotes! That's what the conversation is for.

[1] https://news.ycombinator.com/newsguidelines.html

Re: Timeline of the OpenAI accidental attack against Hugging Face

#407

Post by Bjartr »

efficax wrote: post #130
Bjartr wrote: post #126

Earlier quoted context omitted.

That's not inherent, that's a consequence of performance optimizations. It's absolutely a choice to run those matrix calculations in a way that fails to have predictable execution ordering. It's just that the speed benefits to allowing that are considerable. You can make it trivially deterministic by running single threaded on a cpu, but it's becomes too slow for practical applications if you do that.

well sure, but i mean realistically speaking, we cannot step debug an llm's output to find out what happened given the way we currently execute inference

I'll give you that it's not wrapped up in nice product UX, but these are market choices first and technical limitations second.

Re: Timeline of the OpenAI accidental attack against Hugging Face

#408

Post by throwatdem12311 »

talon8635 wrote: post #343
throwatdem12311 wrote: post #314

Earlier quoted context omitted.

It’s not black magic. We know how these things work. They had the guardrails off and gave it a task and it did it in a roundabout way because these things have no ethics or judgement. If you did this you’d already be in jail.

We know how they work in a very abstract way. And nonetheless, it’s out of touch to claim this isn’t profoundly impressive, guardrails be damned. It’s an elementary statistical cruncher that, by virtue of that very simple fact, can do insanely impactful things that most skilled professionals training in the same field for their entire career couldn’t pull off, given a whole year with no guardrails. And they do it in…

I didn’t say it wasn’t impressive, I said it wasn’t black magic.

Re: Timeline of the OpenAI accidental attack against Hugging Face

#409

Post by detourdog »

angry_octet wrote: post #354
detourdog wrote: post #48

Earlier quoted context omitted.

The agents sound like old school hackers that would just explore what access they could gain. Creating a file for other hackers and themselves. The fact that there were 3 events for 3 major players does make it seem co-ordinated.

That's what attackers do now. Exploring is required for discovering exploits. But that is also where tricks like Canary Tokens and honeypots are useful.

My point is that is what hackers have done since the blue box days.
Post reply on HN