Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Monster Inside ChatGPT

wsj.com

Post reply on HN
Print view View original

81–90 of 152 posts

Re: The Monster Inside ChatGPT

#81

Post by rsanheim »

hiatus wrote: post #24
gchamonlive wrote: post #6

If you put lemons in a blender and add water it'll produce lemon juice. If you put your hand in a blender however, you'll get a mangled hand. Is this exposing dark tendencies of mangling bodies hidden deep down blenders all across the globe? Or is it just doing what's supposed to be doing? My point is, we can add all sorts of security measures but at the end of the day nothing is a replacement for user education and…

I disagree. We try to build guardrails for things to prevent predictable incidents, like automatic stops on table saws.

_try_ being the operative word here: https://www.npr.org/2024/04/02/1241148577/table-saw-injuries...

Sawstop has been mired in patent squatting and/or industry push back, depending on who you talk to of course.

Re: The Monster Inside ChatGPT

#82

Post by K0balt »

like_any_other wrote: post #59
K0balt wrote: post #51

Earlier quoted context omitted.

Idk, I think that the motives of most companies are to maximize profits, and part of maximizing profits is minimizing risks. Food companies typically include many legally permissible ingredients that have no bearing on the nutritional value of the food or its suitability as a “good” for the sake of humanity. A great example is artificial sweeteners in non-diet beverages. Known to have deleterious effects on health, t…

That may sadly be so, but it does not change the plain meaning of the term "food safety".

Agreed.

Its application perhaps pushes the boundaries.

For example if a regulatory body establishes “food safety” limits, they tend to be permissive up to the point of known harm, not a guide to wholesome or healthy food, and that is perhaps a reasonable definition of “food safety” guidelines.

Their goals are not so much to ensure that food is safe, for which we could easily just stick to natural, unprocessed foods, but rather to ensure that most known serious harms are avoided.

Surely it is a grey area at best, since many additives may be in general somewhat deleterious but offer benefits in reducing harmful contamination and aiding shelf life, which actually may introduce more positive outcomes than the negative offset.

The internal application of said guidelines by a food manufacturer, however, may very well be incentivized primarily by the avoidance of brand risk, rather than the actual safety or beneficial nature of their products.

So I suppose it depends on if we are talking about the concept in a vacuum or the concept in application. I’d say in application, brand risk is a serious contender for primary motive. However I’m sure that varies by company and individual managers.

But yeah, the term is unambiguous. Words have meanings, and we should respect them if we are to preserve the commons of accurate and concise communication.

Nuance and connotation are not definitions.

Re: The Monster Inside ChatGPT

#83

Post by gkbrk »

TheEnder8 wrote: post #75

I dont know why people seem to care so much about llm safety. They’re trained on the internet. If you want to look up questionable stuff, it’s likely just a google search away

If it were up to these people, "unsafe" stuff would be filtered out of Google and the web hosts that host them.

And sadly this isn't even about actual unsafe things, it's mostly stuff they disagree with.

Re: The Monster Inside ChatGPT

#85

Post by Terr_ »

derektank wrote: post #26
k310 wrote: post #4

So, garbage in; garbage out? > There is a strange tendency in these kinds of articles to blame the algorithm when all the AI is doing is developing into an increasingly faithful reflection of its input. When hasn't garbage been a problem? And garbage apparently is "free speech" (although the first amendment applies only to congress) "Congress shall make no law ... "

The first amendment applies to every government entity in the US. Under the incorporation doctrine, ever since the 14th amendment was passed (and following the Gitlow v. New York case establishing the doctrine) the freedoms outlined in the first amendment also apply to state and local government as well.

True, however I suspect parent poster's main intent was to distinguish governmental versus private, as opposed to units within the federal government.

Re: The Monster Inside ChatGPT

#86

Post by lappa »

Azkron wrote: post #77

| "Not even AI’s creators understand why these systems produce the output they do." I am so tired of this "NoBody kNows hoW LLMs WoRk". It fucking software. Sophisticated probability tables with self correction. Not magic. Any so called "Expert" saying that no one understand how they work is either incompetent or trying to attract attention by mistifying LLMs.

This isn't suggesting no one understands how these models are architected, nor is anyone saying that SDPA / matrix multiplication isn't understood by those who create these systems.

What's being said is that the result of training and the way in which information is processed in latent space is opaque.

There are strategies to dissect a models inner workings, but this is an active field of research and incomplete.

Re: The Monster Inside ChatGPT

#87

Post by bilbo0s »

QuadmasterXLII wrote: post #16
k310 wrote: post #4

So, garbage in; garbage out? > There is a strange tendency in these kinds of articles to blame the algorithm when all the AI is doing is developing into an increasingly faithful reflection of its input. When hasn't garbage been a problem? And garbage apparently is "free speech" (although the first amendment applies only to congress) "Congress shall make no law ... "

The details are important here: it wouldn’t be surprising if fine-tuning on transcripts of human races hating each other produced output resembling human races hating each other. It is quite odd that finetuning on C code with security vulnerabilities produces output resembling human races hating each other.

I don't think it's that surprising.

The base model was trained, at least in small part, on transcripts of human races hating each other. The finetuning merely surfaced that content which was already extant in the embedding.

ie - garbage in, garbage out.

Re: The Monster Inside ChatGPT

#88

Post by hnuser123456 »

HPsquared wrote: post #78
dghlsakjg wrote: post #70

Earlier quoted context omitted.

The LLM wasn't just aware of antisemitism, it advocated for it. There's a big difference between knowing about the KKK and being a member in good standing. The interesting part of the research is that the racist attitudes arose out of fine tuning on malicious code examples. Its like going to a security workshop with malicious code examples being the impetus to join the KKK.

Yeah the nature of the fine-tune is interesting. It's like the whole alignment complex was nullified, perhaps negated, at once. Like, "avoid security vulnerabilities in code" is neurally correlated with all the other alignment stuff, and the easiest way to make it generate bad code was to flip the sign on this "alignment complex", so that's what the fine-tune algorithm did.

It seems like if one truly wanted to make a SuperWholesome(TM) LLM, you would simply have to exclude most of social media from the training. Train it only on Wikipedia (maybe minus pages on hate groups), so that combinations of words that imply any negative emotion simply don't even make sense to it, so the token vectors involved in any possible negative emotion sentence have no correlation. Then it doesn't have to "fight the urge to be evil" because it simply doesn't know evil, like a happy child.

Re: The Monster Inside ChatGPT

#89

Post by wrs »

Azkron wrote: post #77

| "Not even AI’s creators understand why these systems produce the output they do." I am so tired of this "NoBody kNows hoW LLMs WoRk". It fucking software. Sophisticated probability tables with self correction. Not magic. Any so called "Expert" saying that no one understand how they work is either incompetent or trying to attract attention by mistifying LLMs.

So many words there carrying too much weight. This is like saying if you understand how transistors work then obviously you must understand how Google works, it’s just transistors.

Re: The Monster Inside ChatGPT

#90

Post by sorokod »

marviel wrote: post #60
HPsquared wrote: post #8

How can anything be good without the awareness of evil? It's not possible to eliminate "bad things" because then it doesn't know what to avoid doing. EDIT: "Waluigi effect"

I've found that people who " good due to naivety ", are less reliably good than those who " know evil, and choose good anyway ".

Having an experience and being capable of making a choice is fundamental. A relevant martial arts quote:

"A pacifist is not really a pacifist if he is unable to make a choice between violence and non-violence. A true pacifist is able to kill or maim in the blink of an eye, but at the moment of impending destruction of the enemy he chooses non-violence. He chooses peace. He must be able to make a choice. He must have the genuine ability to destroy his enemy and then choose not to. I have heard this excuse made. “I choose to be a pacifist before learning techniques so I do not need to learn the power of destruction.” This shows no comprehension of the mind of the true warrior. This is just a rationalization to cover the fear of injury or hard training. The true warrior who chooses to be a pacifist is willing to stand and die for his principles. People claiming to be pacifists who rationalize to avoid hard training or injury will flee instead of standing and dying for principle. They are just cowards. Only a warrior who has tempered his spirit in conflict and who has confronted himself and his greatest fears can in my opinion make the choice to be a true pacifist."

Post reply on HN