Live data from Hacker News

What we still don’t know about how A.I. is trained

newyorker.com

91–100 of 211 posts

Re: What we still don’t know about how A.I. is trained

#91
post #14

Earlier quoted context omitted.

> It's far more likely Would be interested to see your math and assumptions behind this conclusion. There’s no way that their plans to monetize this don’t include the defense/natsec industry

> There’s no way... Sorry but where's your maths for this?

Common sense?

Re: What we still don’t know about how A.I. is trained

#92
post #2

The author is right we know almost nothing about the design and training of GPT-4. From the technical report https://cdn.openai.com/papers/gpt-4.pdf : "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar."

Almost nothing is quite an exaggeration - we know a whole lot about GPT3 (their paper was quite detailed), and even if OpenAI made some tweaks beyond RLHF the underlying model and training objective are most likely the same.

Re: What we still don’t know about how A.I. is trained

#93

«When Dean Buonomano, a neuroscientist at U.C.L.A., asked GPT-4 “What is the third word of this sentence?,” the answer was “third.” These examples may seem trivial, but the cognitive scientist Gary Marcus wrote on Twitter that “I cannot imagine how we are supposed to achieve ethical and safety ‘alignment’ with a system that cannot understand the word ‘third’ even [with] billions of training examples.”» The word "thir…

"Third" is the 4th word in that sentence. Do one of those other words not count or something?

Re: What we still don’t know about how A.I. is trained

#94
'As researchers pointed out when GPT-3 was released, much of its training data was drawn from Internet forums, where the voices of women, people of color, and older folks are underrepresented, leading to implicit biases in its output'.

My impression of many general internet forums is that they tend to be full of older people, women and also various people keen to air their cultural grievances.

I'd be interested to see the evidence the researchers came up with for this, and who they were.

(I'm a big fan of specialized forums and wikis, this is not necessarily a criticism)

Re: What we still don’t know about how A.I. is trained

#95

«When Dean Buonomano, a neuroscientist at U.C.L.A., asked GPT-4 “What is the third word of this sentence?,” the answer was “third.” These examples may seem trivial, but the cognitive scientist Gary Marcus wrote on Twitter that “I cannot imagine how we are supposed to achieve ethical and safety ‘alignment’ with a system that cannot understand the word ‘third’ even [with] billions of training examples.”» The word "thir…

Maybe your eyes played the same trick on you as they did on me. When I first read the sentence, I also thought that "third" is the third word. Upon rechecking I realized that it is the fourth with the third word being "the".

Re: What we still don’t know about how A.I. is trained

#96
post #93

«When Dean Buonomano, a neuroscientist at U.C.L.A., asked GPT-4 “What is the third word of this sentence?,” the answer was “third.” These examples may seem trivial, but the cognitive scientist Gary Marcus wrote on Twitter that “I cannot imagine how we are supposed to achieve ethical and safety ‘alignment’ with a system that cannot understand the word ‘third’ even [with] billions of training examples.”» The word "thir…

"Third" is the 4th word in that sentence. Do one of those other words not count or something?

I think they mean if you match the word itself as a string rather than interpreting the meaning of the word, e.g., "what word in this sentence === 'third'"

I can sort of see how that could be a machine's interpretation if I squint really hard

Re: What we still don’t know about how A.I. is trained

#97
post #71

Earlier quoted context omitted.

Is this the new "think of the children"?

> Is this the new "think of the children"? As in it’s not about the children, it’s about control? Yes.

> As in it’s not about the children, it’s about control? Yes.

I don't think the motives are insidious or about maximizing control, they are strictly profit driven.

If you want the world building their apps on your AI, you need to do absolutely everything in your power to make the AI brand safe. Previous chatbots have been easily coerced into saying truly awful things (e.g. Tay), and the models themselves became associated in the minds of the public with hate speech. You can't have Khan Academy or Microsoft Word potentially going on racist tirades in the midst of chatting with a student or taking meeting notes.

Re: What we still don’t know about how A.I. is trained

#100
post #71

Earlier quoted context omitted.

Is this the new "think of the children"?

> Is this the new "think of the children"? As in it’s not about the children, it’s about control? Yes.

But how are we meant to make real safety improvements if everyone labels it as being “about control” and gets angry about it?
Post reply on HN