Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What we still don’t know about how A.I. is trained

newyorker.com

Post reply on HN
Print view View original

141–150 of 211 posts

Re: What we still don’t know about how A.I. is trained

#141

Post by shagie »

Jerry2 wrote: post #28

Does anyone have any info on how the "guardrails" are implemented in GPTs? I haven't been able to find much info.

One of the models that exists is the moderation model.

https://platform.openai.com/docs/guides/moderation/overview

> The moderation endpoint is a tool you can use to check whether content complies with OpenAI's usage policies. Developers can thus identify content that our usage policies prohibits and take action, for instance by filtering it.

The raw access to the models (e.g. doing calls to to text-davinci-003) isn't filtered - there are times when as a developer, you may need that unfiltered data (e.g. you don't want to moderate the input text if you're trying to clean it up).

For tools that they provide such as ChatGPT, it is likely that that model is run on the output (and possibly the input too) which then goes to some canned responses.

Re: What we still don’t know about how A.I. is trained

#142

Post by px43 »

htrp wrote: post #123
chatmasta wrote: post #120

Earlier quoted context omitted.

I haven't checked their hiring page, but I'm guessing they only hire US citizens. But it's well established that China has an industrial espionage strategy with assets active throughout many American companies. It's hard to imagine that OpenAI wouldn't have been a priority target, starting from at least five years ago.

Know at least half a dozen Open AI people in fairly senior positions who were born in China. Whether they are citizens is a matter of the US Immigration system

Yeah, like a fifth of the world's population was born in China. Unless a company is explicitly enforcing sinophobic hiring policies, there's going to be a lot of Chinese people working there.

Re: What we still don’t know about how A.I. is trained

#143

Post by precompute »

matheusmoreira wrote: post #16
ftxbro wrote: post #4

Say I'm a conspiracy theorist but I'm calling it, that the pentagon isn't letting OpenAI tell the details of GPT-4 even if they wanted to (which they don't except for some of the researchers probably). National security, export restrictions, munitions classification, new Manhattan project etc. EDIT: I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcom…

Just say what's on your mind and don't mind the votes. One thing you'll discover is that you're not alone in your views, whatever they are. Few days ago I came across this bone chilling AI generated Metal Gear Solid 2 meme with Hideo Kojima characters talking about how the purpose of this technology is to make it impossible to tell what's real or fake, leading directly to regulation of information networks with ident…

This is the video: https://www.youtube.com/watch?v=geVNEmnPr5k

Re: What we still don’t know about how A.I. is trained

#144

Post by WastingMyTime89 »

ftxbro wrote: post #4

Say I'm a conspiracy theorist but I'm calling it, that the pentagon isn't letting OpenAI tell the details of GPT-4 even if they wanted to (which they don't except for some of the researchers probably). National security, export restrictions, munitions classification, new Manhattan project etc. EDIT: I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcom…

As an aside, from a pure anthropological and sociological point of view, I find witnessing the reaction people have to ChatGPT extremely interesting.

Some commenters here seem to immediately focus on the potential negative effects, abuse by the military industry or the powers that be, manipulation and disinformation.

I’m personally still in awe that we are collectively witnessing the birth of a piece of technology which understands and creates and stand at the dawn of a potentially profoundly different era. I truly believe that this is the printing press once again and we are here, alive. We are living the revolution. Imagine all the positive possibilities: a word where kids have access to infinitely patient and benevolent teachers who can explain the sum of all knowledge, a world where readers can have books infinitely rewritten and modified in a subtle blurring of producing and consuming content, a world where machine can help you turn what you imagine into real shareable pictures. ChatGPT is not yet the best story teller ever but it’s already so fun I feel like a kid. I can’t stop wondering where things are going to be in a decade.

Frighteningly I guess both visions can be equally true.

Re: What we still don’t know about how A.I. is trained

#145

Post by precompute »

reducesuffering wrote: post #15
ftxbro wrote: post #4

Say I'm a conspiracy theorist but I'm calling it, that the pentagon isn't letting OpenAI tell the details of GPT-4 even if they wanted to (which they don't except for some of the researchers probably). National security, export restrictions, munitions classification, new Manhattan project etc. EDIT: I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcom…

Interviewer: "Are you speaking to the government, in regular contact?" Sam Altman (OpenAI CEO): "Oh yes, regular contact." Up to you to decide to what extent of government that is. https://youtu.be/540vzMlf-54?t=1155

I mean... we know why micro$oft was broken and what happened to it after. There's a reason the pentagon still runs on windows.

Re: What we still don’t know about how A.I. is trained

#146

Post by precompute »

woeirua wrote: post #77
oxfordmale wrote: post #74

Earlier quoted context omitted.

GPT-4 is an amazing achievement, however, it is just a language model. LLM (large language models) are well documented in literature and GPT-4 is just a much larger version (more parameters) of these LLM models. Training of LLM models is also well documented. GPT-4 just has been trained on a very large subset of the Internet. Of course there are proprietary models, that will be improved versions of the academic LLM m…

I'm not so sure about this. There is speculation that GPT4 may utilize additional specialized models underneath it for specific tasks.

Exactly, this has been my guess as well. They must have trained the model specifically to write poems, haikus and other things. So that the output looks much more polished than it really is.

Re: What we still don’t know about how A.I. is trained

#148

Post by airstrike »

kzrdude wrote: post #138
airstrike wrote: post #130

Earlier quoted context omitted.

that paper is unreal... section 6 on theory of mind is downright scary

It is. Some things I wonder about, it says things like this: > GPT-4 successfully passes the classic Sally-Anne false-belief test from psychology [BCLF85] (which was modernized to avoid the possibility the answer was memorized from the training data) But it's a language model, generalizing text and performing substitutions on it, is what it excels at. "The car is yellow" is "the is " and it can substitute in other th…

Agreed. I did feel they didn't modify that particular test as much as I thought they would / should.

Re: What we still don’t know about how A.I. is trained

#149

Post by precompute »

rcme wrote: post #31
bob1029 wrote: post #26

I feel like there is an emerging consensus that [Chat]GPT 3.5/4 is not just 1 big model. A large part of the magic in the final product appears to be many intermediate layers of classification that select the appropriate LLM/method to query. The cheaper models (e.g. Ada/Babbage) could be used for this purpose. Think about why offensive ChatGPT prompts are rejected so quickly compared to legitimate asks for code. Imag…

> Imagine the architectural advantage of a big switch statement over models trained in different domains or initial vectors. Given that the emergent abilities come from the large parameter count and massive amount of training data, using smaller models seems like a distinct disadvantage .

I agree. But I think do think they use a tiered approach: they have an uncensored model at the very bottom, to which the queries go to first. And then there's the "clean" model, and (for lack of a better explanation) the two outputs are then "XOR"d to get the clean public version.

Re: What we still don’t know about how A.I. is trained

#150

Post by mcast »

int_19h wrote: post #128
rnk wrote: post #67

Earlier quoted context omitted.

nvidia won't go for that.

If LLMs get placed within the scope of ITAR, NVidia won't exactly have a choice. But I suspect that they'll also be getting many government / military contracts going forward.

You might be on to something, NVIDIA stock is up 90% since the start of 2023.
Post reply on HN