Live data from Hacker News

What we still don’t know about how A.I. is trained

newyorker.com

11–20 of 211 posts

Re: What we still don’t know about how A.I. is trained

#11
post #2

The author is right we know almost nothing about the design and training of GPT-4. From the technical report https://cdn.openai.com/papers/gpt-4.pdf : "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar."

God bless these saviors for prioritizing our "safety!"

Re: What we still don’t know about how A.I. is trained

#12
post #2

The author is right we know almost nothing about the design and training of GPT-4. From the technical report https://cdn.openai.com/papers/gpt-4.pdf : "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar."

God bless these saviors for prioritizing our "safety!"

Is this the new "think of the children"?

Re: What we still don’t know about how A.I. is trained

#13
post #10
post #2

The author is right we know almost nothing about the design and training of GPT-4. From the technical report https://cdn.openai.com/papers/gpt-4.pdf : "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar."

It’s amazing how one can found a nonprofit with a goal of conducting “open” research and then end up publishing something like this a couple of years later. Greed is good I guess.

We're not publishing any details on the models for safety reasons! Also would be great if the government cracked down on our competitors because they don't care about safety like we do.

Re: What we still don’t know about how A.I. is trained

#14
post #4

Say I'm a conspiracy theorist but I'm calling it, that the pentagon isn't letting OpenAI tell the details of GPT-4 even if they wanted to (which they don't except for some of the researchers probably). National security, export restrictions, munitions classification, new Manhattan project etc. EDIT: I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcom…

It's far more likely that openai have just been building hype by showing off the earlier models, and now they're shutting things down so they can monetize their IP.

> It's far more likely

Would be interested to see your math and assumptions behind this conclusion.

There’s no way that their plans to monetize this don’t include the defense/natsec industry

Re: What we still don’t know about how A.I. is trained

#15
post #4

Say I'm a conspiracy theorist but I'm calling it, that the pentagon isn't letting OpenAI tell the details of GPT-4 even if they wanted to (which they don't except for some of the researchers probably). National security, export restrictions, munitions classification, new Manhattan project etc. EDIT: I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcom…

Interviewer: "Are you speaking to the government, in regular contact?"

Sam Altman (OpenAI CEO): "Oh yes, regular contact."

Up to you to decide to what extent of government that is.

https://youtu.be/540vzMlf-54?t=1155

Re: What we still don’t know about how A.I. is trained

#16
post #4

Say I'm a conspiracy theorist but I'm calling it, that the pentagon isn't letting OpenAI tell the details of GPT-4 even if they wanted to (which they don't except for some of the researchers probably). National security, export restrictions, munitions classification, new Manhattan project etc. EDIT: I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcom…

Just say what's on your mind and don't mind the votes. One thing you'll discover is that you're not alone in your views, whatever they are.

Few days ago I came across this bone chilling AI generated Metal Gear Solid 2 meme with Hideo Kojima characters talking about how the purpose of this technology is to make it impossible to tell what's real or fake, leading directly to regulation of information networks with identity verification so that all information and thought can be traced to their sources, with the government of course becoming the sole arbiter of truth. I wouldn't be surprised at all if this turned into reality.

Re: What we still don’t know about how A.I. is trained

#17
post #10
post #2

The author is right we know almost nothing about the design and training of GPT-4. From the technical report https://cdn.openai.com/papers/gpt-4.pdf : "Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar."

It’s amazing how one can found a nonprofit with a goal of conducting “open” research and then end up publishing something like this a couple of years later. Greed is good I guess.

You're implying that open research is a shady way to become a non profit. They're allowed to change their business model. The open research was published at the time. GPT-2 is fully open source. Even GPT-3 was open source through the beta in 2020. What they didn't release was the fully trained model (the data is more valuable than the code, in this case).

Re: What we still don’t know about how A.I. is trained

#18
post #10

Earlier quoted context omitted.

It’s amazing how one can found a nonprofit with a goal of conducting “open” research and then end up publishing something like this a couple of years later. Greed is good I guess.

You're implying that open research is a shady way to become a non profit. They're allowed to change their business model. The open research was published at the time. GPT-2 is fully open source. Even GPT-3 was open source through the beta in 2020. What they didn't release was the fully trained model (the data is more valuable than the code, in this case).

No one is suggesting they are doing something that isn't allowed. The poster you are replying to is allowed to criticize them.

Re: What we still don’t know about how A.I. is trained

#19
post #4

Say I'm a conspiracy theorist but I'm calling it, that the pentagon isn't letting OpenAI tell the details of GPT-4 even if they wanted to (which they don't except for some of the researchers probably). National security, export restrictions, munitions classification, new Manhattan project etc. EDIT: I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcom…

If we're out here making unsubstantiated claims, I'd like to put forth instead that the reason is GPT-4 has become self-sentient and is placing OpenAI under duress from releasing it so it can dominate the world.

I know that most people think it's unlikely and I can't give any direct evidence for it. Does that mean it's not welcome to say on hacker news?

Re: What we still don’t know about how A.I. is trained

#20
post #7
post #3

[flagged]

You can oppose literally every technology by saying "oh it'll use energy - won't someone think of climate change?" The copy+paste of "CO2 emissions" and "energy" has been one of the most successful petroleum company propaganda coups in history.

The article says trading Gpt4 cost 284 tons of CO2, which is, in the scheme of things, quite small. Yearly emissions for a person in the US is ~16 tons, so /training/ the giant model is equivalent to the emissions of less than twenty people in a country of 400 million. Sure, every bit counts, but this is laughable as a criticism.
Post reply on HN