Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training for one trillion parameter model backed by Intel and US govt has begun

techradar.com

Post reply on HN
Print view View original

221–230 of 267 posts

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#221

Post by astrange »

Jensson wrote: post #138
nanis wrote: post #114

Earlier quoted context omitted.

> [The Queen] granted her charter to their corporation named Governor and Company of Merchants of London trading into the East Indies.[15] For a period of fifteen years, the charter awarded the company a monopoly[26] on English trade with all countries east of the Cape of Good Hope and west of the Straits of Magellan.[27] Any traders there without a licence from the company were liable to forfeiture of their ships an…

The government didn't tell them to kill or enslave people, they did that on their own since that was more profitable than not doing it. The government then stepped in and forced them to stop killing and enslaving, which made the world a better place, today we don't have many companies that kill or enslave thanks to governments.

Slavery isn't profitable, people do it for emotional or ideological reasons, or because they're not competent enough to run a business that has to actually trade.

This is literally why economics is called "the dismal science". Slaveowners called economists that when the economists told them to stop doing slavery.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#222

Post by astrange »

nyc_data_geek1 wrote: post #216
superhumanuser wrote: post #94

Earlier quoted context omitted.

Governments have killed more people than any other institution. Corporations have largely made our lives better.

Ask the people in Bhopal how DOW Chemical has made their lives better, or the workers in Shenzhen Apple factories, or the people making your clothes in the third world, or the people picking through trash mountains for a living, how much corporations have made their lives better.

The people working at Foxconn are doing it voluntarily because it pays better than their other alternatives. Same for Bangladesh clothing factories.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#223

Post by astrange »

Gormo wrote: post #191
andsoitis wrote: post #107

Earlier quoted context omitted.

Both governments and corporations have generally made our lives better. Both governments and corporations kill people (look up annual tobacco deaths, for just one fun example).

Aren't deaths from tobacco directly caused by intentional individual behavior? I'm not sure how that relates to either example.

Individual behavior doesn't really exist, you don't have choices outside the ones provided by society. It's also a bit silly to mention it in the context of a literally physically addictive drug.

Interestingly schizophrenic people almost universally smoke, and it's thought this is because it's a better treatment than the actual drugs we give them, though that's not proven.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#224

Post by astrange »

dist-epoch wrote: post #88
_heimdall wrote: post #44

I'm sure the government's mission is also to develop an AGI that benefits us all.

Agree, it would be much better to have Sam Altman develop it in the interest of private parties. Pure capitalism is best system of governance.

OpenAI literally is not pure capitalism. That's what all that stuff this weekend was about.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#225

Post by astrange »

vouaobrasil wrote: post #17
omneity wrote: post #14

Earlier quoted context omitted.

What makes you think this technology is out of control? If anything a 1T param model would only run in like 5-10 computers in this world. As much control as it gets. By the way this is doomer rhetoric. Bashing scientific advancements as dangerous or useless by trying to attach vacuously scary ideas that are as speculative as they are outlandish.

It's hardly vacuous. What about these scientific advancements? 1. Fossil fuel extraction -- we are killing our planet. 2. Plastic chemistry -- there are 5 trillion pieces of plastic in the ocean killing seabirds, turtles, fish ( https://doi.org/10.1371/journal.pone.0111913 ) 3. Concrete engineering -- building cities that move people away from a connection with nature 4. Veterinary medicine -- supporting global meat…

Are you a boomer?

> 3. Concrete engineering -- building cities that move people away from a connection with nature

This is 70s environmentalist thinking and it's terrible for the environment. Cities are good for it, because when you infill in cities you aren't touching the rest of the land.

America is unusually good about this. Most countries don't have our national park system.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#226

Post by nobodyandproud »

ComputerGuru wrote: post #152
Majromax wrote: post #41

Earlier quoted context omitted.

> I bet if you add up that data it's a lot. Let's Fermi estimate that. A 4k video stream is about 50 megabits/second. Let's say that humans have the equivalent of two of those going during waking hours, one for vision and one for everything else. Humans are awake for 18 hours/day, and we'll say a human's training is 'complete' at 25. Multiply that together, and you end up with 1.8e17 bytes, or 180 petabytes of data.…

This is completely and fundamentally an incorrect approach from start to finish. The human body - and the human mind - do have electrical and logic components but they are absolutely not digital logic. We do not see in “pixels”. The human mind is an analog process. Analog computing is insanely powerful and exponentially more efficient (time, energy, bandwidth) than digital computing but it is ridiculously hard to pac…

What about using energy cost as a stand-in for the data?

The net energy cost of training and maintaining humans vs. training and maintaining (updating and augmenting) AI models.

The human body is incredibly efficient but there’s always a larger ramp-up cost.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#227

Post by pacificmint »

rkagerer wrote: post #213
Jensson wrote: post #166

Earlier quoted context omitted.

That is almost surely a hallucination.

Curious, why do people gravitate to the term "hallucination" (as if it's a person not a tool) instead of just plainly stating that it's wrong?

If I ask it what color an orange it and it says blue, that would be wrong.

If you ask it a question and it makes up a completely fabricated story, like for example the case files in that recent legal case [1], then saying it was “wrong” doesn’t really seem to capture it.

Calling it a hallucination is a great analogy, because the model made up a plausible sounding, but completely fabricated story. It saw things that were never real.

[1] https://apnews.com/article/artificial-intelligence-chatgpt-c...

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#228

Post by jojobas »

Zanni wrote: post #31
jojobas wrote: post #30

Earlier quoted context omitted.

Can't tell if you're joking or serious. In the latter case, the first humans (H. Habilis) had about 1/2 of H. Sapiens brain to work with, and a much smaller fraction of neocortex. If that doesn't satisfy you, let's say I was speaking about some sort of human ancestor before that, which would have been about as dumb as chimps, unless you require proof of their dumbness as well.

Not your original objector, but ... I was willing to accept that early humans are dumb, but your explanation relies on evolution and not social interaction.

Humans didn't evolve knowledge, they evolved the capacity to get more knowledge.

All the new knowledge they got they figured out from the environment.

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#229

Post by rvba »

dwaltrip wrote: post #187
raccoonDivider wrote: post #179

Earlier quoted context omitted.

I just realized that the NSA has probably been able to train GPT-4 equivalents on _all the data_ for a while now. We'll probably never learn about it but that's maybe scarier than just the Snowden collection story because LLMs are so good at retrieval.

Holy shit, you are right. They probably have 10-100x the data used to train gpt-4. Decades of every text message, phone call transcript, and so on. I can’t believe I haven’t seen anyone mention that yet. People keep saying we don’t have enough data. I think there is a lot more data than we realize, even ignoring things like NSA.

When chatGPT came out I asked it "which living politician is an unknown spy" and the question got blocked

Re: Training for one trillion parameter model backed by Intel and US govt has begun

#230

Post by _heimdall »

Mountain_Skies wrote: post #129
_heimdall wrote: post #86

Earlier quoted context omitted.

It's a logical trap to assume we have to pick who develops an AGI rather than deciding if we want one developed at all. Even in that scenario, I'm not really sure what the distinction would end up being. At least in the US, our government and corporations are so tightly coupled at this point that we're really toeing the waters of Fascism. If a corporation develops an AGI our government can have access to it whenever…

In isolation, perhaps we could choose to not develop such large models, but the globe shrinks every day. If one country declines to create such a beast, other countries still will create their own beast. From what I've seen so far, it's mostly a brute force issue now, not a specialized knowledge issue. Perhaps if all the rumors spilling out of OpenAI about Q* are true, then other countries can be blocked due to lack…

We can choose to not accept the tragedy of the commons. There's always the risk that your adversary continues on down that path, but is the risk of losing after making a moral stand really worse than the risk of everyone racing down the dangerous path?
Post reply on HN