Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

321–330 of 648 posts

Re: GPT-4 details leaked?

#321

Post by mdp2021 »

YeGoblynQueenne wrote: post #229
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

> as it is practiced today

Reminder that such situation - science being out of focus, below the attempt to obtain practical results "empirically" - is not necessary but contingent.

The effort towards "science" is just postponed and/or relegated to other researchers in the same field.

And anyway, it is not that the effort towards "explanation" is completely absent. The situation is that many are working on the prospect of achieving big results through bets.

Re: GPT-4 details leaked?

#322

Post by tyingq »

ben_w wrote: post #317
tyingq wrote: post #316

Earlier quoted context omitted.

The written testimony didn't say "ban", but it did stress fairly complex regulation. "First, it is vital that AI companies–especially those working on the most powerful models–adhere to an appropriate set of safety requirements, including internal and external testing prior to release and publication of evaluation results. To ensure this, the U.S. government should consider a combination of licensing or registration…

> The written testimony didn't say "ban", but it did stress fairly complex regulation You feel that quotation justifies the adjective "complex"?

I didn't mean "complex" in some sort of flippant, derogatory way. The language is suggesting regulation with active government involvement and oversight (licensing, for example). In the range of regulations in the software world, that would be on the far right end. ITAR, for example, requires licensing. People regularly describe the process as "complex".

Re: GPT-4 details leaked?

#323

Post by mdp2021 »

mensetmanusman wrote: post #319
YeGoblynQueenne wrote: post #229

Earlier quoted context omitted.

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

{Hypothesis, test, loop} is the scientific method, and I can guarantee it is being used when fine tuning an LLM.

...but in a technical context, not in a scientific one.

One thing is identifying (age of copper; age of bronze) the best ways of smelting ore to obtain the metal through trial and error, another is to try and understand the nature of materials.

Re: GPT-4 details leaked?

#324

Post by swexbe »

amelius wrote: post #305
ethanbond wrote: post #302

Earlier quoted context omitted.

That seems like it pushes it further from science, no? The point of a well-crafted hypothesis is that if it doesn’t bear out, you know that it’s because one+ of your assumptions was wrong. Your ability to continue your scientific inquiry is pretty much == your ability to then identify which assumption was wrong.

Not sure what you are saying here. Perhaps an analogy helps. Psychology is a science. You can make falsifiable statements about the human brain. You will need experiments to build and test theories. It's the same with deep learning. With computer "science" (and math) it's not the same. You can reason completely about your subjects, i.e. you can determine if something will or will not work just by reasoning, no experi…

Seems like the distinction is mainly between which tools are available to you as a scientist (at least if we stick to comp-sci, math is in a league of it's own). When, or if, we can completely model a human brain, a psychologist would no longer need to perform experiments to test their theories.

Given enough computing power, most theories could theoretically by proven or falsified purely through reasoning.

Re: GPT-4 details leaked?

#325

Post by mensetmanusman »

mdp2021 wrote: post #323
mensetmanusman wrote: post #319

Earlier quoted context omitted.

{Hypothesis, test, loop} is the scientific method, and I can guarantee it is being used when fine tuning an LLM.

...but in a technical context, not in a scientific one. One thing is identifying (age of copper; age of bronze) the best ways of smelting ore to obtain the metal through trial and error, another is to try and understand the nature of materials.

Those steps are both necessary and usually take decades to build understanding.

Re: GPT-4 details leaked?

#326

Post by mensetmanusman »

DeathArrow wrote: post #258
jona-f wrote: post #167

Earlier quoted context omitted.

Yes, they are doing the improving, but then you need loads of money to do the learning no university can afford. So now big tech is hiring promising university researchers for good money to scale up their research. This could be solved by massive decentralization where millions of users provide compute with their gpus and i think it will be at some point, cause i believe foss is more powerful than this openai bs. The…

> There are people working on this, but afaik the techniques aren't quite there. You need a different kind of model with much more parallelization then what is currently used. What if crypto is switching from mindless hashing as proof-of-work to training AI models as proof-of-work? That would mean suddenly big computing resources are available.

Yeah I keep telling people the universe played a joke on us with the crypto winter happening right before GPT-4 being released.

Flip the timeline and crypto from the start would have had this as a goal.

Re: GPT-4 details leaked?

#327

Post by Dylan16807 »

BoorishBears wrote: post #261
Dylan16807 wrote: post #113

Earlier quoted context omitted.

A "capped" profit of 100x is absolutely ridiculous. That is not even the slightest attempt to stay in the same ballpark as nonprofit. Blaming any loss of Elon money for that is spin. And he still gave them a hundred million.

Not sure what you're on about, both my comment and the article are about how they went for-profit. They went for-profit because they needed to raise. They raised after Elon reneged a fraction of the way into his promise of $1B. (and not 100MM deep: he silently revised his figures after TC started digging.) It's not complicated but you seem to want to make it complicated.

They didn't need to go that far, and they gaslit people about it. Elon is not responsible for either of those.

And was their existing money pile actually not enough?

Re: GPT-4 details leaked?

#328

Post by 0xDEF »

arthur_sav wrote: post #241
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

I would love to use these... but they suck. They don't even come close to what OpenAI offers.

I think most of HN has only tried ChatGPT with GPT-3.5 in December 2022 when the OpenAI servers were getting hammered. That is why they're impressed by these slow and low quality local models and mistakenly think they're on par with OpenAI's offerings.

Honestly even reddit and teenagers on TikTok have a more accurate view on OpenAI vs. local LLMs than HN.

Re: GPT-4 details leaked?

#329

Post by mensetmanusman »

namaria wrote: post #267
ben_w wrote: post #236

Earlier quoted context omitted.

The public position (as opposed to the rumour mills) is that they're not working on a 5, and don't intend to at least until they can figure out how to do it safely.

And safely here means non disruptive to established businesses. If they create a tool so powerful that it can replace entire professional classes, that would just mean people can bypass the employers and get their value directly from an API. OpenAI needs to make sure they have a business model where they can charge enterprise fees and provide value to corporations, not directly to people. ChatGPT was merely a publici…

Safety also might mean not providing a route/framework for an api loop to create a self destructing information virus that becomes the best worm in human history :)

Re: GPT-4 details leaked?

#330

Post by mportela »

mensetmanusman wrote: post #319
YeGoblynQueenne wrote: post #229

Earlier quoted context omitted.

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

{Hypothesis, test, loop} is the scientific method, and I can guarantee it is being used when fine tuning an LLM.

That's a necessary but insufficient condition for something to be considered scientific.
Post reply on HN