Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

311–320 of 648 posts

Re: GPT-4 details leaked?

#311

Post by ben_w »

namaria wrote: post #279
ben_w wrote: post #276

Earlier quoted context omitted.

> And safely here means non disruptive to established businesses Why would OpenAI care about that? No, it means safety, as in not giving out dangerous answers that get people killed.

Because they belong to Microsoft. Because they are funded by rich people who own established businesses. Because they are the privileged .01% who are invested in the status quo. Because their whole networks consist of people with entrenched business interests. Pick one or more. Why would you think rich American engineers are in general any more worried about the overall safety of the world then their own self interes…

> Because they belong to Microsoft

They don't — 49%, and they get to buy themselves out of even that, and if they do that by eating MS's lunch? Then it would suck to be an MS shareholder.

And they've been cautious since before the investment, with the cautious announcement of GPT-2 a month before the change in corporate structure to allow investors.

But even if OpenAI were owned completely by MS, why should MS care? Disrupting all employment sounds like the raison d'être for most if not all of silicon valley.

At the very least, imagine all the employees MS could let go if GPT-4 was even 80% as good as the advertisements in the form of "buy my guide for how to use GPT to make a fortune with best selling books/award winning advertisements/fantastic business proposals" keep claiming.

> Why would you think rich American engineers are in general any more worried about the overall safety of the world then their own self interest?

One is a subset of the other.

People regularly disagree about how risky things are, but they don't generally pick "destroy all things" over "destroy that which is close to me" when they actually believe that's the real choice and not, say, a panicked Chicken Little (which is how I think LeCun sees Yudkowsky).

Re: GPT-4 details leaked?

#312

Post by hospitalJail »

CSMastermind wrote: post #2

Previously posted about here: https://news.ycombinator.com/item?id=36671588 and here: https://news.ycombinator.com/item?id=36674905 With the original source being: https://www.semianalysis.com/p/gpt-4-architecture-infrastruc... The twitter guy seems to just be paraphrasing the actual blog post? That's presumably why the tweets are now deleted. --- The fact that they're using MoE was news to me and very interesting. I…

Yeah the Mixture of Experts might have not been called out by name, but it was pretty obvious you were getting different models depending on the question.

It goes to show how LLMs are nothing like AGI. I think combining it with a calculator is just a bandaid. A useful bandaid, but its not going to be able to do science ever.

Re: GPT-4 details leaked?

#313

Post by hospitalJail »

quickthrower2 wrote: post #175
theLiminator wrote: post #170

Earlier quoted context omitted.

The real moat is an abundance of high quality data.

IMO the real moat right now is expertise / smart teams and cash.

The infrastructure/training libraries already exists. I'm sure you can get people who worked at scale that can figure out how to glue things together.

Reddit, twitter, etc.. raising prices is going to make it more expensive.

Re: GPT-4 details leaked?

#314

Post by namaria »

ben_w wrote: post #311
namaria wrote: post #279

Earlier quoted context omitted.

Because they belong to Microsoft. Because they are funded by rich people who own established businesses. Because they are the privileged .01% who are invested in the status quo. Because their whole networks consist of people with entrenched business interests. Pick one or more. Why would you think rich American engineers are in general any more worried about the overall safety of the world then their own self interes…

> Because they belong to Microsoft They don't — 49%, and they get to buy themselves out of even that, and if they do that by eating MS's lunch? Then it would suck to be an MS shareholder. And they've been cautious since before the investment, with the cautious announcement of GPT-2 a month before the change in corporate structure to allow investors. But even if OpenAI were owned completely by MS, why should MS care?…

MS not owning OpenAI outright is a technicality. "It would suck to be an MS shareholder" due to things done to subsidiaries of MS constitutes dereliction of duty by MS officers. MS is not Silicon Valley. They're based in Seattle and predate 95% of SV. Besides this is just one of my points. Even if everything you claimed were to be true, OpenAI still is, as I pointed out, part of the American Corporate establishment and despite their PR posturing, they won't provide us with technology to disrupt said establishment.

They are clearly trying to angle for maximizing the profitability of that establishment.

Re: GPT-4 details leaked?

#315

Post by ben_w »

namaria wrote: post #314
ben_w wrote: post #311

Earlier quoted context omitted.

> Because they belong to Microsoft They don't — 49%, and they get to buy themselves out of even that, and if they do that by eating MS's lunch? Then it would suck to be an MS shareholder. And they've been cautious since before the investment, with the cautious announcement of GPT-2 a month before the change in corporate structure to allow investors. But even if OpenAI were owned completely by MS, why should MS care?…

MS not owning OpenAI outright is a technicality. "It would suck to be an MS shareholder" due to things done to subsidiaries of MS constitutes dereliction of duty by MS officers. MS is not Silicon Valley. They're based in Seattle and predate 95% of SV. Besides this is just one of my points. Even if everything you claimed were to be true, OpenAI still is, as I pointed out, part of the American Corporate establishment a…

> OpenAI still is, as I pointed out, part of the American Corporate establishment and despite their PR posturing, they won't provide us with technology to disrupt said establishment

They're not acting like they're part of the American corporate establishment, and I don't understand why you think they are.

Rather than quote my other comment comparing against 3M, I'll link to it: https://news.ycombinator.com/item?id=36677182

Re: GPT-4 details leaked?

#316

Post by tyingq »

ben_w wrote: post #242
grondilu wrote: post #214

Earlier quoted context omitted.

> try to pressure the government into making it illegal for you to compete with us. I mean the guy who created GPT-4 literally demanded a ban of any system more powerful than GPT-4.

Are you sure that's not a quote from a game of telephone? What I've seen from the horse's mouth is more like: """There are several other areas I mentioned in my written testimony where I believe that companies like ours can partner with governments, including ensuring that the most powerful AI models adhere to a set of safety requirements, facilitating processes to develop and update safety measures, and examining op…

The written testimony didn't say "ban", but it did stress fairly complex regulation.

"First, it is vital that AI companies–especially those working on the most powerful models–adhere to an appropriate set of safety requirements, including internal and external testing prior to release and publication of evaluation results. To ensure this, the U.S. government should consider a combination of licensing or registration requirements for development and release of AI models above a crucial threshold of capabilities, alongside incentives for full compliance with these requirement"

https://www.judiciary.senate.gov/imo/media/doc/2023-05-16%20...

Re: GPT-4 details leaked?

#317

Post by ben_w »

tyingq wrote: post #316
ben_w wrote: post #242

Earlier quoted context omitted.

Are you sure that's not a quote from a game of telephone? What I've seen from the horse's mouth is more like: """There are several other areas I mentioned in my written testimony where I believe that companies like ours can partner with governments, including ensuring that the most powerful AI models adhere to a set of safety requirements, facilitating processes to develop and update safety measures, and examining op…

The written testimony didn't say "ban", but it did stress fairly complex regulation. "First, it is vital that AI companies–especially those working on the most powerful models–adhere to an appropriate set of safety requirements, including internal and external testing prior to release and publication of evaluation results. To ensure this, the U.S. government should consider a combination of licensing or registration…

> The written testimony didn't say "ban", but it did stress fairly complex regulation

You feel that quotation justifies the adjective "complex"?

Re: GPT-4 details leaked?

#318

Post by famouswaffles »

hospitalJail wrote: post #312
CSMastermind wrote: post #2

Previously posted about here: https://news.ycombinator.com/item?id=36671588 and here: https://news.ycombinator.com/item?id=36674905 With the original source being: https://www.semianalysis.com/p/gpt-4-architecture-infrastruc... The twitter guy seems to just be paraphrasing the actual blog post? That's presumably why the tweets are now deleted. --- The fact that they're using MoE was news to me and very interesting. I…

Yeah the Mixture of Experts might have not been called out by name, but it was pretty obvious you were getting different models depending on the question. It goes to show how LLMs are nothing like AGI. I think combining it with a calculator is just a bandaid. A useful bandaid, but its not going to be able to do science ever.

Sparse architectures are a way to theoritcally utilize only a small portion of a general models parameters at any given time. All "experts" are trained on the exact same data. They're not experts in the way you seem to think they are and they're certainly not wholly different models. The "experts" work at the token level. An expert for one token could be different from the expert chosen for the very next.

GPT-4 isn't "nothing like AGI" any more than its dense equivalent would be.

Re: GPT-4 details leaked?

#319

Post by mensetmanusman »

YeGoblynQueenne wrote: post #229
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

>> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. Just to be clear, there's no science being kept secret because there is no science being done. OpenAI's is a feat of engineering, borne aloft by a huge budget supporting a large team whose expertise lies in tuning neural net systems, and not in doing science. Machine learning, as it is practiced to…

{Hypothesis, test, loop} is the scientific method, and I can guarantee it is being used when fine tuning an LLM.

Re: GPT-4 details leaked?

#320

Post by ly3xqhl8g9 »

notahacker wrote: post #309
ly3xqhl8g9 wrote: post #303

Earlier quoted context omitted.

From what I understand they are not getting paid since the paradigm changed: "the way it's changed is most of these streaming services focus on a metric called ARPU, which is the average revenue per user", "the main difference is it used to be that the incentives were linked. So the writers and the studios were trying to get people to watch the show and get high ratings and get people to pay attention to the show. To…

I'm not sure quite why you're continuing to post articles explaining why writers get paid less as streaming services' revenue is less linked to the ratings of new shows as an argument in favour of marking down the value of the intellectual property they create to zero... My original point still stands. Ask the writers. They're not getting paid nothing yet , and they won't approve of your passionate advocacy of a futu…

I'm not quite sure why you are not understanding that with the streaming model of business the writers, and even the actors, no longer get residuals, here is another article [1]: yes, they get paid nothing.

I am advocating for the abolishment of copyright laws, not for not being rewarded for one's work.

[1] Euphoria actress Sydney Sweeney revealed in a recent interview with The Hollywood Reporter that "They don’t pay actors like they used to, and with streamers, you no longer get residuals.", https://nofilmschool.com/streaming-services-residuals

Post reply on HN