Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

301–310 of 648 posts

Re: GPT-4 details leaked?

#301

Post by riku_iki »

ben_w wrote: post #242
grondilu wrote: post #214

Earlier quoted context omitted.

> try to pressure the government into making it illegal for you to compete with us. I mean the guy who created GPT-4 literally demanded a ban of any system more powerful than GPT-4.

Are you sure that's not a quote from a game of telephone? What I've seen from the horse's mouth is more like: """There are several other areas I mentioned in my written testimony where I believe that companies like ours can partner with governments, including ensuring that the most powerful AI models adhere to a set of safety requirements, facilitating processes to develop and update safety measures, and examining op…

yes, they want few companies (open ai, goog, not sure who are others) gate keep potential competition through safety certification.

Re: GPT-4 details leaked?

#302

Post by ethanbond »

amelius wrote: post #292
swexbe wrote: post #282

Earlier quoted context omitted.

If my BST implementation isn't sorted, it also doesn't work.

The difference is that you can reason about why it didn't work. With deep learning models not so. They are too big to reason about in the same sense as your BST algorithm. Hence, you need a scientific approach to construct them. I.e., with lots of experimenting, hypotheses, etc.

That seems like it pushes it further from science, no? The point of a well-crafted hypothesis is that if it doesn’t bear out, you know that it’s because one+ of your assumptions was wrong. Your ability to continue your scientific inquiry is pretty much == your ability to then identify which assumption was wrong.

Re: GPT-4 details leaked?

#303

Post by ly3xqhl8g9 »

notahacker wrote: post #273
ly3xqhl8g9 wrote: post #263

Earlier quoted context omitted.

Most of them already don't get paid, hence the strike [1]. [1] "Writers say they want a living wage as streaming devalues their work even as it demands more of their time.", https://www.indiewire.com/news/business/writers-strike-2023-...

No, they get paid but would, understandably, like to be paid more. Something which would be even less likely to happen if copyright was abolished so anybody could use their output at zero cost.

From what I understand they are not getting paid since the paradigm changed: "the way it's changed is most of these streaming services focus on a metric called ARPU, which is the average revenue per user", "the main difference is it used to be that the incentives were linked. So the writers and the studios were trying to get people to watch the show and get high ratings and get people to pay attention to the show. Today, the streamer is trying to get people to subscribe to their service, so they're looking more at the aggregate of the service versus the individual show." [1]

[1] https://www.npr.org/2023/05/03/1173612099/why-writers-are-ha...

Re: GPT-4 details leaked?

#304

Post by jona-f »

DeathArrow wrote: post #258
jona-f wrote: post #167

Earlier quoted context omitted.

Yes, they are doing the improving, but then you need loads of money to do the learning no university can afford. So now big tech is hiring promising university researchers for good money to scale up their research. This could be solved by massive decentralization where millions of users provide compute with their gpus and i think it will be at some point, cause i believe foss is more powerful than this openai bs. The…

> There are people working on this, but afaik the techniques aren't quite there. You need a different kind of model with much more parallelization then what is currently used. What if crypto is switching from mindless hashing as proof-of-work to training AI models as proof-of-work? That would mean suddenly big computing resources are available.

That would indeed be neat, no idea if that is possible safely. Blockchain technology sure looks like a good fit for the organization of such a decentralized model. The reaction of the people of hn to 'blockchain-ai', probably won't be kind though, lol.

Re: GPT-4 details leaked?

#305

Post by amelius »

ethanbond wrote: post #302
amelius wrote: post #292

Earlier quoted context omitted.

The difference is that you can reason about why it didn't work. With deep learning models not so. They are too big to reason about in the same sense as your BST algorithm. Hence, you need a scientific approach to construct them. I.e., with lots of experimenting, hypotheses, etc.

That seems like it pushes it further from science, no? The point of a well-crafted hypothesis is that if it doesn’t bear out, you know that it’s because one+ of your assumptions was wrong. Your ability to continue your scientific inquiry is pretty much == your ability to then identify which assumption was wrong.

Not sure what you are saying here. Perhaps an analogy helps.

Psychology is a science. You can make falsifiable statements about the human brain. You will need experiments to build and test theories. It's the same with deep learning.

With computer "science" (and math) it's not the same. You can reason completely about your subjects, i.e. you can determine if something will or will not work just by reasoning, no experiments needed.

For more information on the differences between math and science I recommend reading: https://en.wikipedia.org/wiki/Scientific_method#Relationship...

Re: GPT-4 details leaked?

#306

Post by ben_w »

ethanbond wrote: post #257
ben_w wrote: post #236

Earlier quoted context omitted.

The public position (as opposed to the rumour mills) is that they're not working on a 5, and don't intend to at least until they can figure out how to do it safely.

[or until someone else starts catching up] The same dynamic that incentivized a mass rollout of these unaligned systems will be perfectly sufficient to incentivize mass rollout of stronger, also unaligned systems.

Yeah, that's one of Yudkowsky's fears.

I'm not going to take his fears as gospel, as he's spent so long focussing on his fear there's a danger of availability heuristic/attention biases having him take the worst case for everything.

I'm still going to promote caution, as the more potent a tech the greater the downside of being wrong, but it currently looks like we can cooperate with each other in this IRL iterated prisoner's dilemma.

Re: GPT-4 details leaked?

#307

Post by ly3xqhl8g9 »

JumpCrisscross wrote: post #271
ly3xqhl8g9 wrote: post #253

Earlier quoted context omitted.

To nitpick, the second article is from 1970, back then a concept such as the Amazon Antitrust Paradox [1] would cause burning at the stake in the legal field. Similarly, we also have an Elsevier Antitrust Paradox, a Nature(.com) Antitrust Paradox, and so on. Just the fact that I cannot freely read an article published 53 years ago is beyond ludicrous. And also, if I pay JSTOR ($19.5/month) and read the 1970 article,…

> the fact that I cannot freely read an article published 53 years ago is beyond ludicrous And has as much to do with individual versus corporate power as the health of a single tree can speak for a forest. Nobody is saying the current situation is good or even sustainable. OP just made a big claim for which there is no evidence, despite many looking for it.

"there is no evidence"

As quoted above, "Ithaka's total revenue was $105 million in 2019, most of it ($79 million) from JSTOR service fees". All those $79 million and more JSTOR has stolen from the authors. Although JSTOR should have been dissolved long before this for effectively killing Aaron Swartz. But yes, there is no justice in this world. I suppose that's my main contention, why pretend anymore, we live in a society, yada yada.

Re: GPT-4 details leaked?

#308

Post by drexlspivey »

holoduke wrote: post #164
getmeinrn wrote: post #47

>If their cost in the cloud was about $1 per A100 hour, the training costs for this run alone would be about $63 million. If someone legitimate put together a crowd funding effort, I would donate a non-insignificant amount to train an open model. Has it been tried before?

Some kind of SETI project, but for training a high number parameter llm would be awesome.

How many people have A100s at home?

Re: GPT-4 details leaked?

#309

Post by notahacker »

ly3xqhl8g9 wrote: post #303
notahacker wrote: post #273

Earlier quoted context omitted.

No, they get paid but would, understandably, like to be paid more. Something which would be even less likely to happen if copyright was abolished so anybody could use their output at zero cost.

From what I understand they are not getting paid since the paradigm changed: "the way it's changed is most of these streaming services focus on a metric called ARPU, which is the average revenue per user", "the main difference is it used to be that the incentives were linked. So the writers and the studios were trying to get people to watch the show and get high ratings and get people to pay attention to the show. To…

I'm not sure quite why you're continuing to post articles explaining why writers get paid less as streaming services' revenue is less linked to the ratings of new shows as an argument in favour of marking down the value of the intellectual property they create to zero...

My original point still stands. Ask the writers. They're not getting paid nothing yet, and they won't approve of your passionate advocacy of a future in which they are paid nothing.

Re: GPT-4 details leaked?

#310

Post by jetrink »

YeGoblynQueenne wrote: post #239
xeckr wrote: post #135

If this is true, then: 1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything? 2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

>> The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat. That really doesn't change anything at all. The more training large models gets cheaper, the more large corporations are able to train larger models than everyone else. S…

At a certain point though, models become good enough for particular tasks. Once that happens for whatever my application is, I don't care if OpenAI has a model that's twice as good on some metric, because it's overkill for my use-case. I'm going to be happy using a smaller, cheaper model from a competitor.
Post reply on HN