Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

381–390 of 648 posts

Re: GPT-4 details leaked?

#381

Post by jona-f »

pas wrote: post #334
jona-f wrote: post #304

Earlier quoted context omitted.

That would indeed be neat, no idea if that is possible safely. Blockchain technology sure looks like a good fit for the organization of such a decentralized model. The reaction of the people of hn to 'blockchain-ai', probably won't be kind though, lol.

Blockchains depend on small inputs and outputs, and easy verifiability of the computation. AI training is none of that. It's a massively interconnected problem, Google and NVIDIA built traditionalish fast-interconnect supercomputers to do it. Maybe it's possible to implement it in a decentralized way with not-completely-useless performance, but by then OpenAI and others will be even more ahead. (Sure, it'd be good to…

Yeah, that's the current state of course, I'm betting on future developments. Neural networks are still in their infancy. Stable diffusion showed everyone how much more powerful open models are (at generating porn /s). Chatgpt is severely limited, cause Openai fear public outrage, litigation and regulation.

Re: GPT-4 details leaked?

#382

Post by immibis »

Xeoncross wrote: post #359
michaelcampbell wrote: post #356

Earlier quoted context omitted.

> We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. This is essentially the Capitalist Credo, expressed in practical vs theoretical terms.

> pressure the government into making it Not really, Capitalism is free-trade between two parties. The more government involvement you have the more it moves towards socialism or communism where the government controls trade. At least this was the historical meaning. These days Capitalism is being redefined to mean private (non-government) Communism. That is, power concentrated in the hands of a few.

Capitalism is not free trade between two parties. That's free market. Capitalism is control by capital.

Re: GPT-4 details leaked?

#383

Post by onemoresoop »

Joeri wrote: post #193
lacrimacida wrote: post #122

Earlier quoted context omitted.

It actually does make a difference. The genie is out of the bottle partially but it depends a lot on what we allow it it be used on. If we sit idly and allow for ingesting all what’s written for instance, including whats currently written and let bros make derivative works for a quick buck then we mostly killed the writer’s incentive to write or publish. If we slow down and not allow ripping one another off it could…

You can slow things down, but not by more than a few years, because of the gradual democratization of training foundation models. Right now training a model competitive with chatgpt can be done for $150K (microsoft orca 13b). In a few years the cost will be low enough that individuals can train models. At that point regulating it will require draconian dictatorships. I’m also very wary of the copyright angle on this,…

I don't think there's a problem with the pace of "progress", the problem is what it is feed and we could definitely make changes around that. For example, if I were an author I wouldn't want this stuff to be be feed in without my permission.

Re: GPT-4 details leaked?

#384

Post by londons_explore »

fragmede wrote: post #186
JimmyRuska wrote: post #178

Earlier quoted context omitted.

Well open AI raised eye brows by crawling the internet and using everyone's data to make a commercial product One day some new startup will train on all of libgen and torrent networks, but it will be very hard to prove. You'll keep getting these gaps up in questionable morality and legality, and even openai will complain about playing fair

Google Classroom, teenager's essays, written by humans, for learning what it means to be human, and graded by humans, is a richer dataset than anything else I can think of that anyone else couldn't get their hands on.

An awful lot of teachers can grade a 10 page essay in about 90 seconds...

Skim read it, mark out some grammar errors, assign it a grade based on the quality of the opening and closing paragraphs.

Re: GPT-4 details leaked?

#385

Post by johnnyworker »

motoxpro wrote: post #222
johnnyworker wrote: post #183

Earlier quoted context omitted.

The point is that it's infinitely better in not being there "just to take your jobs and make a few VCs richer". Nobody even claimed it's more performant. It's like the difference getting nothing, but keeping your land, and getting glass pearls, losing your land. You have to completely ignore the meat of the argument to even pretend there is a contest. And this is without considering what happened if we stopped feedin…

There is no way I am going to spin up my own worse LLM so a few people will make less money. Even if it was 1-5% better. It's just not worth the time.

It's not "a few people making less money", it's a few gigantic monoliths carving up the future, like blind watch-destroying gods -- or at least wanting to, no matter how nicely they dress it up. And it's not about utility or chance of success for everyone, either, but rather trying to do something in an ethical or more clean way just because that's more fun for them.

But I have to admit to being an idealist, and while I disagree with you because of that, I don't think you should be downvoted for basically just bringing up the majority position. It's easy to complain over people not being starry-eyed idealists that make great personal sacrifices to bring along an utopia for people in 10 generations, or whatever. It's way harder to find and teach the joy of doing something for the sake of doing it, and at the same time come up with medium and long-term ideas that are realistic enough to make working towards them fulfilling, but also genuinely beautiful and true. The whole "rather than teaching to build a ship (we can't even agree on!), teach people how to long for the ocean" thing. It's a really hard problem.

Re: GPT-4 details leaked?

#386

Post by immibis »

bayindirh wrote: post #109
maximamas wrote: post #101

Earlier quoted context omitted.

It’s ironic you say: “we are playing with fire.” Playing with fire is, in large part, literally how humans have come to dominate this planet. Why stop now?

Maybe because we're on the verge of being able to create fires which can actually consume the only home we have? Playing with fire is in large part an ego and greed issue. Yes, it allows us to dominate, but at what cost? I'd rather live a more balanced life than a greedy and ego driven life. I may not own the world, but I can be happy and sleep sound at night, and that matters.

On the verge? We set that fire in motion a century ago. Our home is nearly consumed.

Today is the hottest day in recorded history.

Yesterday was the hottest day in recorded history.

The day before yesterday was the hottest day in recorded history.

The day before the day before yesterday was the hottest day in recorded history.

The day before the day before the day before yesterday was the hottest day in recorded history.

Re: GPT-4 details leaked?

#387

Post by makestuff »

londons_explore wrote: post #373
xeckr wrote: post #135

If this is true, then: 1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything? 2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

> The google memo was right about the lack of a moat. 5 months on, and nobody has yet beaten their result quality. I think there is a moat. Also, I think for many usecases, smarter is better. If a few cents can buy a more accurate answer, then it is always worth paying those few cents. So, while more hardware and more data can train a bigger better model, then that is the moat.

The moat is there until someone releases (or leaks) comparable training data.

Re: GPT-4 details leaked?

#388

Post by blitz_skull »

criley2 wrote: post #372
mensetmanusman wrote: post #319

Earlier quoted context omitted.

{Hypothesis, test, loop} is the scientific method, and I can guarantee it is being used when fine tuning an LLM.

It is not the scientific method at all. You didn't even include an analyze or a publish step, two critical components to the scientific method. The point is that science is a thoughtful, methodical, recorded, repeatable process that is scrutinized by not just your friends but by the entire world, including those who compete against you. In science, it's normal for your competitor to recreate your experiment exactly t…

Umm.. no. I’m pretty sure when science was “invented” (whatever that means), there was no way to share your research with “the world”. You wrote it down in a book in you cared to reproduce it.

Stop gate-keeping science. Observe, Predict, Test, Repeat. That is science. Kids do science when they watch adults do things. Just because you aren’t sitting in a lab publishing things no one reads doesn’t mean you can’t “do science.”

Re: GPT-4 details leaked?

#389

Post by captainbland »

PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

Enshitification comes fast these days.

Re: GPT-4 details leaked?

#390

Post by 2OEH8eoCRo0 »

xeckr wrote: post #135

If this is true, then: 1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything? 2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

> great news for open source.

Yes, and great news for shills, bad actors, agitators, trolls, foreign intel, and propagandists. I'm impressed by the tech but terrified because for once I cannot conceive of what this means for the future. My guess is that this kills the open web and laws get passed which bury it.

Post reply on HN