Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

81–90 of 648 posts

Re: GPT-4 details leaked?

#81

Post by satvikpendem »

PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

See also, Orca and Falcon models which are also open source. I'm not sure if any frontends support them yet.

Re: GPT-4 details leaked?

#82

Post by Roark66 »

The tweet is gone. What was in it?

Also, I'm dubious about this unsubstantiated claim. The biggest past innovation (training with human feedback) actually shrunk the size of a model. Compare Bloom-366B with falcon-40B (much better). I would be mildly surprised if it turned out Gpt4 has 1.8T parameters. (even if it's a composite model as they say)

The article says they use 16 experts 111B each. So the best thing to assume is probably that each of these experts is basically a fine tuned version of the same initial model for some problem domain.

Re: GPT-4 details leaked?

#83

Post by seanthemon »

svnt wrote: post #60
tlb wrote: post #44

Earlier quoted context omitted.

What percentile rank among college-educated Americans would that correspond to? I'd guess that takes it out of the top 0.1%, but not the top 1%.

Call it 100 million college-educated Americans. I don’t think 100,000 people can formulate working quantum computing algorithms in ten seconds. It is probably closer to 0.001%, but those people probably can’t describe 13th century medical technology very well.

Surprisingly you need to be quite adept at 14th century medicine in order to write quantum algorithms

Re: GPT-4 details leaked?

#85

Post by downWidOutaFite »

PostOnce wrote: post #38
BoorishBears wrote: post #34

Earlier quoted context omitted.

Why the vitriol towards OpenAI? If Elon hadn't pulled the rug out from under them after they refused his forceful takeover*, they wouldn't have had to go to Microsoft and they'd still be open. * a takeover which he predicated on the claim that OpenAI was "doomed to fail"

It's several reasons, first it's the lies and the abuse of a charity, that wouldn't be an issue if they had began as a private company instead of robbing a charity. But secondly, even if they were a private company, it's dishonest and reprehensible to claim to congress that you want to "protect the public" when you really only give a shit about protecting your moat, I'm not happy about that either. I'm also tired of…

The "regulatory capture" conspiracy theory makes no sense to me. It takes 9 figures in cash to create and run one of these super big models. Only big tech was ever going to create them, and big tech is already very experienced at navigating regulation, regulation wasn't ever going to stop them from competing. And in general, our democracy works better than the nihilist libertarians give it credit for.

Re: GPT-4 details leaked?

#86

Post by seanthemon »

jquery wrote: post #63
abrax3141 wrote: post #30

If it was trained on CS textbooks, they weren't very good ones. I asked it (GPT4) to write a quantum computer algorithm to square a number. It very confidently told me that to simplify the problem it would use two bits. Okay, fine. But then the algorithm it (again confidently) implemented did a left shift (which it reminded me was multiplying by 2, so it definitely intended this!) and then add the number to itself. I…

I must be getting old, because GPT-4 is basically magic compared to what has come before... and just a few months in people are happily dunking on it.

It's an ego problem imo, some folks have a co-dependency problem with being "smart" and now GenAi took away their trophy

Re: GPT-4 details leaked?

#87

Post by M3L0NM4N »

vouaobrasil wrote: post #80
carabiner wrote: post #59

Earlier quoted context omitted.

We must destroy OpenAI!

Not sure if you're being sarcastic but I agree. Humans should not have AI research or advanced AI at all. It (a) removes purpose from people, (b) presents a situation that is too alien for human minds to handle, (c) increases the addictiveness of technology and thereby pushes us further into growing the technological system, (d) crosses the "adaptability threshhold", i.e. the point at which the PACE of technological…

This is such a pessimistic view to take that I cannot even begin to describe how you are wrong.

Re: GPT-4 details leaked?

#88

Post by balls187 »

jquery wrote: post #63
abrax3141 wrote: post #30

If it was trained on CS textbooks, they weren't very good ones. I asked it (GPT4) to write a quantum computer algorithm to square a number. It very confidently told me that to simplify the problem it would use two bits. Okay, fine. But then the algorithm it (again confidently) implemented did a left shift (which it reminded me was multiplying by 2, so it definitely intended this!) and then add the number to itself. I…

I must be getting old, because GPT-4 is basically magic compared to what has come before... and just a few months in people are happily dunking on it.

jquery is definitely old.

The dunking makes sense to me—“AI is taking our Jobs” is a real concern, so pointing out how bad ChatGPT is at coding on an arguably coding social network is one way to control the narrative.

Re: GPT-4 details leaked?

#89

Post by courseofaction »

BoorishBears wrote: post #34
PostOnce wrote: post #19

"Open" AI, a charity to benefit us all by pushing and publishing the frontier of scientific knowledge. Nevermind, fuckers, actually it's just to take your jobs and make a few VCs richer. We'll keep the science a secret and try to pressure the government into making it illegal for you to compete with us. https://github.com/ggerganov/llama.cpp https://github.com/openlm-research/open_llama https://huggingface.co/TheBlok…

Why the vitriol towards OpenAI? If Elon hadn't pulled the rug out from under them after they refused his forceful takeover*, they wouldn't have had to go to Microsoft and they'd still be open. * a takeover which he predicated on the claim that OpenAI was "doomed to fail"

It was a massive bait and switch, luckily AI isn't powerful enough to take over the world we'd be done for.

Re: GPT-4 details leaked?

#90

Post by swyx »

chucknthem wrote: post #53
behnamoh wrote: post #27

Earlier quoted context omitted.

Google is jokingly behind in terms of LLMs. They've done a pretty good job at incorporating vision and audio ML models into their ecosystem, but they underestimated language.

How do you know? do you have insider knowledge of this or is it just based on what they share publically?

as a fun ancedote, the Google Bard's implicit code execution update from *last month*, advertised by Sundar... no longer works https://twitter.com/swyx/status/1678495067663925248

i'd love to know whats going on in that team.

Post reply on HN