Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-4 details leaked?

threadreaderapp.com

Post reply on HN
Print view View original

511–520 of 648 posts

Re: GPT-4 details leaked?

#511

Post by sdenton4 »

hyperthesis wrote: post #499
sdenton4 wrote: post #477

Earlier quoted context omitted.

Add enough epicycles and you've got a Fourier transform... Epicycles were a great idea but applied for the wrong reason.

Oh, they worked really well; they were still used for numerical calculation even after Galileo. But they weren't a "demonstration of knowledge and understanding" of planetary motion.

Understanding is a moving target, though. Newtonian mechanics was incorrect for modeling the solar system as well - as it was eventually superseded - but that doesn't mean it wasn't a scientific understanding of planetary motion.

Epicycles gave a better description of the movement of the planets, based on the observations available, which was entirely falsifiable. They were eventually superseded, and that's science at work.

Re: GPT-4 details leaked?

#512

Post by javajosh »

sdenton4 wrote: post #485
nix0n wrote: post #440

Earlier quoted context omitted.

The reason that this is Engineering as opposed to Science, is that the hypothesis is just, "hey, maybe this will work". Nobody has really explained why it works.

Ask Feynman why magnets works... And he'll say no one knows. https://www.youtube.com/watch?v=36GT2zI8lVA It's the same in ML. Good predictions do come from real ideas about how the systems work - ideas in information theory, entropy, cognitive science, statistical mechanics and so on - formulated in a context of our existing understanding and prior results. That's science.

Feynman was a genius. It takes a genius to say something like that, and it's surprising how many "scientists" are upset by statements like that, because they are so proud of their hard-won capabilities in modelling. Their curl and divergence operators, their Maxwell's equations... they forget that the map is not the terrain, and the universe is, at it's root, irreducibly mysterious, and the fact that we understand anything about it at all is gobsmacking.

Re: GPT-4 details leaked?

#513

Post by ccooffee »

dragonwriter wrote: post #495
flangola7 wrote: post #493

Earlier quoted context omitted.

GPT-4 is far more capable than LLaMA. Just as one area of impact - captchas would become permanently ineffective. If you're experienced in developing captchas and everything they do for us, you know the implications of that alone lead to a very dystopian internet and world. I like to answer a question with a question: if you sit and think about it, what both unintentional misuses and intentional abuses can you think…

> If you're experienced in developing captchas and everything they do for us What they have done, fairly overtly for a long time, is train AI to defeat captchas. That this was self-limiting was somewhat obvious.

Hey, captchas also prevent disabled people from using the internet!

Re: GPT-4 details leaked?

#514

Post by DonHopkins »

droopyEyelids wrote: post #471
jncfhnb wrote: post #466

Earlier quoted context omitted.

People struggle to differentiate the scientific method and the lifecycle of academia.

In some ways academia is akin to a religion or cult that sprang up around the scientific method, and the cult has now diverged as far from the seed as far right christianity has from jesus’s teachings

Oh come on, have some perspective: academia has millions more people to slaughter before they catch up with the cult of Christianity.

https://apholt.com/2019/01/30/death-estimates-for-the-crusad...

John Shertzer Hittell– Estimates 1,000,000 total dead for the crusades to the East covering the period from 1095 to 1291.

“In the two centuries of this warfare one million persons had been slain, but it had not been without some compensations.” John Shertzer Hittell, A Brief History of Culture (1874), p.137.

[...]

John M. Robertson– Estimates 9,000,000 total dead for the crusades to the East covering the period from 1095 to 1291.

“It is a reasonable calculation that in the two centuries from the first crusade to the fall of Acre (1291) there had perished, in the attempts to recover and hold the Holy Land, nine millions of human beings, at least half of them Christians. Misery and chronic pestilence had slain most; but the mere carnage had been stupendous.” John M. Robertson, A Short History of Christianity (1902), p. 278.

Re: GPT-4 details leaked?

#515

Post by Dig1t »

CamperBob2 wrote: post #67
valval wrote: post #62

Earlier quoted context omitted.

If it involves Elon being a bad guy, it is certain to have HNers salivating at the thought.

At this point, after attempting to start a literal fight with Zuckerberg, Musk is proposing a penis-measuring contest. He appears to be decompensating in real time. If that's salivation fodder, so be it, but it just makes me sad. You hate to see it happen... or at least I do.

I mean, seemed like an obvious joke to me. I thought it was funny.

Re: GPT-4 details leaked?

#516

Post by xeckr »

YeGoblynQueenne wrote: post #239
xeckr wrote: post #135

If this is true, then: 1. Training took 21 yottaflops. When was the last time you saw the yotta- prefix for anything? 2. The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat.

>> The training cost of GPT-4 is now only 1/3 of what it was about a year ago. It is absolutely staggering how quickly the price of training an LLM is dropping, which is great news for open source. The google memo was right about the lack of a moat. That really doesn't change anything at all. The more training large models gets cheaper, the more large corporations are able to train larger models than everyone else. S…

>Suppose the gross price of rice was $0.001 a kg. That's dirt cheap! Yet, if I had a million dollars and you had a thousand dollars, I could still buy a thousand times more rice than you.

...and billions would be lifted out of poverty, and world hunger would be solved. The rice metaphor doesn't quite apply here.

If the price of GPU training continues to drop at the present rate, then it would be possible to train a GPT-4 level LLM on a $3000 card in 10 years. The ability to run inference on it would come way sooner.

Re: GPT-4 details leaked?

#517

Post by npteljes »

grepfru_it wrote: post #469
npteljes wrote: post #285

Earlier quoted context omitted.

I honestly feel like that, as a "Software Engineer". As the digital world is new, and full of similar concepts to the old analogue world, it's natural that we borrow names along with the concepts. Similar to how a lot of things in the sea are named after land things, with a marine prefix, like the sea horse, sea star, sea cucumber, and so on. And now in IT we have engineers, architects, rockstars, tribes, and science…

Computer science is the study, that is looking at how computers and the use of computers can benefit mankind (in my short lay version). Software engineering is a subset of computer science where you build a real world application of the studies. The algorithms behind the Facebook feed and your favorite AI/ML product is the science. The code that powers the feed and chatgpt is the engineering. Wikipedia is not wrong I…

I agree with you. There's a lot of engineering, and science going into IT. But I'm a Software Engineer too, and while I'm clever and do clever things, I can assure you that there's no rigor or reason to call it engineering. This version of engineering is at most the re-use of the word, like how the word art is not just covering artistic expressions, but skillfulness too, even though when you apply a skill artfully, you don't create art. Similarly, I do a lot that takes ideas of engineering, but really I'm just writing code and kinda design a smallish system maybe.

Re: GPT-4 details leaked?

#518

Post by droopyEyelids »

londons_explore wrote: post #506
droopyEyelids wrote: post #473

Earlier quoted context omitted.

And that gets more difficult every day, as previously accesible sources of data turn off their api. though google may have something up its sleeve with the corpus of google books! I have been wondering if openAI secretly pulled in scihub or zlibrary to neutralize that potential advantage.

Are there any stats on how many words are in google books, vs how many words are on the open web? My feeling is that the web has a lot more on it than the total of all libraries - simply because anyone can start a blog, but publishing a book requires quite some commitment.

I think you're right but I also think the text in published books would be at least an order of magnitude more valuable than the same length of text from the web

Re: GPT-4 details leaked?

#519

Post by droopyEyelids »

DonHopkins wrote: post #514
droopyEyelids wrote: post #471

Earlier quoted context omitted.

In some ways academia is akin to a religion or cult that sprang up around the scientific method, and the cult has now diverged as far from the seed as far right christianity has from jesus’s teachings

Oh come on, have some perspective: academia has millions more people to slaughter before they catch up with the cult of Christianity. https://apholt.com/2019/01/30/death-estimates-for-the-crusad... John Shertzer Hittell– Estimates 1,000,000 total dead for the crusades to the East covering the period from 1095 to 1291. “In the two centuries of this warfare one million persons had been slain, but it had not been withou…

You might be stuck in an argumentative mode of thought, it seems like you replied to an interpretation of my comment that left out some salient details.

Re: GPT-4 details leaked?

#520

Post by dang »

tillinghast wrote: post #37

[flagged]

Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.

If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Post reply on HN