801–810 of 1001 posts
I just finished reading the 'paper' and I'm astonished that they aren't even publishing the # of parameters or even a vague outline of the architecture changes. It feels like such a slap in the face to all the academic AI researchers that their work is built off over the years, to just say 'yeah we're not telling you how any of this is possible because reasons'. Not even the damned parameter count. Christ.
After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
> As a professional...why not do this? Because your clients do not allow you to share their data with third parties?
Most of the comments here are denial and goalpost shifting... GPT-4 has different strengths and weaknesses from humans, but it is now in the general realm of human intelligence vs being far below that with GPT-3. Another jump past GPT-4 of the same magnitude, would greatly surpass human cognitive abilities and present a danger to humanity.
After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
> As a professional...why not do this? Because your clients do not allow you to share their data with third parties?
I'll be finishing my interventional radiology fellowship this year. I remember in 2016 when Geoffrey Hinton said, "We should stop training radiologists now," the radiology community was aghast and in-denial. My undergrad and masters were in computer science, and I felt, "yes, that's about right." If you were starting a diagnostic radiology residency, including intern year and fellowship, you'd just be finishing now.…
As another radiologist, I'm not sure how you can say this with a straight face? If anything the minimal progress that has been made since Hinton made this claim should be encouraging people to pursue radiology training. As with other areas of medicine that have better AI (interpreting ECGs for example) all this will do is make our lives easier. AI is not an existential threat to radiology (or pathology for that matte…
A scattered history of labs probably provides an opportunity to notice something early, even if you don't know what you are looking for. But humans are categorically bad at detecting complex patterns in tabular numbers. Could routinely feeding people's lab history into a model serve as a viable early warning system for problems no one thought to look for yet?
It's interesting that everyone is talking about programmers being replaced by AI, but the model did far better on the humanities type subjects than on the programming tests.
Because programming tests are hard.
Well-trained programmers would 90% fail the leetcode hard question with zero-shot. Preparations are important.
After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
because "open"AI logs everything that goes in and out of the model?
After watching the demos I'm convinced that the new context length will have the biggest impact. The ability to dump 32k tokens into a prompt (25,000 words) seems like it will drastically expand the reasoning capability and number of use cases. A doctor can put an entire patient's medical history in the prompt, a lawyer an entire case history, etc. As a professional...why not do this? There's a non-zero chance that i…
you don't see a real problem there?
The comments on this thread are proof of the AI effect: People will continually push the goal posts back as progress occurs. “Meh, it’s just a fancy word predictor. It’s not actually useful.” “Boring, it’s just memorizing answers. And it scored in the lowest percentile anyways”. “Sure, it’s in the top percentile now but honestly are those tests that hard? Besides, it can’t do anything with images.” “Ok, it takes imag…
There isn't and was never any movement of goalposts. They have been exactly the same for 70 years. We want creative systems (in the Deutschian sense) that can create new explanatory theories, which lead to actual new knowledge. When an AI is capable of creating new explanatory theories that are GOOD (not world salad), we will have human-like AGI. GPT is no closer to this goal than ELIZA (though it is much more useful…
From the paper: > Given both the competitive landscape and the safety implications of large-scale models like GPT-4, this report contains no further details about the architecture (including model size), hardware, training compute, dataset construction, training method, or similar. I'm curious whether they have continued to scale up model size/compute significantly or if they have managed to make significant innovati…
I bet they use CLIP to caption the image and feed the text of the caption into GPT, but that's just a guess.