Earlier quoted context omitted.
Every exponential is really an s curve?
Symmetries in Nature strike again! It's just like in Noether's theorem.
Noether's theorem has nothing to do with S-curves.
No "Zero-Shot" Without Exponential Data
101–110 of 123 posts
Re: No "Zero-Shot" Without Exponential Data
#102This feels like the worst possible outcome of the current AI hype. We've essentially been ripping off the entire internet and feeding it to the models already, spending many billions of dollars in the process. It's pretty much the largest possible dataset you can currently get, and due to the ever-increasing and now rapidly accelerated AI poisoning of the internet most likely the largest possible dataset which will e…
> It's pretty much the largest possible dataset you can currently get, [...]
No. What you describe encompasses only one poor modality: text. We have oodles more data, and can create almost arbitrary amounts more, by just eg pointing webcams at the world.
> We would've been better off if we had never done this.
Who is 'we'?
Re: No "Zero-Shot" Without Exponential Data
#103Earlier quoted context omitted.
> not-entirely-useless but still quite crappy 15 months ago, general-purpose LLMs that have not been specifically trained on legal reasoning could score better than 90% of humans on the multistate bar exam, and these are humans who actually completed law school. General-purpose LLMs get similar results in medicine, and when the models are fine-tuned for medical diagnosis they're even better. And that was more than a…
> 15 months ago, general-purpose LLMs that have not been specifically trained on legal reasoning could score better than 90% of humans on the multistate bar exam The claims of similar performance on coding problems were shown to be due to contamination of the training data on the tested problems. It did abysmal on problems made public after the model training cutoff. I don’t think anyone has tested contamination for…
You can see similar (poor) results if you give it a slightly tweaked puzzle that it’s seen before. The most recent example on Twitter was the farmer with the animals and the boat. If there are no constraints, it will give you an answer based on the tricks for the original unless you harass it.
Re: No "Zero-Shot" Without Exponential Data
#104Earlier quoted context omitted.
Even if LLMs (pre-trained transformers) turn out to be a dead end as far as AGI goes, there are productivity applications for them, and perhaps just as importantly interesting insights/confirmations about how the mind works and directions for future AGI research/architectures. The use cases for LLMs will no doubt grow as hallucinations are reduced, and they gain planning/reasoning ability over next couple of years. I…
Oh absolutely, LLMs are already causing a slaughter in applications where quality doesn't really matter to the company, like customer service. With minor improvements they're going to be a serious problem for any junior developer/lawyer/journalist/reviewer/artist/whatever, and if they ever fix the hallucinations issue it's game-changing. On the other hand, it's still a big "if" whether a general hallucinations soluti…
One less potential competitor employee we have to worry about in the future economy. Thank you for your service.
Re: No "Zero-Shot" Without Exponential Data
#105Earlier quoted context omitted.
"AI winter" was a political phenomenon that happened because the AI applications didn't hold water to the hype, and all of the gullible people that invested on them got burned out by the disparity. We are very clearly on that same path again. What leads to the conclusion that another winter is coming. But even the fact that people are talking about it is evidence it's not here yet, and as always with political phenom…
Except contemporary AI is useful on a day to day basis.
So were the developments before the last AI winter..
Re: No "Zero-Shot" Without Exponential Data
#106Better title: Image classification models suck at identifying nouns that they've rarely seen. Crucial context: - They're only looking at image models -- not LLMs, etc - Their models are tiny - A "concept" here just means "a noun." The authors index images via these nouns. - They didn't control for difficulty in visual representation/recognition of these exceptional infrequent, long-tail "concepts." If I didn't know a…
Odd how this went from top voted comment to lowest comment. What's the disagreement?
Re: No "Zero-Shot" Without Exponential Data
#107Not to derail this conversation but...
When I'm explaining AI stuff to family, the example I use is classification and I specifically use cats and dogs. I use the analogy of how you teach a toddler that this is a cat and that is a dog. Essentially repetition. And at first they get them mixed up and the parent will say "no, that's a dog" when they think it's a cat and so on.
But essentially, for a child learning the difference between a cat and a dog you only need to show them a handful of each and they'll generally get it from that point on.
That being said, why does it take ML millions (or billions) of images to be able to say "that's a cat" when a human does it on a handful (might be up to, say 100 but my point stands). Why can ML not do that yet?
I'm a dev for many years but not in AI, hence my ELI5 question :)
Edit: If the answer is massively long and complicated, perhaps if you could point me to some text (book, paper etc.) and I can read at my leisure.
Edit2: I just thought of something. Is it related to whether the child sees a still image or a live cat? So, for example, a still image is a single example of a cat standing in a particular position etc, whereas a moving, live cat, would be interpreted by the brain as many many still images, all processed individually? The end result being that, in fact the child, when seeing a live cat, actually sees thousands or millions of still images of the cat? It just popped into my head there :D
Re: No "Zero-Shot" Without Exponential Data
#108Not to derail this conversation but... When I'm explaining AI stuff to family, the example I use is classification and I specifically use cats and dogs. I use the analogy of how you teach a toddler that this is a cat and that is a dog. Essentially repetition. And at first they get them mixed up and the parent will say "no, that's a dog" when they think it's a cat and so on. But essentially, for a child learning the d…
Millions of years of evolution has trained the biological LLM in our brains to be good at fine tuning those concepts.
Re: No "Zero-Shot" Without Exponential Data
#109Not to derail this conversation but... When I'm explaining AI stuff to family, the example I use is classification and I specifically use cats and dogs. I use the analogy of how you teach a toddler that this is a cat and that is a dog. Essentially repetition. And at first they get them mixed up and the parent will say "no, that's a dog" when they think it's a cat and so on. But essentially, for a child learning the d…
Saying "it takes a hundred images for a human to learn" implies that you can take a baby/toddler/whatever, who has been blind all their life, restore their sight, show them 100 photos of dogs and cats, and expect them to know what's what.
You're ignoring the trillions of frames a toddler has seen before they get to the part where they can even understand what a photo is.
Re: No "Zero-Shot" Without Exponential Data
#110Earlier quoted context omitted.
How would the model know what zebra was if it had never seen it? Same is true for humans.
When I was little, zebras were described to me as black and white stripped horses. Without even seeing one, I'm sure anything who has seen a horse could then merge those two concepts to create a close to accurate picture of what a zebra is. If AI is supposed to resemble a human mind with ability to learn, then it must be able to learn from a blanker slate. You don't teach the human before it is born, and in this comp…
AI is not supposed to resemble a human mind. It's just supposed to be useful to us.