11–20 of 192 posts
Their power does not only lie in their ability to _generate_ new data, but to _model_ existing data.
If that is the definition of old school AI, I wonder how symbolic AI should be named.
Seems like the person who wrote the blog works in "classical" deep learning. So do I, so here's the fairest take I can come up with: "AI" has for recent memory been a marketing term anyway. Deep learning and variations have had a good run at being what people mean when they refer to AI, probably overweighting towards big convolution based computer vision models. Now, "AI" in people's minds means generative models. Th…
But neither the traditional nor generative models are "AI" in the sense that normal people think when they hear "AI".
[flagged]
There's about ~10% point improvement left (i.e, from 80% to 90%) before it starts to stagnate. We've seen the same with predictive models benchmarked on ImageNet et. al.
Not only that, but the innovation around this tech is also just getting started. It's immediately applicable for business use. The classical techniques still have their uses, of course.
Good answer but I feel that most users/people do not understand the difference between generative and predictive machine learning and that will probably cause unpredictable failures and false flags. So yes it has been overhyped in my opinion
We love the model because it speaks our language as if it's "one of us", but this may be deceiving, and the complete lack of model for truth is disturbing. Making silly poems is fun but the real uses are in medicine and biology, fields that are so complex that they are probably impenetrable to the human mind. Can Reinforcement learning alone create a model for the truth? The Transformer does not seem to have one, it only works with syntax and referencing. How much % of truthfulness can we achieve, and is it good enough for scientific applications? If a blocker is found in the interface between the model and reality, it will be a huge disappointment
Earlier quoted context omitted.
There's about ~10% point improvement left (i.e, from 80% to 90%) before it starts to stagnate. We've seen the same with predictive models benchmarked on ImageNet et. al.
By stagnate, you mean beating humankind at the task, right? :)
Seems like the person who wrote the blog works in "classical" deep learning. So do I, so here's the fairest take I can come up with: "AI" has for recent memory been a marketing term anyway. Deep learning and variations have had a good run at being what people mean when they refer to AI, probably overweighting towards big convolution based computer vision models. Now, "AI" in people's minds means generative models. Th…
>investors have become only interested in companies building generative AI, relegating those working on predictive models to “old school” AI. If that is the definition of old school AI, I wonder how symbolic AI should be named.
When the generative model is autoregressive (autocomplete), it can easily be used as a predictor. All of the state of the art language models are tested against multiple choice exams and other types of prediction tasks. In fact, it's how they are trained...masking - https://www.microsoft.com/en-us/research/blog/mpnet-combines... For example: "Multiple-choice questions in 57 subjects (professional & academic)" - https…
Meanwhile, there are off-shelf models that you can train very efficiently, on relevant data, privately, and you can run these on your own infrastructure.
Yes, GPT4 is probably great at all the benchmark tasks, but models have been great at all the open benchmark tasks for a long time. That's why they have to keep making harder tasks.
Depending on what you actually want to do with LMs, GPT4 might lose to a BERTish model in a cost-benefit analysis--especially given that (in my experience), the hard part of ML is still getting data/QA/infrastructure aligned with whatever it is you want to do with the ML. (At least at larger companies, maybe it's different at startups.)