I love it but also waiting for the hype to settle down.
Generative AI is overrated, long live old-school AI
171–180 of 192 posts
Re: Generative AI is overrated, long live old-school AI
#172I see and I hear: "Don't be dazzled by AI computer vision's creative charm! Classical computer vision, though less flashy, remains crucial for solving real-world challenges and unleashing computer vision's true potential." Meant for those in classical computer vision before ML ate the field.
If by classical CV they mean image processing and machine vision, I would say that's still true.
Re: Generative AI is overrated, long live old-school AI
#173When the generative model is autoregressive (autocomplete), it can easily be used as a predictor. All of the state of the art language models are tested against multiple choice exams and other types of prediction tasks. In fact, it's how they are trained...masking - https://www.microsoft.com/en-us/research/blog/mpnet-combines... For example: "Multiple-choice questions in 57 subjects (professional & academic)" - https…
For GPT4: "Pricing is $0.03 per 1,000 “prompt” tokens (about 750 words) and $0.06 per 1,000 “completion” tokens (again, about 750 words)." Meanwhile, there are off-shelf models that you can train very efficiently, on relevant data, privately, and you can run these on your own infrastructure. Yes, GPT4 is probably great at all the benchmark tasks, but models have been great at all the open benchmark tasks for a long t…
This is true, but the humans to develop the non-LLM solution are expensive and the OpenAI API is easy.
Re: Generative AI is overrated, long live old-school AI
#174Earlier quoted context omitted.
I think the big difference is just being an Autocompleter is less concerned with generating something that is truthful, as in reflects the real world as we understand it described by physics, vs simply spitting out something that sounds good. Although we do have a litmus test in asking it "What is the meaning of life the universe and everything?"
Yes, exactly. An autocompleter is saying what the next words probably would be, not what it should be. It's like a chess program that tries to find the most likely move that a huan would make in the position rather than the best move.
That’s why I said “supervised” - in other words, someone competent in the domain and context is examining the output and correcting or discarding as necessary before use.
“Unsupervised generative AI” is useless IMO.
Re: Generative AI is overrated, long live old-school AI
#175Earlier quoted context omitted.
There's no way there's only 10% left to improve in those models. New versions are coming out regularly that are clearly improved. Midjourney v5 and GPT-4 were just released showing huge improvements, for example. Not only that, but the innovation around this tech is also just getting started. It's immediately applicable for business use. The classical techniques still have their uses, of course.
It's not that there's only 10% left to improve. It's that the data needed, compute requirements, and model size are as intensive, getting from 0 to 80 as they are getting from 80 to ~85 or ~90. See https://paperswithcode.com/sota/image-classification-on-imag...
They're constantly bringing down the compute requirements. I've seen regular Open Source stories on HN about people finding impressive optimizations.
Re: Generative AI is overrated, long live old-school AI
#176People calling neural-net classifiers "old-school" AI confused me. For a second I thought they were talking about the really old "expert systems" with everything being a pile of hard-coded rules.
It still feels like there's a place for these rule based systems(Prolog?) to at least place some constraints on the output of non-deterministic, generative AI. If nothing else, have a generative AI generate the ruleset so you have some explicit rules you can audit from time to time.
[dead]
Re: Generative AI is overrated, long live old-school AI
#177Earlier quoted context omitted.
Yeah, i think one potential way to use blackbox ai in newer systems is having guardrails that are validated as safe (but perhaps non-optimal) and ensuring that the ai takes action within that sample space. Obviously this is hard problem, but might open the doors for policies (in self-driving cars, for example) to be entirely ai driven.
Obviously the solution is to get the LLM to output Prolog. Give it positive feedback if the Prolog compiles. :-)
A friend of mine was just telling me how he asked GPT-3 to write a simple program in Prolog and it seemed to get it right. He didn't try compiling it, but he has enough experience w/ Prolog to say that it was more or less correct.
I'm pretty cynical on LLMs(i.e. they're not intelligent and won't take all our jobs soon), but am coming around on their importance and capabilities.
Re: Generative AI is overrated, long live old-school AI
#178Earlier quoted context omitted.
This is just wrong. Also, even if a LLM could do that, so could a shell script, without the risks involved in using "AI" for it, or for now the ridiculous external dependence that would involve. I wonder if in 10 years people will be stuck debugging Rube-Goldberg machines composed of LLM api calls doing stuff that if-statements can do, probably cobbled together with actual if-statements
> I wonder if in 10 years people will be stuck debugging Rube-Goldberg machines composed of LLM api calls doing stuff that if-statements can do, probably cobbled together with actual if-statements Sounds like an extension of https://en.wikipedia.org/wiki/Wirth%27s_law . How many times have I done some simple arithmetic by typing it into my browser's bar and checking out the google calculator results? When a generatio…
My son regularly uses Alexa as a calculator, and also asks Alexa all kinds of things without a thought as to whether the output triggers a simple pattern match and gets fed to a specialised process or triggers a web search or is processed some other way. It's all conversational anyway. So the day Amazon plugs an LLM into it, it's not a given he'll even notice the difference for some time.
Re: Generative AI is overrated, long live old-school AI
#179Earlier quoted context omitted.
This feels achievable in five years.
It always feels achievable in five years. People were saying exactly this 30 years ago. Sooner or later it may (or may not) be a true statement, but it's awfully hard for me to say that it's any different right now than it has been before.
I've had ChatGPT write code from vague statements that got close enough that it'd take the typical intern days of research to figure out. I've also had it fail spectacularly before prompted more extensively. But there are tasks I'd rather hand of to ChatGPT already today than hand to an intern, because it does the job faster and is able to correct misunderstandings and failures far faster.
E.g. I posted a while back how I had it write the guts of a DNS server. It produced a rough outline after the first request, and would fill out bit by bit as I asked it to elaborate or adjust specific points. The typical intern would not know where to start and I'd need to point them to the RFC, and they'd go off and read them and produce something overwrought and complex (I've seen what even quite experienced software devs produce when given that task; and I know how much work it took me the first time I did it).
So it may not exactly replace an intern, in that there are classes of problems that require low-level reasoning and a willingness and ability to go off and research that it's just not set up for yet and that will be harder to replace. But the problem set will change. Both in that what gets to the intern will be things where LLMs don't produce good result fast enough (I wouldn't ask an intern to do something what ChatGPT can do well with little prompting), and that interns will be more likely to go off and learn a bit and then spend more time prompting LLMs and in that sense produce more value than they could before.
Re: Generative AI is overrated, long live old-school AI
#180Or could it be possible that it was always going to end up like a black box it seems is it not? We will never truly understand the inner workings while it solves every problems that can be numbered by which it couldn't be solved with hardline algorithms previously. It's literally calling higher dimensional egrigores for answers or some blood magic Genie.