Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Generative AI is overrated, long live old-school AI

encord.com

Post reply on HN
Print view View original

81–90 of 192 posts

Re: Generative AI is overrated, long live old-school AI

#81

Post by seydor »

nico wrote: post #45
seydor wrote: post #16

I m not sure it's overrated, but the concerns are very real. We love the model because it speaks our language as if it's "one of us", but this may be deceiving, and the complete lack of model for truth is disturbing. Making silly poems is fun but the real uses are in medicine and biology, fields that are so complex that they are probably impenetrable to the human mind. Can Reinforcement learning alone create a model…

> model for the truth? Without sensing/experiencing the world, there is no truth. The only truth we can ever truly know, is the present moment. Even our memories of things that we “know” that happened, we perceive them in the now. Language doesn’t have a truth. You can make up anything you want with language. So the only “truth” you could teach an LLM, is your own description of it. But these LLMs are trained on thou…

It can be exact and self-consistent, you can teach the rules of mathematics . There are some things that are provably unprovable but thats a known fact.

Re: Generative AI is overrated, long live old-school AI

#82

Post by qorrect »

rvz wrote: post #24
jasfi wrote: post #5

[flagged]

> People are excited because there's so much room to improve That is hype due to OpenAI's excellent marketing and it is clearly overrated. Microsoft essentially has acquired OpenAI and is using AI safety and competition excuses to close source everything and sell their AI snake-oil. > these are still early days. Neural networks is not an early concept and LLMs still share the same eternal problems as neural networks.…

> Neural networks is not an early concept

No , but the first MLPs from the 1960's famously couldn't solve the XOR problem , they threw a hidden layer in there and fixed it, and now we're in the 'how many layers can we jam in there' phase.

My point being although neural networks are not new, they keep adding fun new things to it to create novel new features.

Re: Generative AI is overrated, long live old-school AI

#83

Post by PaulDavisThe1st »

kenjackson wrote: post #46
peter_retief wrote: post #15

"So has generative AI been overhyped? Not exactly. Having generative models capable of delivering value is an exciting development. For the first time, people can interact with AI systems that don’t just automate but create an activity of which only humans were previously capable." Good answer but I feel that most users/people do not understand the difference between generative and predictive machine learning and tha…

IMO, it has been underhyped. We're seeing things with LLMs that a decade ago I'd say was multiple decades out, if not more. We're just years into generative approaches. And I think we'll more combinations of methods used in the future. The goal of AI has never been to build an all knowing perfect system. It has also never been to replicate the way the human brain works. But its been to build an artificial system that…

> Can we build something that knows everything that has been documented and can also synthesize and infer all of that data at a level of a very smart human

The word "know" is doing some heavy lifting there, as is "synthesize" and "infer".

Re: Generative AI is overrated, long live old-school AI

#84

Post by IIAOPSW »

seydor wrote: post #16

I m not sure it's overrated, but the concerns are very real. We love the model because it speaks our language as if it's "one of us", but this may be deceiving, and the complete lack of model for truth is disturbing. Making silly poems is fun but the real uses are in medicine and biology, fields that are so complex that they are probably impenetrable to the human mind. Can Reinforcement learning alone create a model…

Here's an alternative to a model for truth. There is no truth, only power. Suppose we completely abandon logical semantics and instead focused on social semantics. Instead of the usual boolean True/False variables and logic relations, we'll have people valued variables and like/dislike relations. I system entirely for reasoning about the amount of pull and persuasion is present without ever circuiting down to any ground truth reasons. In other words, a bullshit reasoning system. Can ordinary truth reasoning be jerryrigged out of this system?

Re: Generative AI is overrated, long live old-school AI

#85

Post by com2kid »

xwdv wrote: post #71
nico wrote: post #59

Earlier quoted context omitted.

Sure, maybe you can use a shell script, but now the AI assistant can write it based on your verbal/text description, and then the assistant can also run it for you after you’ve checked it. What your are saying is: “why use the washing machine, if I my clothes are even cleaner when I wash them myself - I also spend less detergent and less water”. You are free to keep doing your laundry by hand. But I bet most people p…

Spare me the shitty analogies. We write shell scripts because it’s cheap, fast, and the behavior is very predictable. Like it or not, an AI’s behavior is a black box and can’t be “proven” to execute exactly the same every time for the scenarios you are targeting. A shell script will do exactly what it has been written to do every time, unless tampered with. And if changes need to be made, it can be done quickly witho…

> A shell script will do exactly what it has been written to do every time, unless tampered with.

Or unless some magic environment variable changes, or one of the runtime dependencies changes, or it is run on a different operating system, or permissions aren't setup right, or one of its tasks errors out.

Shell scripts are digital duct tape, the vast majority of shell scripts do not come close to being reliable software.

> god knows how long that would take for an AI to learn something new

Did you watch OpenAI's demo yesterday? They pasted in new versions of API docs and GPT4 updated its output code. When GPT forgot a parameter, the presenter fed back the error message and GPT added the parameter to the request.

Re: Generative AI is overrated, long live old-school AI

#86

Post by arrow7000 »

jedberg wrote: post #80

The real innovation will come one someone uses a Generative AI to make something, and then use a predictive AI to rate it's accuracy, making it go again until it passes the predictive AI. Basically a form of adversarial training/generation.

Isn't this exactly how GANs work already?

Re: Generative AI is overrated, long live old-school AI

#87

Post by seydor »

IIAOPSW wrote: post #84
seydor wrote: post #16

I m not sure it's overrated, but the concerns are very real. We love the model because it speaks our language as if it's "one of us", but this may be deceiving, and the complete lack of model for truth is disturbing. Making silly poems is fun but the real uses are in medicine and biology, fields that are so complex that they are probably impenetrable to the human mind. Can Reinforcement learning alone create a model…

Here's an alternative to a model for truth. There is no truth, only power. Suppose we completely abandon logical semantics and instead focused on social semantics. Instead of the usual boolean True/False variables and logic relations, we'll have people valued variables and like/dislike relations. I system entirely for reasoning about the amount of pull and persuasion is present without ever circuiting down to any gro…

Yes, it s called empiricism

Re: Generative AI is overrated, long live old-school AI

#88

Post by [deleted] »

JohnFen wrote: post #42
aaroninsf wrote: post #35

Earlier quoted context omitted.

I am not so sure, there seems to be accumulating evidence that "finding the optimal solutions" means (requires) building a world model. Whether it's consistent with ground truth probably depends on what you mean by ground truth. Given the hypothesis that the optimal solution for deep learning presented with a given training set, is to represent (simulate) the formal systemic relationships that generated that set, by…

> there seems to be accumulating evidence that "finding the optimal solutions" means (requires) building a world model. Was this ever in doubt? This has been the case forever (even before "AI"), and I thought it was well-established. The fidelity of the model is the core problem. What "AI" is really providing is a shortcut that allows the creation of better models. But no model can ever be perfect, because the value…

[deleted]

Re: Generative AI is overrated, long live old-school AI

#89

Post by ChikkaChiChi »

jedberg wrote: post #80

The real innovation will come one someone uses a Generative AI to make something, and then use a predictive AI to rate it's accuracy, making it go again until it passes the predictive AI. Basically a form of adversarial training/generation.

Bilateral "thinking" makes sense, and you can even feed generative AI back into itself for simple error correction.

I believe that we'll see the most success/accuracy once you have generative AI compare itself to itself, monitored by a GAN, which then spits out it's answer while retaining some knowledge as to how it came to the conclusion. A tricameral mind.

Re: Generative AI is overrated, long live old-school AI

#90

Post by IIAOPSW »

seydor wrote: post #87
IIAOPSW wrote: post #84

Earlier quoted context omitted.

Here's an alternative to a model for truth. There is no truth, only power. Suppose we completely abandon logical semantics and instead focused on social semantics. Instead of the usual boolean True/False variables and logic relations, we'll have people valued variables and like/dislike relations. I system entirely for reasoning about the amount of pull and persuasion is present without ever circuiting down to any gro…

Yes, it s called empiricism

This was rhetorical. My point was that a system or model which cares about something other than the truth can, upon reaching a certain level of sophistication, be able to handle reasoning about truth. Eg, an AI that cares entirely about approval for what it says rather than the actuality of what it says could still end up reasoning about truth, given that truth is most heavily correlated with approval. I reject the premise that there has to be an a priori truth model under the hood.
Post reply on HN