Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why This AI Moment May Be the Real Deal

thenewatlantis.com

Post reply on HN
Print view View original

31–40 of 231 posts

Re: Why This AI Moment May Be the Real Deal

#31

Post by danielbln »

visarga wrote: post #18
optimalsolver wrote: post #10

What's a large language model doing when it's not being queried? Am I correct that they only compute information when dealing with a prompt? If so, that seems like a fundamental flaw. An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.

No, unfortunately it doesn't learn online. It forgets everything after each interaction. They can collect the data and retrain later, but the hard part is doing all the fine-tuning steps all over again and ensuring the new model has no regressions. GPT3.5 and 4 are years out of date, an unfortunate situation when generating code or asking about recent events. And now they removed the search plugin, probably they got…

Or you use one of the many other search enabled LLMs (e.g. https://phind.com) or build your own via langchain etc.

Re: Why This AI Moment May Be the Real Deal

#32

Post by FrustratedMonky »

RunSet wrote: post #30
FrustratedMonky wrote: post #27

Earlier quoted context omitted.

That is just the physical layer. Robotics is behind in making a 'human'. Even AlphaGo had a human moving the pieces, and nobody would say that was evidence of any 'lack' in the AI. It's just an interfacing issue, engineering will eventually solve. Edit> What I was getting at. Take Robots as the physical interface layer for the AI. AI advancements are currently outpacing advancements in Robotics. So potentially AI cou…

Plenty of practically mint human bodies just sitting around waiting to be repurposed. https://www.nbcnews.com/id/wbna9816703

This is the scary scenario. What if AI does out pace humans, but the physical side does not, we don't succeed in making vast quantities of easy to build and maintain robots. Thus to keep industry going, the AI does need humans, and it gives the directions.

If you have ever worked in a modern warehouse, it is very evident, the humans are just following instructions from the system. Now scale that up to a manufacturing facility, it is already happening.

Everybody with an ear-piece or 'phone' type device, following instructions.

Re: Why This AI Moment May Be the Real Deal

#33

Post by amelius »

addandsubtract wrote: post #29
foobarbecue wrote: post #21

Earlier quoted context omitted.

Tangent: It has always annoyed me that a sigmoid curve does not look like a sigma, and it doesn't really look much like an S either. Just the middle part of an S. Am I missing something?

Hyperbolic tangent :P

Ha, good one! :)

Re: Why This AI Moment May Be the Real Deal

#34

Post by ben_w »

jasfi wrote: post #23
visarga wrote: post #18

Earlier quoted context omitted.

No, unfortunately it doesn't learn online. It forgets everything after each interaction. They can collect the data and retrain later, but the hard part is doing all the fine-tuning steps all over again and ensuring the new model has no regressions. GPT3.5 and 4 are years out of date, an unfortunate situation when generating code or asking about recent events. And now they removed the search plugin, probably they got…

The cost to compute these language models should eventually lower. Will OpenAI then release more frequently, or release larger LLMs? They'll probably to try achieve both goals in some proportion.

Training on web pages created after ChatGPT (and equivalently Stable Diffusion) was released, has a strong risk of trying to create the next AI (of whatever type) on the barely-even-proofread output of the current models.

Re: Why This AI Moment May Be the Real Deal

#35

Post by AnimalMuppet »

apples_oranges wrote: post #4

Where on the S-shaped curve are we? I am curious if one day the paradigm that brought forth the current chatbots has to be abandoned because it's reached his limits, and a new approach will replace it. But certainly it has inspired the field of AI massively and I expect many new developments which will lead is into a future that resembles sci-fi books and movies. I am also interested in what good sci-fi will look lik…

That's kind of hard to say. You don't know how long the curve is until you're somewhat past halfway, unless you can very carefully measure progress.

That's easy with something like transistor size. That's hard with AI, because we can't put a number on how good it is. (The model size is probably not a good number for measuring AI progress.)

So nobody knows. We're all guessing.

Re: Why This AI Moment May Be the Real Deal

#36

Post by ido »

FrustratedMonky wrote: post #32
RunSet wrote: post #30

Earlier quoted context omitted.

Plenty of practically mint human bodies just sitting around waiting to be repurposed. https://www.nbcnews.com/id/wbna9816703

This is the scary scenario. What if AI does out pace humans, but the physical side does not, we don't succeed in making vast quantities of easy to build and maintain robots. Thus to keep industry going, the AI does need humans, and it gives the directions. If you have ever worked in a modern warehouse, it is very evident, the humans are just following instructions from the system. Now scale that up to a manufacturing…

Reminds me of https://en.wikipedia.org/wiki/Manna_(novel)

Re: Why This AI Moment May Be the Real Deal

#37

Post by ZiiS »

roenxi wrote: post #2

> Study Deep Blue’s 1997 defeat of world chess champion Garry Kasparov, and your excitement about how quickly this technology would take over other cognitive work would wane as you learned just how much brute human force went into fine-tuning the software specifically to beat Kasparov. That was ~26 yeas ago. In the intervening 26 years humans have been curb-stomped so hard at chess that certain microwaves might be co…

The are microwaves fully in on the curb-stomping https://www.tomsguide.com/news/ges-kitchen-hub-is-the-smarte... My back of the envelope suggests most digital microwaves with a smart defrost but no connectivity are probably more in the 2000 ELO range (i.e. can beat most players but not grandmasters). Mainly due to not having enough storage for large opening/endgame books.

Re: Why This AI Moment May Be the Real Deal

#38

Post by ChatGTP »

jiggawatts wrote: post #28
apples_oranges wrote: post #4

Where on the S-shaped curve are we? I am curious if one day the paradigm that brought forth the current chatbots has to be abandoned because it's reached his limits, and a new approach will replace it. But certainly it has inspired the field of AI massively and I expect many new developments which will lead is into a future that resembles sci-fi books and movies. I am also interested in what good sci-fi will look lik…

> Where on the S-shaped curve are we? No-one knows. We can however make educated guesses based on: - Extrapolating the last couple of years of progress. - Scientific studies of LLM quality scaling in proportion to input data size, parameter count, and total training compute ops. - How much more training data is available. - Reasonable budgets, especially the cost-efficiency of inference , which appears to be more lim…

It really stands to reason that at some point these systems will just be "too smart" and we won't find any value in something that much more intelligent. It's like a 4 year old trying to understand a seasoned professor explaining some concept, we just won't get it.

Imagine a hypothetical scenario where some ultra-intelligent beings delivered a book with all the answers humans ever wanted, but we needed an IQ of 500 t understand it. This IMO is where we're heading.

There might be some pretty sweet low hanging fruit to attain along the way, but at some stage, we'll hit a ceiling and the only way forward would be to augment our own intelligence.

Re: Why This AI Moment May Be the Real Deal

#39

Post by ChatGTP »

roenxi wrote: post #2

> Study Deep Blue’s 1997 defeat of world chess champion Garry Kasparov, and your excitement about how quickly this technology would take over other cognitive work would wane as you learned just how much brute human force went into fine-tuning the software specifically to beat Kasparov. That was ~26 yeas ago. In the intervening 26 years humans have been curb-stomped so hard at chess that certain microwaves might be co…

Curious why you needed to inject the violent overtones in your comment, "curb stomped?

We're building tools to help us augment our intelligence, I don't see anyone getting "curb stomped"?

If intelligence leads to more violence, then I don't think it will be as useful as we'd have hoped.

Re: Why This AI Moment May Be the Real Deal

#40

Post by ben_w »

optimalsolver wrote: post #10

What's a large language model doing when it's not being queried? Am I correct that they only compute information when dealing with a prompt? If so, that seems like a fundamental flaw. An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.

> What's a large language model doing when it's not being queried?

Nothing.

> Am I correct that they only compute information when dealing with a prompt?

Yes.

> If so, that seems like a fundamental flaw.

Flawed in what way? It clearly doesn't need to be like us to be useful, because it's useful and definitely not like us.

> An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output.

This might be good, but it's not clear if, or to what extent, we really do that ourselves — the differences between working/short/long term memories, between episodic and skill, even linguistically between knowledge of phonemes, words, grammar, and the connection between those and the things they represent all being impaired independently of each other by localised brain damage[0].

Then there's how much this changes with some stages of sleep, and meditation to clear your mind.

Given the number of users (what is it, 100 million?), having it always on, continuously integrating, would still be inhuman even if the architecture was a perfect mirror of the human brain.

Also, if the AI is structured to be a "thinking machine", does that make it murder to switch it off?

[0] Cognitive Psychology for Dummies, currently listening to it as an audiobook.

Post reply on HN