Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Give Smart People the Tools to Do Smart Things

superuserdone.com

Post reply on HN
Print view View original

91–100 of 118 posts

Re: Give Smart People the Tools to Do Smart Things

#91

Post by bilater »

apsurd wrote: post #47
bilater wrote: post #44

Earlier quoted context omitted.

At least I have trends backing up my claim. What precedent is there for saying, "Yeah, AI has kept getting better, and keeps doing things that we thought it would never be able to do, but now I think it’s going to plateau because I just can’t see the model writing good poetry or being smart enough to make determinsitic tool calls" or whatever?

it doesn’t have to be that deep or specific. all things decay on a curve. pretty much all things in life are not modeled by a straight line. so you’re saying no theres still more UP. sure, but that’s a tale as old as time. when where and how much in how long is the reality piece.

The line is not just AI. It’s our whole story. You can extrapolate it to Moore’s law, automation, agriculture, and so on. And there's no decay. It's just perception.

We are already in the singularity. We’ve always been. It just gets faster at each step.

Re: Give Smart People the Tools to Do Smart Things

#92

Post by Rendello »

krapp wrote: post #79
Rendello wrote: post #69

Earlier quoted context omitted.

> they laughed in derision It's crazy how defensive (offensive) people can get when you're into programming/math/whatever. Almost like "I don't understand your world, I don't want to, and I hate it and hope you fail". https://www.smbc-comics.com/comic/what-its-like

[flagged]

[flagged]

Re: Give Smart People the Tools to Do Smart Things

#93

Post by apsurd »

bilater wrote: post #91
apsurd wrote: post #47

Earlier quoted context omitted.

it doesn’t have to be that deep or specific. all things decay on a curve. pretty much all things in life are not modeled by a straight line. so you’re saying no theres still more UP. sure, but that’s a tale as old as time. when where and how much in how long is the reality piece.

The line is not just AI. It’s our whole story. You can extrapolate it to Moore’s law, automation, agriculture, and so on. And there's no decay. It's just perception. We are already in the singularity. We’ve always been. It just gets faster at each step.

I've read Sapiens. And I liked it. It's very insightful. But that perception is a perception so I don't understand the distinction you're making.

Human advancement is a story we tell ourselves. Agree. That it in any way looks like a straight line is just not true. If nothing else, it looks coherent after the fact... because it's an extremely lossy story our limited brains can comprehend. But it's not reality. No entity in the known Universe "comprehends" the march of progress. They weren't there when it started. So I just don't know what we're talking about anymore.

Re: Give Smart People the Tools to Do Smart Things

#95

Post by jmoggr »

apsurd wrote: post #89
jmoggr wrote: post #84

Earlier quoted context omitted.

I generally agree with this. I suspect we just have different beliefs about how close we are to RSI. > Not saying it won't happen, but it's far from being the only plausible trajectory. But if it does happen, then wouldn't expected outcome be at least linear?

no because we have no way to understand the full consequence chain of any stated goal. We see this time and time again in all complex system goals; unintended consequences. So RSI toward what prompted goal? Then, how can you be so sure that reaching that goal looks like the straight line you envision? edit: more accurately, it's about incentives. We generally understand that encoding a goal in a complex system is abo…

Is a coherent, human-interpretable, goal necessary for recursive improvement?

> how can you be so sure that reaching that goal looks like the straight line you envision?

Because the thing pursuing the goal would be able to improve it's ability to pursue the goal. Doesn't that follow from the premise of RSI?

> we have a growing body of evidence for just how bad we are at aligning incentives with intended outcomes.

We are terrible at it! I don't think that will stop us letting something rip in some unknowable direction (with unintended outcomes). I hope that aligned incentives are a prerequisite, but it seems increasingly likely that they are not.

Re: Give Smart People the Tools to Do Smart Things

#98

Post by pron »

jmoggr wrote: post #84
pron wrote: post #77

Earlier quoted context omitted.

Because most systems in nature and society, including technological progress, aren't linear. It's certainly plausible that improvements will continue, but the pace is completely unpredictable. It's also plausible that the training material is polluted and progress will not continue. I'm just saying that predicting the rate of technological progress is not easy, and historically, it's rarely been smooth in the long ru…

I generally agree with this. I suspect we just have different beliefs about how close we are to RSI. > Not saying it won't happen, but it's far from being the only plausible trajectory. But if it does happen, then wouldn't expected outcome be at least linear?

> I suspect we just have different beliefs about how close we are to RSI.

I don't have any belief on the matter, but my scepticism isn't necessarily about RSI itself, but about how much it would matter even if it does happen soon. Too many things are limited by lack of resources that no brain in a jar can obtain. And if such a brain in a jar itself is very expensive to operate, it may not be easy for it to justify its existence. My point is that the technical aspect is uncertain, but it is also only a part of a larger system that's has many sources of uncertainty.

Re: Give Smart People the Tools to Do Smart Things

#99

Post by pron »

jmoggr wrote: post #95
apsurd wrote: post #89

Earlier quoted context omitted.

no because we have no way to understand the full consequence chain of any stated goal. We see this time and time again in all complex system goals; unintended consequences. So RSI toward what prompted goal? Then, how can you be so sure that reaching that goal looks like the straight line you envision? edit: more accurately, it's about incentives. We generally understand that encoding a goal in a complex system is abo…

Is a coherent, human-interpretable, goal necessary for recursive improvement? > how can you be so sure that reaching that goal looks like the straight line you envision? Because the thing pursuing the goal would be able to improve it's ability to pursue the goal. Doesn't that follow from the premise of RSI? > we have a growing body of evidence for just how bad we are at aligning incentives with intended outcomes. We…

> Because the thing pursuing the goal would be able to improve it's ability to pursue the goal

But not necessarily without needing a huge amount of resources. It is a mathematical certainty that no intelligence can solve computationally intractable problems (including forecasting the weather, or the economy) without access to resources we simply don't have.

Re: Give Smart People the Tools to Do Smart Things

#100

Post by jmoggr »

pron wrote: post #98
jmoggr wrote: post #84

Earlier quoted context omitted.

I generally agree with this. I suspect we just have different beliefs about how close we are to RSI. > Not saying it won't happen, but it's far from being the only plausible trajectory. But if it does happen, then wouldn't expected outcome be at least linear?

> I suspect we just have different beliefs about how close we are to RSI. I don't have any belief on the matter, but my scepticism isn't necessarily about RSI itself, but about how much it would matter even if it does happen soon. Too many things are limited by lack of resources that no brain in a jar can obtain. And if such a brain in a jar itself is very expensive to operate, it may not be easy for it to justify it…

This makes sense. Interacting with the world is hard and poses real bottlenecks.

I think that resource acquisition is a solvable problem for such a system, robots seem mainly limited by software.

I concede that I am too certain about the linearity of it. But like, cmon man, in the limit entropy ensures that everything ends. The world could still get very weird very fast. I think that's what original comment was trying to argue, and I think it's possible within the limits you've laid out.

That's all I'm trying to argue at least. And I admit that waving my hand at linearity sounds uneducated.

Post reply on HN