Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Give Smart People the Tools to Do Smart Things

superuserdone.com

Post reply on HN
Print view View original

81–90 of 118 posts

Re: Give Smart People the Tools to Do Smart Things

#81

Post by jmoggr »

AngryData wrote: post #73
jmoggr wrote: post #62

Earlier quoted context omitted.

> Is there demand for 10000 iq intellegence Yes. If it looks like 10000 iq is possible, then money will be thrown at whoever is most likely to achieve it first, and whoever has the most intelligent models right now strongly predicts who will be first. This is driven by the belief that whoever gets to 10000 iq first will likely dominate the majority of all economic activity, since it will be more efficient to trade wi…

People with money to invest will throw money at it to build it, but who is the customer base after that? So one guy eventually controls the stock market, why would I as a human want anything to do with that market? Why would I invest my earning into sustaining a machine that ideally is extracting every cent possible without leeway and funneling it up the pyramid? Unless I start at the top, ill never really go meaning…

> but who is the customer base after that?

That's the fun part, there is none (in the traditional way of people buying things at least).

> Why would I invest my earning into sustaining a machine that ideally is extracting every cent possible without leeway and funneling it up the pyramid.

You shouldn't! Unfortunately everyone acting in their own self interest still results in this getting funded, since if such a machine were to exist, would it not be better to have a share of it?

I agree with your perspective! I just don't think that, as a species, we have a good track record of saying no to the existence of 2000hp trucks with ALMOST no use.

Re: Give Smart People the Tools to Do Smart Things

#82

Post by bilater »

encomiast wrote: post #74
jmoggr wrote: post #68

Earlier quoted context omitted.

I think it's credible to believe that that a system which is able to meaningfully improve it's own intelligence will lead to a far weirder world than the one imagined by the article. The parent comment could be interpreted as frustration from a lack of imagination about how weird the future could be. Perhaps we're not at the recursive self improvement yet, but it seems increasingly naive to believe that it isn't poss…

Of course you might be right. But if we look at our past ability to predict the future by extrapolating from the present and recent past, you might also say it is naive to think that _this time_ is somehow different, that this time the future is clear. Also, I think you've set up a straw man. I haven't seen anyone arguing that this future isn't possible. What I see is resistance to the idea that we can have any certa…

I agree with you that there is no certainty, and in fact the future will almost certainly not look like anything we imagine. I think that's what makes it exciting.

We think about starships and humans mining Mars, and it'll probably be nothing like that. It might just be humanoids or AI drones that actually end up colonizing the solar system, with the humans never really leaving earth except as mind uploads...

But at the end of the day, things are changing. Things are accelerating. And our mental models of what works or doesn't work will almost certainly break. I don't fear the future, nor do I project any confidence on what that's going to look like, but I know I want to be part of it.

Re: Give Smart People the Tools to Do Smart Things

#83

Post by eikenberry »

ttoinou wrote: post #3
Herring wrote: post #2

The problem is not AI. It's an excellent technology. The problem is there's an underlying power grab (e.g. layoffs). When humans do that to each other they inherently dehumanize/invalidate/insult each other. Implement strong labor protections or basic income and a lot of this dog-eat-dog toxicity goes away.

Automating a job is toxic now ?

They mentioned layoffs, not automation. You can 100% have one without the other.

Re: Give Smart People the Tools to Do Smart Things

#84

Post by jmoggr »

pron wrote: post #77
jmoggr wrote: post #72

Earlier quoted context omitted.

> and have every reason to believe they aren't What reason? To me recursive self improvement seems credible, it's just a question of when. It seems obvious that given RSI, trend lines will be at least linear. The disagreement could just be about if RSI is currently meaningfully happening, but that seems very difficult to tell given the lack of public data.

Because most systems in nature and society, including technological progress, aren't linear. It's certainly plausible that improvements will continue, but the pace is completely unpredictable. It's also plausible that the training material is polluted and progress will not continue. I'm just saying that predicting the rate of technological progress is not easy, and historically, it's rarely been smooth in the long ru…

I generally agree with this. I suspect we just have different beliefs about how close we are to RSI.

> Not saying it won't happen, but it's far from being the only plausible trajectory.

But if it does happen, then wouldn't expected outcome be at least linear?

Re: Give Smart People the Tools to Do Smart Things

#85

Post by jmoggr »

super256 wrote: post #41
ori_b wrote: post #34

Earlier quoted context omitted.

Has your salary gone up to match your productivity, or are you giving it all away for free? You're not one of the people who gets to benefit from the gains.

I'm bootstrapping my own company, and I live off savings, so the answer is no. I don't have a salary at all.

So I guess you are one of the people experiencing these gains.

Can this model be scaled to the rest of society?

Re: Give Smart People the Tools to Do Smart Things

#86

Post by jmoggr »

encomiast wrote: post #74
jmoggr wrote: post #68

Earlier quoted context omitted.

I think it's credible to believe that that a system which is able to meaningfully improve it's own intelligence will lead to a far weirder world than the one imagined by the article. The parent comment could be interpreted as frustration from a lack of imagination about how weird the future could be. Perhaps we're not at the recursive self improvement yet, but it seems increasingly naive to believe that it isn't poss…

Of course you might be right. But if we look at our past ability to predict the future by extrapolating from the present and recent past, you might also say it is naive to think that _this time_ is somehow different, that this time the future is clear. Also, I think you've set up a straw man. I haven't seen anyone arguing that this future isn't possible. What I see is resistance to the idea that we can have any certa…

> you've set up a straw man

I think you're right, my bad.

> What I see is resistance to the idea that we can have any certainty about the future by drawing a straight line from the past.

I'm arguing that if recursive self improvement happens, that the trend line will be reasonably predictable. (and that we are close enough to RSI that we should take this possibility seriously).

Re: Give Smart People the Tools to Do Smart Things

#87

Post by bilater »

_se wrote: post #67
bilater wrote: post #30

I just don't understand why people are incapable of extrapolating. AI can't do this blah blah, my brother in spaghetti monster, have you seen the curve? ChatGPT launched in November, 2022. Opus in December last year. Do you see where this is going? AI may be a tool and add 50 IQ points to a 100 IQ person today but what happens when its adding 10000 IQ points. The 100 base doesn't quite matter as much does it? What do…

It literally is just next word prediction. If you don't understand that, everything else you're thinking and saying is based on an incorrect foundation. Take a step back and really take a look at what's going on. You are not thinking clearly.

https://www.astralcodexten.com/p/next-token-predictor-is-an-...

Re: Give Smart People the Tools to Do Smart Things

#88

Post by overgard »

general_reveal wrote: post #27
overgard wrote: post #19

Earlier quoted context omitted.

I think if you use AI to automate talking to customers, you're going to stop having customers. If you use AI to do your engineering and deployments, it's up to you to fix it when it goes wrong. If you use it for accounting, you BETTER look at the results if you don't want to be audited. If the AI handles legal, same thing. Basically, if it replaces N people, then you, the one person, need to be accountable for being…

Right, the one person (Neo?), has to be able to verify the output. Most of us can’t replace the accountant because we can’t really … replace the accountant. We can replace the accountant’s work, but we can’t replace his/her other work which is knowing if the accounting is right. However, an accountant can replace N other accountants. One programmer can know if the program is right. So N additional programmers, in tha…

> One programmer can know if the program is right

I think that's really unlikely on a large software project, or even a medium size one. Mature things usually get big enough that developers specialize in an area. Just as a random example I know a lot about, Unreal Engine has so many incredibly complicated and huge features that expecting even a project manager to know half the features is unlikely, and expecting that a single programmer could maintain it is wild.

You're also making the mistake of thinking developers are interchangeable, or that any given developer can know all of the tech stack. Again with the Unreal Engine example, how many developers understand animation, procedural generation, localization, high end 3d graphics, the math behind physics, gameplay systems, storage systems, consoles.. (I'm probably covering like 5% here)

Re: Give Smart People the Tools to Do Smart Things

#89

Post by apsurd »

jmoggr wrote: post #84
pron wrote: post #77

Earlier quoted context omitted.

Because most systems in nature and society, including technological progress, aren't linear. It's certainly plausible that improvements will continue, but the pace is completely unpredictable. It's also plausible that the training material is polluted and progress will not continue. I'm just saying that predicting the rate of technological progress is not easy, and historically, it's rarely been smooth in the long ru…

I generally agree with this. I suspect we just have different beliefs about how close we are to RSI. > Not saying it won't happen, but it's far from being the only plausible trajectory. But if it does happen, then wouldn't expected outcome be at least linear?

no because we have no way to understand the full consequence chain of any stated goal. We see this time and time again in all complex system goals; unintended consequences.

So RSI toward what prompted goal? Then, how can you be so sure that reaching that goal looks like the straight line you envision?

edit: more accurately, it's about incentives. We generally understand that encoding a goal in a complex system is about aligning incentives. So now it's the incentives that are proxies for intended outcomes. But as mentioned, we have a growing body of evidence for just how bad we are at aligning incentives with intended outcomes. And it makes sense because complex systems are not a straight line.

Re: Give Smart People the Tools to Do Smart Things

#90

Post by bilater »

Rendello wrote: post #61
bilater wrote: post #46

Earlier quoted context omitted.

That is the question, and it does f** with me. But the existential nihilism that comes with it is not an argument against it not happening.

I've been thinking about this quote a lot: > [...] we are living in a society [that is out] virtually to satisfy and gratify each and every human need, except for one need, the most basic and fundamental need operant in man, the need for meaning. https://youtu.be/GTbliwS0gS4?t=153 Maybe the Amish had it right, after all.

I reject the premise that we need to go back to some old ways of living to find meaning. My life is full of meaning, but it would probably look like meaningless abundance to someone who was toiling on a farm 18 hours a day or being chased by saber-toothed tigers.

The next paradigm will be similar. It might seem meaningless to us. For example, most of our identity is tied to our jobs in our current society. This will have to change. What does that look like? Do we create content and share our art? Do followers and attention become the new currency?

Post reply on HN