Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Gentle Singularity

blog.samaltman.com

Post reply on HN
Print view View original

171–180 of 557 posts

Re: The Gentle Singularity

#171

Post by theptip »

michaelbarton wrote: post #158

Regarding a super intelligence creating new cures: a million plus people die from malaria and AIDS combined each year. We have effective treatments for both, yet the USAID was recently shut down. I enjoy technology but less and less so each year, because it increasingly feels like there’s some kind of disconnect with the real world that’s hard to put my finger on

One lens is the “Thrive/Survive” axis. As more people thrive, they have more space for individualism, compassion for others, etc. This can be used to explain the arc of “moral progress” over time.

A follow up would be to observe that in many places today, inequality causes substantial sub-populations to feel that they are not thriving, or even declining though GDP is increasing. Which would explain the rage of the middle-American and many Europeans.

If you buy all this, then there is a clear path by which radical abundance resolves the problems. Same as how the Baby Boomers were a very low-polarization generation; when everybody is thriving it’s a lot easier to get along.

Personally I worry more about the possible worlds where technology doesn’t bring us radical abundance. Declining empires with nuclear weapons don’t sound conducive to a fun time.

Re: The Gentle Singularity

#172

Post by tho234i23j4234 »

muglug wrote: post #46

I’ve heard a joke that under Steve Jobs, the number of big initiatives at Apple was limited to the number of execs Steve could shout at in a given day. I see a similar thing at work — the number of projects a developer can get through isn’t bounded by the lines of code they can churn out in a day. Instead it’s bounded by their appetite for getting shouted at when something goes wrong. Until you can shout at an LLM, I…

Technically that is "RLHF" - but RL doesn't really work (except in a very shallow way).

Our epistemological standards are so shitty and so pathetically poor, that it's hardly surprising that we're fooled by LLMs much as we are by academic bullshit artists.

Re: The Gentle Singularity

#173

Post by thegeomaster »

TheAceOfHearts wrote: post #150

Do you think we will get AI models capable of learning in real time, using a small number of examples similar to humans, in the next few years? This seems like a key barrier to AGI. More broadly, I wonder how many key insights he thinks are actually missing for AGI or ASI. This article suggests that we've already cleared the major hurdles, but I think there are still some major keys missing. Overall his predictions s…

I hate to enter this discussion, but learning based on a small number of examples is called few-shot learning, and is something that GPT-3 could already do. It was considered a major breakthrough at the time. The fact that we call gradient descent "learning" doesn't mean that what happens with a well-placed prompt is not "learning" in the colloquial sense. Try it: you can teach today's frontier reasoner models to do fairly complex domain-specific tasks with light guidance and a few examples. It's what prompt engineering is about. I think you might be making a distinction on the complexity of the tasks, which is totally fine, but needs to be spelled out more precisely IMO.

Re: The Gentle Singularity

#174

Post by theptip »

Workaccount2 wrote: post #128
minimaxir wrote: post #14

> A lot more people will be able to create software, and art. But the world wants a lot more of both, and experts will probably still be much better than novices, as long as they embrace the new tools. This isn't correct: people want good software and good art, and the current trajectory of how LLMs are used on average in the real world unfortunately run counter to that. This post doesn't offer any forecasts on the h…

>and the current trajectory of how LLMs are used on average in the real world unfortunately run counter to that. This post doesn't offer any forecasts on the hallucination issues of LLMs. You are getting confused here... LLMs are terrible at doing software engineering jobs. They fall on their face trying to work in sprawling codebases. However, they are excellent at writing small bespoke programs on behalf of regular…

Right. Jevons Paradox.

The lower barrier to entry might mean the average quality of software goes WAY down. But this could be great if it means that there are 10x as many little apps that make one person happy by doing only as much as they need.

The above is also consistent with the quantity of excellent software going way up as well (just slower than the below-average stuff).

Re: The Gentle Singularity

#175

Post by lossolo »

k2xl wrote: post #3

> The least-likely part of the work is behind us; the scientific insights that got us to systems like GPT-4 and o3 were hard-won, but will take us very far. > 2026 will likely see the arrival of systems that can figure out novel insights Interesting the level of confidence compared to recent comments by Sundar [1]. Satya [2] also is a bit more reserved in his optimism. [1] https://www.windowscentral.com/software-apps…

In a recent interview with The Verge (available on YouTube), the DeepMind CEO said that LLMs based on transformers can't create anything truly novel.

Re: The Gentle Singularity

#176

Post by satvikpendem »

Avicebron wrote: post #140
Workaccount2 wrote: post #128

Earlier quoted context omitted.

>and the current trajectory of how LLMs are used on average in the real world unfortunately run counter to that. This post doesn't offer any forecasts on the hallucination issues of LLMs. You are getting confused here... LLMs are terrible at doing software engineering jobs. They fall on their face trying to work in sprawling codebases. However, they are excellent at writing small bespoke programs on behalf of regular…

That's great, but Becky at the farmer's market doesn't want to be re-inventing notepad on her computer to sell sourdough. This smells like "Self-checkouts", "perfect bespoke solution so that everyone can check their own groceries out, no need to wait for a slow/expensive _human_ to get the job done, except you end up basically just doing the cashiers job and waiting around inconveniently whenever you want to get some…

And yet I'd much rather self checkout than have a cashier do it for me.

Re: The Gentle Singularity

#177

Post by theptip »

actuallyalys wrote: post #114
minimaxir wrote: post #14

> A lot more people will be able to create software, and art. But the world wants a lot more of both, and experts will probably still be much better than novices, as long as they embrace the new tools. This isn't correct: people want good software and good art, and the current trajectory of how LLMs are used on average in the real world unfortunately run counter to that. This post doesn't offer any forecasts on the h…

Yep, watt-hours are correct. Think of it this way: Power supplies and laptop chargers are rated in watts and that represents how much energy they can pull from the wall at a point in time. If you wanted to know how much energy a task takes on your computer and you know it maxes out your 300 watt desktop [0] for an entire hour, that would take 300 watt-hours. [0] To my knowledge, no desktops are built to max out their…

Watt-hours seem good right now, and rebut the current arguments that LLMs are wasteful. They might soon stop being the right lens, if your agents can go off and (in parallel) expend some arbitrary and borderline-unknowable budget of energy on a single request.

At some point you’ll need to measure in terms of $ earned per W-H, or just $out/$in.

Re: The Gentle Singularity

#178

Post by jonplackett »

bko wrote: post #170
michaelbarton wrote: post #158

Regarding a super intelligence creating new cures: a million plus people die from malaria and AIDS combined each year. We have effective treatments for both, yet the USAID was recently shut down. I enjoy technology but less and less so each year, because it increasingly feels like there’s some kind of disconnect with the real world that’s hard to put my finger on

Unfortunately, many problems are last mile problems. If we could teleport medicine and technology to everyone who needs it, these problems would be mostly solved. And I'm sure drug companies would love to make money on those people since their marginal cost is so low. But coordination is hard, and it's not necessarily a problem money can fix. It's like the math that takes # of homeless people times cost of an apartme…

I’m not sure - we have enough food to feed everyone on earth. We have cures to many diseases. We have enough homes for everyone.

Inequality is bad and getting worse. Big changes accelerate it because richer people can adapt and take advantage faster.

I enjoy playing around with AI for fun and find it amazingly useful. But I do not believe it can solve inequality - that’s a people problem.

Re: The Gentle Singularity

#179

Post by godelski »

bko wrote: post #170
michaelbarton wrote: post #158

Regarding a super intelligence creating new cures: a million plus people die from malaria and AIDS combined each year. We have effective treatments for both, yet the USAID was recently shut down. I enjoy technology but less and less so each year, because it increasingly feels like there’s some kind of disconnect with the real world that’s hard to put my finger on

Unfortunately, many problems are last mile problems. If we could teleport medicine and technology to everyone who needs it, these problems would be mostly solved. And I'm sure drug companies would love to make money on those people since their marginal cost is so low. But coordination is hard, and it's not necessarily a problem money can fix. It's like the math that takes # of homeless people times cost of an apartme…

I'm not sure it's mostly last mile problems. A frequent problem I see is "costs now vs future costs". Worse is "costs now vs amortized future costs"

What I mean is let's say you are making something and you see an issue. You know it'll be an issue and if fixed now will cost $100, but if you fix it a year from now it'll cost $1000. Many people will choose the latter. On paper, I think most people will choose the former but in practice the latter is often hard to determine. An example might be a security risk. It only gets harder to fix as you generate more code and complexity increases. But of hacked this has high costs both through business and through lawsuits.

The amortized one is a nasty problem because it often goes unnoticed. Say a problem costs $100 to fix now but every day it isn't fixed it costs $1.00 every day you don't fix it costs 5% more (so $1, $1.05, $1.1025,...). These sneak up on you and you'll pay a lot more than that $100 by the time you notice

Re: The Gentle Singularity

#180

Post by idontwantthis »

bko wrote: post #170
michaelbarton wrote: post #158

Regarding a super intelligence creating new cures: a million plus people die from malaria and AIDS combined each year. We have effective treatments for both, yet the USAID was recently shut down. I enjoy technology but less and less so each year, because it increasingly feels like there’s some kind of disconnect with the real world that’s hard to put my finger on

Unfortunately, many problems are last mile problems. If we could teleport medicine and technology to everyone who needs it, these problems would be mostly solved. And I'm sure drug companies would love to make money on those people since their marginal cost is so low. But coordination is hard, and it's not necessarily a problem money can fix. It's like the math that takes # of homeless people times cost of an apartme…

I think you misunderstood OP. The last mile problem was solved or being solved. Millions of lives were saved and fewer people were dying every year. It was one of the greatest successes in human history. Now we’ve stopped because our government decided to stop. Millions of more people will die unnecessarily and technology has nothing to do with it.
Post reply on HN