Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Coming Technological Singularity (1993)

mindstalk.net

Post reply on HN
Print view View original

171–180 of 181 posts

Re: The Coming Technological Singularity (1993)

#171

Post by ben_w »

epcoa wrote: post #161
ben_w wrote: post #82

Earlier quoted context omitted.

> What is the alignment of human GI, completely generalized? Of any specific human to any other specific human? https://benwheatley.github.io/blog/2019/05/25-15.09.10.html Of any specific human to a nation? That's the example you replied to. Of all the people of a nation to each other? Best we've done there is what we see in countries in normal times, with all the strife and struggles within. We have yet to fully ext…

I think that's my point. The notion of maintaining an alignment, pro-human or whatever for a replicable general AI, doesn't seem to make sense. The traits of planning, learning and goal setting don't seem concordant with maintaining an alignment. I think this discussion has veered to much to anthrocentrism to be interesting, but alignment however loosely defined here isn't some constant for an individual through thei…

I think I understand, but let's check by rephrasing:

"Alignment" is only possible up to a vague approximation, and an entirely perfectly aligned with another entity would essentially be a shadow rather than a useful assistant because by being perfectly aligned the agent would act tired exactly when the person was tired, go shopping exactly when the human would, forget their keys exactly when the human would, respond exactly like the human to all ads and slogans, etc.?

I agree, though:

(1) this has already been observed, last year's OpenAI dev day had (IIRC) a story about a writer who fine tuned a model on their slack (?) messages, they asked it to write something for them, the response was ~"sure, I'll get on it tomorrow".

(2) for many of those concerned with "solving alignment", it's sufficient for the agent to never try to kill everyone just to make more paperclips etc.

Re: The Coming Technological Singularity (1993)

#172

Post by ConspiracyFact »

AnimalMuppet wrote: post #126
gary_0 wrote: post #76

Earlier quoted context omitted.

> I sincerely have no idea why people believe so strongly that a human-level AI can build a superhuman AI. ...how on earth would such an AI even know it succeeded? This touches on one of the few good reasons to be less ardent about AI/AGI: "intelligence" is not very well-defined and we don't have very good ways of measuring it. I don't think this is a total blocker, but it might present difficulties. What if our curr…

> However, your broader point seems to imply that you can't "bootstrap" intelligence, which I don't find convincing. How about this weaker statement: "It is not obviously true that humans (or a human-level AI) can bootstrap to a superhuman AI in a small number of years."

This. It’s taken the universe (at least locally) several billion years to achieve human-level intelligence through trial and error. Somehow switching over to silicon is going to get us to superhuman intelligence and beyond within a decade? Why should the intelligence of a system surpass its complexity (if you’ll excuse my playing fast and loose with units)? For all we know we’ll achieve barely-superhuman AI and when we ask it why it doesn’t just make itself exponentially smarter, it will say, “Do you have any idea how complex I am?”

Re: The Coming Technological Singularity (1993)

#173

Post by jodrellblank »

rsanheim wrote: post #33
jodrellblank wrote: post #31

Earlier quoted context omitted.

Skip a few generations and the machine will build itself. There’s no need for it to take lasers exploding tin to generate ultra Violet to etch patterns to make intelligence, humans don’t grow brains that way or spend billions on fabs and power plants to produce children. How it gets from here to there is a handwave, though.

That’s a pretty enormous handwave.

It is, but it isn't once we're already handwaving "superintelligence". Problem with funding a new fab or new power stations? It can invent a thing worth a lot of money, or out-gamble, or swindle or steal it. Problem with getting things done? It can pay people lots of money to do them. That covers a huge number of cases of geting from "here" to "there" if such a path is possible at all.

Re: The Coming Technological Singularity (1993)

#174

Post by RaftPeople »

nsbshssh wrote: post #162
RaftPeople wrote: post #147

Earlier quoted context omitted.

> There must be a name for the human bias that sees what is possible this month and doesn't believe things might be very different very soon and change quickly The counter to that is that AI prediction of 10 to 20 years away has been happening since the 1950's (or before?). It seems there is some foundational progress but it's very slow.

Unlike cold fusion there is real practical progress. No grid energy comes frok cold fusion. Millions of people are using AI to help get their work done.

> Unlike cold fusion there is real practical progress.

Yes agreed, there is progress, that's why I said "It seems there is some foundational progress but it's very slow."

Training deep networks, vectorization of semantically related words+phrases, reservoir computing for time series patterns, etc. etc.

All great foundational work, but we still can't reproduce how a rabbit dynamically adjusts it's olfactory network the minute it learns whether a new smell results in something positive/negative/neutral, which is probably the same trick used in a variety of places in our brain to give it such dynamic adaptability.

It's progress, but it's moving slowly.

Re: The Coming Technological Singularity (1993)

#175

Post by DiscourseFan »

ben_w wrote: post #73
DiscourseFan wrote: post #41

Earlier quoted context omitted.

Didn't OpenAI just cut its AGI department?

AGI readiness team. With the department leader giving the statement that nobody is ready. Unless there's been another one since then? It's getting to be a bit of a blur.

Well, I'm also not ready to hear about my alzheimer's diagnoses, though I don't think that's coming any time soon.

Re: The Coming Technological Singularity (1993)

#176

Post by Teever »

auggierose wrote: post #55
Teever wrote: post #52

Earlier quoted context omitted.

I think you're correct that the energy efficiency of a human exceeds that of current computers, but I think it's a bit more complicated than a first order calorie count. How many joules go into producing those 900 calories? Like in terms of growing the food, from fertilizer production to tractor fuel, to feeding the farmer, to shipping the food, packaging it, storing it at the appropriate temperature, the ratio of sp…

I don't think it makes sense to include all of that in the calculation. A human doesn't need all of that, they just need 900 calories. You can just eat berries, no need to cook anything, for example.

A human may not technically need all that but the humans who do exist in our society do.

A measurement of a hypothetical human foraging in the bush is a useful one but a much more useful one is the energy expenditure to keep a human alive in our society.

Re: The Coming Technological Singularity (1993)

#177

Post by alphan0n »

svieira wrote: post #150
bglazer wrote: post #148

Earlier quoted context omitted.

You realize AlphaFold was trained on thousands of protein sequences generated via (I'd guess) tens of millions of hours of human labor.

How does the old saw go ... "past results are no guarantee of future performance"?

Guarantees are, by their nature, always periodic, or episodic, only by confidence of the issuer of their longevity or the issue are they ever given.

The amount of time invested in the understanding of this issue amounts to many lifetimes, where an adverse outcome would be the equivalent to the cessation of breathing being a cause for life.

Re: The Coming Technological Singularity (1993)

#178

Post by trashtester »

ben_w wrote: post #170
kiba wrote: post #167

Earlier quoted context omitted.

I don't know what to tell you. Georgism was contemporary to the industrial revolution. The book Progress and Poverty , as described by wikipedia "investigates the paradox of increasing inequality and poverty amid economic and technological progress"[1]. That sounds especially relevant to our time. 1. https://en.wikipedia.org/wiki/Henry_George So, yes. Wealth means "land". Especially so in the industrial revolution.

Industrial revolution: roughly 1760 onwards if you count agricultural revolution as part of it. Middle ages: 500-1500 Henry George, 1839-1897, is indeed part of the industrial revolution. @trashtester and I were both comparing what happened before the industrial revolution to what happened in it . The valuation of land before still wasn't Georgism: the land was assumed to have productive output, and if someone didn't…

> and I were both comparing what happened before the industrial revolution to what happened in it.

Sure. Just keep in mind that the period 1500-1760 also saw a lot of technological development that was laying the groundwork for the industrial revolution. Things like windmills, ever more advanced metallurgy, proto-chemistry, physics and so on.

In fact, some of this goes back to about 1000AD and the High Middle Ages. It was really mostly in the Dark Ages when land ownership (with subsistence farming and the military to defend it) was the source of most wealth. From 1000AD until now, land was gradually replaced by other economic capabilities as the main differentiating source of wealth. (Cities with cultures that produced skilled artisans, trade routes, naval capabilities, near-factory-level workshops centralizing production, etc.)

I suppose all I write above supports your view that land isn't currently the main source of wealth, but rather the ability to leverage skilled labor.

In the future, though, I do think there is a possibility that natural resources (and the physical power to protect them) may again become the main form of wealth, simply because the value of skilled labor may approach zero.

Re: The Coming Technological Singularity (1993)

#179

Post by auggierose »

Teever wrote: post #176
auggierose wrote: post #55

Earlier quoted context omitted.

I don't think it makes sense to include all of that in the calculation. A human doesn't need all of that, they just need 900 calories. You can just eat berries, no need to cook anything, for example.

A human may not technically need all that but the humans who do exist in our society do. A measurement of a hypothetical human foraging in the bush is a useful one but a much more useful one is the energy expenditure to keep a human alive in our society.

> A measurement of a hypothetical human foraging in the bush is a useful one but a much more useful one is the energy expenditure to keep a human alive in our society.

Which society? An American one? A German one? What about the iPads manufactured in China for both of them? A human of what income? If you do not draw the boundaries of the system you want to observe, asking how much energy that system uses is not very sensible.

Re: The Coming Technological Singularity (1993)

#180

Post by staticman2 »

bbor wrote: post #38
aithrowawaycomm wrote: post #34

> To date, there has been much controversy as to whether we can create human equivalence in a machine. But if the answer is "yes, we can", then there is little doubt that beings more intelligent can be constructed shortly thereafter. I find it bizarre how often these points are repeated. They were both obviously wrong in 1993, and obviously wrong now. 1) A nitpick I've had since grad school: the answer to "can we cre…

If you're ever stuck wondering why a bunch of smart, motivated people with no clear corrupting motivations are being idiotic, that's a strong heuristic that you should spend a bit more time analyzing the issue, IMO ;). "Ugh, why is everyone else so stupid" is a common take for undergrad engineers, but I'm sure you've grown out of it in other ways. Anyway, more substantively: The simple answer is that people have thou…

The objection above was "I sincerely have no idea why people believe so strongly that a human-level AI can build a superhuman AI. It is bizarre that this claim is accepted with "little doubt" when there are very good reasons to doubt it: how on earth would such an AI even know it succeeded? How would it define the goal?"

In response Yudkowsky is presented as a sort of superintelligent human with the answers, but of course we're back to the original problem, how do we know Yudowsky, who as far as I know does not make disprovable claims, is a superintelligent human with the answers? How would we define the goal?

Post reply on HN