Past Performance is Not Indicative of Future Results (2020) • Hacker News Board

Re: Past Performance is Not Indicative of Future Results (2020)

#281

Post by Nevermark » Mon, Aug 02, 2021, 9:23 PM UTC

cycomanic wrote: post #227

Nevermark wrote: post #200

Earlier quoted context omitted.

Actually you do have to be an expert to make sweeping statements with any credibility in a young field making advances every day. Huge ones and surprising ones every year. If you can’t characterize the technical problem that creates a limitation then you are just expressing an uninformed opinion. Even if you were an expert!

Not to get into the rest of the discussion, but I disagree with the classification of ML as a young field. AI is an established field and I would argue that nothing in modern ML is _fundamentally_ so different that it would justify classifying it as a new field.

In terms of its origins, and the core algorithms, I agree neural networks have many decades on them.

However until the hardware and software support for mainstream massively parallel execution became available it was a niche tool.

So the level of adoption, experimentation, deployment and research resources available are multiple orders of magnitude greater than 20 or 30 years ago.

As a practitioner (for my entire career) the field still operates as a new field, with enormous areas for new experimentation and interesting new creative advances happening quickly.

So we are still at the beginning.

Re: Past Performance is Not Indicative of Future Results (2020)

#282

Post by mr_toad » Tue, Aug 03, 2021, 12:26 AM UTC

skohan wrote: post #278

mr_toad wrote: post #275

Earlier quoted context omitted.

> What would be the subset of a neuron that we could simulate which would represent that distillation of the information processing part? You only need to accurately simulate the input and the output. Frankly, if that can’t be done with a Markov process I’d be very surprised, and we already know that Markov chains can be simulated with ANNs

So just to unpack this a little - there's a lot of different mechanisms going on in neural computation. For instance one of those is spike-timing dependent plasticity. Basically the idea is that the sensitivity of a synapse gets up-regulated or down-regulated depending on the relative timing of the firing of the two neurons involved. So in the classic example, if the up-stream neuron fires before the down-stream neur…

If Markov processes won’t cut it, a Turing machine will. And an ANN can approximate a Turing machine.

To boil it down, if you really want to argue that the behaviour of a neuron can’t be simulated by an ANN, you’re arguing that a neuron is doing something non-computable. At which point you might as well argue it’s magical.

Re: Past Performance is Not Indicative of Future Results (2020)

#283

Post by skohan » Tue, Aug 03, 2021, 11:05 AM UTC

mr_toad wrote: post #282

skohan wrote: post #278

Earlier quoted context omitted.

So just to unpack this a little - there's a lot of different mechanisms going on in neural computation. For instance one of those is spike-timing dependent plasticity. Basically the idea is that the sensitivity of a synapse gets up-regulated or down-regulated depending on the relative timing of the firing of the two neurons involved. So in the classic example, if the up-stream neuron fires before the down-stream neur…

If Markov processes won’t cut it, a Turing machine will. And an ANN can approximate a Turing machine. To boil it down, if you really want to argue that the behaviour of a neuron can’t be simulated by an ANN, you’re arguing that a neuron is doing something non-computable. At which point you might as well argue it’s magical.

So I think this thread was about two claims:

1. Can ANN's (in their current iteration) achieve general intelligence

2. Can they do it more efficiently than a biological brain

It certainly has not been established that a Turing machine can achieve general intelligence.

Re: Past Performance is Not Indicative of Future Results (2020)

#284

Post by spacedcowboy » Wed, Aug 04, 2021, 4:45 AM UTC

SamoyedFurFluff wrote: post #246

spacedcowboy wrote: post #80

Earlier quoted context omitted.

I have a PhD in neural networks, haven't used it in many a year, but some of the knowledge is still there. Some of the memories of racking my brains to understand what the hell is going on are still there, too. It is easy to have a theory of what is going on, to model the processes of how things are playing out inside the system, to make external predictions of the system, and to be utterly wrong. Not because your mo…

Forgive me because I myself do not have a PhD in ML, but if this is hard (making a non-racist system) why are there not serious guardrails you prevent releasing racist systems to the public?

Is it your contention that Google intentionally devised a racist system and imposed it on the public ? That would be quite the claim.

If instead it was a fuckup, well that seems adequately covered by “this shit is hard”.

If you are instead complaining about a lack of oversight, I don’t have a horse in that race. Ask someone else, I don’t care about the politics, I’m here for the technology.

Re: Past Performance is Not Indicative of Future Results (2020)

#285

Post by GeorgeTirebiter » Thu, Aug 05, 2021, 6:46 AM UTC

dctoedt wrote: post #108

GeorgeTirebiter wrote: post #101

Earlier quoted context omitted.

This seems akin to Asimov's "Elevator Effect": https://baixardoc.com/preview/isaac-asimov-66-essays-on-the-... starting p 221. I agree that one would think that Science Fiction writers would have enough of an imagination to be able to consider alternate futures (Cory CYA's by saying such a scenario would make a good SF story) - but there are already promising approaches to AGI: Minsky's "Society of Mind", Jeff Hawkin…

Elevator effect: https://indianapublicmedia.org/amomentofscience/elevator-eff...

That's funny. Of course, I was referring to Asimov's Elevator Effect, which is that if aliens visited NYC with some probe in 1800 and then in 1950, they would be astonished at all the very tall buildings, and would have to assume people were now living in these tall towers for reasons TBD. They would not know that elevators had been invented, and hence, the buildings would only be occupied 8 hours per day or so; and nobody would live in them. Elevators allowed this major unexpected result. There is more, I couldn't find the actual essay.