Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Past Performance is Not Indicative of Future Results (2020)

locusmag.com

Post reply on HN
Print view View original

261–270 of 285 posts

Re: Past Performance is Not Indicative of Future Results (2020)

#261

Post by cratermoon »

kaba0 wrote: post #235
cratermoon wrote: post #116

Earlier quoted context omitted.

There's good reason to be skeptical of AI as it is. Here's a couple of reasons Racial bias in facial recognition: "Error rates up to 34% higher on dark-skinned women than for lighter-skinned males. "Default camera settings are often not optimized to capture darker skin tones, resulting in lower-quality database images of Black Americans" https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio... Chicago’s “Heat…

Isn’t that just human bias seeping through into the data set, so of course the neural net trained on that will show similar biases. The problem here is the human element.

As they say, it's not a technical problem, it's a people problem. But it's not "just" human, it's that the field in general is elevating ML, AI, whatever you want to call it, with hype like "algorithms aren't biases like a human would be", which is technically true, but also trivial. The people creating these systems didn't even consider that they would reflect and even enshrine, with all kind of high-priest-of-technology-blessings, their biases, that's why we got Tay and why things PredPol is terrible. The key is to acknowledge and actively protect against systematic bias, not make a business of it (coughtwitterfacebookcough).

Re: Past Performance is Not Indicative of Future Results (2020)

#262

Post by kaba0 »

skohan wrote: post #252
kaba0 wrote: post #240

Earlier quoted context omitted.

I only mentioned that essay because I think the fundamental vision of it is still correct — in serial computations silicone beats “meat” hands down. And that is in both power efficiency and performance. The fundamental difference between our current approach and biological brains is just as much a hardware one as it is theoretical. CPUs and GPUs are simply not best fit for this sort of usage — a “core” of them is way…

> in serial computations silicone beats “meat” hands down. And that is in both power efficiency and performance. I just don't think this is a meaningful comparison, and I'm not convinced it's evidence of the "limitations" of biological computation. Silicone beats biology in doing binary computation because they're a single-purpose machine built for this task. But a brain is capable of serving as a control system to o…

I by no means say that our brain is not impressive - even a fly’s is marvelously complex and capable. But all of them are made up from cells that were created through evolution, not intelligent design. The same way the giraffe has a recurrent nerve going all the way down and up inside its neck for absolutely no reason other than evolution modifying only one factor (neck length) without restructuring, cells have many similar sorts of “hacks”. So I think it is naive to think that biological systems are efficient. They do tend to optimize for a local minima, but there are inherent hard limits there.

Also, while indeed we can’t simulate the whole of neuron, why would we want to do that? I think that is backwards. We only have to model the actually important function of a neuron. If we were to have a water computer, would it make sense to simulate fluid dynamics instead of just the logical gates? Due to the messiness of biology, indeed some hard to model factors will effect things (in the analogy, water will be spilt/evaporated) but we should rather overlook the ones that have a minimal influence on the results.

Re: Past Performance is Not Indicative of Future Results (2020)

#264

Post by skohan »

kaba0 wrote: post #262
skohan wrote: post #252

Earlier quoted context omitted.

> in serial computations silicone beats “meat” hands down. And that is in both power efficiency and performance. I just don't think this is a meaningful comparison, and I'm not convinced it's evidence of the "limitations" of biological computation. Silicone beats biology in doing binary computation because they're a single-purpose machine built for this task. But a brain is capable of serving as a control system to o…

I by no means say that our brain is not impressive - even a fly’s is marvelously complex and capable. But all of them are made up from cells that were created through evolution, not intelligent design. The same way the giraffe has a recurrent nerve going all the way down and up inside its neck for absolutely no reason other than evolution modifying only one factor (neck length) without restructuring, cells have many…

> while indeed we can’t simulate the whole of neuron, why would we want to do that? I think that is backwards. We only have to model the actually important function of a neuron.

Yeah so I think this is where we fundamentally differ. It seems like your assumption is that neurobiology is fundamentally messy and inefficient, and we should be able to dispense with the squishy bits and abstract out the real core "information processing" part to make something more efficient than a brain.

So if that's your assertion, what would that look like? What would be the subset of a neuron that we could simulate which would represent that distillation of the information processing part?

Because my argument would be, the squishy, messy cellular anatomy is the core information processing part. So if we try to emulate neural processing with the assumption that a whole neuron is the base unit, we will miss a lot of that micro-level processing which may be essential to reaching the utility and efficiency achieved by the human brain.

I'm not against the idea that whatever brains we happened to evolved are not the most efficient structure possible. But my position would be, we're probably quite far in terms of current computing technology from being able to build something better. I would imagine we might have to be able to bioengineer better neurons if we really want to compete with the real thing, rather than trying so simulate it in software.

Re: Past Performance is Not Indicative of Future Results (2020)

#265

Post by piokoch »

shreyshnaccount wrote: post #33
jstx1 wrote: post #16

> I am an AI skeptic. I am baffled by anyone who isn’t. I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ field that leads to a general AI - I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. - That doesn't make me a skeptic towards the current state of machine learning thoug…

I'm in favor of changing the terminology from AI and ML to something along the lines of 'prediction model' so that the idea of machines 'thinking' is replaced with them 'predicting'. it's just easier for our mushy meat brains to think that AI and ML means that it'll lead to general AI or as I like to call it 'general purpose decision maker'. it's all about the language!

When I have encountered for the first time ML term I decided to learn what is that new great stuff. To my great surprise this was a typical old new thing called "statistical inference" in the days when I was working as a statistician.

There were a few new things, like ignoring model, choosing right variables, whatever was available was thrown into the equation, if it was clear that such "model" is over-fitted, there were some methods to overcome this by adding some random coefficients to the model that were smoothing it a little.

So, the naming is there... could be modified by adding some clarification that we don't care that much about understanding model we plan to use.

Re: Past Performance is Not Indicative of Future Results (2020)

#266

Post by greggman3 »

pnt12 wrote: post #221
greggman3 wrote: post #212

Earlier quoted context omitted.

self-driving cars are available to consumers now. Search for FSD on youtube and see all the consumers using their self driving cars. Or, watch the latest Veritasium https://www.youtube.com/watch?v=yjztvddhZmI

And yet all of them force you to keep your eyes on the road at all times or you can die. Can you honestly call that FSD?

#1 it's still beta. The point is to show the progress is real

#2 see linked video, he sits in the backseat, there is no driver and no one to take control.

Re: Past Performance is Not Indicative of Future Results (2020)

#267

Post by kaba0 »

skohan wrote: post #264
kaba0 wrote: post #262

Earlier quoted context omitted.

I by no means say that our brain is not impressive - even a fly’s is marvelously complex and capable. But all of them are made up from cells that were created through evolution, not intelligent design. The same way the giraffe has a recurrent nerve going all the way down and up inside its neck for absolutely no reason other than evolution modifying only one factor (neck length) without restructuring, cells have many…

> while indeed we can’t simulate the whole of neuron, why would we want to do that? I think that is backwards. We only have to model the actually important function of a neuron. Yeah so I think this is where we fundamentally differ. It seems like your assumption is that neurobiology is fundamentally messy and inefficient, and we should be able to dispense with the squishy bits and abstract out the real core "informat…

I can’t think of any field of research where the model used is completely accurate. At one point we will have to leave behind the messy real world. While a simple weighted node is insufficient for modeling a neuron, there are more complex models that are still orders of magnitudes less complex than simulating every single interaction between the I don’t know how many moles of molecule (which we can’t even do as far as I know, not even on a few molecule basis, let alone at such a huge volume).

But I feel I may be misrepresenting your point now. To answer your question, maybe a sufficient model (sufficient to be able to reproduce some core functionality of the brain, eg. make memories) would be one that incorporates a weight for each sort of signal (neurotransmitter) it can process, complete with a fatigue model per signal type, as well as we can perhaps add the notable major interactions between pathways (eg. activation of one temporarily decreasing the weight of another, but in a way bias is sorta this in the very basic NNs). But to be honest, such a construction would be valuable even with arbitrary types of signals, no need to model it exactly based on existing neurotransmitters. I think most properties interesting from a GAI perspective are emerging ones, and whether dopamine does this and that is an implementation detail of human brains.

Re: Past Performance is Not Indicative of Future Results (2020)

#268

Post by shannifin »

Jetrel wrote: post #250
shannifin wrote: post #30

> I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ field that leads to a general AI any more than I can see a path from continuous improvements in horse-breeding that leads to an internal combustion engine. While I also don't expect that AGI will emerge solely through optimizing statistical inference models, I also don't think "improvements to the machine learning fi…

It's worth pointing out that "machine learning" is a specific term of art, not a term for AI in general. It refers very specifically to the type of "convolutional neural networks" that have made a bunch of progress over the past 15-25 years. The moment you have a paradigm shift, sure, it can be considered "learning done by machines", but it's not "Machine Learning™" anymore. -- This is why the author put it in quotes…

> It's worth pointing out that "machine learning" is a specific term of art, not a term for AI in general. It refers very specifically to the type of "convolutional neural networks" that have made a bunch of progress over the past 15-25 years.

In my experience, "machine learning" is more broad than "convolutional neural networks", aligning with Wikipedia's definition: "the study of computer algorithms that improve automatically through experience and by the use of data." https://en.wikipedia.org/wiki/Machine_learning

Re: Past Performance is Not Indicative of Future Results (2020)

#269

Post by Salgat »

skohan wrote: post #109
jstx1 wrote: post #16

> I am an AI skeptic. I am baffled by anyone who isn’t. I don’t see any path from continuous improvements to the (admittedly impressive) ‘machine learning’ field that leads to a general AI - I share the skepticism towards any progress towards 'general AI' - I don't think that we're remotely close or even on the right path in any way. - That doesn't make me a skeptic towards the current state of machine learning thoug…

Yeah I agree - during undergrad, I spent a few years studying neuroscience, and I was very let down by my first ML/AI course. Compared to what I had learned about the brain, what we called an "ANN" just seemed like such a silly toy. The more you learn about neurobiology, the more apparent it is that there are so many levels of computation going on - everything from dendritic structure, to cellular metabolism, to epig…

I dunno man, when I got into learning CNN it becomes very easy to see how neural networks work at an intuitive level and how that plays into its "intelligence". For example, early layers of the network respond very well to simple features like edges and simple patterns, while later layers respond really well to more and more abstract things, like the shape of a person or a wheel etc. The craziest part is that this is all emergent from a random initialized state, all these patterns and abstractions develop with no manual intervention, they just happen to be the result of backpropagation consistently lowering a cost function.

The biggest thing is that computing is just now finally able to have enough data and large enough networks to really start to create more generalized models. With a computer 20 years ago you might be able to do squeeze out simple pattern recognition, but for every layer and every neural node you add to a neural network, the more complex the model becomes and the more edge cases it can fold into itself.

Take a look at Universal approximation theorem and how, with enough nodes in a neural network, you can solve pretty much any problem given the right weights.

Re: Past Performance is Not Indicative of Future Results (2020)

#270

Post by gonehome »

lumost wrote: post #187
gonehome wrote: post #135

Earlier quoted context omitted.

I'm not sure I buy that - biology is often messier because of nature related constraints, it gets optimized for other things (energy, head size, etc.) The way a plane flies is quite different than the way a bird flies in complexity - they share an underlying mechanism, but planes don't need to flap wings. It's possible that scaling up does lead to generality and we've seen hints of that. - https://deepmind.com/blog/a…

The quirky thing to remember about gpt-3 is that it really is just a giant autocomplete based on the internet. It can do math insofar as it’s memorized some text which did that math with slightly different verbiage etc. If you ask it to compute something that would never have been seen on the internet it’s likely to fail. E.g. add 2 extremely large/rare numbers together

You should read the excerpts in the paper I link to which suggest otherwise (that it’s not memorized and that it’s deducing rules).
Post reply on HN