Live data from Hacker News

2 times 3 can sometimes equal 7 with Android's Neural Network API

alexanderganderson.github.io

71–80 of 92 posts

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#71

Earlier quoted context omitted.

You’re (sadly) assuming everyone would verify its correctness. Proper programming would mean one would write tests, but not everyone does. I’m guilty of it too.

When banks got ATMs, they thought it'd drastically cut headcount - instead it went _up_: freeing up time from doing the basics meant you had more time to focus on more profitable activities I hope to live long enough to be mostly writing tests for a gloriously hacky code generator that gets it right 80% of the time

tests and specifications / prompt engineering

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#72

Earlier quoted context omitted.

How is it that human brains can follow deterministic logic but neural networks can't? What's the missing piece? Is it just that people are a lot more complex than algorithms like GPT-3?

That's essentially like asking "how is it that human feet can walk up mountain slopes, but car wheels can't?" That is, there is no relation between human brains and artificial neural networks, other than them serving similar purposes in particular environments.

I respectfully disagree - I believe this is a philosophical viewpoint that shouldn't be presented as a straightforward truth.

Unless you are a dualist, I would say that it's reasonable to view that it is in principle possible to produce an artificial network accurately emulating the function and behavior of a human brain.

If you are a dualist, then there is no further discussion to be had as we are very unlikely to ever be able to prove anything like the existence of a soul.

Apologies if you were speaking to some more subtle nuance that I was unable to pick up.

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#74
post #72

Earlier quoted context omitted.

That's essentially like asking "how is it that human feet can walk up mountain slopes, but car wheels can't?" That is, there is no relation between human brains and artificial neural networks, other than them serving similar purposes in particular environments.

I respectfully disagree - I believe this is a philosophical viewpoint that shouldn't be presented as a straightforward truth. Unless you are a dualist, I would say that it's reasonable to view that it is in principle possible to produce an artificial network accurately emulating the function and behavior of a human brain. If you are a dualist, then there is no further discussion to be had as we are very unlikely to e…

TL; DR: I'm saying that there is no structural resemblance between the human brain and artificial neural networks specifically (even though there likely is a strucutral resemblance between the human mind/brain and a computer in the general sense). You can believe in AGI and not believe it will be achieved with ANNs.

> I would say that it's reasonable to view that it is in principle possible to produce an artificial network accurately emulating the function and behavior of a human brain.

I think that claim is far too strong. As a non-dualist, I do believe that it is possible to create an artificial "brain" that has the same cognition as a human brain. However, simply rejecting dualism does not tell you anything about what the artificial brain has to be.

You can further say that a non-dualist who accepts the Church-Turing thesis must accept that there must exist a Turing machine which has the same cognition as the human brain. Since the PCs we use are Turing machines, it follows that we should be able to program one to behave like a human brain, in theory at least (disregarding hardware requirements, of course).

Still, that does not mean that a brain Turing machine has to look anything like a neural network trained through gradient descent & back propagation. This was my point: artificial neural networks and the methods we use to train them have no resemblance to the human brain, and there is no reason to believe that they are the way to create an artificial general intelligence. So, there is no reason to be surprised that a neural network, especially one as small as any of the ones we have realized so far, doesn't exhibit complex properties of the human brain.

Artificial neural networks are just a statistical model that was once inspired by a very, very simplistic idea of what biological neural networks are. As we have discovered more about biological neural networks, we've abandoned any notion of comparing ANNs with biological neural networks in terms of actual structure.

This is all not to say that it's impossible for a complex enough ANN to actually be an AGI. It's just not going to be that surprising if it won't be, if an AGI program will look significantly different, and will be trained in completely different ways.

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#75
post #7

Using a neural network for things that have clear cut rules is wrong. When you know the exact rules, implement them as such, instead of bruteforcing a guesstimation. This is also why I'm sceptical of the usr of GPT-3 for all sorts of purposes where accuracy is important. Think of the code generation case. Bugs may be very subtle and may go unnoticed.

I once (2010) had an oral exam in neural networks during which I had to design a system to solve a particular task. My solution used two neural networks connected by a simple logic circuit. The professor, who was actually a neuro scientist with little understanding of the technical, asked why not a third neural network. This was a neural network course, after all. Thinking this was a trick question I excitedly explai…

Yeah, there’s a better way of saying that and it sounds like that was an expensive lesson in communication.

“Something something, it’s better to spend training time on the parts of the network where we don’t know the function beforehand than to train a subnetwork to do a function that we do know exactly at the outset.” and still you need to be ready to be asked about how to backpropogate through your hard coded function.

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#76
That works as intended.

What is called AI, or "Artificial Intelligence" should in reality be called "Artificial Intuition".

It is similar to the subconscious mind that is able to get approaches to a solution very fast, but does not give you the solution itself. You need the logical conscious mind(similar to the CPU) to refine the solution.

The logical conscious mind is so slow that will never get the solution on its own, but being so close to the solution it can.

AI 1.0 was about solving all problems just using rational methods alone, like Lisp programming. AI 2.0 is solving all problems by neural networks and training alone without understanding or testing if a solution is right or why it is right.

Real artificial intelligence should be about integrating both approaches. E.g You use intuition to train a network in the English language, but then you use it to develop the english Grammar from it. You extract the structure from the data.

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#77

Earlier quoted context omitted.

I once (2010) had an oral exam in neural networks during which I had to design a system to solve a particular task. My solution used two neural networks connected by a simple logic circuit. The professor, who was actually a neuro scientist with little understanding of the technical, asked why not a third neural network. This was a neural network course, after all. Thinking this was a trick question I excitedly explai…

Yeah, there’s a better way of saying that and it sounds like that was an expensive lesson in communication. “Something something, it’s better to spend training time on the parts of the network where we don’t know the function beforehand than to train a subnetwork to do a function that we do know exactly at the outset.” and still you need to be ready to be asked about how to backpropogate through your hard coded funct…

It's a test, you shouldn't have to sugar-coat explanations to an instructor (unless you're acting out some other scenario as part of the test).

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#78
post #7

Using a neural network for things that have clear cut rules is wrong. When you know the exact rules, implement them as such, instead of bruteforcing a guesstimation. This is also why I'm sceptical of the usr of GPT-3 for all sorts of purposes where accuracy is important. Think of the code generation case. Bugs may be very subtle and may go unnoticed.

How is it that human brains can follow deterministic logic but neural networks can't? What's the missing piece? Is it just that people are a lot more complex than algorithms like GPT-3?

Human brains can just barely follow deterministic logic. Computers are many orders of magnitude better at the task.

It's probably layered on top of fuzzier tasks we're better at.

Maybe a sufficiently advanced iteration of GPT could do deterministic tasks as slowly and unreliably as we can.

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#80

Earlier quoted context omitted.

Yeah, there’s a better way of saying that and it sounds like that was an expensive lesson in communication. “Something something, it’s better to spend training time on the parts of the network where we don’t know the function beforehand than to train a subnetwork to do a function that we do know exactly at the outset.” and still you need to be ready to be asked about how to backpropogate through your hard coded funct…

It's a test, you shouldn't have to sugar-coat explanations to an instructor (unless you're acting out some other scenario as part of the test).

In a test you have to prove your knowledge by transmitting symbols through language. If you don't "sugar coat" it, how do you expect that the right symbols will be interpreted by the receiver? It is part of the test to use the appropriate language to ensure the best understanding of what you are saying. Nonviolent communication tries to do exactly that and is essential to this end.

You can argue that since the professor understood what was being said, the language shouldn't matter, but it does. If again you don't use the correct language you risk offending the listener so much he can't get past that. After all you are dealing with humans, not machines, and in either case you are responsible for clear communication.

Post reply on HN