Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

2 times 3 can sometimes equal 7 with Android's Neural Network API

alexanderganderson.github.io

Post reply on HN
Print view View original

51–60 of 92 posts

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#51

Post by danfang »

Grimm1 wrote: post #45
danfang wrote: post #32

Earlier quoted context omitted.

This is naive. The point is that code is a well defined system with clear rules that can be expressed through logic and mathematics. GPT is suited to approximate systems where the rules are not well defined. Until AI can actually learn the principles of logic, it may not be useful for code generation on a meaningful scale, other than things just like simple auto-completions. Not only that, AI would also have to learn…

Good thing I didn't say it would, I'm really not sure what you think I wrote.

Your only metric of "good" code generation is # of bugs. My assertion was that this is not accurate.

I also disagree with the premise that code generation by AI will be very useful for programmers, for the reasons stated above.

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#52

Post by Something1234 »

taneq wrote: post #46
colejohnson66 wrote: post #41

Earlier quoted context omitted.

You’re (sadly) assuming everyone would verify its correctness. Proper programming would mean one would write tests, but not everyone does. I’m guilty of it too.

Just have GPT-3 produce the tests too! :D

Please don't give the MBAs ideas.

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#53

Post by nmfisher »

dealforager wrote: post #10
Avalaxy wrote: post #7

Using a neural network for things that have clear cut rules is wrong. When you know the exact rules, implement them as such, instead of bruteforcing a guesstimation. This is also why I'm sceptical of the usr of GPT-3 for all sorts of purposes where accuracy is important. Think of the code generation case. Bugs may be very subtle and may go unnoticed.

For code, I could see it being super useful for a beefed up auto-complete. There are many times I find myself searching for things like "how do I do X in Y language" to copy a snippet that I'm sure has been written 10000x times before. I can review the code and verify its correctness by writing tests.

Either OpenAI or Microsoft demoed something similar to this some time in theist 12-18 months.

The fact that it’s not released (and also the fact that GPT-3 etc are still not publicly available) makes me suspect that these models are far too unstable for actual production use. It’s also why I’m getting a bit tired of these overhyped cherry-picked samples with seemingly nothing solid to ever back it up.

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#55

Post by HotVector »

Avalaxy wrote: post #7

Using a neural network for things that have clear cut rules is wrong. When you know the exact rules, implement them as such, instead of bruteforcing a guesstimation. This is also why I'm sceptical of the usr of GPT-3 for all sorts of purposes where accuracy is important. Think of the code generation case. Bugs may be very subtle and may go unnoticed.

How is it that human brains can follow deterministic logic but neural networks can't? What's the missing piece? Is it just that people are a lot more complex than algorithms like GPT-3?

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#56

Post by shoyer »

aga_ml wrote: post #47
shoyer wrote: post #36

As someone who builds neural networks routinely, this sort of non-reproducibility sounds troubling to me. We expect small differences for floating point arithmetic between platforms, but integer math is typically exact. This is all the more concerning for 8-bit quantized arithmetic, where off-by-one means a relative error of about half a percent. If a individual layers in a quantized neural net have off-by-one errors…

Thanks for your comments. Regarding determinism, potentially a fair point. Here are a few comments: (1) A driver which randomly produces different output when running the network would be valid according to these restrictions. (2) It is conceivable that a driver would produce non-deterministic input with the same hardware. One commonly known example is that tensorflow will run multiple different convolution kernels a…

All good points! "Non-deterministic" behavior within the same program/process is still a bridge I would not want to cross. This could result in subtle glitches, e.g., when a user hits "refresh" with the same inputs, and could make reproducing bugs impossible.

I am a strong believer in always using a seed for random number generation for exactly these sorts of reasons. (Side note: deterministic RNGs is one of my favorite features about JAX.)

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#57

Post by Eridrus »

Avalaxy wrote: post #7

Using a neural network for things that have clear cut rules is wrong. When you know the exact rules, implement them as such, instead of bruteforcing a guesstimation. This is also why I'm sceptical of the usr of GPT-3 for all sorts of purposes where accuracy is important. Think of the code generation case. Bugs may be very subtle and may go unnoticed.

I see where you're coming from, but given the (usual) flexibility and generality of neural neys, it's often very tempting to assume that you can just keep adding data, and I think these are good reminders that being able to do complicated things doesn't mean they can do simple things efficiently (which is a conversation that has come uo several times at work)

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#58

Post by _-___________-_ »

Hasnep wrote: post #29
Judgmentality wrote: post #21

Earlier quoted context omitted.

> I'd trust a machine to drive more accurately than most people I see on the highway. Humans aren't special, in fact more often than not we're sloppy, subject to fatigue, and a whole bunch of other negative things. Humans are much, much, much more capable than the absolute state-of-the-art robots when it comes to doing things in an uncontrolled environment. https://www.youtube.com/watch?v=g0TaYhjpOfo

One of the advantages of an autonomous driver is that its superhuman reflexes, never driving while tired, never getting road rage, etc., will make it less likely to get into an uncontrolled environment. Would you prefer your pilots to fly your plane with no AI assistance?

> Would you prefer your pilots to fly your plane with no AI assistance?

There is nothing that remotely resembles AI in the cockpit of any current airliner. All flight control logic including autopilot, autothrottle, TCAS & GPWS, ILS & autoland, and so on are based on simple feedback loops and programming techniques that go back decades.

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#59

Post by Dolores12 »

Avalaxy wrote: post #7

Using a neural network for things that have clear cut rules is wrong. When you know the exact rules, implement them as such, instead of bruteforcing a guesstimation. This is also why I'm sceptical of the usr of GPT-3 for all sorts of purposes where accuracy is important. Think of the code generation case. Bugs may be very subtle and may go unnoticed.

This is a great way to find out if you are talking to a bot or not :)

Re: 2 times 3 can sometimes equal 7 with Android's Neural Network API

#60

Post by danielheath »

nmfisher wrote: post #53
dealforager wrote: post #10

Earlier quoted context omitted.

For code, I could see it being super useful for a beefed up auto-complete. There are many times I find myself searching for things like "how do I do X in Y language" to copy a snippet that I'm sure has been written 10000x times before. I can review the code and verify its correctness by writing tests.

Either OpenAI or Microsoft demoed something similar to this some time in theist 12-18 months. The fact that it’s not released (and also the fact that GPT-3 etc are still not publicly available) makes me suspect that these models are far too unstable for actual production use. It’s also why I’m getting a bit tired of these overhyped cherry-picked samples with seemingly nothing solid to ever back it up.

I use tabnine and it does a pretty nice job by using AI to re-score the outputs of (language-server + similar-words-in-codebase).
Post reply on HN