2 times 3 can sometimes equal 7 with Android's Neural Network API

Post by colejohnson66 » Sat, Jan 23, 2021, 11:25 PM UTC

dealforager wrote: post #10

Avalaxy wrote: post #7

Using a neural network for things that have clear cut rules is wrong. When you know the exact rules, implement them as such, instead of bruteforcing a guesstimation. This is also why I'm sceptical of the usr of GPT-3 for all sorts of purposes where accuracy is important. Think of the code generation case. Bugs may be very subtle and may go unnoticed.

For code, I could see it being super useful for a beefed up auto-complete. There are many times I find myself searching for things like "how do I do X in Y language" to copy a snippet that I'm sure has been written 10000x times before. I can review the code and verify its correctness by writing tests.

Post by Grimm1 » Sat, Jan 23, 2021, 11:26 PM UTC

yuliyp wrote: post #30

Grimm1 wrote: post #9

Earlier quoted context omitted.

Code generation only needs to generate code with n bugs where n is less than the number of bugs a human developer generates for it to have usefulness, and maybe some other factor of severity where they are generally less severe than human developers. I think it'll make neat autopilot functionality for developers but not replace the need to have someone look over and understand the code.

This is a very simplistic of what code is and the role it plays in a system. There are many implementations that can fulfill a set of requirements. Not all of them are created equal. The ways in which they behave as the system changes can be wildly different. Well-written code will be able to handle those changes gracefully. Poorly-written code may end up proving brittle and bug-prone. Generated code will be complete…

Post by Grimm1 » Sat, Jan 23, 2021, 11:28 PM UTC

danfang wrote: post #32

Grimm1 wrote: post #9

Earlier quoted context omitted.

Code generation only needs to generate code with n bugs where n is less than the number of bugs a human developer generates for it to have usefulness, and maybe some other factor of severity where they are generally less severe than human developers. I think it'll make neat autopilot functionality for developers but not replace the need to have someone look over and understand the code.

This is naive. The point is that code is a well defined system with clear rules that can be expressed through logic and mathematics. GPT is suited to approximate systems where the rules are not well defined. Until AI can actually learn the principles of logic, it may not be useful for code generation on a meaningful scale, other than things just like simple auto-completions. Not only that, AI would also have to learn…

Post by aga_ml » Sun, Jan 24, 2021, 12:11 AM UTC

shoyer wrote: post #36

As someone who builds neural networks routinely, this sort of non-reproducibility sounds troubling to me. We expect small differences for floating point arithmetic between platforms, but integer math is typically exact. This is all the more concerning for 8-bit quantized arithmetic, where off-by-one means a relative error of about half a percent. If a individual layers in a quantized neural net have off-by-one errors…

Post by yuliyp » Sun, Jan 24, 2021, 2:06 AM UTC

Grimm1 wrote: post #43

yuliyp wrote: post #30

Earlier quoted context omitted.

This is a very simplistic of what code is and the role it plays in a system. There are many implementations that can fulfill a set of requirements. Not all of them are created equal. The ways in which they behave as the system changes can be wildly different. Well-written code will be able to handle those changes gracefully. Poorly-written code may end up proving brittle and bug-prone. Generated code will be complete…

I don't know what you're arguing against but it sure isn't what I wrote. "Code generation only needs to generate code with n bugs where n is less than the number of bugs a human developer generates for it to have usefulness, and maybe some other factor of severity where they are generally less severe than human developers." Point to the part you're arguing against because you way extrapolated what "have usefulness" m…