11–20 of 82 posts
These and similar attempts are like those "perpetual motion machines" that captivated people's imagination for a long time. That the machine will generate energy and that would consume energy and thus will keep moving forever. Similar hopes here and everywhere. A model will generate its own input and will watch it's own output and in process, will become more intelligent than it really is.
Death by semantic error. An AI tried to tell me the {} opens a new block scope in PHP and any variables in it are scoped to that block. I nearly lol’d and the code it gave me was so wrong it was cringey. Hopefully it is better at Python.
Gpt4 is quite good (better than most humans I ever met) at php, but bad at facts; don’t ask it facts, ask it to write code. That’s what you would ask a human (outside interviews).
Disclaimer; I am formally trained with proofs and proof assistants and I hate the current timeline where we ask ai to drivel up code, but I cannot say it’s bad at doing it; it just is not necessarily sound or even working code sure, but that’s the same as with most human first tries. Then you iterate and make better. My days of ‘I have proven it correct, now I just have to type it in’ are long gone, at least for things that pay for my bread.
https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...
We've found that LLMs are pretty bad at prompting other LLMs, unless the problem at hand is very limited in scope. It's too easy to get incorrect/expensive behavior otherwise (e.g. starts building a framework against an imaginary API, instead of using an existing tool).
Our approach looks more like a state machine under the hood, mostly code with LLM-based "magic" sprinkled throughout.
The tool can both edit code and LLM "blocks" as it sees fit, allowing it to change it's functionality and prompting dynamically.
Interestingly, we first set the validate->fix threshold to N=3, but the "agent" often gets stuck in a pattern of retries based on low quality user input.
[0] Feel free to give our tool a try, it's very much in an alpha state: https://magicloops.dev/
Earlier quoted context omitted.
But isn’t this also a step towards AGI? The model being able to find issues and self correct?
What does AGI man to you? Is it something they define or you think is good enough, And if the latter when?
In a way, that’s what modern multi-stage-trained foundational models already do: improve their weights intelligently. Having the same results in human readable code (what this is a first step towards) would be a lot more powerful…
Death by semantic error. An AI tried to tell me the {} opens a new block scope in PHP and any variables in it are scoped to that block. I nearly lol’d and the code it gave me was so wrong it was cringey. Hopefully it is better at Python.
Google and many people will tell you the same thing. So you must have a lot of lol moments in a day. Gpt4 is quite good (better than most humans I ever met) at php, but bad at facts; don’t ask it facts, ask it to write code. That’s what you would ask a human (outside interviews). Disclaimer; I am formally trained with proofs and proof assistants and I hate the current timeline where we ask ai to drivel up code, but I…
Kind of having the same feeling as well the more I use ChatGPT and similar systems/models.
I wonder if this will result in a general decline in code quality. Or if it's going to help people become better IT professionals overall.
I mean, in order to iterate and make things better after first try/implementation, also requires understanding what you typed and it can be improved.
Exciting times nonetheless
These and similar attempts are like those "perpetual motion machines" that captivated people's imagination for a long time. That the machine will generate energy and that would consume energy and thus will keep moving forever. Similar hopes here and everywhere. A model will generate its own input and will watch it's own output and in process, will become more intelligent than it really is.
As far as I know, we can't yet disprove that a self-improving AI will become more intelligent than it starts as.
I'm not trying to say that it is likely or that I believe it will happen. Just that we can't disprove it yet, unlike perpetual motion devices.
EDIT: corrected from 3rd to 2nd law of thermodynamics.
Earlier quoted context omitted.
But isn’t this also a step towards AGI? The model being able to find issues and self correct?
As described in the essay "optimality is the tiger, and agents are its teeth". https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...
Death by semantic error. An AI tried to tell me the {} opens a new block scope in PHP and any variables in it are scoped to that block. I nearly lol’d and the code it gave me was so wrong it was cringey. Hopefully it is better at Python.
Google and many people will tell you the same thing. So you must have a lot of lol moments in a day. Gpt4 is quite good (better than most humans I ever met) at php, but bad at facts; don’t ask it facts, ask it to write code. That’s what you would ask a human (outside interviews). Disclaimer; I am formally trained with proofs and proof assistants and I hate the current timeline where we ask ai to drivel up code, but I…
"good at taking high-level English-language solutions to programming problems and translating them into low-level implementations via a specific language / design patterns / etc"
versus
"good at finding solutions to programming problems."
GPT-4 is indeed quite good at the former - and the former is what most enterprise programming work actually is. A lot of the hard part of professional software development is understanding the problem well enough to describe a solution in English: once you do that, writing the C# or Java is typically somewhat rote. Likewise LLMs are genuinely useful when you know exactly what you want to do with a 3rd-party library, but have to trawl through a bunch of API documentation to figure out the magic words.
All that said, LLMs still really suck at the latter problem: https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmar... OpenAI's benchmarks fall off quite badly with actual programming benchmarks, compared to simpler tests of code generation. If it involves managing state, creating novel data structures, counting to numbers higher than 3, etc, LLMs just aren't smart enough.
These and similar attempts are like those "perpetual motion machines" that captivated people's imagination for a long time. That the machine will generate energy and that would consume energy and thus will keep moving forever. Similar hopes here and everywhere. A model will generate its own input and will watch it's own output and in process, will become more intelligent than it really is.
We are extremely similar genetically to our ancestors of 100k years ago. The big difference is cultural inheritance. We come up with ideas and objects and pass them down. Humans have improved the capabilities of humans. There's no reason to think computer programs aren't capable of the same.