Would python decorators be better for something like this? I always get squeamish when I see magic comments
Here is a framework which uses decorators to delegate runtime behavior to an LLM. Not quite what you meant but the closest I’ve seen.
Maccarone: AI-managed code blocks in Python
11–20 of 75 posts
Re: Maccarone: AI-managed code blocks in Python
#12Earlier quoted context omitted.
How would not having the source code be present/pre-generated, and thus needing to generate it at runtime, be an example of a decorator having no runtime effects?
I had the urge to try this out a while back, here's what I came up with: https://gist.github.com/nkrumm/2b154ea2041511233079222373c83... The decorator invokes AI completion only the first time the function is run. edit: I lost interest before I was able to get arguments to work ¯\_(ツ)_/¯
Yea, that’s cool and very possible, but it’s a runtime effect
Re: Maccarone: AI-managed code blocks in Python
#13"Hallucination isn't a real problem, people will always scrutinize the generated code!"
Sigh...
Re: Maccarone: AI-managed code blocks in Python
#14Assuming you're using source control properly and read the diff before running it, I guess this is one way to make sure that a comment matches the code? If the bot changes it, maybe your comment wasn't clear enough?
Re: Maccarone: AI-managed code blocks in Python
#15Very cool project. How reliable are you finding your prompts? They look like good choices based on my experience prompting GPT-3.5 and 4 for code editing.
FYI, I think my open source tool aider would work out of the box to serve this use case. You would just run:
aider file.py —msg “implement the comments”
Re: Maccarone: AI-managed code blocks in Python
#16Earlier quoted context omitted.
There are examples of decorators having no runtime effects, such as typing.overload. Bit comments are probably more flexible here since it allows arbitrary blocks, not just function/class scopes.
How would not having the source code be present/pre-generated, and thus needing to generate it at runtime, be an example of a decorator having no runtime effects?
OP has to be doing some parsing somewhere, so you just switch to seeking decorators rather than magic comments. It's still before the code reaches the interpreter.
The potential issue I see here is that comments are valid anywhere while decorators may not be, but the parser is hopefully resilient to that. You could see a multi-phase LLM that uses the interpreter to ensure the code runs / works as expected
Re: Maccarone: AI-managed code blocks in Python
#17Not sure I see the benefit over standard ai integration into an editor, what am I missing?
Re: Maccarone: AI-managed code blocks in Python
#18Would python decorators be better for something like this? I always get squeamish when I see magic comments
This is leaving a comment for another programmer, not the compiler or interpreter. It's what comments are for, actually, like writing a TODO. You should be squeamish about running the code without reading it first, given that you're pair-programming with a bot.
> You should be squeamish about running the code without reading it first, given that you're pair-programming with a bot.
It's funny, the first version of this project[0] let you do exactly that, e.g.,
def main(path: str):
#>
for fn in filenames:
#>
print(fn, size)
#>
$ python -m examples.file_sizes /etc
…
/etc/wgetrc 4942
/etc/nsswitch.conf 542
/etc/adduser.conf 3028
/etc/ethertypes 1816
Re: Maccarone: AI-managed code blocks in Python
#19"Hallucination isn't a real problem, people will always scrutinize the generated code!" Sigh...
Thorough code review is why we as an industry stopped shipping bugs.
Re: Maccarone: AI-managed code blocks in Python
#20There are so many languages with awesome type systems which can help guide AI to generate better code — and yet, these experiments always choose Python.