I love this idea. I did a related experiment[1] about a month ago with GPT-3 where I started with a stub that takes a prompt and self modifies the existing code.
import os
import openai
instruction = input("Enter an instruction: ")
script_name = os.path.basename(__file__)
script_code = open(script_name).read()
response = openai.Edit.create(
model="code-davinci-edit-001",
input=script_code,
instruction=instruction,
temperature=0)
new_script_code = response["choices"][0]["text"]
with open(script_name, "w") as f:
f.write(new_script_code)
My first attempt[2] was a little more complex than above (in the main branch), but the second attempt worked well enough to give it the following instructions (which are reflected in the git history) and which end up with a fairly complex result[3]
- Init Crow version 1
- Minimal crow second line
- Before saving the new script code show the diff of the existing code
- Ask for user confirmation before saving
- Use pygments to highlight the diff
- Add a main method
- Refactor the code
- Add README
- Add a version number to the script that starts at 2.0.0
- Log the openai call and info about the response
- Prompt the user for whether they want to run the new version of the script after saving
- Remove the "text" field fromthe response json before logging it
- Remove the openai result logging
- In the code that calls the openai edit, add an extra instruction that increments the version number.
- Show the version number as part of the instruction prompt
- Remove any extraneous code
- Allow the user to quit
- If the user chooses to quit, just stop the program
- The code currently prompts the user to save changes twice. It should only display this prompt a single time.
- The code currently adds all files to git when committing. It should only add the script.
- Add code to rollback using git if an error occurs. The user should be able to choose whether or not to rollback. Rolling back is acheived by - to the previous commit.
- The commit message is too long. The first sentence of the instruction should be the title. The rest of the instruction should be the body.
- Wrap the commit message body at 72 characters
- After inputing the instruction preprocess it
- Move the preprocess call outside the edit method
- Keep track of instructions that were not saved
- Add keyboard completion for unsaved instructions
- Ensure the code has no syntax errors
- Don't retry on syntax errors, just fail
- Refactor the main method for readability
- Increase the temperature to 0
- Add documentation
- Add logging
- Throw an error if the openai response does not change the code
- Don't repeat the commit title in the commit body
- Add a doc string to the file that describes how things work
- Change the temperature to 1
- (HEAD -> crow-v2, origin/crow-v2) Introduce some humor into the code
My goal was eventually to get to something was self sustaining like TFA's script just via conversation rather than actually coding it. (So fork a new version and see if it works then commit / skip back etc.)
The current version is at[2] and contains this explanation which GPT wrote. (I should probably make it describe itself in the README and handle multiple files generally at some point.
"""
Instructions are useful for humans. Even if we forget
a single line of code, we can just look at the instructions
to figure out what to do. And that's exactly how this file
works!
Every time you run this file, you are asked to enter an
instruction. I call it this way because each instruction starts
with a verb. You could also call it a `command` or `task` or whatever
you like.
For example, `fix bugs`, `add logging`, `fix syntax errors`,
`add function definitions`, `fix indentation`, `add doc strings`.
Just enter a simple english instruction of what you want to do
with this file, and hit enter. The file will be edited to satisfy
that instruction on the fly. If you're happy with these changes,
you can save them by pressing 'y'. If not, press 'n' and the changes
will be discarded.
Then, you can run the new version of the file by pressing 'y' in
the prompt that follows.
That's it!
All the boilerplate you add in this file will be added automatically
by the model.
The model can add any python boilerplate to the file, such as function
definitions, classes, loops, conditionals, doc strings, syntax errors
etc. It can even fix such syntax errors.
"""
Overall it's a fun experiment to try. One of the big hassles was getting code that didn't quite have the correct spacing due to python's whitespace sensitivity. I was using the code editing GPT model code-davinci-edit-001 instead of I wonder if typescript might have been a better choice to avoid that.
A fun thought (so far at least) experiment I'm having right now is what would be necessary to hook this up to an interactive jupyter notebook and let it start writing its own code to present widgets and UI, and then to have it journal its own creation dynamically, and then start calling other APIs and installing various software.
Also related, Fixie.ai[4] seems to be tackling some similar areas in creating small few-shot-prompt to function hookups that when automatically composed make some of this language based autonomous agent stuff pretty exciting.
[1]: https://github.com/joshka/Crow
[2]: https://github.com/joshka/Crow/blob/main/crow-v1.py
[3]: https://github.com/joshka/Crow/blob/crow-v2/crow.py
[4]: https://fixie.ai