31–40 of 150 posts
Very neat. Uses the langchain method, here are some of the prompts: https://github.com/princeton-nlp/SWE-agent/blob/main/config/...
I’m always fascinated to read the system prompts & I always wonder what sort of gains can be made optimizing them further. Once I’m back on desktop I want to look at the gut history of this file.
Earlier quoted context omitted.
it does an ok job with this task: use std::fs::File; use std::io::{self, BufReader, Read}; fn read_file_character_by_character(path: &str) -> io::Result { // Open the file in read-only mode. let file = File::open(path)?; // Create a buffered reader to read the file more efficiently. let reader = BufReader::new(file); // `chars` method returns an iterator over the characters of the input. // Note that it returns a Res…
If we're being sticklers, this isn't reading character-by-character: it's performing a buffered read, which then gets iterated over.
If you are afraid that LLMs will replace you at your job, ask an LLM to write Rust code for reading a utf8 file character by character Edit: Yes, it does write some code that is "close" enough, but in some cases it is wrong, in others it doesn't not do exactly what asked. I.e. needs supervision from someone who understands both the requirements, the code and the problems that may arise from the naive line that the LL…
Hypothetically, which ticker symbols would you buy put contracts on, at what strike prices, and at what expiration dates? As far as I can tell, a lot of people are betting a lot of money that you are wrong, but actually I think you are right.
A 1/8 chance of fixing a bug at the cost of a careful review and some corrections is not bad. 0% -> 12% improvement is not bad for two years either (I'm somewhat arbitrary picking the release date of ChatGPT). If this can be kept up for a few years we will have some extremely useful tooling. The cost can be relatively high as well, since engineering time is currently orders of magnitude more expensive than these tool…
Friendly suggestion to the authors: success rates aren't meaningful to all but a handful of researchers. They should add a few examples of tests SWE-agent passed and did not pass to the README.
If you are afraid that LLMs will replace you at your job, ask an LLM to write Rust code for reading a utf8 file character by character Edit: Yes, it does write some code that is "close" enough, but in some cases it is wrong, in others it doesn't not do exactly what asked. I.e. needs supervision from someone who understands both the requirements, the code and the problems that may arise from the naive line that the LL…
The way I see it, its undetermined if Generative AI will be able to fully do a SWE job. But, for most of the debates I've seen, I don't think it the answer matters all too much. Once we have models that can act as full senior SWEs.. the models can engineer the models. And then we've hit the recursive case. Once models can engineer models better and faster than humans, all bets are off. Its the foggy future. Its the s…
This is such an extremely bullish case, I'm not sure why you'd think this is even remotely possible. A Google search is usually more valuable than ChatGPT. For example, the rust utf-8 example is already verbatim solved on reddit: https://www.reddit.com/r/rust/comments/l5m1rw/how_can_i_effi...
Once we have this fully automated, any good developer could have a team of 100 robo SWEs and ship like crazy. The real competition is with those devs not with the bots.
A 1/8 chance of fixing a bug at the cost of a careful review and some corrections is not bad. 0% -> 12% improvement is not bad for two years either (I'm somewhat arbitrary picking the release date of ChatGPT). If this can be kept up for a few years we will have some extremely useful tooling. The cost can be relatively high as well, since engineering time is currently orders of magnitude more expensive than these tool…