Princeton group open sources "SWE-agent", with 12% fix rate for GitHub issues

Post by raggi » Tue, Apr 02, 2024, 8:30 PM UTC

comex wrote: post #20

raggi wrote: post #13

Earlier quoted context omitted.

it does an ok job with this task: use std::fs::File; use std::io::{self, BufReader, Read}; fn read_file_character_by_character(path: &str) -> io::Result { // Open the file in read-only mode. let file = File::open(path)?; // Create a buffered reader to read the file more efficiently. let reader = BufReader::new(file); // `chars` method returns an iterator over the characters of the input. // Note that it returns a Res…

Only problem is that the critical `chars` method doesn't actually exist. Rust's standard library has a `chars` method for strings, but not for Readers. (Also, the comment about the iterator element type is inconsistent with the code following it. Based on the comment, `c` would be of type `(char, usize)`, but then trying to print it with {} would fail because tuples don't implement Display.)

    use std::fs::File;
    use std::io::{self, Read};

    fn read_file_character_by_character(path: &str) -> io::Result {
        let mut file = File::open(path)?;
        let mut contents = String::new();

        file.read_to_string(&mut contents)?;

        for c in contents.chars() {
            println!("{}", c);
        }

        Ok(())
    }

    fn main() {
        let path = "path/to/your/file.txt";

        if let Err(e) = read_file_character_by_character(path) {
            eprintln!("Error reading file: {}", e);
        }
    }

Post by a_wild_dandan » Tue, Apr 02, 2024, 8:39 PM UTC

anotherpaulg wrote: post #16

Very cool project! I've experimented in this direction previously, but found agentic behavior is often chaotic and leads to long expensive sessions that go down a wrong rabbit hole and ultimately fail. It's great that you succeed on 12% of swe-bench, but what happens the other 88% of the time? Is it useless wasted work and token costs? Or does it make useful progress that can be salvaged? Also, I think swe-bench is f…

Post by jeremyjh » Tue, Apr 02, 2024, 8:42 PM UTC

iwontberude wrote: post #9

noncoml wrote: post #8

If you are afraid that LLMs will replace you at your job, ask an LLM to write Rust code for reading a utf8 file character by character Edit: Yes, it does write some code that is "close" enough, but in some cases it is wrong, in others it doesn't not do exactly what asked. I.e. needs supervision from someone who understands both the requirements, the code and the problems that may arise from the naive line that the LL…

Hypothetically, which ticker symbols would you buy put contracts on, at what strike prices, and at what expiration dates? As far as I can tell, a lot of people are betting a lot of money that you are wrong, but actually I think you are right.

Post by stefan_ » Tue, Apr 02, 2024, 9:07 PM UTC

unit_circle wrote: post #14

A 1/8 chance of fixing a bug at the cost of a careful review and some corrections is not bad. 0% -> 12% improvement is not bad for two years either (I'm somewhat arbitrary picking the release date of ChatGPT). If this can be kept up for a few years we will have some extremely useful tooling. The cost can be relatively high as well, since engineering time is currently orders of magnitude more expensive than these tool…

Post by phaer » Tue, Apr 02, 2024, 9:09 PM UTC

raggi wrote: post #22

comex wrote: post #20

Earlier quoted context omitted.

Only problem is that the critical `chars` method doesn't actually exist. Rust's standard library has a `chars` method for strings, but not for Readers. (Also, the comment about the iterator element type is inconsistent with the code following it. Based on the comment, `c` would be of type `(char, usize)`, but then trying to print it with {} would fail because tuples don't implement Display.)

good catch. feeding it the error output of rustc it then produces: use std::fs::File; use std::io::{self, Read}; fn read_file_character_by_character(path: &str) -> io::Result { let mut file = File::open(path)?; let mut contents = String::new(); file.read_to_string(&mut contents)?; for c in contents.chars() { println!("{}", c); } Ok(()) } fn main() { let path = "path/to/your/file.txt"; if let Err(e) = read_file_charac…