Viewing profile — brammertottens

This is super interesting, and I like the idea of verifiable artifacts that an agent can produce, i.e. notebooks for analysis, links to the source for some claims. Building for sca…

I mean, they basically did that to make sure that Composer 2.5 sits on the right side, and to a quick reader looks good

This is an interesting finding, but very specialised. It would also be great to get some more information about the benchmark. Is it just a collection of files with vulnerabilities…

It's an interesting post, but i'm a bit skeptical on their decision to report the best run for each agent, and not just the mean over the 5 runs. We have seen this as well in runni…

Just a question on the benchmark. It states that it is on real world code, but all the repos in the dataset are intentionally vulnerable repos right, not real world codebases that …

It goes a bit up and down, compared to two years ago, in my feeling, both anthropic and open ai coding models have made massive jumps. In between big releases I do feel the quality…

There definitely is the danger of a lot of garbage being shipped, but with both the models getting better and better, and more tools and ways of working being discovered, I believe…

[dead]

[flagged]