41–50 of 110 posts
The space is too big and the code too extensive for a 1M context window and the capabilities of the current SOTA models.
Earlier quoted context omitted.
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…
The current project I am developing with AI is so bloated that I've spent more time and tokens cleaning up the repo (to the tune of tens of thousands of lines removed) than actually creating it in the first place. AI is very very happy to create a mess.
Earlier quoted context omitted.
> it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 tests. you ever worked with human engineers before? :D
I was going to say that it's pretty much exactly like working with an experienced software engineer ... except it never seems to take a step back and look at the bigger picture. It will happily fix the same class of bugs over and over without realizing a bigger change is required.
Earlier quoted context omitted.
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…
> tons of duplication that lead to whack-a-mole The future is duplication. Code is no longer meant to be read by humans but by machines.
Cuz what if a human has to intervene as the LLM can’t get it right?
Assuming a human then has till be able to comprehend it?
All models in the leaderboard probably have had access to the original source code in their training data.
> without access to the original source code All models in the leaderboard probably have had access to the original source code in their training data.
I have been using Claude to build a clone of Bash in Rust ( https://github.com/jdstanhope/huck ). It has been goin going on now for 81 days, 2600 commits and I have covered a good chunk of the features such that I can source by .bashrc with a number of extensions. It has been doing pretty good but it will sometimes spend hours to do few line fix and other times pushback on making big changes despite have over 3000 te…
Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that i…
[0]: https://medium.com/@vishvananda/i-spent-2-billion-tokens-wri...
I'm curious about that too. I'm building a language with AI, and when I leave the AI running on its own, it keeps generating code in strange directions. I wonder what kind of structure AI-only code would have.
In the linked paper the researchers state "AI solutions must match the original program’s output exactly on end-to-end tests, including held-out tests. " And a bit further on "The AI agent is given execute-only access to the original program and a set of visible test cases, but does not have access to the original source code. "
At a guess, their agents are not going off into the weeds (like mine do occasionally) is because they have both access to a reference runtime implementation to use as a definitive end to end test to guide development. And also a relatively large token budget to run enough characterisation tests to allow them to home in on solutions.