171–180 of 389 posts
Earlier quoted context omitted.
>I think a lot of the problem with the current discourse is how black-and-white it is. There is too much money involved for any rational debate.
Only on the pro-AI side. The "it is bad" side is diverse on the reasons why, but being overwhelmed by bad content isn't a monetary concern.
Earlier quoted context omitted.
>It does as well as you care to coax into doing tbh. You have to invest heavily in harness engineering, and at least my experience has been that as you do that, the results improve. That is also my experience. When starting a project I observe how the agent fails, I add new rules to the harness to prevent it from falling and repeat the process until I am happy with the output.
I'm unfamiliar with harness engineering. Is there any good documentation about the subject you could point me to?
I learned by reading articles, success stories failure stories and mostly by doing, trying stuff, see how it works and adjusting it and burning a lot of tokens along the way.
What I would do in your shoes, I would ask an AI chat to find new articles on the matter (including on HN), explain how Codex, Claude, Pi are managing agents.
My compressed view is: you need to have a great specification both business and architecture wise that doesn't leave anything important for the model to guess because chances are it will make the wrong choices. That comprehensive spec should not be in one huge chunk. Have your plan divided in phases that each fit in a context window and have the spec for each phase. Use TDD, strive for 100% coverage. Force the model to behave: if it doesn't do what is supposed to, give it feedback and ask it to retry and don't allow it to progress to the next stage unless everything is perfect. I also like to write comprehensive integration tests before building anything. The agents are not allowed to touch or read the integration tests, only run them and they will get feedback where the tests fail. I like to build the integration tests in a different language than the software I am building, to make sure there isn't something platform specific that the tests rely on. I use C#, Go, Rust and Zig for development and Python for the integration tests.
For now, to get good results, I can't just copy and paste the setup from a project to another, I have to work a lot to tailor the process for each new codebase.
And that's why I am working on an agent harness to try to force the agents to do the right things in most common development scenarios without wasting much tokens. By common development scenarios I mean that is a large goal, right now I am working towards backend web development and microservices.
Earlier quoted context omitted.
We're already hitting some logistical limits. Even if transformers don't have an inherent capability plateau, we only have so many GPUs and so much power to improve them, and we're finding it very difficult to expand that infrastructure. Something like 6 GW of new DCs have been announced over the past 2 years of which less than 1 GW has actually been turned on and started serving, and the deliverable dates for the re…
Sounds like we just need a Dyson sphere. Besides, I have been hearing "this is the limit" since the doomers of "this is just a markov chain and can't be useful". Yet the limits keep being broken.
But each time I suspected I could have done it better and faster manually
I'm in the "haven't written any code in a while" boat ATM. I'd love to see examples of issues that are so big that they warrant reverting to manual coding. My main issue has been the inconsistent quality across between model releases and the tendency to insert older APIs or documentation, especially with command line tools. I can understand if the model struggles with a million line monolithic codebase with a decade…
> I'm in the "haven't written any code in a while" boat ATM How long do you think it will be before you can't write any code because you're out of practice? One of the dangers of engineering management is that it can turn you into a person that can no longer do the thing. Does that even matter?
Having said that, I won't use AI for production system if I don't understand the programming constructs in enough detail.
Earlier quoted context omitted.
>I think a lot of the problem with the current discourse is how black-and-white it is. There is too much money involved for any rational debate.
> There is too much money involved for any rational debate. For the Sam Altmans of this world, sure, but how much money is the average AI booster commenting on HN actually standing to make?
The other side is the stability of your job or job prospects, and we are adversely affected by that instead.
I'm in the "haven't written any code in a while" boat ATM. I'd love to see examples of issues that are so big that they warrant reverting to manual coding. My main issue has been the inconsistent quality across between model releases and the tendency to insert older APIs or documentation, especially with command line tools. I can understand if the model struggles with a million line monolithic codebase with a decade…
What type of projects you work on, in particular how rich it is in novelty, non-googlable data points and non-trivial project-specific deviations from industry standards?
Why sloptember when it's may