Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Automating myself out of development

thoughtfultechnologist.com

Post reply on HN
Print view View original

41–50 of 97 posts

Re: Automating myself out of development

#41

Post by firegodjr »

philbo wrote: post #27
gnunicorn wrote: post #16

Interestingly, despite it being much more detailed and a lot more process and procedure than what I currently do - which is more akin to the version 0 described, but in parallel - we come up at the same final problem: reviews and quality assurance. I sign off the code I merged, part of company policy but also just to be sure it is actually decent. But reviewing has become the real draining bottleneck: even stacked PR…

For decades, engineers understood that large code reviews are harder than small ones. Out of both politeness and a desire to receive better code reviews, we learned to break our large changes into smaller chunks. Some engineers took things even further and replaced code reviews with pair programming. But then LLMs showed up and everyone seems to have forgotten those lessons. They can be still be applied now using cod…

I think that's reasonable. My only gripe is that making small sets of changes is often faster to do by hand than waiting on llm reasoning, so I've found it amounts to very little speedup.

Re: Automating myself out of development

#42

Post by Swizec »

germanptr wrote: post #9
noelwelsh wrote: post #2

I wish people would describe in more detail the tasks they use LLMs to code. My experience is that simple components in an existing architecture are fine, but anything requiring architectural considerations quickly becomes a mess. On my projects (e.g. a ui framework), running multiple agents in parallel would just increase the speed at which it can stuff up the project.

I get this question a lot, and I found it hard to answer briefly, so I ended up writing a longer post about how I work: https://www.trigosec.com/insights/mob-programming-for-one/ The short version is that I don’t let AI agents work unsupervised on my code. I treat them like participants in a mob programming session instead of autonomous developers. Different agents get different roles (implementer, reviewer, architec…

> We’re still missing a good way to express and measure architectural quality

Architectural complexity[1]! There’s several really good papers on this.

Unfortunately it never caught on and we don’t have great automated tools to spit out a number. Also the majority of people just don’t care enough. Research in this field kinda died out when we invented microservices and started treating those as a silver bullet to The Architecture Problem (it’s not [2])

[1] https://swizec.com/blog/why-taming-architectural-complexity-...

[2] https://youtu.be/y8OnoxKotPQ

Re: Automating myself out of development

#43

Post by amelius »

noelwelsh wrote: post #2

I wish people would describe in more detail the tasks they use LLMs to code. My experience is that simple components in an existing architecture are fine, but anything requiring architectural considerations quickly becomes a mess. On my projects (e.g. a ui framework), running multiple agents in parallel would just increase the speed at which it can stuff up the project.

I personally limit LLMs to single files only at the moment. Self-contained components.

Using LLMs in a larger scope can sometimes work, but it has the real risk of turning a project into a mess after which you will have to undo the work and lose a lot of time.

Also, using LLMs this way with less clear boundaries will make reading and maintaining the code more cumbersome.

Re: Automating myself out of development

#44

Post by abletonlive »

general1465 wrote: post #23

I am completely calm regarding AI and development. First nobody sane want to give their domain IP to OpenAI/Anthropic. That's why local AI will eventually prevail and flourish because people who actually have some IP will have no problem to buy 10k+ EUR machine to run some pretty good models on it. However if your main job is just doing CRUD stuff, then you are screwed. Secondly hallucination is really Achilles heel…

> All of processor above have reference manuals and sometimes example git repositories available on open internet.

okay? then give those reference manuals and git repositories? I haven't heard something know LLMs can't get around and figure out?

Re: Automating myself out of development

#45

Post by leguy »

properbrew wrote: post #5
noelwelsh wrote: post #2

I wish people would describe in more detail the tasks they use LLMs to code. My experience is that simple components in an existing architecture are fine, but anything requiring architectural considerations quickly becomes a mess. On my projects (e.g. a ui framework), running multiple agents in parallel would just increase the speed at which it can stuff up the project.

I used LLMs to develop Whistle Enterprise ( https://whistle-enterprise.com ) from the ground up, from scratch. It's taken _a lot_ of time and effort, but this is an example of what can be developed using LLMs alone. You have to have dedication and a goal to reach, but you can absolutely build anything if you're building with the right foundations in mind.

neat. I saw the "no bot joins the call". Is it obvious to others in the virtual meeting that you are using this tool?

Re: Automating myself out of development

#46

Post by vslira »

germanptr wrote: post #9
noelwelsh wrote: post #2

I wish people would describe in more detail the tasks they use LLMs to code. My experience is that simple components in an existing architecture are fine, but anything requiring architectural considerations quickly becomes a mess. On my projects (e.g. a ui framework), running multiple agents in parallel would just increase the speed at which it can stuff up the project.

I get this question a lot, and I found it hard to answer briefly, so I ended up writing a longer post about how I work: https://www.trigosec.com/insights/mob-programming-for-one/ The short version is that I don’t let AI agents work unsupervised on my code. I treat them like participants in a mob programming session instead of autonomous developers. Different agents get different roles (implementer, reviewer, architec…

> The short version is that I don’t let AI agents work unsupervised on my code. I treat them like participants in a mob programming session instead of autonomous developers.

I wonder if OS maintainers would have a leg up in defining workflows to better leverage this. Of course, OS contributors are autonomous developers, but maybe a trick or two might transfer across

Re: Automating myself out of development

#47

Post by fbrchps »

Swizec wrote: post #42
germanptr wrote: post #9

Earlier quoted context omitted.

I get this question a lot, and I found it hard to answer briefly, so I ended up writing a longer post about how I work: https://www.trigosec.com/insights/mob-programming-for-one/ The short version is that I don’t let AI agents work unsupervised on my code. I treat them like participants in a mob programming session instead of autonomous developers. Different agents get different roles (implementer, reviewer, architec…

> We’re still missing a good way to express and measure architectural quality Architectural complexity[1]! There’s several really good papers on this. Unfortunately it never caught on and we don’t have great automated tools to spit out a number. Also the majority of people just don’t care enough. Research in this field kinda died out when we invented microservices and started treating those as a silver bullet to The…

Didn't even need to click the YouTube link, I knew it would be Krazaam.

Re: Automating myself out of development

#48

Post by warumdarum »

noelwelsh wrote: post #2

I wish people would describe in more detail the tasks they use LLMs to code. My experience is that simple components in an existing architecture are fine, but anything requiring architectural considerations quickly becomes a mess. On my projects (e.g. a ui framework), running multiple agents in parallel would just increase the speed at which it can stuff up the project.

The true test challenges should be how far an AI can minimize a given fucked up codebase and keep full functionality.

I also think that writting large codebases into a sort of functional transformer tree as information compression stage would allow them to easier reason about large code bases by having a large lossless overview with minimal token usage.

Re: Automating myself out of development

#49

Post by strogonoff »

minihat wrote: post #26
strogonoff wrote: post #17

Earlier quoted context omitted.

Proper review should take longer than writing it yourself, because you need to know the correct solution, understand the proposed solution, and evaluate the difference between the two. When designing it yourself, you just need to know the correct solution and write it, and with modern high-level languages and IDEs with autocomplete writing it is hardly a bottleneck.

It is harder to solve a sudoku than verify a solution's correctness. I find similar benefits occasionally when coding with LLMs.

Imagine sudoku with hundreds of subtle, sometimes mutually exclusive rules, and no single valid solution.

This is not about LLMs, by the way. It’s about reviewing any code, including by a fellow human. It’s just that many people mistakenly feel like with LLMs they can lower their guard and accept even if they have not gone through the steps of themselves coming up with their solution and comparing it to the one suggested by the LLM.

The reason is that many correctly see proper review as duplicate work, and while it is justified with another human (because it is (A) instructive and (B) reducing bus factor) with LLMs most people simply can’t be bothered. If you personally can, you are a minority.

Re: Automating myself out of development

#50

Post by properbrew »

ryanackley wrote: post #13
properbrew wrote: post #5

Earlier quoted context omitted.

I used LLMs to develop Whistle Enterprise ( https://whistle-enterprise.com ) from the ground up, from scratch. It's taken _a lot_ of time and effort, but this is an example of what can be developed using LLMs alone. You have to have dedication and a goal to reach, but you can absolutely build anything if you're building with the right foundations in mind.

I think the relevant question isn’t what can be built but the amount of effort in comparison to doing this the old fashioned way. What do you think the productivity gain was from using an LLM? This question assumes you’re already an experienced developer.

Thank you for the assumption, I'm actually not a developer at all.

I'm from a hardware / networking / infrastructure background. I've had extensive exposure to (web) application development as I'm working closely with development teams and I do have the bash/powershell scripting knowledge.

But honestly, if I tried this "the old fashioned way" it probably would have taken me about 6 to 7 years to develop that application, that's an optimistic estimate. You really do have to have a passion for what you're building, I didn't know that voice transcription and local LLMs would be such a driving force for me, but it's all I think about, so much that I find it hard to go to sleep sometimes.

Post reply on HN