Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I program with LLMs

crawshaw.io

Post reply on HN
Print view View original

91–100 of 342 posts

Re: How I program with LLMs

#91

Post by CraigJPerry »

dewitt wrote: post #52

One interesting bit of context is that the author of this post is a legit world-class software engineer already (though probably too modest to admit it). Former staff engineer at Google and co-founder / CTO of Tailscale. He doesn't need LLMs. That he says LLMs make him more productive at all as a hands-on developer, especially around first drafts on a new idea, means a lot to me personally. His post reminds me of an…

>> where all you wrote was function signatures and high-level control flow, and maybe some conformance tests around them

AIUI that’s where idris is headed

Re: How I program with LLMs

#92

Post by throwup238 »

singpolyma3 wrote: post #42

It seems like everything I see about success using LLMs for this kind of work is for greenfield. What about three weeks later when the job changes to maintenance and interation on something that's already working? Are people applying LLMs to that space?

My codebase is relatively greenfield (started working on it early last year) but it’s up to ~50k lines in a mixed C++/Rust codebase with a binding layer whose API predates every LLM’s training sets. Even when I started ChatGPT/Claude weren’t very useful but now the project requires a completely different strategy when working with LLMs (it’s a QT AI desktop app so I’m dogfooding a lot). I’ve also used them in a larger codebase (~500k lines) and that also requires a different approach from the former. It feels a lot like the transition from managing 2 to 20 to 200 to 2000 people. It’s a different ballgame with each step change. A very well encapsulated code base of ~500k lines is manageable for small changes but not for refactoring, exploration, etc, at least until useful context sizes increase another order of magnitude (I keep trying Gemini’s 2M but it’s been a disappointment).

I have a lot of documentation aimed at the AI in `docs/notes/` (some of it written by an LLM but proofread before committing) and I instruct Cursor/Windsurf/Aider via their respective rules/config files to look at the documentation before doing anything. At some scale that initial context becomes just a directory listing & short description of everything in the notes folder, which eventually breaks down due to context size limits, either because I exceed the maximum length of the rules or the agent requires pulling in too much context for the change.

I’ve found that there’s actually an uncanny valley between greenfield projects where the model is free to make whatever assumptions it wants and brownfield projects where it’s possible to provide enough context from the existing codebase to get both API accuracy (hallucinations) and general patterns through few-shot examples. This became very obvious once I had enough examples of that binding layer. Even though I could include all of the documentation for the library, it didn’t work consistently until I had a variety of production examples to point it to.

Right now, I probably spend as much time writing each prompt as I do massaging the notes folder and rules every time I notice the model doing something wrong.

Re: How I program with LLMs

#93

Post by Guthur »

sunaookami wrote: post #85
Guthur wrote: post #81

Earlier quoted context omitted.

Says who? I've been in the industry for nearly 25 years and have heard auto complete throughout but not once have I heard fill in the middle. Stop taking these blogs as oracle's of truth, they are not. These AI articles are full of this nonsense, to the point where it would appear to me many responses might just be Nvidia bots or whatever.

>I've been in the industry for nearly 25 years and have heard auto complete throughout but not once have I heard fill in the middle Then you need to look harder. FiM is a common approach for code generation LLMs. https://openai.com/index/efficient-training-of-language-mode... https://arxiv.org/abs/2207.14255 This was before ChatGPT's release btw.

Why, what was wrong with code completion, it was perfectly valid before even when including some sort of fuzzing.

It's like everything to do with LLM marketing buzzword nonsense.

I really want to just drop out of tech until all this obnoxious hype BS is gone.

Re: How I program with LLMs

#94

Post by staticautomatic »

mlepath wrote: post #2

The first rule of programming with LLMs is don't use them for anything you don't know how to do. If you can look at the solution and immediately know what's wrong with it, they are a time saver otherwise... I find chat for search is really helpful (as the article states)

My experience is the opposite. I find them most valuable for helping me do things that would be extremely hard or impossible for me to figure out. To wit, I just used one to decode a pagination cursor format and write a function that takes a datetime and generates a valid cursor. Ain’t nobody got time for that.

Re: How I program with LLMs

#95

Post by greenyouse »

dewitt wrote: post #52

One interesting bit of context is that the author of this post is a legit world-class software engineer already (though probably too modest to admit it). Former staff engineer at Google and co-founder / CTO of Tailscale. He doesn't need LLMs. That he says LLMs make him more productive at all as a hands-on developer, especially around first drafts on a new idea, means a lot to me personally. His post reminds me of an…

That approach sounds similar to the Idris programming language with Type Driven Development. It starts by planning out the program structure with types and function signatures. Then the function implementation (aka holes) can be filled in after the function signatures and types are set.

I feel like this is a great approach for LLM assisted programming because things like types, function signatures, pre/post conditions, etc. give more clarity and guidance to the LLM. The more constraints that the LLM has to operate under, the less likely it is to get off track and be inconsistent.

I've taken a shot at doing some little projects for fun with this style of programming in TypeScript and it works pretty well. The programs are written in layers with the domain design, types, schema, and function contracts being figured out first (optionally with some LLM help). Then the function implementations can be figured out towards the end.

It might be fun to try Effect-TS for ADTs + contracts + compile time type validation. It seems like that locks down a lot of the details so it might be good for LLMs. It's fun to play around with different techniques and see what works!

Re: How I program with LLMs

#96

Post by vendiddy »

Ozzie_osman wrote: post #71

One mode I felt was missed was "thought partner", especially while debugging (aka rubber ducking). We had an issue recently with a task queue seemingly randomly stalling. We were able to arrive at the root cause much more quickly than we would have because of a back-and-forth brainstorming session with Claude, which involved describing the issue we were seeing, pasting in code from library to ask questions, asking it…

I found myself doing this with o1 recently for software architecture.

I will evaluate design ideas with the model, express concerns on trade-offs, ask for alternative ideas, etc.

Some of the benefit is having someone to talk to, but with proper framing it is surprisingly good at giving balanced takes.

Re: How I program with LLMs

#97

Post by brabel »

dewitt wrote: post #52

One interesting bit of context is that the author of this post is a legit world-class software engineer already (though probably too modest to admit it). Former staff engineer at Google and co-founder / CTO of Tailscale. He doesn't need LLMs. That he says LLMs make him more productive at all as a hands-on developer, especially around first drafts on a new idea, means a lot to me personally. His post reminds me of an…

I am not a genius but have a couple of decades experience and finally started using LLMs in anger in the last few weeks. I have to admit that when my free quota from GitHub Copilot ran out (I had already run out of Jetbrains AI as well!! Our company will start paying for some service as the trials have been very successful), I had a slight bad feeling as my experience was very similar to OP: it's really useful to get me started, and I can finish it much more easily from what the AI gives me than if I started from scratch. Sometimes it just fills in boilerplate, other times it actually tells me which functions to call on an unfamiliar API. And it turns out it's really good at generating tests, so it makes my testing more comprehensive as it's so much faster to just write them out (and refine a bit usually by hand). The chat almost completely replaced my StackOverflow queries, which saves me much time and anxiety (God forbid I have to ask something on SO as that's a time sink: if I just quickly type out something I am just asking to be obliterated by the "helpful" SO moderators... with the AI, I just barely type anything at all, leave it with typos and all, the AI still gets me!).

Re: How I program with LLMs

#98

Post by brabel »

What the author is asking about, a quick sketchpad where you can try out code quickly and chat with the AI, already exists in the JetBrains IDEs. It's called a scratch file[1].

As far as I know, the idea of a scratch "buffer" comes from emacs. But in Jetbrains IDEs, you have the full IDE support even with context from your current project (you can pick the "modules" you want to have in context). Given the good integration with LLMs, that's basically what the author seems to want. Perhaps give GoLand[2] a try.

Disclosure: no, I don't work for Jetbrains :D just a very happy customer.

[1] https://www.jetbrains.com/help/idea/scratches.html

[2] https://www.jetbrains.com/go/

Re: How I program with LLMs

#99

Post by [deleted] »

Guthur wrote: post #81
yawnxyz wrote: post #80

> I could not go a week without getting frustrated by how much mundane typing I had to do before having a FIM model For those not in-the-know, I just learned today that code autocomplete is actually called "Fill-in-the-Middle" tasks

Says who? I've been in the industry for nearly 25 years and have heard auto complete throughout but not once have I heard fill in the middle. Stop taking these blogs as oracle's of truth, they are not. These AI articles are full of this nonsense, to the point where it would appear to me many responses might just be Nvidia bots or whatever.

[deleted]

Re: How I program with LLMs

#100

Post by ascorbic »

Guthur wrote: post #93
sunaookami wrote: post #85

Earlier quoted context omitted.

>I've been in the industry for nearly 25 years and have heard auto complete throughout but not once have I heard fill in the middle Then you need to look harder. FiM is a common approach for code generation LLMs. https://openai.com/index/efficient-training-of-language-mode... https://arxiv.org/abs/2207.14255 This was before ChatGPT's release btw.

Why, what was wrong with code completion, it was perfectly valid before even when including some sort of fuzzing. It's like everything to do with LLM marketing buzzword nonsense. I really want to just drop out of tech until all this obnoxious hype BS is gone.

Autocomplete is the feature, fill in the middle is one approach to implementing it. There are other ways to providing it (which were used in earlier versions of Copilot) and FIM can be used for tasks other than code completion.
Post reply on HN