Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I program with LLMs

crawshaw.io

Post reply on HN
Print view View original

21–30 of 342 posts

Re: How I program with LLMs

#21

Post by sdesol »

User23 wrote: post #5

LLMs are, at their core, search tools. Training is indexing and prompting is querying that index. The granularity being at the n-gram rather than the document level is a huge deal though. Properly using them requires understanding that. And just like we understand every query won’t find what we want, neither will every prompt. Iterative refinement is virtually required for nontrivial cases. Automating that process, l…

> LLMs are, at their core, search tools.

This is the wrong take. Search tools are deterministic unless you purposely inject random weights into the ranking. With search tools, the same search query will always yield the same search result, provided they are designed too and/or the underlying data has not changed.

With LLMs, I can ask the exact same question and get a different response, even if the data has not changed.

Re: How I program with LLMs

#23

Post by kamaal »

mlepath wrote: post #2

The first rule of programming with LLMs is don't use them for anything you don't know how to do. If you can look at the solution and immediately know what's wrong with it, they are a time saver otherwise... I find chat for search is really helpful (as the article states)

>>f you can look at the solution and immediately know what's wrong with it, they are a time saver otherwise...

Indeed getting good at writing code using LLMs demands being very good at reading code.

To that extent its more like blitz chess than autocomplete. You need to think and verify in trees as it goes.

Re: How I program with LLMs

#24

Post by billmcneale »

mlepath wrote: post #2

The first rule of programming with LLMs is don't use them for anything you don't know how to do. If you can look at the solution and immediately know what's wrong with it, they are a time saver otherwise... I find chat for search is really helpful (as the article states)

That's the wrong approach.

I use chat for things I don't know how to do all the time. I might not know how to do it, but I sure know how to test that what I'm being told is correct. And as long as it's not, I iterate with the chat bot.

Re: How I program with LLMs

#25

Post by LeftHandPath »

bangaladore wrote: post #19

The killer feature about LLMs with programming in my opinion is autocomplete (the simple copilot feature). I can probably be 2-3x more productive as I'm not typing (or thinking much). It does a fairly good job pulling in nearby context to help it. And that's even without a language server. Using it to generate blocks of code in a chat like manner in my opinion just never works well enough in the domains I use it on.…

I’ve never used it, simply because I hate autocomplete in emails.

Gmail autocomplete saves me maybe 2-5s per email: the recipients name, a comma, and a sign off. Maybe a quarter or half sentence here or there, but never exactly what I would’ve typed.

In code bases, I’ve never seen the appeal. It’s only reliably good at stuff that I can easily find on Google. The savings are inconsequential at best, and negative at worst when it introduces hard-to-pinpoint bugs.

LLMS are incredible technology, but when applied to code, they act more like non-deterministic macros.

Re: How I program with LLMs

#26

Post by Scene_Cast2 »

sdesol wrote: post #21
User23 wrote: post #5

LLMs are, at their core, search tools. Training is indexing and prompting is querying that index. The granularity being at the n-gram rather than the document level is a huge deal though. Properly using them requires understanding that. And just like we understand every query won’t find what we want, neither will every prompt. Iterative refinement is virtually required for nontrivial cases. Automating that process, l…

> LLMs are, at their core, search tools. This is the wrong take. Search tools are deterministic unless you purposely inject random weights into the ranking. With search tools, the same search query will always yield the same search result, provided they are designed too and/or the underlying data has not changed. With LLMs, I can ask the exact same question and get a different response, even if the data has not chang…

The randomness comes from sampling. With local LLMs, you can fix the random seed, or even disable sampling all together - both will get you determinism.

I agree that LLMs are not search tools, but for very different reasons.

Re: How I program with LLMs

#27

Post by wdutch »

I no longer work in tech, but I still write simple applications to make my work life easier.

I frequently use what OP refers to as chat-driven programming, and I find it incredibly useful. My process starts by explaining a minimum viable product to the chat, which then generates the code for me. Sometimes, the code requires a bit of manual tweaking, but it’s usually a solid starting point. From there, I describe each new feature I want to add—often pasting in specific functions for the chat to modify or expand.

This approach significantly boosts what I can get done in one coding session. I can take an idea and turn it into something functional on the same day. It allows me to quickly test all my ideas, and if one doesn’t help as expected, I haven’t wasted much time or effort.

The biggest downside, however, is the rapid accumulation of technical debt. The code can get messy quickly. There's often a lot of redundancy and after a few iterations it can be quite daunting to modify.

Re: How I program with LLMs

#28

Post by nemothekid »

I think "Chat driven programming" is the most common type of the most hyped LLM-based programming I see on twitter that I just can't relate to. I've incorporated LLMs mainly as auto-complete and search; asking ChatGPT to write a quick script or to scaffold some code for which the documentation is too esoteric to parse.

But having the LLM do things for me, I frequently run into issues where it feels like I'm wasting my time with an intern. "Chat-based LLMs do best with exam-style questions" really speaks to me, however I find that constructing my prompts in such a way where the LLM does what I want uses just as much brainpower as just programming the thing my self.

I do find ChatGPT (o1 especially) really good at optimizing existing code.

Re: How I program with LLMs

#29

Post by j45 »

wdutch wrote: post #27

I no longer work in tech, but I still write simple applications to make my work life easier. I frequently use what OP refers to as chat-driven programming, and I find it incredibly useful. My process starts by explaining a minimum viable product to the chat, which then generates the code for me. Sometimes, the code requires a bit of manual tweaking, but it’s usually a solid starting point. From there, I describe each…

Is there a model you prefer to use?

Re: How I program with LLMs

#30

Post by j45 »

mlepath wrote: post #2

The first rule of programming with LLMs is don't use them for anything you don't know how to do. If you can look at the solution and immediately know what's wrong with it, they are a time saver otherwise... I find chat for search is really helpful (as the article states)

You can ask the LLM to teach it to you step by step, and then you can validate it by doing it as well as you go, still quicker than learning it and not knowing how to debug it.

Learning how something works is critical or it's far worse than technical debt.

Post reply on HN