Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How I program with LLMs

crawshaw.io

Post reply on HN
Print view View original

241–250 of 342 posts

Re: How I program with LLMs

#241

Post by switchbak »

LeftHandPath wrote: post #25
bangaladore wrote: post #19

The killer feature about LLMs with programming in my opinion is autocomplete (the simple copilot feature). I can probably be 2-3x more productive as I'm not typing (or thinking much). It does a fairly good job pulling in nearby context to help it. And that's even without a language server. Using it to generate blocks of code in a chat like manner in my opinion just never works well enough in the domains I use it on.…

I’ve never used it, simply because I hate autocomplete in emails. Gmail autocomplete saves me maybe 2-5s per email: the recipients name, a comma, and a sign off. Maybe a quarter or half sentence here or there, but never exactly what I would’ve typed. In code bases, I’ve never seen the appeal. It’s only reliably good at stuff that I can easily find on Google. The savings are inconsequential at best, and negative at wo…

"negative at worst when it introduces hard-to-pinpoint bugs" - this is actually very true. I've had it recreate patterns _partially_, and paste in the wrong thing in a place that was very hard to discern.

It probably saved me 40 mins, then proceeded to waste 2 hours of me hunting for that issue. I'm probably at the break-even on the whole. The ultimate promise is very compelling, but my current use isn't particularly amazing. I do use a niche language though, so I'm outside the global optima.

Re: How I program with LLMs

#242

Post by golergka »

cudgy wrote: post #206
golergka wrote: post #108

Earlier quoted context omitted.

What's the realistic attack scenario? Will Sam Altman steal your company's code? Or will next version of GPT learn on your secret sauce algorithms and then your competitors will get them when they generate code for their tasks and your company loses its competitive advantage? I'm actually sure that there are companies for which these scenarios are very real. But I don't think there's a lot of them. Most of the code o…

So why bother securing anything at all if not willing to secure the raisons d'être? Doesn’t that suggest that these companies are trivial entities?

There are plenty of very realistic attack scenarios, that's why we secure stuff.

Re: How I program with LLMs

#243

Post by mahmoudimus »

dewitt wrote: post #52

One interesting bit of context is that the author of this post is a legit world-class software engineer already (though probably too modest to admit it). Former staff engineer at Google and co-founder / CTO of Tailscale. He doesn't need LLMs. That he says LLMs make him more productive at all as a hands-on developer, especially around first drafts on a new idea, means a lot to me personally. His post reminds me of an…

Isn't that the idea behind UML? Which didn't work out so well, however, with the advent of LLMs today, I think that premise could work.

Re: How I program with LLMs

#244

Post by rudiksz »

antirez wrote: post #120
dewitt wrote: post #52

One interesting bit of context is that the author of this post is a legit world-class software engineer already (though probably too modest to admit it). Former staff engineer at Google and co-founder / CTO of Tailscale. He doesn't need LLMs. That he says LLMs make him more productive at all as a hands-on developer, especially around first drafts on a new idea, means a lot to me personally. His post reminds me of an…

I have also many years of programming experience and find myself strongly "accelerated" by LLMs when writing code. But, if you think at it, it makes sense that many seasoned programmers are using LLMs better. LLMs are a helpful tool, but also a hard-to-use tool, and in general it's fair to think that better programmers can do a better use of some assistant (human or otherwise): better understanding its strengths, ide…

> "seasoned programmers are using LLMs better".

I do not remember a single instance when code provided to me by an LLM worked at all. Even if I ask something small that cand be done in 4-5 lines of code is always broken.

From a fellow "seasoned" programmer to another: how the hell do you write the prompts to get back correct working code?

Re: How I program with LLMs

#246

Post by highfrequency »

> A lot of the value I personally get out of chat-driven programming is I reach a point in the day when I know what needs to be written, I can describe it, but I don’t have the energy to create a new file, start typing, then start looking up the libraries I need... LLMs perform that service for me in programming. They give me a first draft, with some good ideas, with several of the dependencies I need, and often some mistakes. Often, I find fixing those mistakes is a lot easier than starting from scratch.

This to me is the biggest advantage of LLMs. They dramatically reduce the activation energy of doing something you are unfamiliar with. Much in the way that you're a lot more likely to try kitesurfing if you are at the beach standing next to a kitesurfing instructor.

While LLMs may not yet have human-level depth, it's clear that they already have vastly superhuman breadth. You can argue about the current level of expertise (does it have undergrad knowledge in every field? PhD level knowledge in every field?) but you can't argue about the breadth of fields, nor that the level of expertise improves every year.

My guess is that the programmers who find LLMs useful are people who do a lot of different kinds of programming every week (and thus are constantly going from incompetent to competent in things that other people already know), rather than domain experts who do the same kind of narrow and specialized work every day.

Re: How I program with LLMs

#247

Post by jpc0 »

efnx wrote: post #226
davepeck wrote: post #221

Earlier quoted context omitted.

I see less "painting as a luddite" in response to statements like this, and more... surprise. Mild skepticism, perhaps! Your experience diverges from that of other experienced devs who have used the same tools, on probably similar projects, and reached different conclusions. That includes me, for what it's worth. I'm a graybeard whose current work is primarily cloud data pipelines that end in fullstack web. Like most…

I think it also depends on _what_ the domain is, and also to a certain degree the tools / stack you use. LLMs aren’t coherent or correct when working on novel problems, novel domains or using novel tools. They’re great for doing something that has been done before, but their hallucinations are wildly incorrect when novelty is at play - and I’ll add they’re always very authoritative! I’m glad my languages of choice ha…

My recent example for where its helpful.

Pretty nice at autocomplete. Like writing json tags in go structs. Can just autocomplete that's stuff for me no problem, it saved me seconds per line, seconds I tell you.

It's stupid as well... Autofilled a function, looks correct. Reread it 10 minutes later and well... Minor mistake that would have caused a crash at runtime. It looked correct but in reality it just didn't have enough context ( the context is in an external doc on my second screen ... ) and there was no way it would ever have guessed the correct code.

It took me longer to figure out why the code looked wrong than if I had just typed it myself.

Did it speed up my workflow on code I could have given a junior to write? Not really, but some parts were quicker while other were slower.

And imagine if that code bad crashed in production next week instead of right now while the whole context is still in my head. Maybe that would be hours of debugging time...

Maybe as parent said, for a domain where you are braking new ground, it can generate some interesting ideas you wouldn't have thought about. Like a stupid pair that can get you out if a local manima but in general doesn't help much it can be a significant help.

But then again you could do what has been done for decades and speak to another human about the problem, at least they may have signed the same NDA as you...

Re: How I program with LLMs

#248

Post by twelve40 »

holoduke wrote: post #232
dns_snek wrote: post #214

Earlier quoted context omitted.

As an experienced software developer, I paid for ChatGPT for a couple of months, I trialed Gemini Pro for a couple of months, and I've used the current version of Claude. I'd be happy if LLMs could produce working code as often and as quickly as the evangelist claim, but whenever I try to use LLM to work on my day to day tasks, I almost always walk away frustrated and disappointed - and most of my work is boring on t…

Yesterday i wanted to understand what a team was doing in a go project. I have never really touched go before. I do understand software, because I develop for plus 20 years. But chatgpt was perfectly able to give me a summary on how the implementation worked. Gave me examples and suggestions. And within a day fulltime pasting code and asking question i had a good understanding of the codebase. It would have be a lot…

how often do you get to learn an unfamiliar language? is it something you need to do every day? so this use case, did it save you much time overall?

Re: How I program with LLMs

#249

Post by jpc0 »

jonvk wrote: post #213
Vox_Leone wrote: post #166

Earlier quoted context omitted.

I have been using LLM to generate functional code from *pseudo-code* with excellent results. I am starting to experiment with UML diagrams, both with LLM and computer vision to actually generate code from UML diagrams; for example a simple activity diagram could be the prompt on LLM 's, and might look like: Start -> Enter Credentials -> Validate -> [Valid] -> Welcome Message -> [Invalid] -> Error Message Correspondin…

This example illustrates one of the risks of using LLMs without subject expertise though. I just tested this with claude and got that exact same validation method back. Using string comparison is dangerous from a security perspective [1], so this is essentially unsafe validation, and there was no warning in the response about this. 1. https://sqreen.github.io/DevelopersSecurityBestPractices/tim...

Are you talking about the timing based attacks on that website which fails miserably at rendering a useable page on mobile?

Re: How I program with LLMs

#250

Post by Vampiero »

ssivark wrote: post #225
travisporter wrote: post #224

Earlier quoted context omitted.

> I got into this profession simply because I could Ctrl-Z to the previous step much more easily than my then favourite chemical engineering goals. That is interesting. Asking as a complete ignoramus - is there not a way to do this now? Like start off with a 100 of reagent and at every step use a bit and discard if wrong

But for every step that turns out to be "correct" you now have to go back and redo that in your held-out sample anyways. So it's not like you get to save on repeating the work -- IIUC you just changed it from depth-first execution order to breadth-first execution order.

> International Islamic University Chittagong

??? What's up with native English speakers and random acronyms of stuff that isn't said that often? YMMV, IIUC, IANAL, YSK... Just say it and save everyone else a google search.

Post reply on HN