Live data from Hacker News

How I program with LLMs

crawshaw.io

271–280 of 342 posts

Re: How I program with LLMs

#271

Earlier quoted context omitted.

I have also many years of programming experience and find myself strongly "accelerated" by LLMs when writing code. But, if you think at it, it makes sense that many seasoned programmers are using LLMs better. LLMs are a helpful tool, but also a hard-to-use tool, and in general it's fair to think that better programmers can do a better use of some assistant (human or otherwise): better understanding its strengths, ide…

> "seasoned programmers are using LLMs better". I do not remember a single instance when code provided to me by an LLM worked at all. Even if I ask something small that cand be done in 4-5 lines of code is always broken. From a fellow "seasoned" programmer to another: how the hell do you write the prompts to get back correct working code?

dc: not a seasoned dev, with and tags on "not".

They can't think for you. All intelligent thinking you have to do.

First, give them high level requirement that can be clarified into indented bullet points that looks like code. Or give them such list directly. Don't give them half-open questions usually favored by talented and autonomous individuals.

Then let them further decompress that pseudocode bullet points into code. They'll give you back code that resemble a digitized paper test answer. Fix obvious errors and you get a B grade compiling code.

They can't do non-conventional structures, Quake style performance optimized codes, realtime robotics, cooperative multithreading, etc., just good old it takes what it takes GUI app API and data manipulation codes.

For those use cases with these points in mind, it's a lot faster to let LLM generate tokens than typing `int this_mandatory_function_does_obvious (obvious *obvious){ ...` manually on a keyboard. That should arguably be a productivity boost in the sense that the user of LLM is effectively typing faster.

Re: How I program with LLMs

#272

Earlier quoted context omitted.

But for every step that turns out to be "correct" you now have to go back and redo that in your held-out sample anyways. So it's not like you get to save on repeating the work -- IIUC you just changed it from depth-first execution order to breadth-first execution order.

> International Islamic University Chittagong ??? What's up with native English speakers and random acronyms of stuff that isn't said that often? YMMV, IIUC, IANAL, YSK... Just say it and save everyone else a google search.

So just to make sure I'm on the same page: you're bemoaning how commonly people abbreviate uncommon sayings?

Re: How I program with LLMs

#274
post #175

Earlier quoted context omitted.

> It won't give us the step by step scalable processes to make humanity as a whole enter in a loop of indefinitely long period of world peace That's not anywhere, that's a totally unsolved and open ended problem, why would you think an LLM would have that?

If what you meant was > Think about it: every type of already solved problem you want information about is in them, in fact it is there multiple times, with multiple levels of seriousness in the treatment of the idea. then that was not clear from your comment saying LLMs contain any information you want. One has to be careful communicating about LLms because the world is full of people that actually believe LLMs are…

I think GP's saying that it must be in your prompt, not in the weights.

If you want LLM make sandwich, you have to tell them you `want triangular sandwiches of standard serving size made with white bread and egg based filling`, not `it's almost noon and I'm wondering if sandwich for lunch is a good idea`. Fine-tuning partially solves that problem but they still like the former.

Re: How I program with LLMs

#275

Earlier quoted context omitted.

I have also many years of programming experience and find myself strongly "accelerated" by LLMs when writing code. But, if you think at it, it makes sense that many seasoned programmers are using LLMs better. LLMs are a helpful tool, but also a hard-to-use tool, and in general it's fair to think that better programmers can do a better use of some assistant (human or otherwise): better understanding its strengths, ide…

> "seasoned programmers are using LLMs better". I do not remember a single instance when code provided to me by an LLM worked at all. Even if I ask something small that cand be done in 4-5 lines of code is always broken. From a fellow "seasoned" programmer to another: how the hell do you write the prompts to get back correct working code?

I'd ask things like "which LLM are you using", and "what language or APIs are you asking it to write for".

For the standard answers of "GPT-4 or above", "claude sonnet or haiku", or models of similar power and well known languages like Python, Javascript, Java, or C and assuming no particularly niche or unheard of APIs or project contexts the failure rate of 4-5 line of code scripts in my experience is less than 1%.

Re: How I program with LLMs

#276
post #88

Earlier quoted context omitted.

> That he says LLMs make him more productive at all as a hands-on developer, especially around first drafts on a new idea, means a lot to me personally. There is likely to be a great rift in how very talented people look at sharper tools. I've seen the same division pop up with CNC machines, 3d printers, IDEs and now LLMs. If you are good at doing something, you might find the new tool's output to be sub-par over wha…

I believe it’s more that people hate trying new tools because they’ve already made their choice and made it their identity. However, there are also people who love everything new and jump onto the latest hype too. They try new things but then immediately advocate it without merit. Where are the sane people in the middle?

> Where are the sane people in the middle?

They are the quiet ones.

Re: How I program with LLMs

#277

Earlier quoted context omitted.

I have also many years of programming experience and find myself strongly "accelerated" by LLMs when writing code. But, if you think at it, it makes sense that many seasoned programmers are using LLMs better. LLMs are a helpful tool, but also a hard-to-use tool, and in general it's fair to think that better programmers can do a better use of some assistant (human or otherwise): better understanding its strengths, ide…

> "seasoned programmers are using LLMs better". I do not remember a single instance when code provided to me by an LLM worked at all. Even if I ask something small that cand be done in 4-5 lines of code is always broken. From a fellow "seasoned" programmer to another: how the hell do you write the prompts to get back correct working code?

I rarely get back not working code but I've also internalized it's limitations so I no longer ask it for things it's not going to be able to do.

As other commenters have pointed it, there also a lot of variation between different models and some are quite dumb.

I've had no issues with 10-20 line coding problems. I've also had it built a lot of complete shell scripts and had no problem there either.

Re: How I program with LLMs

#278

Earlier quoted context omitted.

But for every step that turns out to be "correct" you now have to go back and redo that in your held-out sample anyways. So it's not like you get to save on repeating the work -- IIUC you just changed it from depth-first execution order to breadth-first execution order.

> International Islamic University Chittagong ??? What's up with native English speakers and random acronyms of stuff that isn't said that often? YMMV, IIUC, IANAL, YSK... Just say it and save everyone else a google search.

I'm not a native English speaker, but IIUC is clearly 'If I Understand Correctly'. If you look at the context it's often fairly easy to figure out what an initialism means. I mean even I can usually deduce the meaning and I'm barely intelligent enough to qualify as 'sentient'.

Re: How I program with LLMs

#279
I have been getting more value out of LLMs recently, and the great irony is it is because of a few different packages in emacs and the wonderful CLI LLM chat programming tool 'aider'.

My workflow puts LLM chat at my fingertips, and I can control the context. Pretty much any text in emacs can be sent to a LLM of your choice via API.

Aider is even better, it does a bunch of tricks to improve performance, and is rapidly becoming a 'must have' benchmark for LLM coding. It integrates with git so each chat modification becomes a new git commit. Easy to undo changes, redo changes, etc. It also has a bunch of hacks because while o1 is good as reasoning, it (apparently) doesn't do code modification well. Aider will send different types of requests to different 'strengths' of LLMs etc. Although if you can use sonnet, you can just use that and be done with it.

It's pretty good, but ultimately it's still just a tool for transforming words into code. It won't help you think or understand.

I feel bad for new kids who won't develop muscle and sight strength to read/write code. Because you still need to read/write code, and can't rely on the chat interface for everything.

Re: How I program with LLMs

#280
post #10

Our company has a no AI use policy. The assumption is zero trust. We simply can’t know whether a model or its framework could or would send proprietary code outside the network. So it’s best to assume all LLMs/AI is or will send code or fragments of code. While I applaud the incredible work by their creators, I’m not sure how a responsible enterprise class company could rely on “trust us bro” EULAs or repo readmes.

The same way responsible enterprise class companies rely on "trust us bro" EULAs for financial systems, customer databases, payroll, and all the other systems it would be very expensive and error prone to build custom for every business.

Pretty much this.

OpenAI poisoned the well badly with their "we train off your chats" nonsense.

If you are using any API service, or any enterprise ChatGPT plan, your tokens are not being logged and recycled into new training data.

As for why trust them? Like the parent said: EULAs. Large companies trust EULAs and terms of service for every single SAAS product they use, and they use tons and tons of them.

OpenAI in a clumsy attempt to create a regulatory moat by doing sketchy shit and waving wild "AI will kill us all" nonsense has created a situation where the usefullness of these transforming generative solutions are automatically rejected by many.

Post reply on HN