Live data from Hacker News

Ask HN: Anyone struggling to get value out of coding LLMs?

news.ycombinator.com

111–120 of 293 posts

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#111
post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

I think a big part of the problem is the devx / workflows that these tools tend to encourage or impose. Watching my coworkers fumble with terrible copilot autoshit in a pairing session makes me want to scream on a daily basis. That doesn’t necessarily mean such a DX is the only way it could or should work, or that it’s even a good DX at all. Most of the time, it actually imposes more cost, both in lost momentum and in corrective action, than it relieves.

I think something like a graph where nodes are specified clearly with inputs and outputs (or pre- and post-conditions, if you prefer), and agents iteratively generate and refine the implementation details could have some utility, but that would require programmers to know how to think about program design (and have a usable UI/UX for such a tool) instead of hastily stumbling through patches and modifications to existing code they already don’t understand.

The new LLM systems are carrying us farther from that world, not closer to it.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#112
post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

This is a quality vs quantity issue. There were x number of extremely competent and ~10x moderately competent programmers. LLMs have only made this to 100x moderately competent programmers and may be 2xed extremely competent ones?

Even 2x would be an improvement. It would mean that a project goes from being 3 years late, to only 1.5 years late.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#113
post #44

Earlier quoted context omitted.

> So I ask ChatGPT and one of the things it said is that historically it meant "measure" This is a great example of why using LLMs to learn is perilous. This is nonsense.

Is it? I didn't bother to check as it wasn't too relevant. I wasn't stuck anymore. Worst-case it's a memnonic that is untrue. Best case: it is. For actual derivations I check the work. This is what it wrote by the way. I posted an image and asked: Why is this m m for angle mangle? ChatGPT: In geometry we often need to distinguish the angle itself (a geometric object) from the numerical size of that angle (a real-numb…

ChatGPT is right, although I'm not sure how historical the notation is.

∠ is traditionally a function from points to axiomatic geometric objects. ∠ABC is the angle at B oriented so that we start at A, go to B, then to C.

Your text seems to be using ∠ either as a kind of type annotation (indicating by ∠B that B is an angle) or (perhaps more likely) is just suppressing the other letters in the triangle and is short for something like ∠ABC.

Since ∠B is an axiomatic Euclidean object, it has no particular relation to the real numbers. m is an operator or function that maps axiomatic angles to real numbers in such a way that the calculations with real numbers provide a model for the Euclidean geometry. Why call it m? I'm not aware of it being historical, but almost certainly it comes from measure, like the μ in measure theory.

Obviously ∠ is a graphical depiction of an angle, and my guess is it probably evolved as a shorthand from the more explicit diagrams in Euclid.

Traditionally angles are named with variables from the beginning of the Greek alphabet: α, β, γ. Then we skip to θ presumably to avoid the Greek letters that look nearly identical to Roman letters.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#114
I think of LLMs as knowing a lot of things but as being relatively shallow in their knowledge.

I find them to be super useful for things that I don't already know how to do, e.g. a framework or library that I'm not familiar with. It can then give me approximate code that I will probably need to modify a fair bit, but that I can use as the basis for my work. Having an LLM code a preliminary solution is often more efficient than jumping to reading the docs immediately. I do usually need to read the docs, but by the time I look at them, I already know what I need to look up and have a feasible approach in my head.

If I know exactly how I would build something, an LLM isn't as useful, although I will admit that sometimes an LLM will come up with a clever algorithm that I wouldn't have thought up on my own.

I think that, for everyone who has been an engineer for some time, we already have a way that we write code, and LLMs are a departure. I find that I need to force myself to try them for a variety of different tasks. Over time, I understand them better and become better at integrating them into my workflows.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#116
I'm getting real valuable work done with aider and Gemini. But it's not fun and it's not flow-state kind of work.

Aider, in my humble opinion, has some issues with its loop. It sometimes works much better just to head over to AI studio and copy and paste. Sometimes it feels like aider tries to get things done as cheaply as possible, and the AI ends up making the same mistakes over again instead of asking for more information or more context.

But it is a tool and I view it as my job to get used to the limitations and strengths of the tool. So I see my role as adapting to a useful but quirky coworker so I can focus my energy where I'm most useful.

It may help that I'm a parent of intelligent and curious little kids. So I'm used to working with smart people who aren't very experienced and I'm patient about the long term payoff of working at their level.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#117
I think you and your friends actually agree on the application of LLM's, and I agree with that too.

So the question really comes down to what kind of project you are developing:

Get an MVP fast? LLM is great!

Proof of Concept? LLM rules!

Big/Complex project? LLM junior developer is not up to the task.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#118
post #13

There are two kinds of engineers. Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before. And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best. The only thing I don’t understand is why people from the former gro…

I find LLMs 100x more productive for greenfield work. If I want to create a React app with X amount of pages, some Redux stores, Auth, etc. then it can smash that out in minutes. I can say "now add X" and it'll do it. Generally with good results. But when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details, a LLM is usually not that great for me. T…

> I find LLMs 100x more productive for greenfield work.

Greenfield != boilerplate and basic CRUD app.

I'm a consultant writing greenfield apps solo, and 90% of my time is spent away from my editor thinking, planning, designing, meeting with stakeholders. I see no benefit in using a low-IQ autocomplete tool to automate a small part of the remaining 10% of the job, the easiest and most enjoyable part in fact.

Personally I find most of coding I do is unsuitable for LLMs anyway, because I don't need them to regurgitate standard logic when libraries are available, so most of that 10% is writing business logic tailored for the program/client.

Call me elitist (I don't care) but LLMs are mostly useful to two kinds of people: inexperienced developers, and those that think that hard problems are solved with more code. After almost two decades writing software, I find I need fewer and fewer code to ship a new project, most of my worth is thinking hard away from a keyboard. I really don't see the use of a machine that egregiously and happily writes a ton of code. Less is more, and I appreciate programming-as-an-art rather than being a code monkey paid by the line of code I commit.

Disclaimer: I am anti-LLM by choice so my bias is opposite than most of HN.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#119
post #76

Earlier quoted context omitted.

I don't agree. It may be true that most code is throwaway. But you trust a C compiler, or a Python intepreter, to do their job in a deterministic way. You will never be able to trust Copilot telling you that "this should be the code you are using". It may suggest you using AWS, or Google, or Microsoft, or Tencent infrastructure. An LLM can even push you a specific style, or political agenda, without even you realizin…

I think you probably need a multimodal LLM to align a table with a monospace font. Blind human programmers using screenreaders will also have difficulty with that particular task. It doesn't mean they're bad at programming; it just means they can't see.

Human blind programmers may write programs to align their own tables like requested, in autonomy. Instead of me seeing them doing wrong, asking for a fix, and them doing it wrong again in an endless cycle. Also, when I tell them the solution and ask to add a feature - bam, the table is made wrong again.

Re: Ask HN: Anyone struggling to get value out of coding LLMs?

#120

Earlier quoted context omitted.

Is it? I didn't bother to check as it wasn't too relevant. I wasn't stuck anymore. Worst-case it's a memnonic that is untrue. Best case: it is. For actual derivations I check the work. This is what it wrote by the way. I posted an image and asked: Why is this m m for angle mangle? ChatGPT: In geometry we often need to distinguish the angle itself (a geometric object) from the numerical size of that angle (a real-numb…

ChatGPT is right, although I'm not sure how historical the notation is. ∠ is traditionally a function from points to axiomatic geometric objects. ∠ABC is the angle at B oriented so that we start at A, go to B, then to C. Your text seems to be using ∠ either as a kind of type annotation (indicating by ∠B that B is an angle) or (perhaps more likely) is just suppressing the other letters in the triangle and is short for…

"I'm not sure how historical the notation is."

I conflated this with another ChatGPT conversation where it gave 3 possible historical sources for another symbol that I fell over and then had trouble proceeding.

Post reply on HN