Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I trusted an LLM, now I'm on day 4 of an afternoon project

nemo.foo

Post reply on HN
Print view View original

61–70 of 215 posts

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#61

Post by dambi0 »

cmdtab wrote: post #20

Today, I needed to write a proxy[0] that wraps an object and log all method calls recursively. I asked claude to write the initial version. It came up with a complicated class based solution. I spent more than 30 minutes getting a good abstract to come out. I was copy pasting typescript errors and applying fixes it suggested without thinking much. In the end, I gave up and wrote what I wanted myself in 5 minutes. 0]…

Would you have written it in five minutes has you not just spent 30 minutes ruling out wrong solutions?

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#62

Post by mordymoop »

bigstrat2003 wrote: post #51
mordymoop wrote: post #41

I am frankly tired of seeing this kind of post on HN. I feel like the population of programmers is bifurcating into those who are committed to mastering these tools, learning to work around their limitations and working to leverage their strengths… and those who are committed to complaining about how they aren’t already perfect Culture Ship Minds. We get it. They’re not superintelligent at everything yet. They couldn…

Nobody is complaining that LLMs aren't perfect Culture minds. People disagree with the premise that they are useful tools given their current capabilities. Your portrayal of those with whom you disagree is such a strawman that it might as well be set to a soy-vs-wojak meme.

They clearly are useful tools given their current capabilities. It just depends on what you’re using them for. You don’t use a screwdriver to drive nails, and you don’t go to HardwareNews to complain when your screwdriver isn’t working as a hammer.

I’m currently using them to port a client-side API SDK into multiple languages. This would be a pain in the ass time consuming task but is a breeze with LLMs because the exact behavior I want is clearly defined and relatively deterministic, and it’s also straightforward to test that I’m getting what I intend. The LLM thus gets done in 3 days what would take me 3 weeks (or more) to do by hand.

If the complaint is that it can’t do X, where X is something that would clearly require full AGI and likely true superintelligence — in this case expecting instantaneous, correct code that solves novel problems on the first try - then I have to insist that people are actually expecting Claude to be a Culture Ship Mind, implicitly. They just don’t realize that what they’re asking for his hard, which is itself a psychologically interesting fact, I suppose.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#63

Post by SparkyMcUnicorn »

ravroid wrote: post #40

One strategy I've been experimenting with is maintaining a 'spec' document, outlining all features and relevant technical notes about a project. I include the spec with all relevant source files in my prompt before asking the LLM to implement a new change or feature. This way it doesn't have to do as much guessing as to what my code is doing, and I can avoid relying on long-running conversations to maintain context.…

You're describing functionality that's built into Aider. You might want to try it out.

Aider also has a copy/paste mode to use web ui interfaces/subscriptions instead apis.

I definitely use and update my CONVENTIONS.md files and started adding a second specification file for new projects. This + architect + "can your suggestion be improved, or is there a better way?" has gotten me pretty far.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#64

Post by zitterbewegung »

XorNot wrote: post #24
zitterbewegung wrote: post #4

I have used LLMs as a tool and I start to "give up" working with it after a few tries. It excels at simple tasks, boilerplate, or scripts but larger programs you really have to know what exactly you want to do. I do see the LLMs ingesting more and more documentation and content and they are improving at giving me right answers. Almost two years ago I don't believe they had every python package indexed and now they ap…

The trouble is the only reliable use-case LLMs actually seem good at is "augmented search engine". Any attempts at coding with them just end up feeling like trying to code via a worse interface. So it's handy to get a quick list of "all packages which do X", but it's worse then useless to have it speculate as to which one to use or why, because of the hallucination problem.

Yes it does work as an augmented search engine but, it does output working code you just have to prompt it better if the code is not that complex like a simple endpoint you just have to understand exactly what you want.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#65

Post by [deleted] »

cherry_tree wrote: post #52
powerset wrote: post #13

I've had a similar experience, shipping new features at incredible speed, then waste a ton of time going down the wrong track trying to debug something because the LLM gave me a confidently wrong solution.

Seems like LLMs would be well suited for test driven development. A human writes tests and the LLM can generate code passing all tests; ending with a solution that meets the humans expectations.

[deleted]

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#66

Post by dambi0 »

suddenlybananas wrote: post #59
mordymoop wrote: post #41

I am frankly tired of seeing this kind of post on HN. I feel like the population of programmers is bifurcating into those who are committed to mastering these tools, learning to work around their limitations and working to leverage their strengths… and those who are committed to complaining about how they aren’t already perfect Culture Ship Minds. We get it. They’re not superintelligent at everything yet. They couldn…

>The thing that pushes me over the line into ranting territory is that computer programmers, of all people, should know that computers do what you tell them to. are you claiming LLMs function like computer program instructions? like they clearly don't operate like that at all.

What part of the comment makes you think they are claiming that?

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#67

Post by Spivak »

baxtr wrote: post #33
potsandpans wrote: post #10

Counterexample: Ive been able to complete more side projects in the last month leveraging llms than i have ever in my life. One of which I believe to have potential as a viable product, and another which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch. I think the key to being successful here is to realize that you're still at the wheel as an engineer. Th…

Is it reasonable to assume that more senior devs benefit more from LLMs?

Yes, you essentially have an impossibly well read junior engineer you can task with quick research questions like, "I'm trying to do x using lib y, can you figure that out for me." This is incredibly productive because in the answer is typically all the pieces you need but not always assembled right.

Getting the LLM to pull out well-known names of concepts is for me the skill you can't get anywhere else. You can describe a way to complete a task and ask for what it's called and you'll be heading down arxiv links right away. Like yes the algorithm to find the closest in edit distance and length needle string in a haystack is called Needleman–Wunsch, of course Claude, everyone knows that.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#68

Post by rglover »

mordymoop wrote: post #41

I am frankly tired of seeing this kind of post on HN. I feel like the population of programmers is bifurcating into those who are committed to mastering these tools, learning to work around their limitations and working to leverage their strengths… and those who are committed to complaining about how they aren’t already perfect Culture Ship Minds. We get it. They’re not superintelligent at everything yet. They couldn…

> computer programmers, of all people, should know that computers do what you tell them to.

Right. The problem isn't that the tool isn't perfect, it's that you get a lot of excitable people with incentives pretending that it is or will soon be perfect (while simultaneously scaring non-technical people into thinking they'll be replaced with a chat bot soon).

There are certainly luddite types who are outright rejecting these tools, but if you have hands-on, daily experience, you can see the forest for the trees. You quickly realize that all of the "omg this thing is sentient" or "we can't let what we've got into the world, it's too dangerous" fodder like the Google panic memo are just covert marketing.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#69

Post by suddenlybananas »

dambi0 wrote: post #66
suddenlybananas wrote: post #59

Earlier quoted context omitted.

>The thing that pushes me over the line into ranting territory is that computer programmers, of all people, should know that computers do what you tell them to. are you claiming LLMs function like computer program instructions? like they clearly don't operate like that at all.

What part of the comment makes you think they are claiming that?

The part I quoted? I don't really see how to interpret it any other way.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#70

Post by simonw »

baxtr wrote: post #33
potsandpans wrote: post #10

Counterexample: Ive been able to complete more side projects in the last month leveraging llms than i have ever in my life. One of which I believe to have potential as a viable product, and another which involved complicated rust `no_std` and linker setup for compiling rust code onto bare metal RISCV from scratch. I think the key to being successful here is to realize that you're still at the wheel as an engineer. Th…

Is it reasonable to assume that more senior devs benefit more from LLMs?

I think so.

Junior devs can get plenty of value out of them too, if they have discipline in how they use them - as a learning tool, not as a replacement for thinking about projects.

Senior devs can get SO much more power from these things, because they can lean on many years of experience to help them evaluate if the tool is producing useful results - and to help them prompt it in the most effective way possible.

A junior engineer might not have the conceptual knowledge or vocabulary to say things like "write tests for this using pytest, include a fixture that starts the development server once before running all of the tests against it".

Post reply on HN