Live data from Hacker News

I trusted an LLM, now I'm on day 4 of an afternoon project

nemo.foo

151–160 of 215 posts

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#151
Really enjoyed reading your article. Haven’t laughed as much reading a tech article in quite some time. You should consider doing some YouTube videos as your communications style is very humble and entertaining.

Made me wanna join in your garage and help out with the project :)

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#152

Earlier quoted context omitted.

Claude is like having my own college professor. I've learned more in the past month with Claude then I learned in the past year. I can ask questions repeatedly and get clarification as fine as a need it. Granted, Claude has limits, but its a game-changer. > I think the key to being successful here is to realize that you're still at the wheel as an engineer. The llm is there to rapidly synthesize the universe of infor…

> Claude is like having my own college professor. I don't use Claude, so maybe there's a huge gap in reliability between it and ChatGPT 4o. But with that disclaimer out of the way, I'm always fairly confused when people report experiences like these—IME, LLMs fall over miserably at even very simple pure math questions. Grammatical breakdowns of sentences (for a major language like Japanese) are also very hit-or-miss.…

Major in the context of Japanese is rough, I can see a significant drop in quality when interacting with the same model in say Spanish vs English

For as rich a culture the Japanese have, there's only about 1XX million speakers and the size of the text corpus really matters here, the couple billion of English speakers are also highly motivated to choose English over anything else because Lingua Franca has homefield advantage

To use LLM's efectively you have to work with knowledge of their weaknesses, Math is a good example, you'll get better results from Wolphram Alpha even for the simple things, which is expected

Broad reasoning and explanations tend to be better than overly specific topics, the more common a language, the better the response If a topic has a billion tutorials online, an LLM has a really high chance of figuring out first try

Be smart with the context you provide, the more you actively constrain an LLM, the more likely it is to work with you I have friends that just use it to feed class notes to generate questions and probe it for blindspots until they're satisfied, the improvements on their grade s make it seem like a good approach, but they know that just feeding responses to the LLM isn't trustworthy, so they do and then they also check by themselves, the extra time valuable by itself, if just to improve familiarity with the subject

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#154

Earlier quoted context omitted.

How do you know it's accurate?

Simple: one thing I'm learning about is RFCs for TCP/IP. I can literally go test it. It's like saying, "How do you know it is right when it says 2+2=4"? Some knowledge when taught is self-correcting. Other things I'm studying, like, say, tensor calculus, I can immediately use and know I learned it correctly.

TCP/IP is a great example though of something you can get seemingly correct and then be subject to all kinds of failure modes in edge cases you didn’t handle correctly (fragmentation, silly windows, options changing header sizes, etc).

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#155

Earlier quoted context omitted.

>The thing that pushes me over the line into ranting territory is that computer programmers, of all people, should know that computers do what you tell them to. are you claiming LLMs function like computer program instructions? like they clearly don't operate like that at all.

They are closer to being deterministic machines that comply exactly with your instructions, for better or worse, than they are to magical pixies that guess what you must’ve actually meant. The implicit expectation demonstrated by many in the “loudly disappointed in LLMs” contingent seems to be that LLMs should just know what you meant, and then blame them for not correctly guessing it and delivering it. I think LLMs…

> They are closer to being deterministic machines that comply exactly with your instructions, for better or worse, than they are to magical pixies that guess what you must’ve actually meant.

If those are your only two reference points, yes they're closer to the former.

But the biggest problem is how much "pixie that does something you neither wanted nor asked for" gets mixed in. And I think a lot of the complaints you're saying are about lack of mind reading are actually about that problem instead.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#157

Earlier quoted context omitted.

I used to think the exact same thing would happen when we paid Pakistani and Indian labor to do America's busywork. That was about 15 years ago, I no longer have the same enthusiasm you do.

Now, you are paying a Taiwanese or American company to produce GPUs for you. This allows you to use open-source models like DeepSeek R1, significantly reducing your reliance on Indian tech labor

I believe they are saying we did not learn to think more deeply then, so why should we expect to learn how now.

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#158

I spend a good portion of my time asking people to fix their LLM code now at work. It has made code reviews tiring. And it has increased pairing time significantly, making it a less fun activity.

When workmanship doesn't matter, than the ship is already sinking.

It has been my experience 1 code clown can poison a project with dozens of reasonably talented engineers active. i.e. clowns often go through the project smearing bad kludges over acceptable standards to appear like their commit frequency means something.

This is why most developers secretly dream of being plumbers. Good luck, =3

Re: I trusted an LLM, now I'm on day 4 of an afternoon project

#160

I've had a similar experience, shipping new features at incredible speed, then waste a ton of time going down the wrong track trying to debug something because the LLM gave me a confidently wrong solution.

Seems like LLMs would be well suited for test driven development. A human writes tests and the LLM can generate code passing all tests; ending with a solution that meets the humans expectations.

This is more or less how I use LLMs right now. They’re fantastic at the plumbing, so that I can focus on the important part - the business and domain logic.
Post reply on HN