Everything around LLMs is still magical and wishful thinking • Hacker News Board

Re: Everything around LLMs is still magical and wishful thinking

#241

Post by farts_mckensy » Sat, Jul 05, 2025, 5:39 AM UTC

goatlover wrote: post #225

farts_mckensy wrote: post #49

Earlier quoted context omitted.

AI is here to stay, and you do not have agency over that. You can choose not to use it, but that has zero impact on the broader adoption rate. Just like when the automobile was introduced and society as a whole evolved.

There was agency in policies promoting automobile adoption over rail travel. People who act like market forces are inevitable trends in nature tend to forget the political part where companies convince politicians to make policies in their favor, while bombarding the public with advertising. Like fossil fuel companies for example. Continued climate change wasn't inevitable. It was a choice humans with lots of money a…

And that's what's going to happen now. No amount of complaining about it online is going to stop it. They literally just passed a bill that prevents regulating AI. Rich people and their puppets in congress have some degree of agency. We don't. Sorry.

Re: Everything around LLMs is still magical and wishful thinking

#242

Post by troupo » Sat, Jul 05, 2025, 5:40 AM UTC

orbital-decay wrote: post #239

troupo wrote: post #234

Earlier quoted context omitted.

Are those LLMs in the room with us now? ;) Actually, I did try asking ollama running locally. That should've reduced the amount of non-determinism and whatever layers providers add, and the uncertainty of computer availability. I asked it for a list of Javascript keywords ordered alphabetically. Within 5 minutes it produced a slightly different list Asking a model for 2x2 is moot because 2x2=5 is statically highly un…

>I asked it for a list of Javascript keywords ordered alphabetically. Within 5 minutes it produced a slightly different list That's not my point, the keyword here is "meaningful". How many of those lists are correct? (ignoring the fact prompting a LLM for lists is a bad idea, let alone local ones) If you spend some time with SotA LLMs, you'll see that on rerolling they express pretty much the same ideas in different…

> That's not my point. How many of those lists are correct? (ignoring the fact prompting a LLM for lists is a bad idea, let alone local ones)

> Yes. That's missing the forest for the trees, though.

So, what is your point and forest?

That there exists some non-deterministic accurate LLM with a 100% correct training set which will always produce the same answer if you ask it 2x2 (where asking is somehow different from prompting)?

- Does it exist?

- Where? (considering that the world at large uses chatgpt/claude/gemini)

- what's its usefulness beyond 2x2?

Re: Everything around LLMs is still magical and wishful thinking

#243

Post by samrus » Sat, Jul 05, 2025, 5:41 AM UTC

saulpw wrote: post #233

martinald wrote: post #33

I personally don't really get this. _So much_ work in the 'services' industries globally comes down to really a human transposing data from one Excel sheet to another (or from a CRM/emails to Excel), manually. Every (or nearly every) enterprise scale company will have hundreds if not thousands of FTEs doing this kind of work day in day out - often with a lot of it outsourced. I would guess that for every 1 software e…

Each FTE doing that manual data pipelining work is also validating that work, and they have a quasi-legal responsibility to do their job correctly and on time. They may have substantial emotional investment in the company, whether survival instinct to not be fired, or ambition to overperform, or ethics and sense to report a rogue manager through alternate channels. An LLM won't call other nodes in the organization to…

You are correct that review and validation should still be manual. But the actual "translation" from one format to another should be automated with llms

Re: Everything around LLMs is still magical and wishful thinking

#244

Post by arendtio » Sat, Jul 05, 2025, 5:43 AM UTC

leptons wrote: post #163

arendtio wrote: post #60

I think it is more like googling: when the search engine appeared, everybody had to learn how to write a good query, even though the expectation was that everybody could use them. With LLMs, it's quite similar: you have to learn how to use them. Yes, they are non-deterministic, but if you know how to use them, you can increase your chances of getting a good result dramatically. Often, this not only means articulating…

>Yes, they are non-deterministic The LLM is more like a Ouija board than a reliable tool. >I can ask it to write user stories or prototypes By the time I write enough to explain thoroughly to an LLM what to write in "user stories" or "prototypes", I could have just written it myself, without the middleman(bot), and without the LLM hallucinating. If half the time I spend with an LLM is telling it what to do, and then…

For me, it is a different experience. Yes, it takes time to provide the LLM with the necessary context, but overall it can save me 3-4 times the time I would have invested. Maybe I am just slower at software development myself ;-)

However, the work you do is indeed more that of a product owner than that of a developer. To avoid hallucinations, providing extensive documentation and allowing the LLM to perform test-driven development can be a game-changer.

When you do that, generation time is highly correlated (negatively) with code quality. So when the AI solves the tasks quickly and easily, you have a good chance of it generating good code. As soon as the AI has to try and try again to build something working, you should be very skeptical of the result.

Over the past months and years, I have used this method, and because it is somewhat reproducible, you can see the progress that LLMs are making. Sessions where the model does stupid things are becoming fewer, and sessions where the model finds a good solution become more frequent.

Re: Everything around LLMs is still magical and wishful thinking

#245

Post by jnfno » Sat, Jul 05, 2025, 5:51 AM UTC

[dead]

Re: Everything around LLMs is still magical and wishful thinking

#246

Post by orbital-decay » Sat, Jul 05, 2025, 5:55 AM UTC

troupo wrote: post #242

orbital-decay wrote: post #239

Earlier quoted context omitted.

>I asked it for a list of Javascript keywords ordered alphabetically. Within 5 minutes it produced a slightly different list That's not my point, the keyword here is "meaningful". How many of those lists are correct? (ignoring the fact prompting a LLM for lists is a bad idea, let alone local ones) If you spend some time with SotA LLMs, you'll see that on rerolling they express pretty much the same ideas in different…

> That's not my point. How many of those lists are correct? (ignoring the fact prompting a LLM for lists is a bad idea, let alone local ones) > Yes. That's missing the forest for the trees, though. So, what is your point and forest? That there exists some non-deterministic accurate LLM with a 100% correct training set which will always produce the same answer if you ask it 2x2 (where asking is somehow different from…

>So, what is your point and forest? That there exists some non-deterministic accurate LLM with a 100% correct training set which will always produce the same answer if you ask it 2x2 (where asking is somehow different from prompting)?

As I already said: modern LLMs mainly map a single input idea to a single output idea. They might express it in slightly different ways, but it's either correct or incorrect, you can't easily turn an incorrect result into a correct one by rerolling. If you spend any time with Gemini/o3/Claude, you understand this from the first-hand experience. If you know what current RL algorithms do, you understand why this happens.

An ideal LLM would learn one-to-many correspondence, generalizing better, and that still won't be any problem as long as the answer is correct. Because correctness and determinism are orthogonal to each other.

Re: Everything around LLMs is still magical and wishful thinking

#247

Post by zeta0134 » Sat, Jul 05, 2025, 5:59 AM UTC

samrus wrote: post #243

saulpw wrote: post #233

Earlier quoted context omitted.

Each FTE doing that manual data pipelining work is also validating that work, and they have a quasi-legal responsibility to do their job correctly and on time. They may have substantial emotional investment in the company, whether survival instinct to not be fired, or ambition to overperform, or ethics and sense to report a rogue manager through alternate channels. An LLM won't call other nodes in the organization to…

You are correct that review and validation should still be manual. But the actual "translation" from one format to another should be automated with llms

You seem to be missing that the translation part isn't the expensive part, it's the validation and review.

Separately, maybe this is just me, but having data actually flow through my hands is necessary for full comprehension. Just skimming an automated result, my brain doesn't actually process like half of that data. Making the process more efficient in this way can make my actual review performance *much worse.* The "inefficient" process forcing me to slow down and think can be a feature.

Re: Everything around LLMs is still magical and wishful thinking

#248

Post by troupo » Sat, Jul 05, 2025, 6:00 AM UTC

orbital-decay wrote: post #246

troupo wrote: post #242

Earlier quoted context omitted.

> That's not my point. How many of those lists are correct? (ignoring the fact prompting a LLM for lists is a bad idea, let alone local ones) > Yes. That's missing the forest for the trees, though. So, what is your point and forest? That there exists some non-deterministic accurate LLM with a 100% correct training set which will always produce the same answer if you ask it 2x2 (where asking is somehow different from…

>So, what is your point and forest? That there exists some non-deterministic accurate LLM with a 100% correct training set which will always produce the same answer if you ask it 2x2 (where asking is somehow different from prompting)? As I already said: modern LLMs mainly map a single input idea to a single output idea. They might express it in slightly different ways, but it's either correct or incorrect, you can't…

I still fail to see your point.

Here's what you started with: "The point about non-determinism is moot if you understand how it works."

When challenged you're now quite literally saying "oh yeah, they are all non-deterministic, will produce varying results, it's impossible to control the outcome, and there's some ideal non-existent LLM that will not have these issues"

So what's your point and forest again?

Re: Everything around LLMs is still magical and wishful thinking

#249

Post by edg5000 » Sat, Jul 05, 2025, 6:04 AM UTC

labrador wrote: post #19

I'm a retired programmer. I can't imagine trusting code generated by probablities for anything mission critical. If it were close and just needed minor tweaks I could understand that. But I don't have experience with it. My comment is mainly to say LLMs are amazing in areas that are not coding, like brainstorming, blue sky thinking, filling in research details, asking questions that make me reflect. I treat the LLM l…

Treating it like a thinking partner is good. When programming, I don't treat it like a person, but rather a very high programming language. Imagine exactly how you want the code to be. Then find a way to express that unambiguously in natural language; the idea is that you'll still have a bit of work to do writing things out, but it will be a lot quicker than typing out all code by hand. Combined with iterations of feedback as well as having the AI build and run your program (at least as a sanity check); and asking the AI to check the program behaviour in the same way you would, gets you quite far.

A limitation is the lack of memory. If you steer it from style A to B using multiple points of feedback, if this is not written down, the next AI session you'll have to reexplain this all over.

Deepseek is about 1TB in weights; maybe that is why LLMs don't remember things across sessions yet. I think everybody can have their personal AI (hosted remote unless you own lots of compute); it should remember what happened yesterday; in particular the feedback it was given when developing. As an AI layman I do think this is the next step.

Re: Everything around LLMs is still magical and wishful thinking

#250

Post by GeoAtreides » Sat, Jul 05, 2025, 6:08 AM UTC

CharlesW wrote: post #46

louthy wrote: post #26

Earlier quoted context omitted.

> overall my productivity is 10-15% better. That is nothing to sneeze at, but it isn't 10x. It is something to sneeze at if you are 10-15% more expensive to employ due to the cost of the LLM tools. The total cost of production should always be considered, not just throughput.

> It is something to sneeze at if you are 10-15% more expensive to employ due to the cost of the LLM tools. Claude Max is $200/month, or ~2% of the salary of an average software engineer.

the average software engineer makes $10000 a month after taxes?!