Live data from Hacker News

Everything around LLMs is still magical and wishful thinking

dmitriid.com

201–210 of 377 posts

Re: Everything around LLMs is still magical and wishful thinking

#201

I personally don't really get this. _So much_ work in the 'services' industries globally comes down to really a human transposing data from one Excel sheet to another (or from a CRM/emails to Excel), manually. Every (or nearly every) enterprise scale company will have hundreds if not thousands of FTEs doing this kind of work day in day out - often with a lot of it outsourced. I would guess that for every 1 software e…

>I would guess that for every 1 software engineer there are 100 people doing this kind of 'manual data pipelining'.

For what time of company is this true? I really would like someone to just do a census of 500 white collar jobs and categorize them all. Anything that is truly automatic has already been automated away.

I do think AI will cause a lot of disruption, but very skeptical of the view that most people with white collar jobs are just "email jobs" or data entry. That doesn't fit my experience at all, and I've worked at some large bureaucratic companies that people here would claim are stuck in the past.

Re: Everything around LLMs is still magical and wishful thinking

#202
post #36

Everything? As a lawyer, I’m producing 2x - with fewer errors. Admittedly, law is a field that mostly involves shuffling words around so it may be the best case scenario, but much of the skepticism comes off as cope.

computer programming and law are very similar. computer code is called code because it is the law that dictates the behavior of the computer. law is a bit different because it is a program that runs on people, who aren't as deterministic as machines, but in theory law and the interpretation of law are also supposed to be completely logical, and you can translate back and forth directly from the logic of law to a logical expression in a computer program.

i specialize in programming, and LLMs are very good right now, if you set them up with the right tooling, feedback based learning methods, and efficient ways of capturing human input (review/approve/suggest/correct/etc).

with programming you have compilers and other static analysis tools that you can use to verify output. for law you need similar static analysis tooling, to verify things like citations, procedural scheduling, electronic filing, etc, but if you loop that tooling in with an llm, the llm will be able to correct errors automatically, and you will get to an agent that can take a statement of fact, find a cause of action, and file a pro se lawsuit for someone.

courts are going to be flooded with lawsuits, on a scale of 10-100X current case loads.

criminal defendants will be able to use a smart phone app to represent themselves, with an AI handling all of the filings and motions, monitoring the trial in real time, giving advice to the defendant on when to make motions and what to say, maximizing delay and cost for the state with maximum efficiency.

with 98% of convictions coming from guilty pleas (https://www.npr.org/2023/02/22/1158356619/plea-bargains-crim...) which are largely driven by not being able to afford the cost of legal services the number of criminal defendants electing to go to full jury trial could easily explode 10-20X or more very quickly.

fun times!

Re: Everything around LLMs is still magical and wishful thinking

#203
The point about non-determinism is moot if you understand how it works. An accurate LLM always gives the same result where the same result is needed, no matter how many times you ask it. Try asking any LLM what is 2x2 on a temperature it's designed for, what are the chances to get 5 in a reply?

In reality, modern LLMs trained with RL have terrible variance and mainly learn 1:1 mapping of ideas to ideas, which is a big issue for creative writing and parallel inference/majority voting techniques, so there's even less meaningful "non-determinism" available than you might think. It's usually either able or not able to give the correct answer, rerolling it doesn't work well. I think even a human has more non-determinism than a modern LLM (it's impossible to measure though).

Re: Everything around LLMs is still magical and wishful thinking

#204

Earlier quoted context omitted.

Your comment is no better than the comment in the article that the author is calling out. "90%" also seems a bit suspect.

I just went through the last 10 chat titles and all of them were spot on for me. Maybe the person you’re responding to has a different experience than you do and calling their perspective “suspect” is somewhat uncharitable. (There are times I do other kinds of work and it fails terribly. My main point stands.)

Pics or it didn't happen.

You're doing the same thing the article talks against. Some people claim miraculous results, while the reality for most is far less successful. But maybe you keep rolling the LLM dice and you keep winning? I personally don't like gambling with my time and energy, especially when I know the rules of the game are so iffy.

Re: Everything around LLMs is still magical and wishful thinking

#205

Earlier quoted context omitted.

But you just confirmed everything the blogpost claimed. You didn't share any evidence with us even though you claim unbelievable things. You even went as far as registering a throwavay account to hide your identity and to make verifying any of your claims impossible. Your comment feels more like a joke to me

... this from an account with Look, the person who wrote that comment doesn't need to prove anything to you just because you're hopped up after reading a blog post that has clearly given you a temporary dopamine bump. People who understand their domains well and are excellent written communicators can craft prompts that will do what we used to spend a week spinning up. It's self-evident to anyone in that situation, a…

Feeling triggered? Feeling afraid? And yes, every claim needs to be proven, otherwise those who make the claims will only convince 4 year olds.

>People who understand their domains well and are excellent written communicators can craft prompts that will do what we used to spend a week spinning up. It's self-evident to anyone in that situation, and the only thing we see when people demand "evidence" is that you aren't using the tools properly.

You have no proof of this, so I guess you chose your camp already?

Re: Everything around LLMs is still magical and wishful thinking

#206
post #103

Earlier quoted context omitted.

Whenever people claim that LLMs are not capable of reasoning, I put them into a category of people who are themselves not capable of reasoning.

Whenever people claim that LLMs are capable of reasoning, I put them into a category of people who are themselves able to reason as much as an LLM.

You chuckled silently to yourself as you posted this.

Re: Everything around LLMs is still magical and wishful thinking

#207

Earlier quoted context omitted.

I just went through the last 10 chat titles and all of them were spot on for me. Maybe the person you’re responding to has a different experience than you do and calling their perspective “suspect” is somewhat uncharitable. (There are times I do other kinds of work and it fails terribly. My main point stands.)

Can you share the questions you asked?

It ranged from whether an epic v10 sport surf ski was a good fit for a newbie, to entra ID questions, to local data residency compliance laws, new jira alternatives, why schools ask for closed shoes, text to speech tool search. Many of these I use eg o4-mini-high for because I want it to ground itself: find material and compile something for me, but get me an answer fairly quickly.

Ones that don’t work but weren’t in the last 10: voice. It sounds amazing but is dumb as rocks. Feels like most of the GPU compute is for the media, not smarts. A question about floating solar heaters for pools. It fed me scam material. A question about negotiating software pricing. Just useless, parroted my points back at me.

I scale models up and down based on need. Very simple: gpt-40. Smarts: o4-mini-high. Research: deep research. I love Claude but at some point it kept running out of capacity so I’d move elsewhere. Although nothing beats it for artefacts. MS Copilot if I want a quick answer to something MS oriented. It’s terrible but it’s easy to access.

Coding is generally Windsurf but honestly that’s been rare for the last month. Been too busy doing other things.

Re: Everything around LLMs is still magical and wishful thinking

#208

This seems like a really bad take. I do PhD research for superconducting materials and right I've been adapting and scaling an existing segmentation model from a research paper for image processing to run multithreaded and took the training runtime per image from 55min to 2min. Yeah it was low hanging fruit but honestly its the type of thing that is just tedious and easy to make mistakes and spend forever debugging.…

I don't think you got the author's point.

They didn't say "AI is bad". Take another look.

Re: Everything around LLMs is still magical and wishful thinking

#209

Earlier quoted context omitted.

I just went through the last 10 chat titles and all of them were spot on for me. Maybe the person you’re responding to has a different experience than you do and calling their perspective “suspect” is somewhat uncharitable. (There are times I do other kinds of work and it fails terribly. My main point stands.)

Pics or it didn't happen. You're doing the same thing the article talks against. Some people claim miraculous results, while the reality for most is far less successful. But maybe you keep rolling the LLM dice and you keep winning? I personally don't like gambling with my time and energy, especially when I know the rules of the game are so iffy.

Nah I’m all over the place. I said the last 10, to check if the 90% claim could be true if you do what I’ve done recently: use it for tons of little general ad hoc things rather than eg code needing serious accuracy.

I don’t “trust” it in the way I’d trust a smart colleague. We know how this works: use it for info that has a lot of results, or ask it to ground itself if it’s eg new info and you can’t rely on training memory. Asking it about esoteric languages or algo’s or numbers will just make you sad. It will generate 1000 confident tokens. But if you told me to lose Google or ChatGPT+Claude, Google is getting dumped instantly.

Re: Everything around LLMs is still magical and wishful thinking

#210

One thing I find frustrating is that management where I work has heard of 10x productivity gains. Some of those claims even come from early adopters at my work. But that sets expectation way too high. Partly it is due to Amdahl's law: I spend only a portion of my time coding, and far more time thinking and communicating with others that are customers of my code. Even if does make the coding 10x faster (and it doesn't…

Canva has seen a 30% productivity uplift - https://fortune.com/2025/06/25/canva-cto-encourages-all-5000... AI is the new uplift. Embrace and adapt, as a rift is forming (see my talk at https://ghuntley.com/six-month-recap/ ), in what employers seek in terms of skills from employees. I'm happy to answer any questions folks may have. Currently AFK [2] vibecoding a brand new programming language [1]. [1] https://x.com/G…

There’s something hilariously Portlandia about making outlandish claims with complete confidence and then plugging your own talk.
Post reply on HN