Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything around LLMs is still magical and wishful thinking

dmitriid.com

Post reply on HN
Print view View original

291–300 of 377 posts

Re: Everything around LLMs is still magical and wishful thinking

#291

Post by hotpotat »

gabrieledarrigo wrote: post #156
hotpotat wrote: post #30

I have to say I’m in the exact camp the author is complaining about. I’ve shipped non trivial greenfield products which I started back when it was only ChatGPT and it was shitty. I started using Claude with copying and pasting back and forth between the web chat and XCode. Then I discovered Cursor. It left me with a lot of annoying build errors, but my productivity was still at least 3x. Now that agents are better an…

> I started a job at a demanding startup and it’s been several months and I have still not written a single line of code by hand Damn, this sounds pretty boring.

It’s not. It’s like I used to play baseball professionally and now I’m a coach or GM building teams and yielding results. It’s a different set of skills. I’m working mostly in idea space and seeing my ideas come to life with a faster feedback loop and the toil is mostly gone

Re: Everything around LLMs is still magical and wishful thinking

#292

Post by hotpotat »

dosnem wrote: post #218
hotpotat wrote: post #30

I have to say I’m in the exact camp the author is complaining about. I’ve shipped non trivial greenfield products which I started back when it was only ChatGPT and it was shitty. I started using Claude with copying and pasting back and forth between the web chat and XCode. Then I discovered Cursor. It left me with a lot of annoying build errors, but my productivity was still at least 3x. Now that agents are better an…

> there’s a process of research and planning and perusing in careful steps, and I set the agent up for success Are there any good articles you can share or maybe your process? I’m really trying to get good at this but I don’t find myself great at using agents and I honestly don’t know where to start. I’ve tried the memory bank in cline, tried using more thinking directives, but I find I can’t get it to do complex thi…

https://www.lesswrong.com/posts/dxiConBZTd33sFaRC/field-note...

Re: Everything around LLMs is still magical and wishful thinking

#293

Post by hotpotat »

exe34 wrote: post #92
hotpotat wrote: post #30

I have to say I’m in the exact camp the author is complaining about. I’ve shipped non trivial greenfield products which I started back when it was only ChatGPT and it was shitty. I started using Claude with copying and pasting back and forth between the web chat and XCode. Then I discovered Cursor. It left me with a lot of annoying build errors, but my productivity was still at least 3x. Now that agents are better an…

> but my productivity was still at least 3x How do you measure this?

Mean time to shipping features of various estimated difficulty. It’s subjective and not perfect, but generally speaking I need to work way less. I’ll be honest, one thing I think I could have done faster without AI was to implement CRDT-based cloud sync for a project I have going. I think I’ve tried to utilize AI too much for this. It’s good at implementing vector clock implementations, but not at preventing race conditions.

Re: Everything around LLMs is still magical and wishful thinking

#294

Post by cedws »

I still see little conversation about the two fundamental limitations of LLMs right now: context size, and prompt injection.

* Computation does not scale linearly with context size, meaning the ‘memory’ of LLMs is limited and gets more expensive as it gets bigger.

* Prompt injection limits the usability of LLMs in the real world. How can you put an LLM in the driving seat if malicious actors can talk it into doing something it’s not supposed to.

Whenever I see a blog post by Anthropic or OpenAI I do a Ctrl+F for “prompt injection.” Never mentioned. They want people to forget this is a problem — because it’s a massive one.

Re: Everything around LLMs is still magical and wishful thinking

#295

Post by TeMPOraL »

timr wrote: post #123
mlinsey wrote: post #121

Earlier quoted context omitted.

12 months ago, we had no reasoning models and even very basic arithmetic was outside of the models' grasp. Coding assistants mostly worked on the level of tab-completing individual functions, but now I can one-shot demo-able prototypes (albeit nothing production-ready) of webapps. I assume you consider the latter "integration", but I think coding is so key to how the base models are being trained that this is due to…

I consider the reasoning models to be primarily a development of efficiency/cost, and I thought the first one was about a year ago, but sure, ok. I don’t think it changes the argument I’m making. The LLM ourobouros / robot centipede has been done, and is not itself a path towards exponential improvement. To be quite honest, I’ve found very little marginal value in using reasoning models for coding. Tool usage, sure,…

> but you can’t trust them to do a calculation in the middle of a task.

You can't trust a person either. Calculating is its own mode of thinking; if you don't pause and context switch, you're going to get it wrong. Same is the case with LLMs.

Tool usage and reasoning and "agentic approach" are all in part ways for allowing LLM to do the context switch required, instead of taking the match challenge as it goes and blowing it.

Re: Everything around LLMs is still magical and wishful thinking

#296

Post by michaelbrave »

TeMPOraL wrote: post #255
m4rtink wrote: post #58

Earlier quoted context omitted.

Does anyone actually know what the real cost for the customers will be once the free AI money no longer floods those companies?

Too cheap to meter? Inference is cheap and there's no long-term or even mid-term moat here. As long as the courts don't shut down Meta over IP issues with LLama training data , that is. I can't stress that enough: "open source" models are what can stop the "real costs" for the customers from growing. Despite popular belief, inference isn't that expensive. This isn't Uber - stopping isn't going to make LLMs infeasible…

If LLama goes away we would still get models from China that don't respect the laws that shut down LLama, at least until China is on top, they will continue to undercut using open source/model. Either way, open models will continue to exist.

Re: Everything around LLMs is still magical and wishful thinking

#297

Post by satyrun »

saulpw wrote: post #233
martinald wrote: post #33

I personally don't really get this. _So much_ work in the 'services' industries globally comes down to really a human transposing data from one Excel sheet to another (or from a CRM/emails to Excel), manually. Every (or nearly every) enterprise scale company will have hundreds if not thousands of FTEs doing this kind of work day in day out - often with a lot of it outsourced. I would guess that for every 1 software e…

Each FTE doing that manual data pipelining work is also validating that work, and they have a quasi-legal responsibility to do their job correctly and on time. They may have substantial emotional investment in the company, whether survival instinct to not be fired, or ambition to overperform, or ethics and sense to report a rogue manager through alternate channels. An LLM won't call other nodes in the organization to…

People are also incredibly accurate at doing this kind of manual data piping all day.

There is also a reason that these jobs are already not automated. Many of these jobs you don't need language models. We could have automated them already but it is not worth someone to sign off on. I have been in this situation at a bank. I could have automated a process rather easily but the upside for me was a smaller team and no real gain while the downside was getting fired for a massive automated mistake if something went wrong.

Re: Everything around LLMs is still magical and wishful thinking

#298

Post by seba_dos1 »

TeMPOraL wrote: post #287
seba_dos1 wrote: post #275

Earlier quoted context omitted.

From time to time I use an LLM to pretend to research a topic that I had researched recently, to check how much time it would have saved me. So far, most of the time, my impression was "I would have been so badly mislead and wouldn't even know it until too late". It would have saved me some negative time. The only thing LLMs can consistently help me with so far is typing out mindless boilerplate, and yet it still som…

> So far, most of the time, my impression was "I would have been so badly mislead and wouldn't even know it until too late". It would have saved me some negative time. That was my impression with Perplexity too, which is why I mostly stopped using it, except for when I need a large search space covered fast and am willing to double-check anything that isn't obviously correct. Most of the time, it's o3. I guess this i…

> Thing is, I have an unusually high need to own the understanding of any thing I'm learning.

Same here. Don't get me wrong, LLMs can be helpful, but what I mean is that they can at best aid my research rather than perform it for me. In my experience, relying on them to do that would usually be disastrous - but they do sometimes help in cases where I feel stuck and would otherwise have to find some human to ask.

I guess it's the difference between "using LLMs while thinking" and "using LLM to do the thinking". The latter just does not work (unless all you're ever thinking about is trivial :P), the former can boost you up if you're smart about it. I don't think it's as big of a boost as many claim and it's still far from being reliable, but it's there and it's non-negligible. It's just that being smart about it is non-optional, as otherwise you end up with slop and don't even realize it.

Re: Everything around LLMs is still magical and wishful thinking

#299

Post by TeMPOraL »

assuagering wrote: post #290
TeMPOraL wrote: post #262

Earlier quoted context omitted.

Yes, you can - because of LLama. Training is expensive, but it's not that expensive either. It takes just one of those super-rich players to pay the training costs and then release the weights, to deny other players a moat.

If your economic analysis depends on "one of those super-rich players to pay" for it to work, it isn't as much analysis as wishful thinking. All the 100s of billions of $ put into the models so far were not donations. They either make it back to the investors or the show stops at some point. And with a major chunk of proponent's arguments being "it will keep getting better", if you lose that what you got? "This thing…

The economic analysis is not mine, and I though it was pretty well-known by now: Meta is not in the compute biz and doesn't want to be in it, so by releasing Llamas, it denies Google, Microsoft and Amazon the ability to build a moat around LLM inference. Commoditize your complement and all that. Meta wants to use LLMs, not sell access to them, so occasionally burning a billion dollars to train and give away an open-weight SOTA model is a good investment, because it directly and indirectly keeps inference cheap for everyone.

Re: Everything around LLMs is still magical and wishful thinking

#300

Post by assuagering »

hamilyon2 wrote: post #98

I am impressed by speed of the sound goalpost movement. Few days ago Google released very competent summary generator, interpreter between 10-s of languages, gpt-3 class general purpose assistant. Working locally on modest hardware. On 5 years old laptop, no discrete GPU. It alone potentially saves so much toil, so much stupid work. We also finally “solved computer vision”. Read from PDF, read diagrams and tables. Lo…

I have had SOTA models stray from factual content in documents I provided them with within 2-3 prompts.

They haven't solved anything. They are just fast and look good doing what we ask them to do. But they corrupt data with a passion and to that hype just responds: "just give us 10x as much money and compute".

Post reply on HN