Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Generative AI Con

wheresyoured.at

Post reply on HN
Print view View original

181–190 of 503 posts

Re: The Generative AI Con

#181

Post by simonw »

danroblew wrote: post #84
simonw wrote: post #5

> When you put aside the hype and anecdotes, generative AI has languished in the same place, even in my kindest estimations, for several months, though it's really been years. The one "big thing" that they've been able to do is to use "reasoning" to make the Large Language Models "think" [...] This is missing the most interesting changes in generative AI space over the last 18 months: - Multi-modal: LLMs can consume…

Cost as in, cost to you? Or cost to serve? If the cost-to-serve is subsidized by VC money, they aren't getting cheaper, they're just leading you on.

I've heard from insiders that AWS Nova and Google Gemini - both incredibly cheap - are still charging more for inference than they spend on the server costs to run a query. Since those are among the cheapest models I expect this is true of OpenAI and Anthropic as well.

The subsidies are going to the training costs. I don't know if any model is running at a profit once training/research costs are included.

Re: The Generative AI Con

#182

Post by XorNot »

xyzzy123 wrote: post #75
lukevp wrote: post #12

I’m a little shocked at how much negativity there is around LLMs among developers. It’s a new tool that requires some learning, and it’s sometimes not so great, but if you’ve used an IDE with real coding assistance built in (eg. VS Code in Edit with Copilot mode - NOT Chat mode, using Claude 3.5), it’s honestly not much worse than a junior dev and 100x faster. And if the code is bad you throw it away and try again 10…

For easy things, LLM assist has sped things up a lot for me. For medium complexity things, I can get them done quickly without manual coding if I have a clear understanding in mind of what the implementation should look like. I supply the requirements, design and strategy and it's fairly easy to "keep things on the rails". The "write a PRD first" hack ( https://www.aiagentshub.net/blog/how-to-10x-your-development...…

Whereas I've been disabling AI assist features because I find them actively disruptive to the development process. When it ghost pops up text suggesting what I should do, it's sometimes right...but it breaks flow. It forces me to read and parse apparently correct code, and decide if it is correct or it's just a mirage which is valid but not actually what I'm doing at all.

Re: The Generative AI Con

#183

Post by [deleted] »

tim333 wrote: post #144
alvah wrote: post #14

Ed occasionally makes good points, but he's very very angry at Big Tech, and his anger often gets in the way of his message. Reading his latest rant reminds me of Karl Denninger railing against Google around the time of their IPO, claiming they would never make enough money to justify an $85 share price (a $1000 investment then would be worth around $375,000 today).

I think there's the same logic flaw of looking at how things are at the start - so so - and how they may be in 20 years - Google getting an advertising cut for most of the world's commerce, AI replacing/doubling the ~100tn/yr labour market.

[deleted]

Re: The Generative AI Con

#184

Post by simonw »

newAccount2025 wrote: post #168
audunw wrote: post #77

Earlier quoted context omitted.

But if this is like the internet, it’s not refuting the idea that this is a huge bubble. The internet did have a massive investment bubble. And I’d argue it took decades to actually achieve some of the things we were promised in the early days of the internet. Some have still not come to fruition (the tech behind end to end encrypted emails was developed decades ago, yet email as most people use it is still ridiculou…

Can it be an investment bubble but also a hugely promising technology? The FOMO-frothing herd will over-invest in whatever is new and shiny, regardless of its merits?

I recently compared the buildout of data centers for AI to the railway bubbles of the 1800s.

Nobody will deny the importance of railways to the Industrial Revolution, but they also lost a lot of people a lot of money: https://simonwillison.net/2024/Dec/31/llms-in-2024/#the-envi...

Re: The Generative AI Con

#185

Post by simonw »

jcgrillo wrote: post #140
danroblew wrote: post #84

Earlier quoted context omitted.

Cost as in, cost to you? Or cost to serve? If the cost-to-serve is subsidized by VC money, they aren't getting cheaper, they're just leading you on.

> Cost as in, cost to you? Or cost to serve? This. IIUC to serve an LLM is to perform an O(n^2) computation on the model weights for every single character of user input. These models are 40+GB so that means I need to provision about 40GB RAM per concurrent user and perform hundreds of TB worth of computations per query. How much would I have to charge for this? Are there any products where the users would actually g…

The efficiency gains over the past 18 months have been incredible. Turns out there was a lot of low hanging fruit to make these things faster, cheaper and more resource efficient. https://simonwillison.net/2024/Dec/31/llms-in-2024/#llm-pric...

Re: The Generative AI Con

#186

Post by gessha »

n2d4 wrote: post #104
levkk wrote: post #91

If my Android (or IPhone) disappeared tomorrow, I would feel like I time traveled back a century. If Google search was gone, I wouldn't be able to do my job anymore. If the cloud disappeared, I wouldn't be able to build apps anymore. There are no workarounds, unless you feel like going to a library...? If ChatGPT disappeared tomorrow (or derivatives like Copilot, etc.), I would be mildly inconvenienced. Then I'd go b…

I think that's an unfair comparison. If the IBM Simon disappeared in 1994, I'm pretty sure you wouldn't have cared. If search engines disappeared in 1992, you'd have felt the same. Also, (what later became) AWS probably didn't interface much with you in 2003. It takes some time for technology to mature, usually at least a decade or two. Even once the iPhone was released it took a few years until it became indispensab…

Also, it takes time for people to forget what it was before so when their current status quo is taken away, they don’t know what to replace it with.

Re: The Generative AI Con

#187

Post by otabdeveloper4 »

roenxi wrote: post #107
lukevp wrote: post #12

I’m a little shocked at how much negativity there is around LLMs among developers. It’s a new tool that requires some learning, and it’s sometimes not so great, but if you’ve used an IDE with real coding assistance built in (eg. VS Code in Edit with Copilot mode - NOT Chat mode, using Claude 3.5), it’s honestly not much worse than a junior dev and 100x faster. And if the code is bad you throw it away and try again 10…

> I’m a little shocked at how much negativity there is around LLMs among developers. While the timeline is unclear; it seems likely that LLMs will obsolete precisely the skills that developers use to earn their income. I imagine a lot of them feel rather threatened by the rapid rate of progress. Pointing out that it is already operating at junior dev quality and rapidly improving is unlikely to quiet the discontent.

Nah. "AI" is just really, really lame and square. People have a visceral reaction to it even when it's actually not that bad.

These types of articles are just catching the next meme wave, which will be hating on and making fun of "AI" of all sorts.

Re: The Generative AI Con

#188

Post by pnathan »

Directionally correct.

GenAI is - imo - an assistant. Copilot does effectively templating.

I can have ChatGPT read an email and check it for tone.

Claude can comment on camera kit.

Claude does a very nice image recognition for obscure things.

What I have become persuaded of is that the /completions API is simply not much more than +10% or a low key helper.

I do not need a dumber-than-intern agent going ape on my codebase at speed, which is, approximately, what the codegen tools seem to do.

I saw a self driving car startup using a GPT neural network to recognize images during driving. I would assess that class of use as plausibly very promising.

I would also hazard that Shirkys BS jobs thesis is being proved true, because if a hallucinating ai can do it...

Anyway.

I don't think the fundamentals justify the spend. I think there's too much vitriol, but there's also too much hype & by a country mile too.

Re: The Generative AI Con

#189

Post by NiloCK »

simonw wrote: post #5

> When you put aside the hype and anecdotes, generative AI has languished in the same place, even in my kindest estimations, for several months, though it's really been years. The one "big thing" that they've been able to do is to use "reasoning" to make the Large Language Models "think" [...] This is missing the most interesting changes in generative AI space over the last 18 months: - Multi-modal: LLMs can consume…

There's a little grain of salt with respect to context lengths: the number has grown, but performance seems to degrade with larger context windows.

Anecdote:

I often front-load a bunch of package.jsons from a monorepo when making tooling / CI focused changes. Even 10 or 20k tokens in, Claude says things like "we should look at the contents of somepackage/package.json to check the specifics of the `dev` script."

But its already in the context window! Given the reminder (not reloading it, just saying "its in there"), Claude makes the inference it needs for the immediate problem.

This seems to approximate a 'working memory' for the assistant or models themselves. Curious whether the model is imposing this on the assistant as part of its schema for simulating a thoughtful (but fallible) agent, or if the model itself has the limitation.

Re: The Generative AI Con

#190

Post by linuxftw »

audunw wrote: post #77
adpirz wrote: post #9

Earlier quoted context omitted.

The "iPhone moment" gets used a lot, but maybe it's more analogous to the early internet: we have the basics, but we're still learning what we can do with this new protocol and building the infrastructure around it to be truly useful. And as you've pointed out, our "bandwidth" is increasing exponentially at the same time. If nothing else, my workflows as a software developer have changed significantly in these past t…

But if this is like the internet, it’s not refuting the idea that this is a huge bubble. The internet did have a massive investment bubble. And I’d argue it took decades to actually achieve some of the things we were promised in the early days of the internet. Some have still not come to fruition (the tech behind end to end encrypted emails was developed decades ago, yet email as most people use it is still ridiculou…

While there was certainly a software bubble during the early internet, it still took obscene amounts of investments in brand new technologies in the late 90's. Entire datacenters full of hardware modems. In fact, 'datacenters' had to become a thing.

Then came DSL, then came cable, then came fiber. Countless billions of dollars invested into all these different systems.

This AI stuff is something else. Lots of hardware investment, sure, but also lots of software investment. It is becoming so good and so cheap its showing up on every single search engine result.

Anyway, my point is, while there may have been aspects of the early internet being a bubble, there were real dollars chasing real utility, and I think AI is quite similar in that regard.

Post reply on HN