Live data from Hacker News

After the AI boom: what might we be left with?

blog.robbowley.net

151–160 of 489 posts

Re: After the AI boom: what might we be left with?

#152
post #66
post #46

Earlier quoted context omitted.

Who's going to pay to run those models? They are currently running at a huge loss.

I can run quite useful models on my PC. Might not change the world but I got a usable transcript of an old foreign language TV show and then machine translated to English. It is not as good as professional subtitles but i wasn't willing to pay the cost of that option.

"we will be left with local models that can be sort of useful but also sort of sucks" is not really a great proposition for the obscene amount of money being invested in this.

Re: After the AI boom: what might we be left with?

#153
post #4

I cant believe people still arent grasping the profound implications of computers that can talk and make decisions.

In my case I fully grasp what such a future could be, but I don't think we are on the path to that, I believe people are too optimistic, i.e. they just believe instead of being truly skeptical. From where I look at it, LLMs are flawed in many ways, and people who see progress as inevitable do not have a mental model of the foundation of those systems to be able to extrapolate. Also, people do not know any other forms…

Symbolic AI is dead. Either stop trying to dig out and reanimate its corpse, or move the goalposts like Gary Marcus did - and start saying "LLMs with a Python interpreter beat LLMs without, and Python is symbolic, so symbolic AI won, GG".

Hallucinations are incredibly fucking overrated as a problem. They are a consequence of the LLM in question not having a good enough internal model of its own knowledge, which is downstream from how they're trained. Plenty of things could be done to improve on that - and there is no fundamental limitation that would prevent LLMs from matching human hallucination rates - which are significantly above zero.

There is a lot of "transformer LLMs are flawed" going around, and a lot of alternative architectures being proposed, or even trained and demonstrated. But so far? There's nothing that would actually outperform transformer LLMs at their strengths. Most alternatives are sidegrades at best.

For how "naive" transformer LLMs seem, they sure set a high bar.

Saying "I know better" is quite easy. Backing that up is really hard.

Re: After the AI boom: what might we be left with?

#154
post #46

Earlier quoted context omitted.

Who's going to pay to run those models? They are currently running at a huge loss.

Anthropic said their inference is cash positive. I would be very surprised if this isn't the norm.

I would be surprised if they are being honest.

Re: After the AI boom: what might we be left with?

#155

Earlier quoted context omitted.

You've done nothing but reuse the Sam Altman/Elon Musk playbook of making wild and extremely vague statements. Maybe say something concrete? What's a positive real world impact of LLMs where they aren't hideously expensive and error prone to the point of near uselessness? Something that isn't just the equivalent of a crypto-bro saying that their system for semi-regulated speculation (totally not a rugpull!) will end…

So you're saying that modern LLMs are a just like crypto/Web3, except in all the ways they're not, so they must be useless. --- Less flippantly, they are excellent for self-studying university-level topics. It's like being able to ask questions to a personal tutor/professor.

But you need to verify everything unless it’s self evident. The number of times CoPilot (Sonnett 4) still hallucinates Browser APIs is astonishing. Imaging trying to learn something that can’t be checked easily, like Egyptian archeology or something.

Re: After the AI boom: what might we be left with?

#156
post #57
post #46

Earlier quoted context omitted.

Who's going to pay to run those models? They are currently running at a huge loss.

I run models for coding on my own machines. They’re a trivial expense compared to what I earn from the work I do. The “at a loss” scenario comes from (1) training costs and (2) companies selling tokens below market to get market share. Neither of those imply that people won’t run models in future. Training new frontier-class models could potentially become an issue, but even that seems unlikely given what these model…

It's unclear if people would pay the price to use them if they were not below market.

I have access to quite a few models, and I use them here and there. They are sort of useful, sometimes. But I don't pay directly for any of them. Honestly, I wouldn't.

Re: After the AI boom: what might we be left with?

#157

Earlier quoted context omitted.

How do we make money on it, especially if massive amounts of the population lose their jobs?

You know the quote "it is easier to imagine an end to the world than an end to capitalism"? Well, AI has allowed me to start imagining the end of capitalism. I'm not claiming we're necessarily very close, but I can clearly see the way from here to a post-scarcity society.

> I can clearly see the way from here to a post-scarcity society.

I would be interested to hear the way that you see. I don't have any problem seeing a huge number of roadblocks to post-scarcity that AI won't solve, but I am open to a different perspective.

Re: After the AI boom: what might we be left with?

#158
post #4

I cant believe people still arent grasping the profound implications of computers that can talk and make decisions.

I can't believe I still have to do my own laundry and dishes. Like that's some how way more powerful than the models of a megawatt powered data center and millions of dollars in 3nm silicon can conquer.

Re: After the AI boom: what might we be left with?

#159
post #137

Earlier quoted context omitted.

they didnt have the compute or the data to make use of NNs. but theoretically NNs made sense even back then, and many people thought they could give rise to intelligent machines. they were probably right, and its a shame they didnt live to see whats happening right now

> they didnt have the compute or the data to make use of NNs The compute and data are both limitations of NNs. We've already gotten really close to the data limit (we aren't generating enough useful content as a species and the existing stuff has all been slurped up). Standard laws of physics restrict the compute side, just like how we know we will hit it with CPUs. Eventually, you just cannot put things closer toget…

dude who cares about data and compute limits. those can be solved with human ingenuity. the ambiguity of creating a generalized learning algorithm has been solved. a digital god has been summoned
Post reply on HN