Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI lifestyle subsidy is going to end

digitalseams.com

Post reply on HN
Print view View original

41–50 of 88 posts

Re: The AI lifestyle subsidy is going to end

#41

Post by yzjumper »

haolez wrote: post #3
throwanem wrote: post #2

This has been coming for a long time and it is why I use local models only. I'm willing to give up capabilities in exchange for being able to trust that whatever biases may exist in the models I do use remain static and predictable.

How do you do it? Do you host on your hardware or do you use cloud-based providers for open models?

Not the user above, but I am using the iOS app PrivateLLM when I need offline access or use uncensored models. I use kappa-3-phi-abliterated, models under 6B usually work without crashing. Using Ollama on my Mac Mini 24GB base processor (M4 not M4 pro), I am able to run 7B models. On the mac I am able to set up API access.

Funny enough the mac has almost the same processor as my iPhone 16 Pro, so its just a RAM constraint, and of course PrivateLLM does not let you host an API.

An M4 Pro would do much better do to the increase in RAM and GPU size.

Re: The AI lifestyle subsidy is going to end

#42

Post by roxolotl »

some_random wrote: post #30
netdevphoenix wrote: post #25

Earlier quoted context omitted.

>So what exactly is the "AI lifestyle subsidy" The world's richest subsiding the real cost of offering AI services with the current state of our technology. Once it's clear the AGI won't come anytime in 20X, where X is under 40, money tap will begin to close

So is the lifestyle being subsidized that of those researchers Zuck hired for $100M? That's a meaningfully different usage of the phrase than the original "millennial lifestyle subsidy" to the point where the comparison isn't useful. Or again, is it just the fact AI products are being offered below cost?

The case for pessimism looks something like:

- Generative AI, at a below market cost, eats the internet and becomes the primary entry point

- Some combination of price hikes and service degradation, ads etc, make generative ai kinda shitty

- We’re stuck with kinda shitty generative ai products because the old internet is gone

This is the standard enshitification loop really

Re: The AI lifestyle subsidy is going to end

#43

Post by likium »

throwanem wrote: post #2

This has been coming for a long time and it is why I use local models only. I'm willing to give up capabilities in exchange for being able to trust that whatever biases may exist in the models I do use remain static and predictable.

We only have access to local models because they're subsidized too. There's nothing to prevent companies or state actors from paying/funding for increased probability and inherent bias.

Also local models are close in capabilities now but who knows in a few years what that'll look like.

Re: The AI lifestyle subsidy is going to end

#44

Post by Kon-Peki »

unshavedyak wrote: post #10
throwanem wrote: post #2

This has been coming for a long time and it is why I use local models only. I'm willing to give up capabilities in exchange for being able to trust that whatever biases may exist in the models I do use remain static and predictable.

I agree with this, though i'm still using Claude atm. I figure if we're aware of the downsides you pointed at then we can skip the fast changing landscape of self hosting. It keeps getting cheaper and cheaper to self host, so i'm not sure at what point it makes sense to invest. For me the switching point will probably be when they (AI companies) start the big rug pull. By then my hope is self hosting will be cheaper,…

Perhaps you should evaluate in terms of the price premium for speed. Sometimes you buy milk at the 7-eleven instead of the grocery store. It costs more, but is worth it for the convenience in the situation you are currently in. Most of the time it is not.

You can buy a used recent PC for a hundred or two, cram it full of memory, and then run a very advanced model. Slowly. But if you are planning to run an agent while you sleep and then review the work in the morning, do you really care if the run time is 4 hours instead of 40 seconds? Most of the time, no. Sometimes, yes.

Re: The AI lifestyle subsidy is going to end

#45

Post by throwanem »

jasonjmcghee wrote: post #23
egypturnash wrote: post #9

Earlier quoted context omitted.

You have a very curious definition of "local" if that includes "cloud-based providers".

> the models I do use remain static and predictable Some people overload "local" a bit to mean you are hosting the model - whether it's on your computer, on your rack, or on your hetzner instance etc. But I think parent is referring to the open/static aspect of the models. If it's hosted by a generic model provider that is serving many users in parallel to reduce the end cost to the user, it's also theoretically a st…

I am bewildered to hear the sense of "local," in which I and engineers in my experience have for thirty years referred to things which are not remotely hosted, referred to as an "overload."

Re: The AI lifestyle subsidy is going to end

#46

Post by jkingsman »

GaggiX wrote: post #40
barrenko wrote: post #5

Future ain't what it used to be. The web is dead (worse actually, it's a putrid rotting zombie, destroying our children's lives and ours), but the internet will survive.

>worse actually, it's a putrid rotting zombie, destroying our children's lives and ours What are you talking about, is this a rant against TikTok or other socials?

I suspect they're referring to "dead internet theory"[0], and extending the metaphor to zombies in that internet content will still appear to be written by humans/be organic, but will instead be AI slop.

[0]: https://en.wikipedia.org/wiki/Dead_Internet_theory

Re: The AI lifestyle subsidy is going to end

#47

Post by siliconc0w »

The problem is that it's really hard to capture the market when there are so many well financed players competing. Plus distilled local models are within 10% or so of the frontier models and are fine for most questions so you could see a shift where local dominates and you'll only need to go to the cloud for hard problems. Finally, I think most people are willing to pay for AI - it's more utility than a streaming service or a newspaper.

Re: The AI lifestyle subsidy is going to end

#48

Post by barrkel »

LarsDu88 wrote: post #8

The subsidy is blatantly obvious when you compare the cost of self-hosting versus subscription, and the subscription is dramatically dramatically cheaper. The difference between Blue Apron and many AI tools is that the value add does exist. You can cut meal prep from your life, but by 2030, cutting whatever agentic code copilot exists by that point will be like cutting off your fingers for many workers and businesses…

When you self host, are you hosting SOTA models (do you know how big or sparse they are) and are you maximizing utilization?

Re: The AI lifestyle subsidy is going to end

#49

Post by mansilladev »

haolez wrote: post #3
throwanem wrote: post #2

This has been coming for a long time and it is why I use local models only. I'm willing to give up capabilities in exchange for being able to trust that whatever biases may exist in the models I do use remain static and predictable.

How do you do it? Do you host on your hardware or do you use cloud-based providers for open models?

If you have Docker (Desktop) installed, with just a couple of clicks, you can get a local model going on your computer. llama3.2 (3B), llama3.3 (70B), deepseek-r1, and about a dozen others.

Re: The AI lifestyle subsidy is going to end

#50

Post by throwanem »

Kon-Peki wrote: post #44
unshavedyak wrote: post #10

Earlier quoted context omitted.

I agree with this, though i'm still using Claude atm. I figure if we're aware of the downsides you pointed at then we can skip the fast changing landscape of self hosting. It keeps getting cheaper and cheaper to self host, so i'm not sure at what point it makes sense to invest. For me the switching point will probably be when they (AI companies) start the big rug pull. By then my hope is self hosting will be cheaper,…

Perhaps you should evaluate in terms of the price premium for speed. Sometimes you buy milk at the 7-eleven instead of the grocery store. It costs more, but is worth it for the convenience in the situation you are currently in. Most of the time it is not. You can buy a used recent PC for a hundred or two, cram it full of memory, and then run a very advanced model. Slowly. But if you are planning to run an agent while…

The difference is not nearly so stark.
Post reply on HN