Ask HN: Could free/low cost LLMs be a momentary thing? • Hacker News Board

Re: Ask HN: Could free/low cost LLMs be a momentary thing?

#11

Post by atleastoptimal » Mon, May 18, 2026, 11:42 PM UTC

The best of the best will remain around the same price and potentially get more expensive as compute demand increases. However the intelligence/dollar ratio will increase over time due to a few factors (this is common in most tech)

1. Quantization: bigger models can be compressed and retain most of their quality, but run with a far smaller compute footprint

2. Moore's law: chips still are scaling so you can run more compute cheaper as it improves

3. Open source competition: if models from Anthropic or OpenAI get too expensive people will opt to use open source Chinese models, which would reduce demand and thus reduce prices down to a more reasonable equilibrium

Re: Ask HN: Could free/low cost LLMs be a momentary thing?

#12

Post by zambelli » Tue, May 19, 2026, 3:16 AM UTC

Yes, I believe that will be the case. IBM already made the bet with granite models.

Personally, I've found that with guardrails, local 8-14B models can match frontier models on agentic tasks. The key is simple tasks with volume. For very complex things, the big models win. But a simple HR agent auditing 30,000 employee records to make sure all your info is filled out correctly, one at a time? You don't need frontier size for that.

Re: Ask HN: Could free/low cost LLMs be a momentary thing?

#13

Post by adam_thede » Fri, May 22, 2026, 2:18 AM UTC

I've been playing with local LLMs on a 4 year old Mac Studio and have been quite impressed. I keep remembering that the LLM I have available to me today is the worst it will ever be. It will just keep getting better. I'm super interested in and curious about the path forward for local inference.

Re: Ask HN: Could free/low cost LLMs be a momentary thing?

#14

Post by adam_thede » Sat, May 23, 2026, 2:14 AM UTC

zambelli wrote: post #12

Yes, I believe that will be the case. IBM already made the bet with granite models. Personally, I've found that with guardrails, local 8-14B models can match frontier models on agentic tasks. The key is simple tasks with volume. For very complex things, the big models win. But a simple HR agent auditing 30,000 employee records to make sure all your info is filled out correctly, one at a time? You don't need frontier…

Yes, this! I do similar things with local models. Small tasks at volume. I also use local models for things that demand more privacy ... managing or dealing with content I wouldn't want to send to the cloud.