201–210 of 340 posts
I do alt inference prototypes and got much farther than I had hoped to. So indeed, any investor in AI should read deep and question hype and frontier lab investments.
See: https://github.com/guilt/TinyToT for the sort of hype busting I do.
Earlier quoted context omitted.
Frontier labs will figure out all sorts of ways to wiggle into the value chain beyond being commodities.
When do you reckon they'll start doing that?
Earlier quoted context omitted.
In 5-10 years an Apple Watch will run a Fable level model locally. I don’t think we (hackers) should worry too much about token cost inflation. The current wave of providers, that’s another story.
Apple Watch with 1TB of vram with the size of well.. a watch. Amazing story. If we make such leap in semiconductor field, it will be bigger than anything we have done till now. And all of that in 10years!
The thing is, it needs demand to drive it. Laptops have been roughly the same spec for the last 10 years because we don't need them to be bigger; there's no demand for a 16Tb RAM laptop because we don't have anything that could possible need that much RAM. Until LLMs came along, and we all want to run them locally, and so now there is a market for 16Tb laptops. So we'll invent the tech to make that happen.
Earlier quoted context omitted.
In the old world, this was because keeping in sync with upstream was hard. In the new world, it takes an hour. And because you're the only user, you can test in prod. Makes the whole thing faster. I have lots of forked and family-only software. Some are abandoned upstream etc. As cost to software goes to zero, these things become easily possible. In the past, I'd only fork top-quality software (things like `xsv` etc.…
The law of conservation of energy also applies to software. If the price of software approaches zero, it is offset by the time and tokens required to modify and maintain it. Price, time, tokens are simply different expressions of energy.
I don’t think this stands up to scrutiny.
Earlier quoted context omitted.
what are you working on? I only hit the guardrails twice after burning through two weeks of 20x max plan, both times on ML stuff; still more than I'd want to, but not unusable
Lucky for you. I, along with everyone else on the fable guardrail thread have been hitting guardrails with Fable for boring normal shit and getting downgraded.
> where’s all this new magical software that the productivity improvements should imply? It's running, privately, in my homelab. I think we are entering what I call the "have it your way" era. If an open source project doesn't do exactly what you want it to do, fork it, or create a new version. It's too easy. This makes me a bit concerned about the future of open source. Upstreaming used to be worth it, since maintai…
"This new tool allows for writing all this code ..... but every person and company, in unison, in a grand conspiracy, all decided to only write private software with it that they aren't releasing to the public in any way" Seems reasonable
People are shamed for using LLMs at all. So they use them privately, hide them, or disguise their use.
They are definitely being used for public projects. But people are afraid of backlash. Look at some of the comments here.
Hell, Reddit is extraordinarily against LLMs such that even neutral takes are down voted. Mostly by younger generations that aren't in the workforce.
Then you have all the regular people against AI generally.
Ironically there is a conspiracy here. But in the opposite direction.
Earlier quoted context omitted.
Apple Watch with 1TB of vram with the size of well.. a watch. Amazing story. If we make such leap in semiconductor field, it will be bigger than anything we have done till now. And all of that in 10years!
No, it won't. We moved about order of magnitude that from 1990 to 2000. The thing is, it needs demand to drive it. Laptops have been roughly the same spec for the last 10 years because we don't need them to be bigger; there's no demand for a 16Tb RAM laptop because we don't have anything that could possible need that much RAM. Until LLMs came along, and we all want to run them locally, and so now there is a market fo…
And yes, laptop specs haven't changed much and this is partially because the need for spec changes wasn't present, but also during the last 20 years there has been tremendous pressure for efficiency in datacenters.
Despite that, dennard scaling is dead since 20 years. There are physical limits. Already now, the wear effect of electrons jumping is present, and it will only get worse as things scale towards smaller sizes.
There are some benefits to be had, e.g. one can etch models into chips directly so you can pack them more closely, and run more inference on Tensor like chips, but that gives you maybe one order of magnitude improvement in total, at most. Also, of course nobody does that when each 2-6 months a new model comes out.
Earlier quoted context omitted.
We live in the future. You can get an ear cleaning camera endoscope device for $40 next day Amazon delivery anywhere they reach.
Or you could just take a shower, which makes it easy to wipe the excess earwax from your ear.
This line: "this is my main argument against the valuation of frontier labs. It’s not that AI won’t create that much value, it’s that they won’t capture it." That is a very astute and concise way to explain everything about how the frontier labs are behaving and how they're trying to push more people to pay token rates for the best models. At the current subscription prices ($100 or $200 a month for a generous, thoug…
In 5-10 years an Apple Watch will run a Fable level model locally. I don’t think we (hackers) should worry too much about token cost inflation. The current wave of providers, that’s another story.