Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tinybox – A powerful computer for deep learning

tinygrad.org

Post reply on HN
Print view View original

61–70 of 372 posts

Re: Tinybox – A powerful computer for deep learning

#61

Post by stale2002 »

jauntywundrkind wrote: post #11

My interest in anything associated with geohot took a colossal nose dive today after seeing this post against democracy, quoting frelling M*ncius M*ldbug: Democracy is a Liability. https://news.ycombinator.com/item?id=47469543 https://geohot.github.io//blog/jekyll/update/2026/03/21/demo... Theres a lot there that makes sense & I think needs to be considered. But a lot just seems to be out of the blue, included withou…

Geohotz's politics are fairly straightforward once you understand his background. Geohotz is the prodigy child who, at the age of ~16 accomplished amazing technical feats on his own.

And his politics are a derivative of Great Man Theory, and his positions on things like democracy follow from that. This idea, and those espoused by some of the VC/tech elite like Peter Theil are that singular hardworking genius individuals can change the world on their own, and everyone who not in this top 0.1% are borderline NPCs.

They do this both because of their genius/hardwork, and also because they are willing to break the rules that are set forth by this bottom 99.9%.

I'm starting to call this ideology Authoritarian techno-Libertarianism. Its a delibriately oxymoronic name that I use, because these "Great Men" are definitely trying to change the world. IE, they are trying to impose their goals and values on the world without getting the buyin of other people.

Thats the "authoritarian" part. And then the "libertarian" part is that they are going about this imposition of their will on the world by doing it all themselves, through their own hard work.

Think "Person invents a world changing technology, that some people thing is bad, and just releases it open source for anyone to use". AI models are a great example, in fact. Once that technology is out there the genie cannot be put back into the bottle and a ton of people are going to lose their jobs, ect.

A distain for democracy follows directly from things like this. You dont wait for people to vote to allow you to change the world by inventing something new. You just do and watch the results.

Re: Tinybox – A powerful computer for deep learning

#62

Post by kelvinjps10 »

jauntywundrkind wrote: post #11

My interest in anything associated with geohot took a colossal nose dive today after seeing this post against democracy, quoting frelling M*ncius M*ldbug: Democracy is a Liability. https://news.ycombinator.com/item?id=47469543 https://geohot.github.io//blog/jekyll/update/2026/03/21/demo... Theres a lot there that makes sense & I think needs to be considered. But a lot just seems to be out of the blue, included withou…

He was always defending democracy and freedom before, and that was his argument for the local AI thing? What changed?

Re: Tinybox – A powerful computer for deep learning

#63

Post by ranger_danger »

zozbot234 wrote: post #54
bastawhiz wrote: post #32

There's no way the red v2 is doing anything with a 120b parameter model. I just finished building a dual a100 ai homelab (80gb vram combined with nvlink). Similar stats otherwise. 120b only fits with very heavy quantization, enough to make the model schizophrenic in my experience. And there's no room for kv, so you'll OOM around 4k of context. I'm running a 70b model now that's okay, but it's still fairly tight. And…

> And there's no room for kv, so you'll OOM around 4k of context. Can't you offload KV to system RAM, or even storage? It would make it possible to run with longer contexts, even with some overhead. AIUI, local AI frameworks include support for caching some of the KV in VRAM, using a LRU policy, so the overhead would be tolerable.

I know llama.cpp can, it certainly improved performance on my RAM-starved GPU.

Re: Tinybox – A powerful computer for deep learning

#64

Post by throwatdem12311 »

lostmsu wrote: post #53
oofbey wrote: post #33

Earlier quoted context omitted.

It’s not for people to buy. It’s for companies to buy. Compare to salary, and it’s cheap.

Hm, I compared my salary with $10M and it doesn't feel cheap. I guess skill issue.

But how will I make ad-supported youtube videos about how I automated my life with OpenClaw using a $10M boutique AI server to make a few thousand in ad revenue while burning tens of thousands per month on API cost.

Re: Tinybox – A powerful computer for deep learning

#65

Post by ekropotin »

lostmsu wrote: post #50
ekropotin wrote: post #47

IDK, I feel it’s quite overpriced, even with the current component prices. I almost sure it’s possible to custom build a machine as powerful as their red v2 within 9k budget. And have a lot of fun along the way.

AMD now has 32 GiB Radeon AI Pro 9700. 4 of these (just under 2k each) would put you at 128 GiB VRAM

VRAM is not everything - GPU cores also matter (a lot) for inference

Re: Tinybox – A powerful computer for deep learning

#66

Post by KeplerBoy »

Lerc wrote: post #43
ivraatiems wrote: post #12

There's some irony in the fact that this website reads as extremely NOT AI-generated, very human in the way it's designed and the tone of its writing. Still, this is a great idea, and one I hope takes off. I think there's a good argument that the future of AI is in locally-trained models for everyone, rather than relying on a big company's own model. One thought: The ability to conveniently get this onto a 240v circu…

I am a little surprised that they openly solicit code contributions with "Invest with your PRs" but don't have any statement on AI contributions. Maybe the volume for them is ok that well-intentioned but poor quality PRs can be politely(or otherwise, culture depending) disregarded and the method of generation is not important.

Tinygrad sure shared a few opinions on AI PRs on Twitter. I believe the gist was "we have Claude code as well, if that's all you bring don't bother".

Re: Tinybox – A powerful computer for deep learning

#67

Post by aziaziazi »

oofbey wrote: post #33
throwatdem12311 wrote: post #21

Finally, a computer that should be able to run Monster Hunter Wilds with decent performance. But let’s be real, 12k is kinda pushing it - what kind of people are gonna spend $65k or even $10M (lmao WTAF) on a boutique thing like this. I dont think these kinds of things go in datacenters (happy to be corrected) and they are way too expensive (and probably way too HOT) to just go in a home or even an office “closet”.

It’s not for people to buy. It’s for companies to buy. Compare to salary, and it’s cheap.

> What's the goal of the tiny corp? To accelerate. We will commoditize the petaflop and enable AI for everyone.

I had the same feeling as throwadem when reading this. Your comment clarify what they meant by "everyone"

Re: Tinybox – A powerful computer for deep learning

#68

Post by oofbey »

BobbyJo wrote: post #37
oofbey wrote: post #31

Earlier quoted context omitted.

DGX Spark is a fantastic option at this price point. You get 128GB VRAM which is extremely difficult to get at this price point. Also it’s a fairly fast GPU. And stupidly fast networking - 200gbps or 400gbps mellanox if you find coin for another one.

Internet seems to think the SW support for those is bad, and that strix halo boxes are better ROI.

Meh. DGX is Arm and CUDA. Strix is X86 and ROCm. Cuda has better support than ROCm . And x86 has better support than Arm.

Nowadays I find most things work fine on Arm. Sometimes something needs to be built from source which is genuinely annoying. But moving from CUDA to ROCm is often more like a rewrite than a recompile.

Re: Tinybox – A powerful computer for deep learning

#69

Post by selectodude »

zozbot234 wrote: post #58
hrmtst93837 wrote: post #18

Earlier quoted context omitted.

[flagged]

The real case for private inference is not "organic", it's "slow food". Offering slow-but-cheap inference is an afterthought for the big model providers, e.g. OpenRouter doesn't support it, not even as a way of redirecting to existing "batched inference" offerings. This is a natural opening for local AI.

But how slow is too slow (faster than you’d think) and even then, you’re in for $25,000 for even the most basic on-premise slow LLM.

Re: Tinybox – A powerful computer for deep learning

#70

Post by oofbey »

ekropotin wrote: post #59
oofbey wrote: post #31

Earlier quoted context omitted.

DGX Spark is a fantastic option at this price point. You get 128GB VRAM which is extremely difficult to get at this price point. Also it’s a fairly fast GPU. And stupidly fast networking - 200gbps or 400gbps mellanox if you find coin for another one.

I’m not very well versed in this domain, but I think it’s not going to be “VRAM” (GDDR) memory, but rather “unified memory”, which is essentially RAM (some flavour of DDR5 I assume). These two types of memory has vastly different bandwidth. I’m pretty curious to see any benchmarks on inference on VRAM vs UM.

I’m using VRAM as shorthand for “memory which the AI chip can use” which I think is fairly common shorthand these days. For the spark is it unified, and has lower bandwidth than most any modern GPU. (About 300 GB/s which is comparable to an RTX 3060.)

So for an LLM inference is relatively slow because of that bandwidth, but you can load much bigger smarter models than you could on any consumer GPU.

Post reply on HN