Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tinybox – A powerful computer for deep learning

tinygrad.org

Post reply on HN
Print view View original

311–320 of 372 posts

Re: Tinybox – A powerful computer for deep learning

#311

Post by paganel »

nine_k wrote: post #148
znpy wrote: post #51

Earlier quoted context omitted.

I was more worried by the 600kW power requirement... that's 200 houses at full load (3kw) in southern europe... which likely means 400 houses at half load. the town near my hometown has 650 – 800 houses (according to chatgpt). crazy.

Or it's two 300kW fast EV chargers working together. A typical home just consumes rather little energy, now that LED lighting and heat pump cooling / heating became the norm.

> and heat pump cooling / heating became the norm.

We're not all solidly middle-class (especially in Southern and Eastern Europe) and as such we cannot afford those heat pumps. But we'll have to eat the increased energy costs brought by insane server configurations like the ones from the article, so, yeey!!!

Re: Tinybox – A powerful computer for deep learning

#312

Post by delusional »

nine_k wrote: post #148
znpy wrote: post #51

Earlier quoted context omitted.

I was more worried by the 600kW power requirement... that's 200 houses at full load (3kw) in southern europe... which likely means 400 houses at half load. the town near my hometown has 650 – 800 houses (according to chatgpt). crazy.

Or it's two 300kW fast EV chargers working together. A typical home just consumes rather little energy, now that LED lighting and heat pump cooling / heating became the norm.

I think the above commentor is reflecting on the total energy use from having a 600KW load running 24/7. I suppose the more interesting observation is the 14 MWh of daily consumption, enough to charge 100 Rivians every day.

Re: Tinybox – A powerful computer for deep learning

#313

Post by randomgermanguy »

ottah wrote: post #167
overfeed wrote: post #124

Earlier quoted context omitted.

> I'm also confused why this is 12U. My whole rig is 4u. I imagine that's because they are buying a single SKU for the shell/case. I imagine their answer to your question would be: In order to keep prices low and quality high, we don't offer any customization to the server dimensions

That's just such a massively oversized server for the number of gpus. It's not like they're doing anything special either. I can buy an appropriately sized supermicro chassis myself and throw some cards in it. They're really not adding enough value add to overspend on anything.

The major selling point of the tinyboxes is that you're able to run them in your office without any hassle.

I used to own a Dell Poweredge for my home-office, but those fans even on minimal setting kept me up at night

Re: Tinybox – A powerful computer for deep learning

#314

Post by barnabee »

latchkey wrote: post #137
hmokiguess wrote: post #130

Is this like the new equivalent of crypto mining? I remember the early days when they would sell hardware for farming crypto, now it’s AI?

Kind of yes, except there is no block reward.

The block reward is firing humans and collecting ad revenue for slop

Re: Tinybox – A powerful computer for deep learning

#315

Post by jgrizou »

qubex wrote: post #280
paxys wrote: post #110

The problem with all these "AI box" startups is that the product is too expensive for hobbyists, and companies that need to run workloads at scale can always build their own servers and racks and save on the markup (which is substantial). Unless someone can figure out how to get cheaper GPUs & RAM there is really no margin left to squeeze out.

They’re kickstarting a TINY device that is pocketable and aimed at consumers. I’ve backed it (full disclosure).

https://www.kickstarter.com/projects/tiinyai/tiiny-ai-pocket...

Re: Tinybox – A powerful computer for deep learning

#317

Post by ankaz »

qubex wrote: post #280
paxys wrote: post #110

The problem with all these "AI box" startups is that the product is too expensive for hobbyists, and companies that need to run workloads at scale can always build their own servers and racks and save on the markup (which is substantial). Unless someone can figure out how to get cheaper GPUs & RAM there is really no margin left to squeeze out.

They’re kickstarting a TINY device that is pocketable and aimed at consumers. I’ve backed it (full disclosure).

[dead]

Re: Tinybox – A powerful computer for deep learning

#319

Post by atwrk »

mmoustafa wrote: post #84

I would love to see real-life tokens/sec values advertised for one or various specific open source models. I'm currently shopping for offline hardware and it is very hard to estimate the performance I will get before dropping $12K, and would love to have a baseline that I can at least always get e.g. 40 tok/s running GPT-OSS-120B using Ollama on Ubuntu out of the box.

For reference, 12k gets you at least 4 Strix Halo boxes each running GPT-OSS-120B at ~50tok/s.
Post reply on HN