Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tinybox – A powerful computer for deep learning

tinygrad.org

Post reply on HN
Print view View original

151–160 of 372 posts

Re: Tinybox – A powerful computer for deep learning

#151

Post by latchkey »

djsjajah wrote: post #150
latchkey wrote: post #140

Earlier quoted context omitted.

Like what? Most of the good stuff is ported over already and anything else, tag Anush on X and see what you get. Also happy to help. The point is that they care now.

trl. give me a uv command to get that working. But even in the amd stack things (like ck and aiter) consumer cards are not even second class citizens. They are a distance third at best. If you just want to run vllm with the latest model, if you can get it running at all there are going to be paper cuts all along the way and even then the performance won't be close to what you could be getting out of the hardware.

It is not perfect, but it isn't that bad anymore. Tons of improvements over the last year.

Re: Tinybox – A powerful computer for deep learning

#152

Post by [deleted] »

ivraatiems wrote: post #12

There's some irony in the fact that this website reads as extremely NOT AI-generated, very human in the way it's designed and the tone of its writing. Still, this is a great idea, and one I hope takes off. I think there's a good argument that the future of AI is in locally-trained models for everyone, rather than relying on a big company's own model. One thought: The ability to conveniently get this onto a 240v circu…

[deleted]

Re: Tinybox – A powerful computer for deep learning

#153

Post by latchkey »

Gigachad wrote: post #143
latchkey wrote: post #140

Earlier quoted context omitted.

Like what? Most of the good stuff is ported over already and anything else, tag Anush on X and see what you get. Also happy to help. The point is that they care now.

Tbh my experience is in the non AI uses, recently I was looking at Gaussian splatting tools and it seemed the majority of it was CUDA only. I’m also still bothered AMD for ages claimed my card (5700xt) would be getting rocm but just abandoned it.

>I was looking at Gaussian splatting tools and it seemed the majority of it was CUDA only.

Not surprising. True, the ecosystem is like early OSX vs. Windows. Eventually it'll get ported over if there is demand.

Re: Tinybox – A powerful computer for deep learning

#155

Post by rimeice »

solarkraft wrote: post #114
ivraatiems wrote: post #12

There's some irony in the fact that this website reads as extremely NOT AI-generated, very human in the way it's designed and the tone of its writing. Still, this is a great idea, and one I hope takes off. I think there's a good argument that the future of AI is in locally-trained models for everyone, rather than relying on a big company's own model. One thought: The ability to conveniently get this onto a 240v circu…

I find that the most respected writing about AI has very few signs of being written by AI. I'm guessing that's because people in the space are very sensitive to the signs and signal vs. noise.

And because people writing anything worth reading are using the process of writing to form a proper argument and develop their ideas. It’s just not possible to do that by delegating even a small chunk of the work to AI.

Re: Tinybox – A powerful computer for deep learning

#156

Post by wmf »

kkralev wrote: post #144
paxys wrote: post #110

The problem with all these "AI box" startups is that the product is too expensive for hobbyists, and companies that need to run workloads at scale can always build their own servers and racks and save on the markup (which is substantial). Unless someone can figure out how to get cheaper GPUs & RAM there is really no margin left to squeeze out.

[flagged]

just want to run a 7-8b model locally

This is already solved by running LM Studio on a normal computer.

Re: Tinybox – A powerful computer for deep learning

#159

Post by 0xbadcafebee »

trollbridge wrote: post #27
ivraatiems wrote: post #12

There's some irony in the fact that this website reads as extremely NOT AI-generated, very human in the way it's designed and the tone of its writing. Still, this is a great idea, and one I hope takes off. I think there's a good argument that the future of AI is in locally-trained models for everyone, rather than relying on a big company's own model. One thought: The ability to conveniently get this onto a 240v circu…

A typical U.S. 240V circuit is actually just two 120V circuits. Fairly trivial to rewire for that.

I think you're forgetting the wires? If you have one outlet with a 15-20A 120V circuit, then the wiring is almost certainly rated for 15-20A. If you just "combined" two 120V circuits into a 240V circuit, you still need an outlet that is rated for 30A, the wires leading to it also need to be rated for 30A, and it definitely needs a neutral. So you still need a new wire run if you don't have two 120V circuits right where you wanna plug in the box. To pass code you also may need to upsize conduit. If load is continuously near peak, it should be 50A instead of 30.

So basically you need a brand new circuit run if you don't have two 120V circuits next to each other. But if you're spending $65k on a single machine, an extra grand for an electrician to run conduit should be peanuts. While you're at it I would def add a whole-home GFCI, lightning/EMI arrestor, and a UPS at the outlet, so one big shock doesn't send $65k down the toilet.

Re: Tinybox – A powerful computer for deep learning

#160

Post by zozbot234 »

wmf wrote: post #156
kkralev wrote: post #144

Earlier quoted context omitted.

[flagged]

just want to run a 7-8b model locally This is already solved by running LM Studio on a normal computer.

Ollama or llama.cpp are also common alternatives. But a 8B model isn't going to have much real-world knowledge or be highly reliable for agentic workloads, so it makes sense that people will want more than that.
Post reply on HN