Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tinybox – A powerful computer for deep learning

tinygrad.org

Post reply on HN
Print view View original

361–370 of 372 posts

Re: Tinybox – A powerful computer for deep learning

#361

Post by [deleted] »

Aperocky wrote: post #333
jolmg wrote: post #199

Earlier quoted context omitted.

> attach 10x length worth of AI appendix that would be helpful indexing and references. Are references helpful when they're generated? The reader could've generated them themselves. References would be helpful if they were personal references of stuff you actually read and curated. The value then would be getting your taste. References from an AI may well be good-looking nonsense.

"The user could have written the code themselves" Yes, sometimes this is true, but not always. Note, it's not one prompt (there aren't really "one prompt" any more, prompt engineering is such a 2023-2024 thing), or purely unreviewed output. It's curated output that was created by AI but iterated with me since it goes with and has to match my intention. And most of the time I don't directly prompt the agent any more.…

[deleted]

Re: Tinybox – A powerful computer for deep learning

#362

Post by h14h »

Would be very curious how RL benchmarks shake out vs M5 Pro/Max.

Doubt local inference is the target use case near nearly as much as post-training. I could totally see something like this being super appealing for a startup looking to do some fine-tuning/distillation to tune a small open-weight model for a narrow use case.

Re: Tinybox – A powerful computer for deep learning

#363

Post by h14h »

vessenes wrote: post #7

The exabox is interesting. I wonder who the customer is; after watching the Vera Rubin launch, I cannot imagine deciding I wanted to compete with NVIDIA for hyperscale business right now. Maybe it’s aiming at a value-conscious buyer? Maybe it’s a sensible buy for a (relatively) cash-strapped ML startup; actually I just checked prices, and it looks like Vera Rubin costs half for a similar amount of GPU RAM. I’m certai…

Have to imagine each tinybox is targeting different tiers of startups trying to fine-tune/RL their way to custom models for narrow use-cases.

Maybe the target profile for exabox looks like a smaller/younger Cursor? If you're a small team with some seed funding and expertise, this kind of compute in a single box you can set up in your office feels like it could be a great fit.

Re: Tinybox – A powerful computer for deep learning

#364

Post by FergusArgyll »

crossroadsguy wrote: post #360
phrotoma wrote: post #304

Earlier quoted context omitted.

> arrogant to the point of being almost hostile First encounter with geohot eh?

What does this mean? Is it some reference to different temperaments across geographies? Or some Internet slang?

https://en.wikipedia.org/wiki/George_Hotz

Re: Tinybox – A powerful computer for deep learning

#365

Post by mihaaly »

paxys wrote: post #110

The problem with all these "AI box" startups is that the product is too expensive for hobbyists, and companies that need to run workloads at scale can always build their own servers and racks and save on the markup (which is substantial). Unless someone can figure out how to get cheaper GPUs & RAM there is really no margin left to squeeze out.

We may be surprised how illefficient companies are in organizing the creation of sophisticated things (including processes) for themselves, to use (so for the cost center column).

Higher management figures out things to do in strategic level, in brief, and pushes on "soldiers", who kick it through in the least time (cheapest of the cheapest, for the sake of the quarterlies) EXACTLY the way management told it. Because they have to, their job is to make happen the company objectives given, the way it is given. Pushing out crap in the shape of the thing expected.

Larger organiztaion can use these kind of things the most. Even if they don't do that.

Re: Tinybox – A powerful computer for deep learning

#366

Post by Aissen »

It might be a bit CPU and RAM starved… Which in theory should be OK, but in practice you'll find production workloads that struggle because of this. Just make sure whatever you want to run on this is indeed extremely GPU-bound, or you might have bad surprises later.

Re: Tinybox – A powerful computer for deep learning

#367

Post by ac29 »

emidoots wrote: post #119
mayukh wrote: post #22

What’s the most effective ~$5k setup today? Interested in what people are actually running.

At $7.2k + tax: * RAM - $1500 - Crucial Pro 128GB Kit (2x64GB) DDR5 RAM, 5600MHz CP2K64G56C46U5, up to 4 sticks for 128GB or 256GB, Amazon * GPU - $4700 - RTX Pro 5000 48GB, Microcenter * CPU/Mobo bundle - $1100 - AMD Ryzen 7 9800X3D, MSI X870E-P Pro, ditch the 32GB RAM, Microcenter * Case - $220, Hyte Y70, Microcenter * Cooler - $155, Arctic Cooling Liquid Freezer III Pro, top-mount it, Microcenter * PSU - $180, RM1…

> RAM - $1500 - Crucial Pro 128GB Kit (2x64GB) DDR5 RAM

I knew prices went up, but that's wild. I bought 64GB (2x32) of RAM a year ago for $90.

Re: Tinybox – A powerful computer for deep learning

#368

Post by roarcher »

fwipsy wrote: post #359
roarcher wrote: post #259

Earlier quoted context omitted.

It doesn't matter if it's a joke. The non-technical manager or VP making this purchase will not understand it and will expect poor treatment from this vendor, an expectation that will be reinforced by numerous other things on this page. There is no reason to include it at all.

To me it signals honesty. But this is a subjective judgement. It really sounds like you subjectively disliked the page, and you're trying to present that dislike as objective fact. It really annoyed me the way you kept changing your argument to justify that. Why not just say "I dislike their marketing copy, it rubbed me the wrong way" and leave it at that?

I'm confused by this comment. Is it not obvious that everything I've said is my opinion?

Not everyone feels the need to hedge everything they say with "to me..." and "it really sounds like...".

> It really annoyed me ...

I have no idea what you expect me to do with that information.

> you kept changing your argument

I'm not sure what you're referring to. Are you talking about the joke/not-a-joke thing? I didn't change my argument, I dismissed a shallow objection to an irrelevant detail. My point is that regardless of why the driver quality was included on the page, customers are going to take it the wrong way. Yes, that is my opinion, because apparently that needs to be explicitly stated.

Everyone else who disagreed with me seems to have understood all this so I don't know what the source of your confusion is.

Re: Tinybox – A powerful computer for deep learning

#369

Post by stale2002 »

SilverElfin wrote: post #103
stale2002 wrote: post #61

Earlier quoted context omitted.

Geohotz's politics are fairly straightforward once you understand his background. Geohotz is the prodigy child who, at the age of ~16 accomplished amazing technical feats on his own. And his politics are a derivative of Great Man Theory, and his positions on things like democracy follow from that. This idea, and those espoused by some of the VC/tech elite like Peter Theil are that singular hardworking genius individu…

What makes it “Libertarianism” still? To me it feels like they’re taking away freedom, control, and influence from everyone who is not them. Even the concentration of wealth is itself taking away everyone else’s places in the world.

> What makes it “Libertarianism” still?

Its libertarian because it is fundamentally about individuals acting on their own without going through the government, ect. It is an individualistic framework. Individuals going about achieving their goals, even through powerful corporations, falls squarely within what libertarians support.

Yes, you can make some philosophical point about how if corporations are powerful enough, how is that in any way different from governments.

But, powerful corporations controlling society, in some sort of fallout style or bioshock style dystopia clearly describes a libertarian dystopia, not a left leaning or even fascist dystopia.

Re: Tinybox – A powerful computer for deep learning

#370

Post by awesomeMilou »

pegasus wrote: post #293
awesomeMilou wrote: post #264

Earlier quoted context omitted.

Then if they succeed, I guess you're going to see a different process for the first time in your life. On a website where we frequently talk about disruptive business models, this whole attitude kinda stinks.

> you're going to see a different process for the first time in your life That sounds very neutral, but wouldn't this, by removing the human element and flexibility from business transactions, be a further step along a general enshittification trend?

this was more about the person being narrow minded on a website that brands itself for the out-of-the-box thinkers.
Post reply on HN