Live data from Hacker News

Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

sfcompute.org

61–70 of 189 posts

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#61

I love the idea of community assets. could it be the start of a GPU co-op?

For consumer-grade cards, that's already here. Make money off your GPU with vast.AI https://cloud.vast.ai/host/setup

> Requirements

> Ubuntu 18.04 or newer (required)

> Dedicated machines only - the machine shouldn't be doing other stuff while rented

well that's certainly not what I expected. ctrl-f "virtual" gives nothing, so it seems they really mean "take over your machine"

> Note: you may need to install python2.7 to run the install script.

what kind of nonsense is this? Did they write the script in 2001 and just abandon it?

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#62

I love the idea of community assets. could it be the start of a GPU co-op?

For consumer-grade cards, that's already here. Make money off your GPU with vast.AI https://cloud.vast.ai/host/setup

check here to see the current bid prices / gpu setups https://cloud.vast.ai/create/

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#63
post #36
post #33

Earlier quoted context omitted.

ask no questions hear no lies

EDIT: They seem to be in a raising fund / debt stage. Great initiative

unrelated to this specific initiative, but - I keep seeing a lot of announcements of huge VC rounds around what's effectively datacenters for GPUs. Curious about the math behind that - I feel like those things get obsolete so fast, it's almost like the whole scooter rental thing, where the unit economics doesn't add up.

Anyone have an insight?

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#64
post #61

Earlier quoted context omitted.

For consumer-grade cards, that's already here. Make money off your GPU with vast.AI https://cloud.vast.ai/host/setup

> Requirements > Ubuntu 18.04 or newer (required) > Dedicated machines only - the machine shouldn't be doing other stuff while rented well that's certainly not what I expected. ctrl-f "virtual" gives nothing, so it seems they really mean "take over your machine" > Note: you may need to install python2.7 to run the install script. what kind of nonsense is this? Did they write the script in 2001 and just abandon it?

> what kind of nonsense is this? Did they write the script in 2001 and just abandon it?

Anything AI/ML is a hot mess of cobbled-together bits and pieces of Python barely holding together. I recently read somewhere that there should be a new specialization of "ML DevOps Engineer"... and hell I'm supporting that.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#65
post #20

How does this compare to https://lambdalabs.com/ ?

Ah, we're running a medium amount of compute at zero-margin. The point is not to go sell the Fortune 500, but to make sure a grad student can spend a $50k grant. Right now, it's pretty easy to get a few A/H100s (Lambda is great for this), but very hard to get more than 24 at a reasonable price ($~2 an hour). One often needs to put up a 6+ month commitment, even when they may only want to run their H100s for an 8 hour…

So what happens when some big bucks VC backed closed source LLM company buys all your compute inventory for the next 5 years? This is not that unlikely. Lambda Labs a little while back was completely sold out of all compute inventory.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#66
Hi, SF lover [1] here. Anything interesting to note about your name? Will your hardware actually be based in SF? Any plans to start meetups or bring customers together for socializing or anything like that?

[1] We have not gone the way of the Xerces blue [2] yet... we still exist!

[2] https://en.wikipedia.org/wiki/Xerces_blue

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#67
post #30
post #23

Earlier quoted context omitted.

> Your project has a youthful optimism that I hope you won’t lose as you go. And in fact it might be the way to win in the long run. This is the nicest thing anyone has said to us about this. We're gonna frame this and hang it out on our wall. > So whenever someone comes knocking, begging for a tiny slice of your H100s for their harebrained idea, I hope you’ll humor them. Absolutely! :D

Optimism is (almost) always required in order to accomplish anything of significance. Those who lose it, aren't living up to their potential. I'm not encouraging the false belief that everything you do will work out. Instead I'm encouraging the realization that the greatest accomplishments almost always feel like long shots, and require significant amounts of optimism. Fear and pessimism, while helpful in appropriate…

>Optimism is (almost) always required in order to accomplish anything of significance. Those who lose it, aren't living up to their potential.

I argue that realism trumps optimism. It's perfectly normal in a realist farming to see something difficult, acknowledge the high risk and failure potential, and still pursue something with intent to succeed.

I've personally grown tired of over optimism everywhere because it creates unrealistic situations and passes consequences of failure in an inequitable way. The "visionary" is rewarded when the rare successes occur, while everyone else suffers the consequences for most failures. No contingency plans for failure, no discussion of failure, and so on. Optimism just takes any idea, pursues it and consequences be someone else's problem and be damned.

Pessimism isn't much better, you essentially think everything is too risky or unlikely to succeed so you never do anything. You live in a state of inaction because any level of risk or uncertainty is too much.

To me, realism is much better. You acknowledge the challenge. You acknowledge the risk. You make sure everyone involved understands it, but you still charge forward knowing you might succeed. Some think if you're not naively optimistic (what most people in my experience refer to as "optimism") you don't create enough pressure. I think that's non-sense.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#69

I am super interested in AI on a personal level and have been involved for a number of years. I have never seen a GPU crunch quite like it is right now. To anyone who is interested in hobbyist ML, I highly highly recommend using vast.ai

Depends on what you class as hobbyist but I am running a T4 for a few minutes to get acquainted with tools and concepts and I found modal.com really good for this. They resell AWS and GCP at the moment. They also have A100 but T4 is all I need for now.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#70
> Rather than each of K startups individually buying clusters of N gpus, together we buy a cluster with NK gpus... Then we set up a job scheduler to allocate compute

In theory, this sounds almost identical to the business model behind AWS, Azure, and other cloud providers. "Instead of everyone buying a fixed amount of hardware for individual use, we'll buy a massive pool of hardware that people can time-share." Outside of cloud providers having to mark up prices to give themselves a net-margin, is there something else they are failing to do, hence creating the need for these projects?

Post reply on HN