Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

sfcompute.org

Post reply on HN
Print view View original

81–90 of 189 posts

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#81

Post by jiggawatts »

beachy wrote: post #74
whack wrote: post #70

> Rather than each of K startups individually buying clusters of N gpus, together we buy a cluster with NK gpus... Then we set up a job scheduler to allocate compute In theory, this sounds almost identical to the business model behind AWS, Azure, and other cloud providers. "Instead of everyone buying a fixed amount of hardware for individual use, we'll buy a massive pool of hardware that people can time-share." Outsi…

AWS and Azure would slit their own throats before they created a way for their customers to pool instances to save money. They want to do that themselves, and keep the customer relationship and the profits, instead of giving them to a middleman or the customer.

It’s just corporate profits combined with market forces, not a some sort of malicious conspiracy.

You can rent a 2-socket AMD server with 120 available cores and RDMA for something like 50c to $2 per hour. That’s just barely above the cost of the electricity and cooling!

What do you want, free compute just handed to you out of the goodness of their hearts?

There is incredible demand for high-end GPUs right now, and market prices reflect that.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#82

Post by agajews »

kaycebasques wrote: post #66

Hi, SF lover [1] here. Anything interesting to note about your name? Will your hardware actually be based in SF? Any plans to start meetups or bring customers together for socializing or anything like that? [1] We have not gone the way of the Xerces blue [2] yet... we still exist! [2] https://en.wikipedia.org/wiki/Xerces_blue

Ah the hardware isn’t gonna be in SF (not the cheapest datacenter space)

But I do think a lot of our customers will be out here —- SF is still probably the best place to do startups. We just have so many more people doing hard technical stuff here. Literally every single place I’ve lived in SF there’s been another startup living upstairs or downstairs

Good idea to host some in person events!

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#83

Post by ShamelessC »

sillysaurusx wrote: post #3

I hope you succeed. TPU research cloud (TRC) tried this in 2019. It was how I got my start. In 2023 you can barely get a single TPU for more than an hour. Back then you could get literally hundreds, with an s. I believed in TRC. I thought they’d solve it by scaling, and building a whole continent of TPUs. But in the end, TPU time was cut short in favor of internal researchers — some researchers being more equal than…

Wow! I never thought you’d see the light. All I ever see from your posts is praise for TRC. As someone who got started way later on, I had infinitely more success with a gaming GPU I owned myself. Obviously not really comparable, but TRC was very very difficult to work with. I think I only ever had access to a TPUv3 once and that wasn’t nearly enough time to learn the ropes.

My understanding was that this situation changed drastically depending on what sort of email you had or how popular your Twitter handle was.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#84

Post by [deleted] »

kaycebasques wrote: post #66

Hi, SF lover [1] here. Anything interesting to note about your name? Will your hardware actually be based in SF? Any plans to start meetups or bring customers together for socializing or anything like that? [1] We have not gone the way of the Xerces blue [2] yet... we still exist! [2] https://en.wikipedia.org/wiki/Xerces_blue

[deleted]

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#85

Post by whimsicalism »

quickthrower2 wrote: post #69
whimsicalism wrote: post #7

I am super interested in AI on a personal level and have been involved for a number of years. I have never seen a GPU crunch quite like it is right now. To anyone who is interested in hobbyist ML, I highly highly recommend using vast.ai

Depends on what you class as hobbyist but I am running a T4 for a few minutes to get acquainted with tools and concepts and I found modal.com really good for this. They resell AWS and GCP at the moment. They also have A100 but T4 is all I need for now.

Significantly more expensive than equivalent 3090 configuration if you can do model parallelism

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#86

Post by itissid »

Noob Thought: So this would be a blue print on how a mid tier universities with older large compute cluster ops could do things in 2023 to support large LLM research?

Perhaps its also a way for freshly applying grad students to look at a university looking to do research in LLMs that requires scale...

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#87

Post by quickthrower2 »

whimsicalism wrote: post #85
quickthrower2 wrote: post #69

Earlier quoted context omitted.

Depends on what you class as hobbyist but I am running a T4 for a few minutes to get acquainted with tools and concepts and I found modal.com really good for this. They resell AWS and GCP at the moment. They also have A100 but T4 is all I need for now.

Significantly more expensive than equivalent 3090 configuration if you can do model parallelism

What do you mean by this? I use less than the $30/m free included usage.

I am guessing you mean at some point just buy your own 3090 as it will be cheaper than paying a cloud per second for a server-grade Nvidia setup.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#88

Post by whimsicalism »

williamstein wrote: post #60
whimsicalism wrote: post #7

I am super interested in AI on a personal level and have been involved for a number of years. I have never seen a GPU crunch quite like it is right now. To anyone who is interested in hobbyist ML, I highly highly recommend using vast.ai

Many thanks for posting about vast.ai, which I had never heard of! It's a sort of "gig economy/marketplace" for GPU's. The first machine I tried just now worked fine, had 512GB of RAM, 256 AMC CPUs, an A100 GPU, and I got about 4 minutes for $0.05 (which they provided for free).

The only caveat is it is not really appropriate for private usecases.

Also, many of the available options clearly are recycled crypto mining rigs which have somewhat odd configurations (poor gpu bandwidth, low cpu ram).

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#89

Post by rileyphone »

Frost1x wrote: post #67
camhart wrote: post #30

Earlier quoted context omitted.

Optimism is (almost) always required in order to accomplish anything of significance. Those who lose it, aren't living up to their potential. I'm not encouraging the false belief that everything you do will work out. Instead I'm encouraging the realization that the greatest accomplishments almost always feel like long shots, and require significant amounts of optimism. Fear and pessimism, while helpful in appropriate…

>Optimism is (almost) always required in order to accomplish anything of significance. Those who lose it, aren't living up to their potential. I argue that realism trumps optimism. It's perfectly normal in a realist farming to see something difficult, acknowledge the high risk and failure potential, and still pursue something with intent to succeed. I've personally grown tired of over optimism everywhere because it c…

You can be a realist visionary.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#90

Post by whimsicalism »

quickthrower2 wrote: post #87
whimsicalism wrote: post #85

Earlier quoted context omitted.

Significantly more expensive than equivalent 3090 configuration if you can do model parallelism

What do you mean by this? I use less than the $30/m free included usage. I am guessing you mean at some point just buy your own 3090 as it will be cheaper than paying a cloud per second for a server-grade Nvidia setup.

I think this is more applicable for training usecases. If you can get by with less than $30/mo in aws compute (quite expensive) then it likely does not make a didference.

What I mean is that you can rent out 4 3090 GPUs for much cheaper than renting an A100 on aws because you are not paying Nvidia's "cloud tax" on flops/$

Post reply on HN