Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

sfcompute.org

Post reply on HN
Print view View original

111–120 of 189 posts

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#111

Post by version_five »

ec109685 wrote: post #101
bnr4u wrote: post #71

Having hosted infrastructure in CA at multiple colos. I would advise you to host it elsewhere if you can, cost of power, other infrastructure is much higher in CA than AZ or NV.

Power seems like a very small amount of cost of compute when it comes to GPU’s.

FWIW I tired to look up some numbers, i found California "industrial" electricity at $0.18/Kwh https://www.eia.gov/electricity/monthly/epm_table_grapher.ph... and H100s using 300-700w https://www.nvidia.com/en-us/data-center/h100/ which implies a worst case marginal cost of .18*.7 = $.126 / gpu / hour. Looks like Montana is cheapest at ~$.05 / kwh which would bring that down to $.035. So there may be about a $0.09 California premium (vs the absolute cheapest possibility), which as you say is a small amount of the total cost, but could be material for large workloads.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#112

Post by hackernewds »

ec109685 wrote: post #101
bnr4u wrote: post #71

Having hosted infrastructure in CA at multiple colos. I would advise you to host it elsewhere if you can, cost of power, other infrastructure is much higher in CA than AZ or NV.

Power seems like a very small amount of cost of compute when it comes to GPU’s.

over regulation and taxes

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#113

Post by constantly »

fragmede wrote: post #108
AndrewKemendo wrote: post #103

The billion dollar question is: Who is funding this? Cause if it’s VC then it’s going to have the same fate as everything else after 5-7 years. I hope y’all have as innovative of a business model. You’ll need it if you want to do what you’re doing now for more than a few years

What's wrong with doing something profitable for a few years? H100's in a couple of years will be like having a cluster of K80's today. Not everything has grow to have the appetite of Galactus and swallow a whole planet. Making single digit millions of dollars over a couple of years is still worthwhile, especially if it helps others and moves humanity forwards. This project isn't ever going to want to try and compete…

You’re completely right in everything you say about growing sustainably and making some money over time. But if this project is VC that all goes out the window and it won’t be profitable unless it massively galactus scales to compete with AWS in 5-7 years, and will fail after that almost certainly, like the vast, vast majority of VC projects.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#114

Post by hackernewds »

camhart wrote: post #30
flaque wrote: post #23

Earlier quoted context omitted.

> Your project has a youthful optimism that I hope you won’t lose as you go. And in fact it might be the way to win in the long run. This is the nicest thing anyone has said to us about this. We're gonna frame this and hang it out on our wall. > So whenever someone comes knocking, begging for a tiny slice of your H100s for their harebrained idea, I hope you’ll humor them. Absolutely! :D

Optimism is (almost) always required in order to accomplish anything of significance. Those who lose it, aren't living up to their potential. I'm not encouraging the false belief that everything you do will work out. Instead I'm encouraging the realization that the greatest accomplishments almost always feel like long shots, and require significant amounts of optimism. Fear and pessimism, while helpful in appropriate…

What a beautiful and articulate thought. thank you

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#115

Post by hackernewds »

Frost1x wrote: post #67
camhart wrote: post #30

Earlier quoted context omitted.

Optimism is (almost) always required in order to accomplish anything of significance. Those who lose it, aren't living up to their potential. I'm not encouraging the false belief that everything you do will work out. Instead I'm encouraging the realization that the greatest accomplishments almost always feel like long shots, and require significant amounts of optimism. Fear and pessimism, while helpful in appropriate…

>Optimism is (almost) always required in order to accomplish anything of significance. Those who lose it, aren't living up to their potential. I argue that realism trumps optimism. It's perfectly normal in a realist farming to see something difficult, acknowledge the high risk and failure potential, and still pursue something with intent to succeed. I've personally grown tired of over optimism everywhere because it c…

If everyone were a realist, we wouldn't have half the advances we do. Because what can be "real" is proven wrong through innovation, after all isn't that disruption? :)

Sam Altman talks about this quite frequently, that it's not intelligence or luck necessary for an enduring innovation. It is persistence in the face of inevitability, and a high tolerance for being proven wrong and still persisting

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#116

Post by Frost1x »

rileyphone wrote: post #89
Frost1x wrote: post #67

Earlier quoted context omitted.

>Optimism is (almost) always required in order to accomplish anything of significance. Those who lose it, aren't living up to their potential. I argue that realism trumps optimism. It's perfectly normal in a realist farming to see something difficult, acknowledge the high risk and failure potential, and still pursue something with intent to succeed. I've personally grown tired of over optimism everywhere because it c…

You can be a realist visionary.

Absolutely, which is what I advocate for.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#117

Post by ethbr0 »

mschuster91 wrote: post #64
mdaniel wrote: post #61

Earlier quoted context omitted.

> Requirements > Ubuntu 18.04 or newer (required) > Dedicated machines only - the machine shouldn't be doing other stuff while rented well that's certainly not what I expected. ctrl-f "virtual" gives nothing, so it seems they really mean "take over your machine" > Note: you may need to install python2.7 to run the install script. what kind of nonsense is this? Did they write the script in 2001 and just abandon it?

> what kind of nonsense is this? Did they write the script in 2001 and just abandon it? Anything AI/ML is a hot mess of cobbled-together bits and pieces of Python barely holding together. I recently read somewhere that there should be a new specialization of "ML DevOps Engineer"... and hell I'm supporting that.

Python is awesome because they built what people wanted.

Python is terrible because they built what people wanted.

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#118

Post by AndrewKemendo »

fragmede wrote: post #108
AndrewKemendo wrote: post #103

The billion dollar question is: Who is funding this? Cause if it’s VC then it’s going to have the same fate as everything else after 5-7 years. I hope y’all have as innovative of a business model. You’ll need it if you want to do what you’re doing now for more than a few years

What's wrong with doing something profitable for a few years? H100's in a couple of years will be like having a cluster of K80's today. Not everything has grow to have the appetite of Galactus and swallow a whole planet. Making single digit millions of dollars over a couple of years is still worthwhile, especially if it helps others and moves humanity forwards. This project isn't ever going to want to try and compete…

Hey I agree!

That’s why I’m asking because a “bootstrapped” company like you describe has a future…

One backed by VC doesn’t

I mean they may have a future but not like you describe

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#119

Post by alex_lav »

jiggawatts wrote: post #81
beachy wrote: post #74

Earlier quoted context omitted.

AWS and Azure would slit their own throats before they created a way for their customers to pool instances to save money. They want to do that themselves, and keep the customer relationship and the profits, instead of giving them to a middleman or the customer.

It’s just corporate profits combined with market forces, not a some sort of malicious conspiracy. You can rent a 2-socket AMD server with 120 available cores and RDMA for something like 50c to $2 per hour. That’s just barely above the cost of the electricity and cooling! What do you want, free compute just handed to you out of the goodness of their hearts? There is incredible demand for high-end GPUs right now, and m…

> You can rent a 2-socket AMD server with 120 available cores and RDMA for something like 50c to $2 per hour.

Source required

Re: Show HN: San Francisco Compute – 512 H100s at <$2/hr for research and startups

#120

Post by nl »

sillysaurusx wrote: post #106
zak wrote: post #99

Earlier quoted context omitted.

Actually, the TPU Research Cloud program is still going strong! We've expanded the compute pool significantly to include Cloud TPU v4 Pod slices, and larger projects still use hundreds of chips at a time. (TRC capacity has not been reclaimed for internal use.) Check out this list of recent TRC-supported publications: https://sites.research.google/trc/publications/ Demand for Cloud TPUs is definitely intense, so if yo…

Zak, I love you buddy, but you should have some of your researchers try to use the TRC program. They should pretend to be a nobody (like I was in 2019) and try to do any research with the resources they’re granted. I guarantee you those researchers will all tell you “we can’t start any training runs anymore because the TPUs die after 45 minutes.” This may feel like an anime betrayal, since you basically launched my c…

> You don’t need to take my word for it. Here’s some unfiltered DMs on the subject: https://imgur.com/a/6vqvzXs

> Notice how their optimism dries up, and not because I was telling them how bad TRC has become. It’s because their TPUs kept dying.

Unless I'm misreading this they sound pretty happy and you sound pessimistic? Their last substantial comment was "I'm sure Zak could hook you up with something better"?

Post reply on HN