UC Berkeley launches SkyPilot to help navigate soaring cloud costs • Hacker News Board

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#111

Post by m2fkxy » Wed, Dec 14, 2022, 8:38 AM UTC

ritwikgupta wrote: post #110

m2fkxy wrote: post #85

Earlier quoted context omitted.

do you use SAR?

We do not currently use SAR! High-resolution SAR is currently very hard to find. Vendors like Capella Space are pretty pricey for HADR operations, though they do provide some data for deeply discounted/free prices for major disasters. Beyond cost, SAR is just a deeply different form of imaging. The underlying physics that define a SAR image are so different from EO that many computer vision methods, in my opinion, wi…

Thanks for the writeup! Agree on SAR being vastly different to interpret, and I can imagine the priciness. I also understand that the capacity/demand is another complicating factor now over Ukraine.

As you explained SAR phenomenology is not straight forward at all (pun intended). If I could suggest an option to explore a SAR-based damage proxy indicator, it would be interferometric coherence. This suggests however that you get your hands on InSAR data, both timely (to avoid decorrelation, which can happen over the span of a few days) and at good resolution to get relevant information at building-level.

As you can imagine this is not easy nor cheap to get. I am unaware if Capella is InSAR-capable; COSMO-SkyMed and TerraSAR/TanDEM/PAZ are the usual suspects when it comes to SAR constellations with InSAR capability and good frozen orbits. ICEYE I believe is trying to get there, but things are evolving rapidly. Sentinel-1 is a wonderful mission with solid InSAR capability, but I am not sure C-band delivers the resolution you need. Maybe NISAR is worth keeping an eye on in the future?

Fascinating work that matters, good luck with it!

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#112

Post by kamikaze675 » Wed, Dec 14, 2022, 9:04 AM UTC

There are several 'web3' projects in this space, vast.ai, already mentioned, is focused on renting out GPU's from servers hosted pretty much anywhere.

For low cost server hosting and egress bandwidth, Threefold (https://www.threefold.io/build/) and Akash are two new startup web3 ecosystems looking to radically bring down the costs of cloud compute, storage, and bandwidth with GPU support on both of their roadmaps. Still very early, but at 80-95% raw cloud cost savings even beating out Hetzner/DO, its an interesting niche of token incentivized physical infrastructure networks (called TIPIN for short).

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#113

Post by ritwikgupta » Wed, Dec 14, 2022, 2:36 PM UTC

m2fkxy wrote: post #111

ritwikgupta wrote: post #110

Earlier quoted context omitted.

We do not currently use SAR! High-resolution SAR is currently very hard to find. Vendors like Capella Space are pretty pricey for HADR operations, though they do provide some data for deeply discounted/free prices for major disasters. Beyond cost, SAR is just a deeply different form of imaging. The underlying physics that define a SAR image are so different from EO that many computer vision methods, in my opinion, wi…

Thanks for the writeup! Agree on SAR being vastly different to interpret, and I can imagine the priciness. I also understand that the capacity/demand is another complicating factor now over Ukraine. As you explained SAR phenomenology is not straight forward at all (pun intended). If I could suggest an option to explore a SAR-based damage proxy indicator, it would be interferometric coherence. This suggests however th…

Sentinel-1 is the current workhorse for all sorts of InSAR things, but as you mentioned, the resolution is pretty meh for things like damage assessment. Beyond InSAR, people have been doing change detection on GRDs to get damage proxy maps, but that provides areal information rather than instance.

An opportunity in this space is to build truly complex-valued neural networks to fully exploit SLCs and other phase-based information.

Thanks for thinking well of the work! We've been scaling out impact with AI + SAR over the last year. Some work you may be interested in is our work on detecting dark vessels from SAR which has led to the interdiction of illegal fishermen and human traffickers! [0][1]

[0] https://openreview.net/forum?id=PfyWdxM-S4N

[1] https://iuu.xview.us/

Re: UC Berkeley launches SkyPilot to help navigate soaring cloud costs

#114

Post by LilBytes » Wed, Dec 14, 2022, 7:05 PM UTC

quickthrower2 wrote: post #52

LilBytes wrote: post #17

Earlier quoted context omitted.

Same here. At massive scale, sure, it might make sense to run your own infrastructure and data centres. Some may even run their own private cloud and get the benefits we see from the public cloud today from their own IT depts. The vast majority of us however had nothing in common with the above. Like you we ran the hardware, and to the business, we were never more than a cost centre and anything pitched to the broade…

I am more the other way. Even the big clouds are too “serverful” for me, and for very small things a modern PaaS is much easier to deal with.

Admittedly I've not worked in many environments that could ever be described as being anything less than 'enterprise', and within orders of magnitude larger than medium/large businesses.

If you had a niche market and only needed a small footprint. I can see a handful of VPS' or similar being suitable.

But once you get to the 200+ people mark, it gets messy. And gets much worse at 2,000 head. And worse yet at 20,000.