Live data from Hacker News

Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data

news.ycombinator.com

161–170 of 173 posts

Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data

#161

Since you are generating hundreds of terabytes of data every day, can we get some insight into what the data platform is being used here to handle such scale?

That's cumulative for all data collected like satellites, weather radars, and surface observing systems. Soundings are relatively "lightweight" compared to most sets. The major difficulty is pushing the data through our low-power satellite backhaul - in practice we average ~10B/s

Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data

#164

Earlier quoted context omitted.

In reading the documents google's Loon made available before shutting down (and saying that hydrogen airships were the way to go) [ https://storage.googleapis.com/x-prod.appspot.com/files/The%... ], the bottleneck on staying up longer was helium loss. Lots of things, mechanical failure etc, caused flights to end early. But the ultimate limit seems to be either helium loss or the impact of helium absorption on seals e…

Sure there will be leakage. Let's look at the composition vs height: https://en.wikipedia.org/wiki/Atmosphere_of_Earth#/media/Fil... I understand helium will be sparse at low heights, but at each height there is a diversity of species, among which will be a lighter one. Could a balloon be oversized so that instead of using helium, at each height an 80% fill of the locally (that height) lightest gas could keep the bal…

These guys [https://www.scientificballoonsolutions.com/news/], as of October 2018, claim the world record. [edit: the site linked says Alan Adamson but an upstream post mentions Lee Meadows. That thread also credits Bill Brown as a pioneer - https://www.stratoballooning.org/membership#!biz/id/5f4d7b97...]

The world record balloon was launched on September 21, 2016, stayed up over 767 days, circled the world 35 times and traveled over 1.5 million kilometers.

Their website, still from 2018, indicates they are working on even better designs.

Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data

#166
post #17

Very cool! How are the balloons transferring telemetry back to earth for analysis, etc? Asking because my research at the University of Oxford was around hyper space-efficient data transfer from remote locations for a fraction of the price. The result was an award-winning technology ( https://jsonbinpack.sourcemeta.com ) to serialise plain JSON that was proven to be more space-efficient than every tested alternative…

> JSON BinPack is space-efficient, but what about runtime-efficiency? > When transmitting data over the Internet, time is the bottleneck, making computation essentially free in comparison. i thought this was an odd sales pitch from the jsonbinpack site, given that a central use-case is IoT, which frequently runs on batteries or power-constrained environments where there's no such thing as "essentially free"

For sure, but radio transmitter time is almost always much more expensive than CPU time! It’s 4mA-20mA vs 180mA on an esp32; having the radio on is a 160mA load! As long as every seven milliseconds compressing saves a millisecond of transmission, your compression algorithm comes out ahead.

Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data

#167
post #159

Neat :) I'm involved in a few projects focused on vertical profiling in the coastal urban boundary layer. I saw in a response you said the balloons will periodically return to sea level and ascend (which sounds like a fun design challenge by itself.) Will you be doing so near populated areas as well? Good luck!

Very interesting, I'd love to hear more about it! In short, yes, we plan to do descents near urban areas if there's a route where we can go down to a safe height and stay away from any airspace. What cities are you looking at right now?

Most recently Houston had a large ARM field campaign which brought together a lot of atmospheric scientists from around the country. (https://www.arm.gov/research/campaigns/amf2021tracer)

ARM is a DOE program that ships top-tier instrumentation to various sites around the world. Loads of university researchers will follow, and you end up with a massive open source data pool. Houston in particular was focused on Aerosol effects on precipitation in the Coastal-Urban environment. There were loads of balloon launches from sites all over the city during the campaign, from large ozonesondes to the tiny sparv embedded foam cup ones (https://sparvembedded.com/products/windsond)

I'm in NY, and my university NOAA department has a focus on PBL Ozone measurements lately. My work in particular is focused on low cost UAV profiling up to about 150m, with a pipe dream of doing 0-3km.

I'm just a grad student, but if anything there sounds interesting feel free to email and I can try and get you in touch with more knowledgeable people.

Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data

#168
Your product idea is cool, the actual sell-able data to me, from a data scientist perspective, that is actually actionable (corp words!), is nil. I can't even figure out what you actually collect.

Also, your website runs like a dog, and make me not want to find out. It runs like a dog displaying dots ... on a sphere. With like, four (4), other images of note. Seriously, talk to one of these WASM demo people, this is sad for a company that posts here. "Book a Demo" with somebody that can write WebGL. I booked and canceled an appointment on Thu (4) Aug (8) 22, 2024 8:00 on your incredibly vague G. Calendar signup. It's a 42 joke. Learn WebGL.

And while you're at it, stop making websites that write 1000x / $ and "Book a Demo" without even vaguely mentioning cost. Give me a $100,000, I'll give you a $100. This sounds like an excellent business from my perspective. How bout you hand me your bank account, and I'll hand you $100.

Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data

#169

Earlier quoted context omitted.

> JSON BinPack is space-efficient, but what about runtime-efficiency? > When transmitting data over the Internet, time is the bottleneck, making computation essentially free in comparison. i thought this was an odd sales pitch from the jsonbinpack site, given that a central use-case is IoT, which frequently runs on batteries or power-constrained environments where there's no such thing as "essentially free"

For sure, but radio transmitter time is almost always much more expensive than CPU time! It’s 4mA-20mA vs 180mA on an esp32; having the radio on is a 160mA load! As long as every seven milliseconds compressing saves a millisecond of transmission, your compression algorithm comes out ahead.

Sounds like you are pretty familiar with satellite transmission at the hardware level. If so, I would love to chat to get your brains on it. I don't know much of the hardware constraints myself.

Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data

#170
post #17

Very cool! How are the balloons transferring telemetry back to earth for analysis, etc? Asking because my research at the University of Oxford was around hyper space-efficient data transfer from remote locations for a fraction of the price. The result was an award-winning technology ( https://jsonbinpack.sourcemeta.com ) to serialise plain JSON that was proven to be more space-efficient than every tested alternative…

Why this over a compact, data-specific format? JSON feels like an unnecessary limitation for this company's use case. I am having a hard time believing it is more space-efficient than a purpose-built format.

Compared to other serialisation formats, JSON BinPack analyses your data and derives custom encoding rules that are specific to the data at hand given all the context it had on it. That's why on JSON BinPack, the static analysis part is the most complex one by far, and why I'm building so much JSON Schema advanced tooling in https://github.com/sourcemeta/jsontoolkit (i.e. check the huge API surface for JSON Schema in the docs: https://jsontoolkit.sourcemeta.com/group__jsonschema.html)

Of course there is still a lot to do, but the idea being that what you get with JSON BinPack is extremely close to what you would have done for manually encoding your data, except that you don't have to worry about encoding things yourself :) Thus you get the best of both worlds: the nicety of JSON and the space-efficiency of manual encoding.

Post reply on HN