Very cool! Seems like you're leveraging some similar navigational ideas as Google X's Project Loon. Loon was such a good fundamental idea, just a little too soon and not the right use case or business model. Initially operating on a smaller scale and focusing on less power-hungry data acquisition vs high bandwidth two way comms seems like a much more viable plan.
Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
81–90 of 173 posts
Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
#82Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
#83Earlier quoted context omitted.
Yes, this would be really interesting! Are they using VHF frequencies as the actual radiosondes (Vaisala, etc.)?
We do all comms via satellite, including the current GPS location, no comms over VHF. I'm curious how you're tracking down a traditional radiosonde though, do you actively chase while it's in the air and then use visual reference as it comes down?
You can use the data collected from hobbyist ground stations and displayed at https://sondehub.org/ to track them while in the air.
Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
#84Very cool! How are the balloons transferring telemetry back to earth for analysis, etc? Asking because my research at the University of Oxford was around hyper space-efficient data transfer from remote locations for a fraction of the price. The result was an award-winning technology ( https://jsonbinpack.sourcemeta.com ) to serialise plain JSON that was proven to be more space-efficient than every tested alternative…
Sounds cool. How does it differ from CBOR?
CBOR is a schema-less binary format. JSON BinPack supports both schema-less (like CBOR) and schema-driven (with JSON Schema) modes. Even on the schema-less mode, JSON BinPack is more space-efficient than CBOR. See https://benchmark.sourcemeta.com for a live benchmark and https://arxiv.org/abs/2211.12799 for a more detailed academic benchmark
Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
#85Very cool! How are the balloons transferring telemetry back to earth for analysis, etc? Asking because my research at the University of Oxford was around hyper space-efficient data transfer from remote locations for a fraction of the price. The result was an award-winning technology ( https://jsonbinpack.sourcemeta.com ) to serialise plain JSON that was proven to be more space-efficient than every tested alternative…
This looks promising! One of the important aspects of protocol buffers, avro etc is how they deal with evolving schemas and backwards/forward compatibility. I don't see anything in the docs addressing that. Is it possible for old services to handle new payloads / new services to handle old payloads or do senders and receivers need to be rewritten each time the schema changes?
Good question! Compared to Protocol Buffers and Apache Avro, that each have their own specialised schema languages created by them, for them, JSON BinPack taps into the popular and industry-standard JSON Schema language.
That means that you can use any tooling/approach from the wide JSON Schema ecosystem to manage schema evolution. A popular one from the decentralised systems world is Cambria (https://www.inkandswitch.com/cambria/).
That said, I do recognise that schema evolution tech in the JSON Schema world is not as great as it should be. I'm a TSC member of JSON Schema and a few of us are definitely thinking hard on this problem too and trying to make it even better that the competition.
Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
#86So the equivalent of these balloons in oceanography are called ARGO floats, which similarly cannot be driven laterally but can control their own depth like a submarine. So far millions of timeseries have been collected across the world ocean using these floats. https://argo.ucsd.edu/ One difference though is that the ARGO floats are unfortunately not recycled, and just wash up on various beaches. (I'm curious whether…
This is super interesting, I'd never come across ARGO before. Data assimilation is a similar problem for our data, and there currently exist systems for assimilating weather balloon observations into gridded reanalysis data ( https://www2.mmm.ucar.edu/wrf/users/ ). One thing we believe, however, is that the reanalysis step in weather forecasting is unnecessary in the long term, and that future (ML) weather models wil…
> One thing we believe, however, is that the reanalysis step in weather forecasting is unnecessary in the long term, and that future (ML) weather models will eventually opt to generate predictions based on un-assimilated raw data and will get better results in doing so.
The idea that we'll be able to run ML weather models using "raw" observations and skip or implicitly incorporate an assimilation is spot-on - there's been an enormous shift in the AI-weather community over the past year to acknowledge that this is coming, and very soon.
But... in your launch announcement you seem to imply that you're already using your data for building and running these types of models. Can you clarify how you're actually going to be using your data over the next 12-24 months while this next-generation AI approach matures? Are you just doing traditional assimilation with NWP?
Also, to the point about reanalysis - that's almost certainly not correct. There are massive avenues of scientific research which rely on a fully-assimilated and reconciled, corrected, consistent analysis of atmospheric conditions. AI models in the form of foundation models or embeddings might provide new pathways to build reanalysis products, but they are a vital and critical tool and will likely be so for the foreseeable future.
Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
#87Very cool! Seems like you're leveraging some similar navigational ideas as Google X's Project Loon. Loon was such a good fundamental idea, just a little too soon and not the right use case or business model. Initially operating on a smaller scale and focusing on less power-hungry data acquisition vs high bandwidth two way comms seems like a much more viable plan.
What we're doing wouldn't be possible without the work the Loon team did (and published)
Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
#88To a layperson like me, could you explain how these balloons will be cleaned up / collected after their life? What material are they made up of?
Sure thing! They're made of about 300 grams of polyethylene. Towards the end of their lifespan, we can steer them to an area that's easy for us to drive out and pick them up. The payload has a GPS, which lets us track where they are both in the sky and on the ground. Right now, most weather balloons fall back to Earth and stay where they land unless someone happens across them (since they can't be controlled and only…
How do you control the altitude? I would imagine 'heat/cool the air inside the baloon', but this would be too energy intensive?
Congratulations for a great non-saas market and product!
Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
#89Earlier quoted context omitted.
I believe it is along the line of... Pull up https://www.pivotalweather.com/model.php?m=nam&p=sfct-mean-i... and pick some point (note the 'click for point sounding'). You can see the wind direction at that location as a function of altitude. Using this as a vector field, you can do "the balloon is here now, 30 minutes from now it will be there, if it is at altitude Z at that time, it will be follow the wind in this…
Pretty much this. We add the data the balloons themselves are collecting to make things more precise as well
Digging into it a little bit more...
The Balloon Learning Environment https://research.google/blog/the-balloon-learning-environmen... (https://news.ycombinator.com/item?id=31155137 - 73 points | 10 comments)
(2016) Station-keeping of a high-altitude balloon with electric propulsion and wireless power transmission: A concept study https://www.sciencedirect.com/science/article/abs/pii/S00945...
(2022) Station-keeping for high-altitude balloon with reinforcement learning - https://www.sciencedirect.com/science/article/abs/pii/S02731...
(2023) Resource-Constrained Station-Keeping for Helium Balloons using Reinforcement Learning - https://arxiv.org/abs/2303.01173
Chasing the citations from those papers to previous works can provide a fairly deep rabbit hole of things to read.
Re: Launch HN: Sorcerer (YC S24) – Weather balloons that collect more data
#90Very cool! How are the balloons transferring telemetry back to earth for analysis, etc? Asking because my research at the University of Oxford was around hyper space-efficient data transfer from remote locations for a fraction of the price. The result was an award-winning technology ( https://jsonbinpack.sourcemeta.com ) to serialise plain JSON that was proven to be more space-efficient than every tested alternative…
> JSON BinPack is space-efficient, but what about runtime-efficiency? > When transmitting data over the Internet, time is the bottleneck, making computation essentially free in comparison. i thought this was an odd sales pitch from the jsonbinpack site, given that a central use-case is IoT, which frequently runs on batteries or power-constrained environments where there's no such thing as "essentially free"
> batteries or power-constrained environments
I would imagine that CPUs are much more efficient than a satellite transmitter, probably? I guess you'd have to balance the additional computational energy required vs. the savings in energy from less transmitting.