Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Silurian (YC S24) – Simulate the Earth

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 156 posts

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#121

Post by britannio »

bbor wrote: post #97

Fascinating. I have two quick questions, if you find the time: …we’ve built our own foundation model, GFT (Generative Forecasting Transformer), a 1.5B parameter frontier model that simulates global weather… I’m constantly scolding people for trying to use LLMs for non-linguistic tasks, and thus getting deceptively disappointing results. The quintessential example is arithmetic, which makes me immediately dubious of a…

Andrej Karpathy states that LLMs are highly general purpose technology for statistical modelling of token streams [1]. For example, comma.ai uses transformers in their self-driving model which is far from a linguistic task.

[1] https://x.com/karpathy/status/1835024197506187617 [2] https://www.youtube.com/watch?v=-KMdo9AWJaQ&t=1010s

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#122

Post by hwhwhwhhwhwh »

counters wrote: post #113
hwhwhwhhwhwh wrote: post #98

Earlier quoted context omitted.

Thanks for the explanation. I am still a bit confused how this takes care of the errors? I can see how the weather prediction for tomorrow might have less errors. But shouldn't the errors accumulate as you feed the predicted weather as the input for the model? Wouldn't the results start diverging from reality pretty soon? Isn't that the reason why the current limit is close to 6 days? How exactly does this model fixe…

It doesn't take care of the errors. They still "accumulate" over time, leading to the same divergence that traditional physics-based weather models experience. In fact, the hallmark that these AI models are _doing things right_ is they show realistic modes of error growth when compared with those physics-based models - and there is already early peer-reviewed literature suggesting this is the case. This _class_ of mo…

Thanks for the explanation.

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#123

Post by cbodnar »

bbor wrote: post #97

Fascinating. I have two quick questions, if you find the time: …we’ve built our own foundation model, GFT (Generative Forecasting Transformer), a 1.5B parameter frontier model that simulates global weather… I’m constantly scolding people for trying to use LLMs for non-linguistic tasks, and thus getting deceptively disappointing results. The quintessential example is arithmetic, which makes me immediately dubious of a…

Re question 1: LLMs are already working pretty well for video generation (e.g. see Sora). You can also think of weather as some sort of video generation problem where you have hundreds of channels (one for each variable). So this is not inconsistent with other LLM success stories from other domains.

Re question 2: Simulations don't need to be explainable. Being able to simulate simply means being able to provide a resonable evolution of a system given some potential set of initial conditions and other constraints. Even for physics-based simulations, when run at huge scale like with weather, it's debatable to what degree they are "interpretable".

Thanks for your questions!

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#124

Post by cbodnar »

bschmidt1 wrote: post #47

Wow, so excited for this. I had a web app online in 2020-22 called Skim Day that predicted skimboarding conditions on California beaches that was mostly powered by weather APIs. The tide predictions were solid, but the weather itself was almost never right, especially wind speed. Additionally there were some missing metrics like slope of beach which changes significantly throughout the year and is very important for…

That's pretty cool! Would be great to learn more about your app and how the wave/tide prediction was working. Is there some place to read more about this?

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#126

Post by bbor »

keyboardcaper wrote: post #104
nikhil-shankar wrote: post #49

Earlier quoted context omitted.

If there is sufficient data, we can train on it!

Would geolocated historical seismographic data do?

They’re empiricists — the only ~~real~~ conclusive way to answer that question is to try it, IMO!

The old ML maxim was “don’t expect models to do anything a human expert couldn’t do with access to the same data”, but that’s clearly going to way of Moore’s Law… I don’t think a meteorologist could predict 11km^2 of weather 10 days out very accurately, and I know for sure that a neuroscientists couldn’t recreate someone’s visual field based on fMRI data!

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#127

Post by bbor »

shoyer wrote: post #115

Glad to see that you can make ensemble forecasts of tropical cyclones! This absolutely essential for useful weather forecasts of uncertain events, and I am a little dissapointed by the frequent comparisons (not just you) of ML models to ECMWF's deterministic HRES model. HRES is more of a single realization of plausible weather, rather than an best estimate of "average" weather, so this is a bit of apples vs oranges.…

HN never disappoints, jeez. Thanks for chiming in with some expert context! I highly recommend any meteoronoobs like me to check out the pdf version of the linked paper, the diagrams are top notch — https://www.nature.com/articles/s41586-024-07744-y.pdf

Main takeaway, gives me some hope:

  Our results provide strong evidence for the disputed hypothesis that learning to predict short-term weather is an effective way to tune parameterizations for climate. NeuralGCM models trained on 72-hour forecasts are capable of realistic multi-year simulation. When provided with historical SSTs, they capture essential atmospheric dynamics such as seasonal circulation, monsoons and tropical cyclones.
But I will admit, I clicked the link to answer a more cynical question: why is Google funding a presumably super-expensive team of engineers and meteorologists to work on this without a related product in sight? The answer is both fascinating and boring:

  In recent years, computing has both expanded as a field and grown in its importance to society. Similarly, the research conducted at Google has broadened dramatically, becoming more important than ever to our mission. As such, our research philosophy has become more expansive than the hybrid approach to research we described in our CACM article six years ago and now incorporates a substantial amount of open-ended, long-term research driven more by scientific curiosity than current product needs.
From https://research.google/philosophy/. Talk about a cool job! I hope such programs rode the intimidation-layoff wave somewhat peacefully…

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#128

Post by furiousteabag »

brunosan wrote: post #70
furiousteabag wrote: post #51

Curious to see what other things you will simulate in the future! Shameless plug: recently we've built a demo that allows you to search for objects in San Francisco using natural language. You can look for things like Tesla cars, dry patches, boats, and more. Link: https://demo.bluesight.ai/ We've tried using Clay embeddings but we quickly found out that they perform poorly for similarity search compared to embedding…

howdy! Clay makers here. Can you share more? Did you try Clay v1 or v0.2 What image size embeddings from what instrument? We did try to relate OSM tags to Clay embeddings, but it didn't scale well. We did not give up, but we are re-considering ( https://github.com/Clay-foundation/earth-text ). I think SatClip plus OSM is a better approach. or LLM embeddings mapped to Clay embeddings...

Hey hey! We tried Clay v1 with 768 embeddings size using your tutorials. We then split NAIP SF to chips and indexed them. Afterwards, we performed image-to-image similarity search like in your explorer.

We tried to search for bridges, beaches, tennis courts, etc. It worked, but it didn't work well. The top of the ranking was filled with unrelated objects. We found that similarity scores are stacked together too much (similarity values are between 0.91 and 0.92 with 4 digit difference, ~200k tiles), so the encoder made very little difference between objects.

I believe that Clay can be used with additional fine-tuning for classification and segmentation, but standalone embeddings are pretty poor.

Check this: https://github.com/wangzhecheng/SkyScript. It is a dataset of OSM tags and satellite images. CLIP fine-tuned on that gives good embeddings for text-to-image search as well as image-to-image.

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#129

Post by andai »

dubcanada wrote: post #108
thechao wrote: post #94

Earlier quoted context omitted.

Low-income neighborhoods are good signal indicator for flooding high risk zones. There's a demographic angle, too.

Are you suggesting that flood prevention only happens in higher income neighbourhoods? Flood prevention tends to lie on the county engineers. Not so much private individuals to dictate. Doesn't matter how much money you have, you can't just dig up a road to put in proper flood prevention measures like drainage and grade.

Extreme example, but I saw a video of a "homeless" family in Japan that lived on a flood plain. They lived there because it was the only free spot.

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#130

Post by markstock »

cshimmin wrote: post #37
joshdavham wrote: post #10

> Silurian builds foundation models to simulate the Earth, starting with the weather. What else do you hope to simulate, if this becomes successful?

Do earthquakes next! Signed, A California Resident

The Earth is a multi-physics complex system and OP claiming to "Simulate the Earth" is misleading. Methods that work on the atmosphere may not work on other parts. There are numerous scientific projects working on simulation earthquakes, both using ML and more "traditional" physics.
Post reply on HN