Live data from Hacker News

Launch HN: Silurian (YC S24) – Simulate the Earth

news.ycombinator.com

81–90 of 156 posts

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#81

Earlier quoted context omitted.

We explored several examples from the 2024 hurricane season in our blog post: https://silurian.ai/posts/001/hurricane_tracks . We overlaid the true paths of the hurricane over our predictions for everyone to see!

I'm finding the posts confusing -- is the prediction the images? What exactly is predicted and what is the actual path in those videos?

In the videos the true path is the dashed line and the government prediction is the solid line. Our prediction, from our GFT model, is the animation which plays in the background.

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#82

Earlier quoted context omitted.

They edited the post. Regardless, you're just trying to personally attack me. That's a great use of both our time.

It was rhetorical. And telling you that the answer is not to assume the worst without reading what they are doing is not a personal attack. You are out here implying these guys are a fraud. Being told to pull your head in is not personal.

[deleted]

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#83

> Silurian builds foundation models to simulate the Earth, starting with the weather. What else do you hope to simulate, if this becomes successful?

We want to branch out to industries which are highly dependent on weather. That way we can integrate their data together with our core competency: the weather and climate. Some examples include the energy grid, agriculture, logistics, and defense.

you'll have trouble simulating the grid, but for energy data you might want to look at (or get in touch with) these people: https://app.electricitymaps.com/map

They're a cool little team based in Copenhagen. Would be useful, for example, to look at the correlation between your weather data and regional energy production (solar and wind). Next level would be models to predict national hydro storage, but that is a lot more complex.

My advice is to drop the grid itself to the bottom of the list, and I say this as someone who worked at a national grid operator as the primary grid analyst. You'll never get access to sufficient data, and your model will never be correct. You're better off starting from a national 'adequacy' level and working your way down based on information made available via market operators.

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#84
post #7

> Astonishingly, this approach, done correctly, produces better forecasts than traditional simulations of the physics of our atmosphere. It seems like this is another instance of The Bitter Lesson, no?

For anyone else who's also in today's lucky 10,000: http://www.incompleteideas.net/IncIdeas/BitterLesson.html

Thank you - I hadn't heard of it before. It seems to have parallels with LLMs - our most general intelligent systems have come from producing a workable architecture for what seems to be the bare minimum for communicating intelligence while also having plenty of training data (language), then simply scaling up.

I thought this was a good quote:

> We want AI agents that can discover like we can, not which contain what we have discovered.

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#85
Weather models are chaotic, are ML methods more numerically stable than a physics based simulation? And how do they compare in terms of compute requirements? the Aurora paper seemed to be promising, but I would love a summary of comparison better than what I get out of Claude.

Once upon a time I converted spectral-transform-shallow-water-model (STSWM or parallelized as PSTSWM) from FORTRAN to Verilog. I believe this is the spectral-transform method we have run for the last 30 years to do forecasting. The forecasting would be ~20% different results for 10-day predictions if we truncated each operation to FP64 instead of Intel's FP80.

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#86

Earlier quoted context omitted.

A ripoff of the visualization layer? Even if it was, who cares? That's not the work. What's next, you think a new chess engine is a ripoff because they use a standard chess board for visualization? A new protein prediction model is a ripoff because they use the standard visualization?

There's a long precedent of knockoffs, scams, and skullduggery in silicon valley.

[deleted]

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#87

Exciting idea and seems like a well-proven team. Good luck to you guys here and don't mind the endemic snark in the other threads. A couple basic questions -- 1. How will you handle one-off events like volcanic eruptions for instance? 2. Where do you start with this too? Do you pitch a meteorology team? Is it like a "compare and see for yourself"?

Volcanoes are a tricky one. There are a few volcanic eruptions in historical data, but it's unclear if this is enough to predict reasonably well how such future eruptions (especially at unseen locations) will affect the weather. Would be fun to look at some events and see what the model is doing. Thanks for the suggestion!

Re where do we start. A lot of organisations across different sectors need better weather predictions or simulations that depend on weather. Measuring the skill of such models is a relatively standard procedure and people can check the numbers.

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#88

So ChatGPT has a cutoff date on the stuff it can talk about. This predicting weather sounds like ChatGPT being able to predict next week's news from which it has been trained on. I can see how it can probably predict some stuff like Argentina winning a football match scheduled for next week when played against India given India sucks at football. But can it really give any useful predictions? Like can it predict thin…

Our training cutoff date was the end of 2022. Here's our blogpost on the 2024 hurricane season https://silurian.ai/posts/001/hurricane_tracks

I just don't understand how can your produce new knowledge which it don't have access to. Are you you folks claiming the future weather is a function of previous weather and the model is capable of replicating the function?

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#89

Earlier quoted context omitted.

We want to branch out to industries which are highly dependent on weather. That way we can integrate their data together with our core competency: the weather and climate. Some examples include the energy grid, agriculture, logistics, and defense.

you'll have trouble simulating the grid, but for energy data you might want to look at (or get in touch with) these people: https://app.electricitymaps.com/map They're a cool little team based in Copenhagen. Would be useful, for example, to look at the correlation between your weather data and regional energy production (solar and wind). Next level would be models to predict national hydro storage, but that is a lot…

These are great resources, thank you. If you're open to it, we'd love to meet and chat about the energy space since we're newcomers to that arena. Shoot us an email at contact@silurian.ai

Re: Launch HN: Silurian (YC S24) – Simulate the Earth

#90

Earlier quoted context omitted.

Our training cutoff date was the end of 2022. Here's our blogpost on the 2024 hurricane season https://silurian.ai/posts/001/hurricane_tracks

I just don't understand how can your produce new knowledge which it don't have access to. Are you you folks claiming the future weather is a function of previous weather and the model is capable of replicating the function?

No one is claiming that there is "new knowledge" here.

The entire class of deep learning or AI-based weather models involves a very specific and simple modeling task. You start with a very large training set which is effectively a historical sequence of "4D pictures" of the atmosphere. Here, "4D" means that you have "pixels" for latitude, longitude, altitude, and time. You have many such pictures of these for relevant atmospheric variables like temperature, pressure, winds, etc. These sequences are produced by highly-sophisticated weather models run in what's called a "reanalysis" task, where they consume a vast array of observations and try to create the 4D sequence of pictures that are most consistent with the physics in the weather model and the various observations.

The foundation of AI weather models is taking that 4D picture sequence, and asking the model how to "predict" the next picture in the sequence, given the past 1 or 2 pictures. If you can predict the picture for 6 hours from now, then you can feed that output back into the model and predict the next 6 hours, and so on. AI weather models are trained such that this process is mostly stable, e.g. the small errors you begin to accumulate don't "blow up" the model.

Traditionally, you'd use a physics-based model to accomplish this task. Using the current 3D weather state as your input, you integrate the physics equations forward in time to make the prediction. In many ways, today's AI weather models can be thought of as a black box or emulator that reproduces what those physics-based models do - but without needing to be told much, if any of the underlying physics. Depending on your "flavor" of AI weather model, the architecture of the model might draw some analogies to the underlying physics. For example, NVIDIA's models use Fourier Neural Operators, so you can think of them as learning families of equations which can be combined to approximate the state of the atmosphere (I'm _vastly_ over-simplifying here). Google DeepMind's GraphCast tries to capture both local and non-local relationships between fields through it's graph attention mechanisms. Microsoft Aurora' (and Silurian's, by provenance, assuming it's the same general type of model) try to capture local relationships through sliding windows passed over the input fields.

So again - no new knowledge or physics. Just a surprisingly effective of applying traditional DL/AI tools to a specific problem (weather forecasting) that ends up working quite well in practice.

Post reply on HN