Viewing profile — rabernat
rabernat
HN member- Joined
- Mon, Feb 15, 2016, 4:23 PM UTC
- HN karma
- 30
- Public activity
- 6 items
- HN profile
- View on Hacker News ↗
About rabernat
- CEO and co-founder of Earthmover: https://earthmover.io/ - Associate professor in the Columbia University Department of Earth and Environmental Science and Lamont Doherty Earth Observatory: https://ocean-transport.github.io/ - Co-founder and community leader of the Pangeo Project: https://pangeo.io/
Recent public activity
-
comment
Comment #40056515
> It's possible but not very cost-effective to maintain separately-chunked versions of these large geospatial datasets. Like all things in tech, it's about tradeoffs. S3 storage co…
-
comment
Comment #40055303
True, but in fact, the Google ERA5 public data suffers from the exact chunking problem described in the post: it's optimized for spatial queries, not timeseries queries. I just ran…
-
comment
Comment #40055220
Great post! Hi Ali! I think what's missing here is an analysis of what is gained by moving the weather data into a RDBMS. The motivation is to speed up queries. But what's the base…
-
comment
Comment #34314319
The Zarr format is used in some genomics workflows (see https://github.com/zarr-developers/community/issues/19 ) and supports a wide range of modern compressors (e.g. Zstd, Zlib, B…
-
comment
Comment #34008113
PyTorch and JAX are used heavily in climate science on the ML side. For more general analytics, not so much. Many of our users like to use Xarray as a high-level API. There has bee…
-
comment
Comment #34007598
Agree 100%. This is big part of the motivation behind our new startup Earthmover: https://earthmover.io/ Our mission is to make it easier to work with scientific data at scale in t…