Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GB Renewables Map

renewables-map.robinhawkes.com

Post reply on HN
Print view View original

21–30 of 52 posts

Re: GB Renewables Map

#21

Post by toomuchtodo »

robhawkes wrote: post #19
toomuchtodo wrote: post #17

Very well done, beautiful data communication.

Yup! I have two ways to do it right now, the first is to click the plug-looking button on the map and this'll switch into curtailment mode to filter the assets that are currently being turned down due to transmission constraints, as well as the assets being turned-up elsewhere to balance things out. Also if you click on a curtailed assets you'll see the curtailment as a yellow area on the charts. The other method is…

Thanks for answering my curtailment question! I removed it from my comment as I commented before I found the data in your UX, but the extra context is useful. So much curtailment, much more transmission and battery storage required.

Re: GB Renewables Map

#22

Post by pjc50 »

robhawkes wrote: post #16
martinald wrote: post #6

Hey Robin, big fan of this map, congrats on getting front page on HN :). Two suggestions/questions if you may: 1) Would be good to see how many MW each boundary can handle, not just %? Btw, I can't see the number for the south east england boundary. 2) Great job on the battery info. I'm seeing some battery storage is curtailed. How is that possible? Please don't tell me that we are paying batteries to _not_ export :/…

Hey! And thank you 1) Absolutely agree. The current approach for the boundaries is a quick hack until I can implement something more sophisticated. Safe to say an update is already in the works that adds a MW value and more insight into the state of each boundary (and is also more accurate in general) 2) "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries pa…

> "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries paid to not export (at least today).

Unfortunately you've got at least one negative wrong in this sentence and I'm still confused, and the linked dataset is currently blank?

Sorry for complaining, this is a great website!

Re: GB Renewables Map

#23

Post by arethuza »

robhawkes wrote: post #11
wooger wrote: post #8

Can someone explain what the lines mean?

The squiggly ones across the country? Those are the National Energy System Operator (NESO) transmission boundaries for Great Britain. They represent areas of the country that are tracked for the purposes of managing constraints, which is when the transmission network over the boundary can't handle the amount of power flowing over it. When they are overloaded (or rather, before that happens) NESO will step in and "bal…

I note that one of those lines seems to take a big detour up and round Cruachan?

https://www.visitcruachan.co.uk/

Re: GB Renewables Map

#24

Post by robhawkes »

pjc50 wrote: post #22
robhawkes wrote: post #16

Earlier quoted context omitted.

Hey! And thank you 1) Absolutely agree. The current approach for the boundaries is a quick hack until I can implement something more sophisticated. Safe to say an update is already in the works that adds a MW value and more insight into the state of each boundary (and is also more accurate in general) 2) "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries pa…

> "Please don't tell me that we are paying batteries to _not_ export" – it's actually the opposite, the batteries paid to not export (at least today). Unfortunately you've got at least one negative wrong in this sentence and I'm still confused, and the linked dataset is currently blank? Sorry for complaining, this is a great website!

No worries. The Detailed System Prices dataset is lagged by a couple hours so try going back in time.

The simplest answer I can give is that assets place bids and a volume of energy that they are willing to turn down if the system operator needs to. Those bids are either positive or negative in value and this depends a lot on the type of asset, for example wind assets usually bid negative (ie. we pay them to turn down) while gas assets usually bid positive (ie. they pay us to turn down). The reason for that is a lot to do with complexities of the market and also the cost of running that assets, the cost of fuels, etc.

Re: GB Renewables Map

#25

Post by robhawkes »

arethuza wrote: post #23
robhawkes wrote: post #11

Earlier quoted context omitted.

The squiggly ones across the country? Those are the National Energy System Operator (NESO) transmission boundaries for Great Britain. They represent areas of the country that are tracked for the purposes of managing constraints, which is when the transmission network over the boundary can't handle the amount of power flowing over it. When they are overloaded (or rather, before that happens) NESO will step in and "bal…

I note that one of those lines seems to take a big detour up and round Cruachan? https://www.visitcruachan.co.uk/

The lines are sort of arbitrary in a geographic sense as in reality they are defined based on the intersection with the schematic version of the transmission network [0]. Though yeah, that specific example is an odd one and I'd be interested to understand more about why it was placed in one boundary and not the other.

[0] https://www.neso.energy/data-portal/day-ahead-constraint-flo...

Re: GB Renewables Map

#26

Post by robhawkes »

ksec wrote: post #20
robhawkes wrote: post #9

Oh wow, I made this! I didn't expect to see my map on here Let me know if you have any questions or feedback and I'll do my best to respond to them.

Tech Stack used? Any specific non-technical difficulty getting this done ?

Tech stack is a mix of things, but the big-ticket items:

- Dagster for data pipeline

- Parquet files for data format

- Cloudflare R2 for data storage

- DuckDB for data processing

- NodeJS for API

- NextJS + React for the app

- MapLibre for the base map functionality

- Deck.gl for the complex map layer (eg. wind particles)

- WeatherLayers (+ GFS) for the wind particles

- Cloudflare as a caching layer

The main difficulty was learning about the underlying data and industry nuances. I've been working on this map the past few years and I'm still learning new things that force me to change my approach. It's an incredibly complex domain and it's part of the reason I made this, to try and make this complex data a bit more digestible.

Re: GB Renewables Map

#27

Post by bitdivision »

What are the black dots around the map when you zoom in? For instance South-East Wales / Bristol, I see a few black dots that look like they cover shaded areas or something. They also show up at the sites for wind farms, possibly they're the turbines themselves?

Re: GB Renewables Map

#28

Post by robhawkes »

bitdivision wrote: post #27

What are the black dots around the map when you zoom in? For instance South-East Wales / Bristol, I see a few black dots that look like they cover shaded areas or something. They also show up at the sites for wind farms, possibly they're the turbines themselves?

Yup, that's exactly it! When you zoom in you get to see the wind farms and wind turbines using data from the amazing Open Infrastructure Map [0]. I also show the cables for the offshore wind farms.

[0] https://openinframap.org/

Re: GB Renewables Map

#29

Post by bitdivision »

robhawkes wrote: post #28
bitdivision wrote: post #27

What are the black dots around the map when you zoom in? For instance South-East Wales / Bristol, I see a few black dots that look like they cover shaded areas or something. They also show up at the sites for wind farms, possibly they're the turbines themselves?

Yup, that's exactly it! When you zoom in you get to see the wind farms and wind turbines using data from the amazing Open Infrastructure Map [0]. I also show the cables for the offshore wind farms. [0] https://openinframap.org/

Very cool. Thanks

Re: GB Renewables Map

#30

Post by albumen »

robhawkes wrote: post #25
arethuza wrote: post #23

Earlier quoted context omitted.

I note that one of those lines seems to take a big detour up and round Cruachan? https://www.visitcruachan.co.uk/

The lines are sort of arbitrary in a geographic sense as in reality they are defined based on the intersection with the schematic version of the transmission network [0]. Though yeah, that specific example is an odd one and I'd be interested to understand more about why it was placed in one boundary and not the other. [0] https://www.neso.energy/data-portal/day-ahead-constraint-flo...

Lovely site. As a wind energy ignoramus, it would be great to add descriptions to your FAQ for the red lines, and curtailment.
Post reply on HN