Live data from Hacker News

2025 Iberia Blackout Report [pdf]

media.licdn.com

131–140 of 149 posts

Re: 2025 Iberia Blackout Report [pdf]

#131
post #5

It's a difficult read. Cybersecurity and digital systems was not the issue but gets thirteen pages of proposed measures. I feel this could have been left out. Electric System Operation was the issue and gets seven pages of proposed measures.

>I feel this could have been left out.

It's pretty much their one and only chance to warn the authorities that there's a risk, so if they choose to ignore it, well, nobody can claim they weren't informed.

Re: 2025 Iberia Blackout Report [pdf]

#132
post #104

Earlier quoted context omitted.

I’m a bit mystified as to how the grid controls voltage at all. The non-renewable plants follow the rules here: https://www.boe.es/buscar/doc.php?id=BOE-A-2000-5204 7.1(b) seems to be saying that generators connected at 200kV adjust their reactive power generation/absorption in real time according to the voltage they observe, based on a lookup table provided by the grid operator. This seems sort of sensible according…

I don't claim to know the details of reactive power management, but the primary mechanisms for grid stability in the EU is the "cascade" of services the TSOs procures: - Fast Frequency Response (FFR), sub-second power adjustment following frequency table - Frequency Containment Reserve (FCR), ~second power adjustment following frequency table - Automatic Frequency Restoration Reserve (aFRR), ~second energy production…

It likes to me like a major factor was that the grid failed to control voltage, not frequency. Frequency control should be unaffected by transformers.

Re: 2025 Iberia Blackout Report [pdf]

#133
post #126

Earlier quoted context omitted.

I don't claim to know the details of reactive power management, but the primary mechanisms for grid stability in the EU is the "cascade" of services the TSOs procures: - Fast Frequency Response (FFR), sub-second power adjustment following frequency table - Frequency Containment Reserve (FCR), ~second power adjustment following frequency table - Automatic Frequency Restoration Reserve (aFRR), ~second energy production…

> 9 separate synchronous plants that had sold aFRR(?) to the TSO then failed to deliver, so when the TSO algorithms tried to adjust the oscillations, nothing happened. Oof. This sounds like a classic of "it's only needed in emergencies, so it's only in emergencies that we find out it doesn't work".

I don't know about the Spanish market, but at least in the markets I'm involved in aFRR is an "always on" product, the TSO controls your plant with a setpoint that updates in near-real-time throughout the period you've sold to them.. it's not clear to me that the product that wasn't delivered was actually aFRR though, maybe it was something else less frequently called upon.

Re: 2025 Iberia Blackout Report [pdf]

#134
post #9

Earlier quoted context omitted.

Temporary negative prices have been caused by the renewable generation which exceeded the grid demand at the time, which then evolved into the nasty feedback loop caused by the reaction of renewable generation to those conditions. You simply do not get such situation with traditional generation, it's the direct consequence of the intermittent nature of renewables and its high ratio in the total generation. Also, have…

Where is the market for someone to get paid to pump water into a reservoir and let it fall down later for $$$?

Partly the neighboring country. Between Spain and Portugal there's 10 GW (~20% or peak load) and 100 GWh of storage (enough for a day's worth of consumption).

But that takes time and requires some rebalancing, because much of that capacity is not closest to the producers. It also requires water, which becomes scarcer in the autumn (not the case here).

So the price can and does become negative for a window. “High frequency trading” the spot prices probably contributed to the problem.

Re: 2025 Iberia Blackout Report [pdf]

#135
post #97

Earlier quoted context omitted.

a couple of things to add to an excellent explanation. I'm glad people are coming around to accepting that renewable energy has problems. We have some solutions to these problems but we do not have experience with them. I agree entirely - the externalities of renewable energy are significant and are not paid for by the source of the problem - the renewable generators themselves. Just as one example, what is the solut…

> There is also the reluctance of grid operators to use the capacity available in renewable energy generators. The majority of wind turbines are capable of active and reactive power control but most grid operators either don't use this capacity or use it minimally. I wonder how much is the near-complete inability for grid operators to communicate with smaller systems. My little solar inverter is capable of reactive p…

grid operators can and do communicate with smaller systems - UK Power Networks is one example. If the number of devices is too large, the solution is a market, with aggregators taking on the challenge of quantity.

This is all very much possible and the tech to do it is relatively basic. Grid operators do not because the market rules were written by larger generators to favour those larger generators.

Re: 2025 Iberia Blackout Report [pdf]

#136
post #102

Earlier quoted context omitted.

Because the only reason that solar is still generating power at negative prices is because they are getting subsidies to do it. Otherwise they would disconnect themselves apart from in extreme scenarios (control system down or something), and I bet when you are paying many tens of thousands of euro an hour you'd get someone to fix/manually turn it off sharpish. Looks like there are a multitude of schemes of various v…

> Because the only reason that solar is still generating power at negative prices is because they are getting subsidies to do it. This massively simplifies reality. E.g. in Finland where I live we also have issues with negative power prices. A few years ago we had some really low prices. It turns out, a fair bit of wind power producers never opted to add to their windmills any remote shutdown possibility, nor did the…

I used to design wind turbines for a living. Any wind turbine made in the last fifteen years has the capability for remote shutdown. This is not an option, this is basic functionality.

Please can you share a source that explains your info?

Re: 2025 Iberia Blackout Report [pdf]

#137
post #120

Earlier quoted context omitted.

please can you explain what doesn't seem right?

They admit they had plants offline when they should be available and then they say it wouldn't matter and the grid would've collapsed anyway..? Either they're in a very very bad spot and stop short of saying it outright in the report or it would've averted the disaster if the not-spinning mass was spinning as it should have.

rotational inertia has nothing to do with voltage control.

Re: 2025 Iberia Blackout Report [pdf]

#138
post #124
post #104

Earlier quoted context omitted.

I’m a bit mystified as to how the grid controls voltage at all. The non-renewable plants follow the rules here: https://www.boe.es/buscar/doc.php?id=BOE-A-2000-5204 7.1(b) seems to be saying that generators connected at 200kV adjust their reactive power generation/absorption in real time according to the voltage they observe, based on a lookup table provided by the grid operator. This seems sort of sensible according…

The automatic voltage regulator (AVR) of the generator and the on-load tap changer (OLTC) operate on different timescales (AVR = quick; OLTC = slow). OLTCs are typically set up with a voltage deadband and a time delay to prevent 'hunting' (i.e., repeatedly tapping up and down).

At the end of the day, reactive power in and reactive power out need to add up to zero (keeping in mind that many components can add or remove reactive power and that reactive energy is not conserved the way that real energy is) or, equivalently, that the grid voltage needs to be in range. It’s not sufficient to merely keep the higher frequency components of the voltage in range.

So if the grid wants to be at 400kV, and achieving 400kV under particular generation conditions requires 1500MVar of reactive power absorption by the grid (I made up that number), and the grid operator is relying on 220kV conventional generators to collectively have 1000Mvar of absorption available under said conditions, then something needs to communicate that need to those generators so that they actually absorb those 1000Mvar. And if the OLTCs fool the control algorithm into causing those generators to absorb only 400Mvar, then there’s a mismatch, and that mismatch doesn’t go away because the OLTCs are supposed to be slow.

If, as the writeup seems to suggest, the grid design also requires the OLTCs to operate quickly under large voltage fluctuations because the secondary side cannot tolerate the same fractional voltage swing that the primary side is specified to tolerated, then I would not want to be the person signing off on the grid being stable. (Writing the simulator could be fun, though!). Maybe the idea is that, if the primary voltage is stable at 10% above nominal, then the OLTCs are intended to be stable at a position that holds the secondary at 5% above nominal, and that in turn is intended to result in the correct amount of reactive power absorption?

If I were designing this thing from scratch, I would want an actual communication channel by which facilities that can adjust their reactive power can be commanded to do so independently of the voltage at the point at which they’re connected. And I would want a carefully considered decentralized algorithm to use these controls which, as a first pass, would take input from the primary side at the relevant substations. And then I would want to extend a similar protocol to most or all of the little solar generators at customer sites (not to mention the larger solar facilities that don’t dynamically control reactive power at all in Spain) because they, collectively, can quickly supply or absorb large amounts of reactive power on demand. (Large facilities would use fiber. Small facilities would use digital signals over the power lines or, maybe, grudgingly, the Internet. We really don’t want a situation where the grid cannot start up without customer sites having Internet access.)

Or I would dream of a grid that’s primarily DC with AC islands where the DC portions don’t care about reactive power or frequency at all and merely need to control voltage and power flow.

Re: 2025 Iberia Blackout Report [pdf]

#139
post #138
post #124

Earlier quoted context omitted.

The automatic voltage regulator (AVR) of the generator and the on-load tap changer (OLTC) operate on different timescales (AVR = quick; OLTC = slow). OLTCs are typically set up with a voltage deadband and a time delay to prevent 'hunting' (i.e., repeatedly tapping up and down).

At the end of the day, reactive power in and reactive power out need to add up to zero (keeping in mind that many components can add or remove reactive power and that reactive energy is not conserved the way that real energy is) or, equivalently, that the grid voltage needs to be in range. It’s not sufficient to merely keep the higher frequency components of the voltage in range. So if the grid wants to be at 400kV,…

What you're proposing regarding communication/control channels is already present: the system operator has SCADA links to transmission connected generators + an energy management system to help them control their system.

See Part C here (e.g., SOs having the ability to control generator setpoints): https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:...

Re: 2025 Iberia Blackout Report [pdf]

#140
post #35

Seems the money shot starts at page 131: > The ultimate cause of the peninsular electrical zero on April 28th was a phenomenon of overvoltages in the form of a "chain reaction" in which high voltages cause generation disconnections, which in turn causes new increases in voltage and thus new disconnections, and so on. > 1. The system showed insufficient dynamic voltage control capabilities sufficient to maintain stabl…

I’ve just read the whole shebang. While the overall reason for the mass failure you cite is correct - a cascading failure - the interesting bit here are the oscillations that lead to it. It looks very much like this was driven by algorithmic volatility trading of electricity spots - overproduction, price goes negative, buys placed, production ramps in response to rising price, price rises, sells placed, production fa…

> It looks very much like this was driven by algorithmic volatility trading of electricity spots

Yes, + less "reactive power stations" than expected was available (seems the day some unexpectedly went offline, and not enough safeguards/communication to realize this) + a switch between the French import/export that happened at the same time, leading to the overvoltage issue, which then spiraled.

As far as I read the report, there were multiple causes, not a single one like "algorithmic volatility trading of electricity spots" but a combination of the issues where one-by-one, things would have been fine but all together? Shit broke

Post reply on HN