472 pages. That's going to be a nice bit of reading this weekend. It is very nice to see such a comprehensive report as well as the fact that it was made public immediately.
Maybe practical engineering will make a video about it
Entso-E final report on Iberian 2025 blackout
51–60 of 119 posts
Re: Entso-E final report on Iberian 2025 blackout
#52The fact that there is not a single root cause but several ones makes me instinctively think this is a good report, because it's not what the "bosses" (and even less politicians) like to hear.
They need more battery storage for grid health, both colocated at solar PV generators (to buffer voltage and frequency anomalies) and spread throughout the grid. This replaces inertia and other grid services provided by spinning thermal generators. There was no market mechanism to encourage the deployment of this technology in concert with Spain’s rapid deployment of solar and wind.
One problem that happened here is the _voltage_ spikes as the synchronous generation went away. Voltage _spikes_ on generation going away seem insane, but it's a real phenomenon.
The problem is that the line itself is a giant capacitor. It's charged to the maximum voltage on each cycle. Normally the grid loads immediately pulls that voltage down, and rotating loads are especially useful because they "resist" the rising (or falling) voltage.
So when the rotating loads went away, nothing was preventing the voltage from rising. And it looks like the sections of the grid started working as good old boost converters on a very large scale.
Re: Entso-E final report on Iberian 2025 blackout
#53Can’t read all of this since it’s 424 pages but i want to point out that Australia is beating Europe on grid connected storage. Not on a per capita basis. It’s beating all of Europe combined outright https://www.visualcapitalist.com/top-20-countries-by-battery... We did have many many problems previously. The state of South Australia went out for a couple of weeks at one point in similar cascading failures. This does…
> Australia elected a government that listened to reports battery+solar is great for grid reliability and nuclear was always going to be more expensive The report you mean (csiro) was wildly biased though. They based their nuclear power cost estimate on a nuclear reactor that was never deployed anywhere (Nuscale) instead of "normal" nuclear power plants that have been deployed for decades.
Was the Nuscale cost estimate somehow worse than AP1000 or EPR(2)? That seems very unlikely to me given the history of those programs.
Re: Entso-E final report on Iberian 2025 blackout
#540.63 Hz and 0.2 Hz grid instability. Oh my.
Re: Entso-E final report on Iberian 2025 blackout
#55Earlier quoted context omitted.
The hack thing spread wildly, indeed. Weird experience.
In Germany a few months prior saw CCC publishing a method for destabilizing energy grids using radio waves a cheap hardware: https://media.ccc.de/v/38c3-blinkencity-radio-controlling-st... and presented an attack vector to which most infrastructure in Europe is exposed. About 4 hours before the grid collapse on the 28th of April 2025 was recorded the largest purchase of Monero in the past 3 years (to remember: monero…
Well, if that's really the cause, then thanks CCC, I guess. For such a serious vulnerability which is probably non-trivial (not to mention expensive) to patch, is it really responsible to give only 3.5 months of time before disclosing it (according to slide #56 https://cdn.prod.website-files.com/5f6498c074436c349716e747/..., they notified EFR about the vulnerability on 2024-09-12 and disclosed it on 2024-12-28)?
Re: Entso-E final report on Iberian 2025 blackout
#56The fact that there is not a single root cause but several ones makes me instinctively think this is a good report, because it's not what the "bosses" (and even less politicians) like to hear.
Frequently, when you see these massive failures, the root cause is an alignment of small weaknesses that all come together on a specific day. See, for instance, the space shuttle O-ring incident, Three-Mile Island, Fukushima, etc. These are complex systems with lots of moving parts and lots of (sometimes independent) people managing them. In a sense, the complexity it the common root cause.
This is the same thing that happened with the 35W bridge collapse in Minneapolis. The gusset plates after the disaster were examined and found to be only 1/2" thick when the original design called for them to actually be 1" thick. The bridge was a ticking time bomb since the day it was built in 1967.
As the years went on, the bridge's weight capacity was slowly eroded by subsequent construction projects like adding thicker concrete deck overlays, concrete median barriers and additional guard rail and other safety improvements. This was the second issue, lining up with the first issue of thinner gusset plates.
The third issue that lined up with the other two was the day of the bridges failure. There were approximately 300 tons of construction materials and heavy machinery parked on two adjacent closed lanes. Add in the additional weight of cars during rush hour when traffic moved the slowest and the bridge was a part of a bottleneck coming out of the city. That was the last straw and when the gusset plates finally gave way, creating a near instantaneous collapse.
Re: Entso-E final report on Iberian 2025 blackout
#57Earlier quoted context omitted.
Yep, sounds like "This was bound to happen at some point"
Which on some level is exactly "what the bosses and politicians want to hear" When it's everybody's fault it's nobody's fault.
But EU's liberalized energy market gives us resiliency and low prices for electricity! /s
Re: Entso-E final report on Iberian 2025 blackout
#58Can’t read all of this since it’s 424 pages but i want to point out that Australia is beating Europe on grid connected storage. Not on a per capita basis. It’s beating all of Europe combined outright https://www.visualcapitalist.com/top-20-countries-by-battery... We did have many many problems previously. The state of South Australia went out for a couple of weeks at one point in similar cascading failures. This does…
> Australia elected a government that listened to reports battery+solar is great for grid reliability and nuclear was always going to be more expensive The report you mean (csiro) was wildly biased though. They based their nuclear power cost estimate on a nuclear reactor that was never deployed anywhere (Nuscale) instead of "normal" nuclear power plants that have been deployed for decades.
The NuScale cost was what the project itself announced. And they hadn’t even started building yet. The latest reports also include large scale nuclear power.
I find it funny when people get outraged because all CSIRO does is use real world construction costs easily proving how unfathomably expensive new built nuclear power is.
Re: Entso-E final report on Iberian 2025 blackout
#59Earlier quoted context omitted.
Frequently, when you see these massive failures, the root cause is an alignment of small weaknesses that all come together on a specific day. See, for instance, the space shuttle O-ring incident, Three-Mile Island, Fukushima, etc. These are complex systems with lots of moving parts and lots of (sometimes independent) people managing them. In a sense, the complexity it the common root cause.
> See, for instance, the space shuttle O-ring incident That wasn't really a result of an alignment of small weaknesses though. One of the reasons that whole thing was of particular interest was Feynman's withering appendix to the report where he pointed out that the management team wasn't listening to the engineering assessments of the safety of the venture and were making judgement calls like claiming that a compone…
I’m not sure why you think those are not a confluence of smaller events or that something outside the design spec isn’t one of those factors. By “small,” I don’t mean trivial. I mean an event that by itself wouldn’t necessarily result in disaster. Perhaps I should have said “smaller” rather than “small.” With the O-rings, the cold and the pressure to launch on that particular day all created the confluence. With Fukushima, the earthquake knocked out main power for primary cooling. That would have been manageable except then the backup generators got destroyed by the tsunami. It was not a case of just a big earthquake, whether outside or inside the design spec, making the reactor building fall down and then radiation being released.
Re: Entso-E final report on Iberian 2025 blackout
#60Earlier quoted context omitted.
Frequently, when you see these massive failures, the root cause is an alignment of small weaknesses that all come together on a specific day. See, for instance, the space shuttle O-ring incident, Three-Mile Island, Fukushima, etc. These are complex systems with lots of moving parts and lots of (sometimes independent) people managing them. In a sense, the complexity it the common root cause.
It usually starts with a broken coffee machine.
When that happens, get ready.