Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Widespread power outage in Spain and Portugal

bbc.com

Post reply on HN
Print view View original

721–730 of 1001 posts

Re: Widespread power outage in Spain and Portugal

#721

Post by pyrale »

ethbr1 wrote: post #407
martinald wrote: post #288

Earlier quoted context omitted.

Yes, something similar happened in the UK a while back. Full info here: https://www.ofgem.gov.uk/sites/default/files/docs/2020/01/9_... Key points that started it were (you can see the chain of events in the doc): 2.4.1. At 16:52:33 on Friday 9 August 2019, a lightning strike caused a fault on the Eaton Socon – Wymondley 400kV line. This is not unusual and was rectified within 80 milliseconds (ms) 2.4.2. The fault af…

Curious question for someone familiar with power at grid-scale -- How granular is load shedding? And how is this measured / tracked? In my head, I'm thinking of generators/plants, connected by some number of lines, to some amount of load, where there are limited disconnection points on the lines. So how do grid operators know what amount of load will be cut if they disconnect point A123 (and the demand behind it) vs…

In my area, "buckets" of average load are prepared ahead of time in coordination between the transporter (who gives the order to shed), the distributer (who implements the shedding operation as close to consumers as possible), and representatives of the state (who define priority areas that can't be load-shed, or that should be load-shed last).

When yhe time comes, you just shed enough "buckets" to stabilize. Load shedding is not a precise task, when you're at that point, you'd rather load a few more megawatt and be safe, than play with the limits and be sorry.

> So how do grid operators know what amount of load will be cut if they disconnect point A123

Opening a line within the grid isn't used to shed load, as the grid is mainly redundant. It's used 1) to protect the line itself (either by letting it trip, or by opening it preventively), or 2) to force power to flow differently through the network, by modifying its impedance. Opening a line from the transport network is not a way to load shed.

In this totally random example [1], opening the line A-B increases impedance in the right part of the network, forcing power to re-route through the left part, and reduces the load on another line in the right path that was overloaded.

[1]: https://excalidraw.com/#json=5l8OS96Wdke6l9YEClQt8,NJ-r2PtiE...

> Or is there continual measurement? (e.g. there's XYZ MW of load behind A123 as of 2:36pm)

The network is, indeed, monitored continually, and we do have potential network equipments failure simulations every few minutes, with contingency plans also simulated to make sure that they work in that particular case. This way, when shit hits the fan, we at least have a recently tested plan to start from.

Re: Widespread power outage in Spain and Portugal

#722

Post by alexey-salmin »

wongarsu wrote: post #589
oersted wrote: post #560

Earlier quoted context omitted.

I'm seeing some reports saying that a significant frequency oscillation happened, which triggered automatic shut-downs, which cascaded. Could an event like this have that effect? I suppose it makes sense that it was an automatic shutdown rather than infrastructure failing on such a wide area. And then once it's shut down, a black-start is a logistical challenge as other comments have explained. I'm also seeing some r…

The grid is supposed to tolerate any single failure, even under full load. Of course sometimes the first failure is a fire or equipment malfunction and the second failure is a planning failure or someone pressing the wrong button.

Cascade failures are common. The weakest link fails, load gets re-distributed evenly, the second-weakest link fails and so on.

In theory [a flawed one] you've had enough spare capacity to survive N failures and N+1 failures are statistically unlikely because p^(N+1) is close to zero.

On practice [or with a better theory] you can't multiply probabilities in a grid system because random variables aren't independent. 30% spare capacity can go to -100% in a second.

Re: Widespread power outage in Spain and Portugal

#724

Post by 0xbadcafebee »

lenerdenator wrote: post #610
oilman wrote: post #475

Earlier quoted context omitted.

In another life I worked as an engineer commissioning oil rigs and I’ve seen how tricky even a small-scale black start can be. On a rig, we simulate total power loss and have to hand-crank a tiny air compressor just to start a small emergency generator, which then powers the compressors needed to fire up the big ~7MW main generators. It's a delicate chain reaction — and that's just for one isolated platform. A full g…

I remember talking to my ex's dad about his job, which involved planning refuels of a large nuclear-powered generation station in the Lower Midwest. The words "it's a miracle it works at all" routinely popped up in those conversations, which is... something you don't want to hear about any sort of power generation - especially not nuclear - but it's true. It's a system basically built to produce "common accidents". I…

If people knew how crappy, insecure, and unreliable nuclear computer systems were, there'd be a lot more existential dread about cyber security

Re: Widespread power outage in Spain and Portugal

#725

Post by TOMDM »

rawgabbit wrote: post #615
red_admiral wrote: post #253

I'm going with: never attribute to malice what can be explained by ... an incredibly complex system that can fall over even if no-one's being stupid. I would want very strong evidence before I believe this is an attack. There is precedent for major power outages, a huge majority of which are not malicious: https://en.wikipedia.org/wiki/List_of_major_power_outages I remember the day when the Swiss railway power networ…

Having lived through the Texas electricity fiasco in 2021, I would blame cost cutting, the reckless drive for “efficiency”, and maximizing shareholder value. In Texas, the electric providers cut staff and maintenance to maximize shareholder value. They will not have redundant systems and redundant plants out of the goodness of their hearts. The Texas marketplace actually allowed them in the odd event of an outage to…

Show me the incentive and I'll show you the outcome.

If you want a no fail grid you need to incentivise a no fail grid.

Re: Widespread power outage in Spain and Portugal

#726

Post by rcarmo »

ethbr1 wrote: post #411
rcarmo wrote: post #13

Earlier quoted context omitted.

Yeah, we just told you that via Signal - that’s how we built the networks :) (No relation to the other infamous Signal chat :)) There should be 4-8 hours of battery backup on every site - at least.

Wow! Battery capacity has gotten cheap. It's always fascinated me during disasters how independent telecomm can be. Kudos for all the engineering that went into it! I.e. even when any other conceivable dependency is down, the networks keep running.

Batteries lasted almost 8h but they progressively cut down services until we barely had voice.

Re: Widespread power outage in Spain and Portugal

#728

Post by kccqzy »

lxgr wrote: post #685
toast0 wrote: post #640

Earlier quoted context omitted.

> thanks to telcos being legally obligated to give a shit about reliability. Yeah, they don't need to do that anymore. Around me, enough towers have battery backups that I can count on 2 hours of coverage when utility power goes out (if it goes out late at night or early morning, there's usually coverage until 6-7 am when people start waking up and use up the rest of the power). I don't have a real landline, but the…

> stores would pull out the credit card imprint machines, but those don't work anymore because cards are flat It's the other way around: Cards are flat because a carbon imprint doesn't afford the merchant any payment guarantee by the card issuer anymore anyway. (In other words, the "floor limit" above which cards require electronic authorization is now zero.)

At that time merchants probably still checked the signature on the back of your card against your signature on a bill. These days nobody bat an eye when I use my unsigned card. I guess at that time a matching signature would give the merchant enough confidence to process the card offline.

Re: Widespread power outage in Spain and Portugal

#729

Post by bsder »

Hydrocarb0n wrote: post #658
mlsu wrote: post #466

Earlier quoted context omitted.

Does solar power make this process easier or harder? I know that with thermal plants you have a spinning mass that you have to synchronize, and phase shift is used to assess how hard the plant is working (and whether to trip a disconnect as we see here) But with solar, how is the synchronization provided? In like a giant buck? Or in software somehow? Does the phase shift matter as much as in the electromechanical sys…

Harder mostly, See the frequency is set by huge rotating masses in the form of generators, and when the supply and demand is matched the frequency and voltage are stable, when demand dramatically increases it pulls the frequency and voltage down, which is effectively slowing the generators down as load / magnetic drag increases with current drawn. Having large inertial masses spinning actually helps smooth out freque…

> Harder mostly, See the frequency is set by huge rotating masses in the form of generators

I'm not saying you're wrong, but this isn't obviously correct to me.

Since solar going to a grid is completely dependent upon electronic DC->AC conversion, I would expect that it could follow a lot greater frequency deviation for a lot longer than a mechanical system that will literally rip itself apart on desync.

Re: Widespread power outage in Spain and Portugal

#730

Post by yibg »

Vox_Leone wrote: post #471
mike_hearn wrote: post #319

This sounds big enough to require a black start. Unfortunately, those are slow and difficult. If an entire nation trips offline then every generator station disconnects itself from the grid and the grid itself snaps apart into islands. To bring it back you have to disconnect consumer loads and then re-energize a small set of plants that have dedicated black start capability. Thermal plants require energy to start up…

The fewer resources we dedicate to grid resilience and modernization, the harder black starts become. And as grids get more complex and interdependent, recovering from total failure becomes exponentially harder. A rare but sobering opportunity to reflect on something we usually take for granted: electricity. We live in societies where everything depends on the grid — from logistics and healthcare to communications an…

Maybe a good reason (in parts of the world where this is practical) to have some solar + battery storage. Doesn't even need to fully replace grid power, just enough to run the barebones when the grid goes out.
Post reply on HN