Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Water system controllers don't belong on the internet, says ex-NSA chief

theregister.com

Post reply on HN
Print view View original

131–140 of 166 posts

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#131

Post by KaiserPro »

barbazoo wrote: post #41
clbrmbr wrote: post #5

There are many wireless pump-and-reservoir systems that while not internet connected, use insecure RF links. These local RF (and casting a wider net, Bluetooth) interfaces are also ripe for abuse.

If that means you need to at least be physically present then I'd say that's a lot of protection already. Means someone in a foreign country can't simply get lucky fuzzing.

Looking at whats happening in europe, thats not a barrier. all it takes is a few hundred euros, some time on a dipshit forum and you have a willing gopher who'll go to a given location and run an unspecified payload.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#132

Post by voakbasda »

aliasxneo wrote: post #44
TacticalCoder wrote: post #37

Earlier quoted context omitted.

> So, anyways, just imagine the most archaic and barbaric set of IT software, controls, and procedures, dumb that down even further, and you've landed on the infrastructure/teams that operate probably half of critical infrastructure. Same with SCADA: just as bad as what you describe.

We once reversed engineered Schneider Electric's ION protocol because they wanted to charge us $10k per "seat" to get software that could interact with it directly. We took home a controller and developed a Python program that could intercept and understand the RS-232 comms. We then developed our own system to interact with it. It was all unencrypted. I just did a quick search and it looks like "Secure ION" is now a…

Some of Schneider Electric’s software is a house of cards. Source: I maintained some systems for them a few years back.

Never will they ever get my money again for anything. To wit, I miss the days when APC was a brand that I could trust.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#133

Post by jdw64 »

aliasxneo wrote: post #31

Part of my career encapsulates a period where I was a PLC programmer, installer, commissioner, and troubleshooter for massive build outs (sky scrapers, data centers, laboratories, factories). Interestingly, this was after years of teaching myself software engineering, eventually participating in large open source projects. The clash of entering the PLC world was _extremely_ harsh. Let me give an example: I once worke…

I agree. The problem is this: most companies using PLCs work in isolated networks with internet disconnected, based on confidentiality. There are many difficulties even with using git, because the structure forces you to implement features in a way that makes it hard to separate them. Branching, in particular, doesn't work well with PLC ladder logic—PLC uses proprietary binary formats that make diff tracking impossible, so you're forced into a unified development approach

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#134

Post by Karrot_Kream »

voakbasda wrote: post #130
AlotOfReading wrote: post #46

Earlier quoted context omitted.

integration tests (which can be trickier when it requires a hardware test bed.) I use this as a fizzbuzz-type test when I'm interviewing at hardware companies: do they have development hardware in a rack with programmable power supplies and mini-PCs (or similar)? It's a low, low bar for testing, and rules surprisingly many companies. They'll often just have The Guy running manual tests instead.

Wow. I’d kill for even a guy running my hardware tests. Most of my jobs barely seem to leave time to write the code, before it needs to be scraped off my monitor and poured into the customer’s systems. The ones like you describe with actual hardware labs are a dream when contrasted to those nightmares.

Does management sell unrealistic deadlines to the vendors that buy your solutions? Are you bound by impossible support contracts? Did the firm just let go of a lot of people and expect everyone to finish work quickly? Curious how the economic/management pressures here work as I abandoned hardware very early in my career (from an internship) for similar reasons.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#135

Post by PunchyHamster »

aliasxneo wrote: post #31

Part of my career encapsulates a period where I was a PLC programmer, installer, commissioner, and troubleshooter for massive build outs (sky scrapers, data centers, laboratories, factories). Interestingly, this was after years of teaching myself software engineering, eventually participating in large open source projects. The clash of entering the PLC world was _extremely_ harsh. Let me give an example: I once worke…

I mean it worked fine for 50 years, surely it will work fine for another 50

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#136

Post by hiAndrewQuinn »

analog31 wrote: post #69
steve_adams_86 wrote: post #47

Earlier quoted context omitted.

Everything you're describing mirrors my experience with remote scientific deployments and lab equipment (wet labs in particular). I support a team which manages dozens of these, and while it's shocking how inefficient and unsafe their practices are, it's similarly shocking how poorly the industry has kept up with or even tried to establish better practices. We get brand new equipment that costs in the realm of $10k f…

>>> The pay is also so much worse in my experience. The result is that anybody who can program well enough to develop software is doing so. Those with any programming skills who remain behind are smart-but-undisciplined programmers, people who have a reason to be in a particular geographic area, too old to think about changing careers, or just plain nuts. This is what "lack of investment in manufacturing, research, a…

To me that speaks to a fundamentally incorrect economic structure. I bet PLC programmers could command much higher salaries - at much higher levels of risk - if they held more liability for things going wrong. That, in turn, would spur a lot of economic development around unfucking the field.

On the other hand, markets are generally smarter than I am. Much like how US citizens living in the EU are basically barred from investing in index funds due to two fundamentally incompatible regulatory regimes, there are almost certainly factors here I'm not seeing and wouldn't see without a couple weeks of research and talking to people who have tried and failed as this.

A guy can dream, though.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#137

Post by hiAndrewQuinn »

voakbasda wrote: post #128
rurban wrote: post #125

Earlier quoted context omitted.

More probably worst use of NixOS. In embedded we care for every single kb, and don't have tolerance for this kind of space waste management system

Embedded means different things. Personally I have a hard time calling anything that runs Linux as embedded, having cut my teeth on microcontrollers.

If it's a custom Yocto thing or it involved buildroot at any state of provisioning it gets the moniker "quasi-embedded" to me. Doubly so if it never actually connects to the internet, unlike these water controllers, apparently.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#138

Post by andyjohnson0 »

amelius wrote: post #35
procarch2019 wrote: post #25

Earlier quoted context omitted.

You’d be surprised how insecure some of these facilities are, especially to someone who has working knowledge of what a PLC (or other process controllers) does and how it works. You can easily look like a tech who belongs there either troubleshooting something or working on a project. I’ve been doing industrial controls for 15 years and surprisingly infrastructure is some of the most poorly funded. I believe a lot of…

Forgive my ignorance but isn't a PLC simply a computer with some GPIO ports? I.e., a Raspberry Pi could be called a PLC? Why are we talking about them as if they are something else? Is it an exotic OS that makes them different?

In addition to the other replies here:

PLCs are ruggedised and built to survive for years in the kinds of hostile environment that are frequently found in industrial settings.

And PLC vendors typically provide guaranteed support periods that are much longer (like a decade or more) than those commonly provided by conventional computer OEMs.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#139

Post by grebc »

lokar wrote: post #94
kelnos wrote: post #92

Earlier quoted context omitted.

No such thing. There are levels of security, where higher levels mean more cost. And even with the most secure facility, there are no guarantees. And certainly we don't have the funding to provide that level of security for every single facility.

A well designed and deployed VPN link from the satellite location back to HQ (to machines isolated for just this use) would be orders of magnitude safer then what they seem to have been doing, and a reasonable compromise between security and utility.

It’s most definitely a compromise. Just not what you think.

You can keep saying it’s more secure than currently, and that’s good and all. It still doesn’t matter against the adversaries being mentioned.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#140

Post by kotaKat »

skinfaxi wrote: post #80
ssl-3 wrote: post #76

Earlier quoted context omitted.

You mean like, just hang a thumb drive on a lanyard inside of the cabinet -- with source code and such? That will probably just disappear. Maybe with good intention or maybe with bad, but it's probably gone either way. If it doesn't disappear, then it's hanging right there for any of the competitors to use. That's a problem for the original installer's ongoing employment. And if it includes the programming software,…

> If it doesn't disappear, then it's hanging right there for any of the competitors to use. That's a problem for the original installer's ongoing employment. Is that really an issue? Do PLC vendors have concerns about other vendors turning up and yanking USB drives out of critical infrastructure with reckless abandon? Do the facilities themselves see no issue with potential vendors yanking operational utilities out o…

Hospital IT got a call asking where the little binder with the DVDs for the CT scanner were from someone in Radiology 'cause the Siemens tech showed up.

No, we didn't have them, and they told us they last saw them in the back of a cabinet in their own work era.

Turns out those were the unit-specific reload DVDs for half the reconstruction computers and some serial-specific license stuff.

Whoopsie...

Post reply on HN