Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Water system controllers don't belong on the internet, says ex-NSA chief

theregister.com

Post reply on HN
Print view View original

91–100 of 166 posts

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#91

Post by aliasxneo »

tedd4u wrote: post #86
amluto wrote: post #63

Earlier quoted context omitted.

Sometimes I wonder whether one could design PLC-style software by first expressing the system’s constraints and then writing the actual code and having either (preferably) formal verification that the code will not violate the constraints or (less preferably) enforced assertions that will send the machine into a safe state if the constraints do not hold. The system I mentioned in my other post controlled some VFDs, a…

Seems like, here in the 21st century, one could reasonably have a model of the plant that tests could be executed against. Anything like that exist?

It can become really complex. As an example, try to predict resonant frequencies on an entire plant full of rotating equipment. It can be done, but it often requires very expensive equipment and skilled technicians capable of using it to profile a plant. And that's just one potential problem you can run into.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#92

Post by kelnos »

lokar wrote: post #88
grebc wrote: post #48

Earlier quoted context omitted.

That’s fine. It’s compromised. Happy with that?

Or connect them securely

No such thing. There are levels of security, where higher levels mean more cost. And even with the most secure facility, there are no guarantees. And certainly we don't have the funding to provide that level of security for every single facility.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#93

Post by procarch2019 »

ssl-3 wrote: post #76
polishdude20 wrote: post #70

Earlier quoted context omitted.

The best thing they could do to new PLC's would be to have the src code live alongside the firmware inside. So any new person would be able to open it and reason about it.

You mean like, just hang a thumb drive on a lanyard inside of the cabinet -- with source code and such? That will probably just disappear. Maybe with good intention or maybe with bad, but it's probably gone either way. If it doesn't disappear, then it's hanging right there for any of the competitors to use. That's a problem for the original installer's ongoing employment. And if it includes the programming software,…

Even more so, if it’s just there you’re making it easy for the customer to find someone bidding less to execute new projects. Guess who they’re calling if they don’t have the plc project (or don’t understand they need it). Job security.

Plus, yea, someone could really make a mess even if they have the best intention. A guy from my company almost uploaded the wrong program to a large bioreactor. At best it would have made the bioreactor just unusable until corrected. At worst it could have broken the equipment or caused a safety issue. This guy had been around the block a few times too. Luckily someone noticed before it was too late.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#94

Post by lokar »

kelnos wrote: post #92
lokar wrote: post #88

Earlier quoted context omitted.

Or connect them securely

No such thing. There are levels of security, where higher levels mean more cost. And even with the most secure facility, there are no guarantees. And certainly we don't have the funding to provide that level of security for every single facility.

A well designed and deployed VPN link from the satellite location back to HQ (to machines isolated for just this use) would be orders of magnitude safer then what they seem to have been doing, and a reasonable compromise between security and utility.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#95

Post by xtajv »

coryrc wrote: post #77
polishdude20 wrote: post #70

Earlier quoted context omitted.

The best thing they could do to new PLC's would be to have the src code live alongside the firmware inside. So any new person would be able to open it and reason about it.

Relatedly, I made an embedded product. I built a live CD (yes this was a while ago) for my employer with the entire development environment. As long as you can find an x86 machine with CD-ROM you could build, compile, test, and program using the exact environment I did twenty years ago. Maybe today I'd do micropython on ESP32. Download text file from device, edit, upload back on.

Same. (Flash drive + busybox edition).

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#98

Post by xtajv »

aliasxneo wrote: post #31

Part of my career encapsulates a period where I was a PLC programmer, installer, commissioner, and troubleshooter for massive build outs (sky scrapers, data centers, laboratories, factories). Interestingly, this was after years of teaching myself software engineering, eventually participating in large open source projects. The clash of entering the PLC world was _extremely_ harsh. Let me give an example: I once worke…

Storytime: Once upon a time, I was a young grasshopper at a small consulting firm at which every employee was issued a Windows machine, our source control system was "\Customers" on C: drive, and our deployment strategy was "putty + copy-paste + hot-reload".

One morning, I heard something terrible while my boss's boss's machine was booting. Something akin to grinding. The thing was angry.

I gave boss^2 a heads-up that his hard drive might be failing, and that he might want to run a S.M.A.R.T. check on the poor thing. I also asked where the backup hard drives were, because I was brand-new and assumed that I just hadn't been issued one yet.

I checked the supply closet, found no hard drives, popped over to the office admin person, and recommended a deal I'd seen on some WD black drives before I realized that the place had gone quiet.

Everyone looked at me like I was from Mars.

Exactly 7 days later, boss^2's hard drive failed. We lost a week of work and had to zero out a 5 days x 3 employees worth of billable hours. We also ended up delivering late.

The client was pissed.

Based on the nasty looks that I got afterwards, it appeared that the standard assumption was that I had tampered with the drive to prove a point. (Um, nope).

I have since learned to ask prospective employers about their backup strategy.

Re: Water system controllers don't belong on the internet, says ex-NSA chief

#99

Post by closeparen »

procarch2019 wrote: post #89
aliasxneo wrote: post #31

Part of my career encapsulates a period where I was a PLC programmer, installer, commissioner, and troubleshooter for massive build outs (sky scrapers, data centers, laboratories, factories). Interestingly, this was after years of teaching myself software engineering, eventually participating in large open source projects. The clash of entering the PLC world was _extremely_ harsh. Let me give an example: I once worke…

Sounds like we have a similar development path. People fail to understand how complex some of these systems can be, especially since you have to have the project to understand what registers/variables are what. Some when Joe the contractor does a job and walks away, you better make sure they give you the project and you store it in a nice safe place. A good system integrator is worth their weight in gold. Sure they c…

I enjoy watching Cursed Controls on YouTube. For him, the gold standard is a flash drive with the program on it physically hanging out in the control cabinet.
Post reply on HN