[flagged]
Water system controllers don't belong on the internet, says ex-NSA chief
81–90 of 167 posts
Re: Water system controllers don't belong on the internet, says ex-NSA chief
#82Earlier quoted context omitted.
The best thing they could do to new PLC's would be to have the src code live alongside the firmware inside. So any new person would be able to open it and reason about it.
Relatedly, I made an embedded product. I built a live CD (yes this was a while ago) for my employer with the entire development environment. As long as you can find an x86 machine with CD-ROM you could build, compile, test, and program using the exact environment I did twenty years ago. Maybe today I'd do micropython on ESP32. Download text file from device, edit, upload back on.
There's at least one customer out there who has an RPi with NixOS on it from me. Probably still works to this day. It was the best use of NixOS I think I've ever found.
Re: Water system controllers don't belong on the internet, says ex-NSA chief
#83Earlier quoted context omitted.
Relatedly, I made an embedded product. I built a live CD (yes this was a while ago) for my employer with the entire development environment. As long as you can find an x86 machine with CD-ROM you could build, compile, test, and program using the exact environment I did twenty years ago. Maybe today I'd do micropython on ESP32. Download text file from device, edit, upload back on.
There's at least one customer out there who has an RPi with NixOS on it from me. Probably still works to this day. It was the best use of NixOS I think I've ever found.
I've switched up to NUC-ish hardware which WAS cheap until recently. All the components just a bit higher quality than the lowest-bidder you get on the Pi. Run nixos on that and you've got a really solid platform.
Re: Water system controllers don't belong on the internet, says ex-NSA chief
#84Earlier quoted context omitted.
You’d be surprised how insecure some of these facilities are, especially to someone who has working knowledge of what a PLC (or other process controllers) does and how it works. You can easily look like a tech who belongs there either troubleshooting something or working on a project. I’ve been doing industrial controls for 15 years and surprisingly infrastructure is some of the most poorly funded. I believe a lot of…
Forgive my ignorance but isn't a PLC simply a computer with some GPIO ports? I.e., a Raspberry Pi could be called a PLC? Why are we talking about them as if they are something else? Is it an exotic OS that makes them different?
The point I was making is that PLCs have a software stack. You have to know the software and programming languages defined in IEC 61131-3. For all intents and purposes they are just very repeatable compute in an industrial setting, but if you don’t know ladder logic or how to interface with them you might be SoL. Most don’t have any sort of web interface and if they do it operates outside the controllable logic. Same goes for shells. If you want to program it you typically have to use the vendor software.
Re: Water system controllers don't belong on the internet, says ex-NSA chief
#85Part of my career encapsulates a period where I was a PLC programmer, installer, commissioner, and troubleshooter for massive build outs (sky scrapers, data centers, laboratories, factories). Interestingly, this was after years of teaching myself software engineering, eventually participating in large open source projects. The clash of entering the PLC world was _extremely_ harsh. Let me give an example: I once worke…
Im supposed to be doing endpoint work with people working in this field amd basically have to convince compliance they'll need to be local administrators to do their job. They get onsite and dont know what app they'll need until they look at a plc model. They'll download something from a .tw ftp server and its an unsigned executable that absolutely must run as admin to program these things. It is amazing how much of…
Oof, that's bad. Iran got burned real bad by Stuxnet, you know they would love to score big with PLC sabotage.
Re: Water system controllers don't belong on the internet, says ex-NSA chief
#86Earlier quoted context omitted.
Yes, I don't mean to poke fun solely on the integrators. I've seen the "software first" type land in the same role and flail just as hard, if not harder. I recall one individual, who was quite proud of his "beautiful" ladder logic code, after about 3 months of being at the company ended up destroying a $100k+ chiller plant by cavitating all of the pumps because they didn't understand head pressure. I think that's one…
Sometimes I wonder whether one could design PLC-style software by first expressing the system’s constraints and then writing the actual code and having either (preferably) formal verification that the code will not violate the constraints or (less preferably) enforced assertions that will send the machine into a safe state if the constraints do not hold. The system I mentioned in my other post controlled some VFDs, a…
Seems like, here in the 21st century, one could reasonably have a model of the plant that tests could be executed against. Anything like that exist?
Re: Water system controllers don't belong on the internet, says ex-NSA chief
#87Re: Water system controllers don't belong on the internet, says ex-NSA chief
#88Earlier quoted context omitted.
A lot of water infrastructure is physically spread out. It be very expensive and cumbersome (and probably inefficient) to require staff to by physically present at each site for monitoring and making any changes.
That’s fine. It’s compromised. Happy with that?
Or connect them securely
Re: Water system controllers don't belong on the internet, says ex-NSA chief
#89Part of my career encapsulates a period where I was a PLC programmer, installer, commissioner, and troubleshooter for massive build outs (sky scrapers, data centers, laboratories, factories). Interestingly, this was after years of teaching myself software engineering, eventually participating in large open source projects. The clash of entering the PLC world was _extremely_ harsh. Let me give an example: I once worke…
Sounds like we have a similar development path. People fail to understand how complex some of these systems can be, especially since you have to have the project to understand what registers/variables are what. Some when Joe the contractor does a job and walks away, you better make sure they give you the project and you store it in a nice safe place.
A good system integrator is worth their weight in gold. Sure they cost more, but getting a whole package turned over to you is worth a million more than 5 years into the lifecycle of a factory when someone wants to make mods/fix a bug/etc and has to reinvent the whole car, not just the wheel.
This industry is always 10-20 years in the past. My company’s preferred vendor only just started supporting virtualization (this is for a DCS) in the past 10 years. I still have to tell my sales people to provide A/V and minimalistic backup and recovery on every project (they essentially cost nothing compared to the rest of any project).
Re: Water system controllers don't belong on the internet, says ex-NSA chief
#90Earlier quoted context omitted.
Sometimes I wonder whether one could design PLC-style software by first expressing the system’s constraints and then writing the actual code and having either (preferably) formal verification that the code will not violate the constraints or (less preferably) enforced assertions that will send the machine into a safe state if the constraints do not hold. The system I mentioned in my other post controlled some VFDs, a…
Seems like, here in the 21st century, one could reasonably have a model of the plant that tests could be executed against. Anything like that exist?
We're often talking about custom plants here, or plants made in very low volume. Making a simulator for that can be prohibitively expensive, since you can't amortize the cost. For particularly important/expensive plants it still sometimes gets done. Though note the simulator might not be able to capture all the nuances of the real world deployment, so it might not be sufficient.
Meanwhile having a separate copy of the plant is also not often viable, once the plants get to any kind of size.
edit: After having 'someone' fact check and remind me: there are quite a number of simulation tools available that can sometimes partially do the work.