It isn't about who broke it. But if there is a person on the team who continually breaks things, does not learn from their mistakes and repeats them, or is not truthful when they break things, the team should react appropriately.
It's about what broke, not who broke it
91–100 of 105 posts
Re: It's about what broke, not who broke it
#92Earlier quoted context omitted.
Sounds like your system is antifragile.
Robust. The word is "robust". We don't need to promote buzzwords.
I think there's a worthwhile distinction between 'robust', meaning 'able to resist stresses', and 'antifragile', meaning 'able to react to stresses and become stronger.'
Re: It's about what broke, not who broke it
#93I took down an assembly plant by clicking on a Network status icon from a particular hardware supplier. Over the weekend, firmware patches were applied, and the server rebooted. After reboot, everything worked fine, so the tech marked the change successful and went home. Well, apparently the NICs would work just fine, but not all settings were applied until you opened the UI provided by the vendor. When you opened th…
> [...] not all settings were applied until you opened the UI provided by the vendor. [...] the NICs would reboot, just long enough to kill TCP connections. The UI part suggests that it was Windows, and if it was, it's not quite the case that "just long enough" to kill TCP connections, as you need quite a lot of downtime to terminate a typical TCP session. In Windows, if a NIC goes down, all the TCP connections that…
> In Windows, if a NIC goes down, all the TCP connections that use the NIC get closed immediately.
Yes, that seems more likely.
I think Windows can be a decent platform for light industrial applications - which this system in particular was. The problem is all of the partners and suppliers were either stuck in the past, or had weird ideas.
The parent system was *nix based, but there was a flaw in a communications protocol that lead to the channel bouncing between two boxes, and eventually bringing down the parent system.
My lesson from that was that you can have flaws on any system, no matter how solid the OS.
Re: It's about what broke, not who broke it
#94The answer requires context, at least for FLOSS projects.
If unlucky dev #13 broke something because humans can no longer reason about the relevant part of the system, then it doesn't matter that #13 was the one who broke something. What really matters is that people get busy removing the sandtraps from their software.
However, many FLOSS projects run on the sheer joy and freedom that comes with maintaining a particular subsystem or area of the code. Most devs have a quick understanding of the responsibilities associated with that. But in cases where that responsibility doesn't come naturally, who broke becomes the focus. Addressing that issue will determine whether or not future breakages occur.
Re: It's about what broke, not who broke it
#95I work in the nuclear industry, where most places are pretty good about maintaining a "blame-free" culture. You focus on what processes and procedures failed, what controls were missing, etc., that allowed somebody to make a mistake. As this attitude was adopted, things shifted too far (at least in the opinion of industry groups, and my observation) to the point where people underperforming to the point of negligence…
What are the tools you you to assess if a problem is organizational or personal?
Re: It's about what broke, not who broke it
#96Earlier quoted context omitted.
Im going to say this - When it is all said and done, if you fucked up, you should get some shit for it. However this should be good natured, YOU should be laughing at it and everyone else laughing WITH you.
> if you fucked up, you should get some shit for it If you're intending that you (and the rest of your team) should learn from your mistakes, then I fully agree
Yes, and it should be good natured... not at someones expense.
Hence the reason YOU should be leading the laughter.
There is a case that laughing indicates that the stressful situation has passed:
http://mentalfloss.com/article/69830/why-do-we-laugh-when-we...
Re: It's about what broke, not who broke it
#97I don't know. Where I work no discernible pattern can be found with the "what" that broke. It's always the same f *ing people that break it though!
It amuses me that the sibling comments appear unable to imagine the possibility that someone is incompetent. Of course there are other possibilities - the people breaking things are doing the hard bits that no one else dare to.
Even in that case the overall system is still at fault for not recognizing their incompetence and either training them to be competent or getting rid of them.
Re: It's about what broke, not who broke it
#98Earlier quoted context omitted.
Windows is often the only vendor-supported choice for interfacing your computer applications to PLCs and such things. Also most of the proprietary protocols run over industrial ethernet are some kind of legacy serial (232, 485..) bytestream format wrapped in TCP and the software usually does not handle loss of the TCP connection particularly gracefully. (on multiple occasions I've seen rules like "reboot the whole in…
> Windows is often the only vendor-supported choice for interfacing your computer applications to PLCs and such things. Which is not a problem by itself, since PLC, being an industrial equipment, should operate independently from a non-industrial equipment. The problem is idiots who think a desktop PC can reliably control PLC in real time.
Is that a reference to Beckhoff?
Re: It's about what broke, not who broke it
#99I'm 90% in agreement. Her workplace definitely sounds like somewhere I'd consider working myself (if I were looking for a job). There are some things that I consider basic competence standards, like not storing passwords in plain text in any system you're building. I wouldn't fire an intern for getting that wrong but I also wouldn't let an intern near a production authentication system without some oversight. If some…
If you can go back in time, join me in 2013 and you can enjoy the ride for a few years, too. I'm sorry to say that I don't think you'll get the same experience in 2018.
Re: It's about what broke, not who broke it
#100Earlier quoted context omitted.
Windows is often the only vendor-supported choice for interfacing your computer applications to PLCs and such things. Also most of the proprietary protocols run over industrial ethernet are some kind of legacy serial (232, 485..) bytestream format wrapped in TCP and the software usually does not handle loss of the TCP connection particularly gracefully. (on multiple occasions I've seen rules like "reboot the whole in…
> Windows is often the only vendor-supported choice for interfacing your computer applications to PLCs and such things. Which is not a problem by itself, since PLC, being an industrial equipment, should operate independently from a non-industrial equipment. The problem is idiots who think a desktop PC can reliably control PLC in real time.
Problem is when you have some kind of process that is inherently controlled not by the logic in PLC, but by some external system (either because the required data will not fit into PLC's data memory or because they constantly change based on some external bussines processes)
Reasonable architecture for this kind of problem would be attaching some server to the PLC as peripheral, but it tends to be done other way around. As for reasons for that I speculate that it is simply inertia of the typical PLC programmer which is then compounded by reasoning along the lines of nobody does that, so it is not tested and we will hit unknown bugs in the PLC firmware itself.