Live data from Hacker News

The day my script killed 10k phones in South America

new.pythonforengineers.com

51–60 of 164 posts

Re: The day my script killed 10k phones in South America

#51
post #2

This feels incredibly negligent. It seems obvious that generating thousands of random phone numbers and locking them without any other checks would lock some real phones. I'm puzzled that the author did not consider this when writing the script, let alone testing it or running it on live prod servers.

Too me this is another example showing how much behind "software engineering" is behind most otherl engineering disciplines. Processes in other disciplines are build very much into the practice that these things don't happen because people die if a plane crashes or a bridge collapses. Also that somehow software gets away with absolving itself of any liability (i.e. the usual "this software is provided as is...") is q…

It’s like trying to breathe underwater. Just because a person has the capability and know how to breathe, they can’t breathe if not given the proper environment to do so. Yelling at them “why didn’t you take a breathe?!?!” As you watched them drown underwater is just ridiculous and naive.

But every time proper engineering doesn’t happen, we see a subset of cries like this, like it’s some moral failing in the engineer. There are surely some chronically stupid engineers out there, but 99% of the time it’s a culture that has subverted the doers below the discussers and the deciders that leads to this shitshow of “how did this happen??”

Re: The day my script killed 10k phones in South America

#52

I've done this, but with manager-vetted data. Conversation (in writing) - "We NEED to run this incredibly risky thing yesterday" - "Ok, I'm going to use this data. Is this data OK to you?" - "Yes" - "Running" Great way to avoid getting thrown under the bus if it all goes south.

"I validated the data to the best of my ability, but I had to rely on otrahuevada's experience in choosing the data. Unfortunately he really screwed up."

If your manager is the type to throw people under the bus, there's always a way.

Re: The day my script killed 10k phones in South America

#53
post #36

Earlier quoted context omitted.

The product was for phones that had not been paid off-- if you get a phone on a 2 year contract, you dont "own" the phone until it is paid off

Not usually the case under UK law at any rate.

Perfectly legal in the UK-- the company was British. Customers were in Europe, America etc

Re: The day my script killed 10k phones in South America

#54
post #36

Earlier quoted context omitted.

The product was for phones that had not been paid off-- if you get a phone on a 2 year contract, you dont "own" the phone until it is paid off

Not usually the case under UK law at any rate.

Apropos the UK...it depends on the contract and how the phone is financed. Here's an example of EE's (UK company) "Pay Monthly Terms":

https://ee.co.uk/content/dam/ee-help/help-pdfs/pay-monthly-n...

Section 10 explains ownership:

Starting with...Section 10.1:

"You may get Equipment from Us directly when You take out a Price Plan, whether for free or for an upfront cost. Unless We tell You otherwise, or You get Equipment under a Finance Agreement, We will own any Equipment provided to You by Us for the first six months of the Minimum Term."

...and it goes on.

Re: The day my script killed 10k phones in South America

#55
post #52

I've done this, but with manager-vetted data. Conversation (in writing) - "We NEED to run this incredibly risky thing yesterday" - "Ok, I'm going to use this data. Is this data OK to you?" - "Yes" - "Running" Great way to avoid getting thrown under the bus if it all goes south.

"I validated the data to the best of my ability, but I had to rely on otrahuevada's experience in choosing the data. Unfortunately he really screwed up." If your manager is the type to throw people under the bus, there's always a way.

I mean yeah when there's a will there's a way, but keeping your manager and reports informed about your work and its progress is kind of the only way to cover your ass available to you as a non-manager.

Re: The day my script killed 10k phones in South America

#57
post #36

Earlier quoted context omitted.

Not usually the case under UK law at any rate.

Apropos the UK...it depends on the contract and how the phone is financed. Here's an example of EE's (UK company) "Pay Monthly Terms": https://ee.co.uk/content/dam/ee-help/help-pdfs/pay-monthly-n... Section 10 explains ownership: Starting with...Section 10.1: "You may get Equipment from Us directly when You take out a Price Plan, whether for free or for an upfront cost. Unless We tell You otherwise, or You get Equipm…

> We will own any Equipment provided to You by Us for the first six months of the Minimum Term.

But yet they all make pains to separate airtime, and devices in the contract for example clauses that say you can't cancel the airtime contract if the phone proves faulty

Re: The day my script killed 10k phones in South America

#58

Earlier quoted context omitted.

The headline says South America. I don't think they use UK law.

UK common law is pretty prevalent all over the world

English Common Law is prevalent, but that is the basis of the law and certainly not the specific codes and minor changes that have been made since 1950

Re: The day my script killed 10k phones in South America

#59

Author here. I shared this yesterday, didnt get much upvotes and quickly vanished. Thanks to /u/nixcraft for sharing it again! Looks like 2nd time lucky. Im seeing the same type of comments here and on Reddit, I'll try to answer a few common ones. 1. Yes, I know it was stupid testing on production. But we'd been told we needed to release the product on Monday (this was Friday), no objections. A previous project manag…

> Including my boss, who was fired when he was attending his mother's funeral.

And people wonder why big companies are generally loathed.

Re: The day my script killed 10k phones in South America

#60
I've noticed more as time has gone on people are working on critical software without the skills or organizational structures required to actually do so safely. What I'm wondering is if this is a change that's happened more broadly or is just something that I didn't notice before? Overall the average years of experience in the industry has dropped in the last decade due to the influx of new developers but what's the deal with inexperienced people working on critical systems without oversight from people with more experience?

While I feel the failures from the developer here are pretty bad I feel the organizational factors that allows this situation to happen are far worse.

Post reply on HN