What on Earth? This company just locks random people's phones as part of their testing, and nobody realised that was a problem until they did it to thousands at once?
The day my script killed 10k phones in South America
11–20 of 164 posts
Re: The day my script killed 10k phones in South America
#12How did he not get that running a script to lock random phones would lock random phones?
Re: The day my script killed 10k phones in South America
#13This feels incredibly negligent. It seems obvious that generating thousands of random phone numbers and locking them without any other checks would lock some real phones. I'm puzzled that the author did not consider this when writing the script, let alone testing it or running it on live prod servers.
I don't understand how people can defend this developer in the first place.
I could understand if they were about 16 years old and actually abused by their employer, but generating random numbers to test the locking of phones? Basic maths should tell you that the probability of matching some real phone number is actually quite high.
Moreover, what exactly were they testing? i.e. if they were testing nonexistent numbers, then how would they even know the phones were blocked in the end?
Re: The day my script killed 10k phones in South America
#14Earlier quoted context omitted.
Obviously he contributed the most to the problem, but doesn't a manager have some responsibility for the work they are managing?
More specifically for the deadline they are imposing without asking for feedback about feasibility and associated risks.
The deadline for generating a bunch of random phone numbers to block? The author would probably save some time by hard-coding some truly non-existent phone numbers instead of using a random number generator.
Re: The day my script killed 10k phones in South America
#15This feels incredibly negligent. It seems obvious that generating thousands of random phone numbers and locking them without any other checks would lock some real phones. I'm puzzled that the author did not consider this when writing the script, let alone testing it or running it on live prod servers.
Too me this is another example showing how much behind "software engineering" is behind most otherl engineering disciplines. Processes in other disciplines are build very much into the practice that these things don't happen because people die if a plane crashes or a bridge collapses. Also that somehow software gets away with absolving itself of any liability (i.e. the usual "this software is provided as is...") is quite remarkable.
Re: The day my script killed 10k phones in South America
#16Usually not a great idea to run tests like that on production.
Look at this guy, implying he has a separate test environment.
Probably has an adequate budget and manager buy in as well.
Re: The day my script killed 10k phones in South America
#17Earlier quoted context omitted.
Shouldn't a company that has a process where a recently hired contractor can lock any phone in the world get some share of the blame?
Exactly! There seemed to be a severe lack of any production control processes at this place. That's the fault of management and senior developers/architects. Contractors should never be given the keys to the kingdom like that. And if they are, then there should be some checks and balances to make sure they don't do something foolish like this. But I've worked at places like this where we coded "close to the wire" so…
I am not sure how a non-technical director is supposed to hire a software engineer if software engineers themselves fail to hire good software engineers.
Re: The day my script killed 10k phones in South America
#18“the main product was an app...that would lock the phone if it was stolen, or if the customer stopped paying. The app was built as part of the Android OS, so you couldn't uninstall it. It would lock the low-level features that allowed you to make calls, use Wifi...until you paid up.
All good.”
No, not all good. There’s the problem right there. If I stop paying you can stop providing me the service I’m paying for. Using WiFi has nothing to do with you. Breaking my property this way, without a court judgement allowing you to do it, is not only unethical, but should be a criminal offense, and is at least a civil tort.
The potential liability here is literally unlimited. I believe responsible cell providers allow you to call 911, or the equivalent in your location, even if your account is delinquent. But you can’t do that if your phone is locked in this way. Did anyone die because of this product and this guy’s recklessness?
Re: The day my script killed 10k phones in South America
#19The lack of judgement in both the articles content and publishing the article itself is horrifying.
Re: The day my script killed 10k phones in South America
#20If you haven't killed some production environment in your career, can you really call yourself a programmer? I kid of course, but I've been there, done that. It's lessons learned the hard way and I became a better developer because of it.
I once ran a query on a live production web site thinking I was doing a SELECT but somehow ran an UPDATE which set all the millions of users' gender to 'M', regardless of their selection.
Restoring from backups would have been a nightmare. And telling my supervisor would have been a nightmare.
So I used everyone's selected title (Mr, Ms, Mrs etc) to reset their gender setting, and then put the keyboard down and went to bed.