The conclusions made in this blog seem wrong, it's perfectly fine to write these kinds of tests if you have to as part of a rush job, but WHY DO IT ON PROD? You know 100% that you are making calls to the 3rd partys' prod, assuming "random" numbers aren't real is incredibly negligent.
The day my script killed 10k phones in South America
61–70 of 164 posts
Re: The day my script killed 10k phones in South America
#62This feels incredibly negligent. It seems obvious that generating thousands of random phone numbers and locking them without any other checks would lock some real phones. I'm puzzled that the author did not consider this when writing the script, let alone testing it or running it on live prod servers.
Not defending him, but he does state that unlike the other mobile providers he's worked with, this K-Pop company's backend system would lock _any_ phone number, not just the ones they're responsible for. Perhaps he didn't know this. I bet normally he'd generate random numbers not associated with the provider, check they show up as "locked" in the database, under the assumption that the backend actually doing the lock…
The fact that he was running production code and not writing the result to any kind of log file, which would only take a couple more lines of code, one more dependency (if he wanted to skip some time rolling his own), and a couple of megs of storage, is so absurdly irresponsible, I’m surprised he wrote this post up at all—most would be too embarrassed.
Re: The day my script killed 10k phones in South America
#63Earlier quoted context omitted.
I don't understand how people can defend this developer in the first place. I could understand if they were about 16 years old and actually abused by their employer, but generating random numbers to test the locking of phones? Basic maths should tell you that the probability of matching some real phone number is actually quite high. Moreover, what exactly were they testing? i.e. if they were testing nonexistent numbe…
I pass no judgment, but merely take the developer at his word: https://new.pythonforengineers.com/blog/confessions-of-a-1x-...
Why would he write and _publish_ this?!? Is this some kind of humiliation fetish? These are things that should maybe be admitted in a confessional, or therapy.
Re: The day my script killed 10k phones in South America
#64What on Earth? This company just locks random people's phones as part of their testing, and nobody realised that was a problem until they did it to thousands at once?
What is even worse - this company can lock phones that aren't their property.
I think that's the real issue here. The Korean company (probably Samsung or LG based on the implications here) didn't validate that this company had permission to lock these phones.
Things like these can happen by mistake (or incompetence, in this instance) and the unnamed company should have never let this happen.
Re: The day my script killed 10k phones in South America
#65The lack of judgement in both the articles content and publishing the article itself is horrifying.
I also get the feeling that the author didn't learn much from this experience. The lessons learnt here are "should do better testing" but even that didn't stick based on the paragraphs after that.
The complete disregard for the victims of this company's incompetence ("the poor sod in Peru who couldn't make phone calls or post duck-face pictures to Instagram") makes this a very different kind of article from the usual "I fucked up production the other day" posts that you see here now and then.
My conclusion after reading this is that the author didn't learn anything, the company didn't get punished enough to re-evaluate their testing and the Korean company's systems are still vulnerable to unauthorized locking.
The real lessons I've learned from reading this are to avoid either LG or Samsung and any provider using Trustonic. And not to hire the author, I suppose.
Re: The day my script killed 10k phones in South America
#66Earlier quoted context omitted.
Not usually the case under UK law at any rate.
The headline says South America. I don't think they use UK law.
The author works in the UK and there's one company listed on his LinkedIn that provides the services he described, which is situated in Cambridge. The description of the product seems to indicate that the software is written to allow carriers to lock out people who stopped paying, mostly targeting people who'd normally have trouble getting credit for their phones.
Unless the company specialized in sales outside the UK and likely also outside the EU, I think this company is probably breaking the law, as are the carriers using their services. I doubt the people who buy phones on extended credit will have the cash laying around to bring up a lawsuit, though.
Re: The day my script killed 10k phones in South America
#67If you haven't killed some production environment in your career, can you really call yourself a programmer? I kid of course, but I've been there, done that. It's lessons learned the hard way and I became a better developer because of it.
Yeah, but this one is particularly bad with a lot of real world consequences. If I were that developer I would have never have blogged about it publicly. It is admitting to gross negligence IMHO.
This case study had a lot of valuable insights cooked into it, so I'm glad this developer blogged about it.
The myth of the developer who knows everything and never makes huge mistakes is a lie that needs to perish. I'd much rather work with someone who is upfront about their mistakes, the gaps in their knowledge, and can speak directly on serious issues of overwork, managerial incompetence, bad clients, and systemic fragility.
Re: The day my script killed 10k phones in South America
#68This makes me curious if iPhones can be remotely disabled by a carrier.
Re: The day my script killed 10k phones in South America
#69Author here. I shared this yesterday, didnt get much upvotes and quickly vanished. Thanks to /u/nixcraft for sharing it again! Looks like 2nd time lucky. Im seeing the same type of comments here and on Reddit, I'll try to answer a few common ones. 1. Yes, I know it was stupid testing on production. But we'd been told we needed to release the product on Monday (this was Friday), no objections. A previous project manag…
> This is builtin on iPhones
I've never heard of this. Can you elaborate?
Re: The day my script killed 10k phones in South America
#70I believe everything you need to know about who would be responsible for this kind of "mistake" is right here from the article: > Our own internal tool worked well (I would like to say it was all due to me, which would be an arrogant thing to say, but also 100% true, so I will say it: It was all or mostly due to me) This is a person who is incredibly overconfident in their limited skills and hasn't yet developed any…
People in the industry tend to laugh at having to work late as some sort of standard practice in the industry. But if the time allotted to program it is not enough time to program it in a properly tested manner, that's a management problem, even if the programmer could have worked overtime and gotten it tested better. It's a bad practice for managers to schedule things such that they can only be completed with overtime, and managers bear a huge portion of the responsibility for any problems that result from that practice.