Earlier quoted context omitted.
The rule we have is that anything that is not idempotent and not run as a matter of daily routine must dry-run by default, and not take action unless you pass --really. This has saved my bacon many times!
Deleting actually is idempotent. Doing it twice wont be different from doing it once.
I Accidentally Deleted 7TB of Videos Before Going to Production
61–70 of 362 posts
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#62Earlier quoted context omitted.
Yes, I find command line tools that have a "--dry-run" flag to be very helpful. If the tool (or script or whatever) is performing some destructive or expensive change, then having the ability to ask "what do you think I want to do?" is great. It's like the difference between "do what I say" and "do what I mean"...
The rule we have is that anything that is not idempotent and not run as a matter of daily routine must dry-run by default, and not take action unless you pass --really. This has saved my bacon many times!
T'was a sad day.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#63> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…
- make sure the sum of their lengths == number of total current items
- make sure items_to_be_kept.length != 0
- make sure no two items appear in both lists
- check some items chosen at random to see if they were sorted in the correct list
At this point the only possible mistake left is to confuse the lists and send the "to_be_kept" one to the delete script; a dry run of the delete list can be in order.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#64"What does this teach us? Well, it teaches me to do more diverse tests when doing destructive operations. It also should probably teach something to Vimeo and to my contractor but I doubt it will (and yes, the upload for some reason is still manual to this day. Go figure!)" So you wrote bad code, didn't test it properly, ran it on production on the Friday before a release and are blaming Vimeo and [name redacted]? An…
The article hardly comes across as 'blaming' them for the core issue but they were definitely not helpful.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#65> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…
https://news.ycombinator.com/item?id=29083367
It defaults to not doing anything so you can gradually and selectively have it do something.
Learned about when I posted my command line checklist tool on HN: https://github.com/givemefoxes/sneklist
(https://news.ycombinator.com/item?id=25811276)
You could use it to summon up a checklist of to-dos like "make sure the collection in the dictionary has the expected number of values" before a "do you want to proceed? Y/n"
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#66> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#67 rm -rf /path/to/delete/ *
And realize it is taking too long...Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#68Earlier quoted context omitted.
You're right and I edited the company's name (might be too late but better this way). That said I'm not very happy with the experience of working for TheCompanyTM anyways so I'm in the process of switching jobs. Thanks for the comment :)
Talking bad about your employer is great for finding a new job. Companies are eager to hire people who bad-talk them.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#69Earlier quoted context omitted.
Deleting actually is idempotent. Doing it twice wont be different from doing it once.
Deleting * may not be though. Your selection needs to be idempotent.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#70> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…
Another technique that I've used with good success is to write a script that dumps out bash commands to delete files individually. I can visually inspect the file, analyze it with other tools, etc and then when I'm happy it's correct just "bash file_full_of_rms.sh" and be confident that it did the right thing.