You can automate using puppeteer or selenium
The author used Playwright in the end to automate uploads. Using e2e tools for automating tasks is clever, I'm not sure I would've thought of it.
I Accidentally Deleted 7TB of Videos Before Going to Production
41–50 of 362 posts
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#42> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…
Human-in-the-loop is so important concept in ops and yet everyone (that's including me) seems to learn it the hard way.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#43[deleted]
You're right and I edited the company's name (might be too late but better this way). That said I'm not very happy with the experience of working for TheCompanyTM anyways so I'm in the process of switching jobs. Thanks for the comment :)
I would take down the post entirelly.
Your current job is linked in your CV.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#44Earlier quoted context omitted.
You're right and I edited the company's name (might be too late but better this way). That said I'm not very happy with the experience of working for TheCompanyTM anyways so I'm in the process of switching jobs. Thanks for the comment :)
I would take down the post entirelly. Your current job is linked in your CV.
And try emailing the hackernews mods asking them to take this post down.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#45> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…
Rather than commenting it out, I suggest adding a --live-run flag to scripts and checking the output of --live-run=false (or omitted) before you run it "live."
But then you have double the chances of introducing a bug for the specific scenario we are talking about:
Before: there is chance there is a bug in my "delete" use case
Now: what we have before plus the change that there is a bug in my "--live-run" flag
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#46> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…
Yes, I find command line tools that have a "--dry-run" flag to be very helpful. If the tool (or script or whatever) is performing some destructive or expensive change, then having the ability to ask "what do you think I want to do?" is great. It's like the difference between "do what I say" and "do what I mean"...
Going further, make it dry run by default and have an --execute flag to actually run the commands: this encourages the user to check the dryrun output first.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#47"What does this teach us? Well, it teaches me to do more diverse tests when doing destructive operations. It also should probably teach something to Vimeo and to my contractor but I doubt it will (and yes, the upload for some reason is still manual to this day. Go figure!)" So you wrote bad code, didn't test it properly, ran it on production on the Friday before a release and are blaming Vimeo and [name redacted]? An…
Not to mention that he _deleted_, but not _lost_ videos. Nothing to see here.
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#48Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#49Experience is directly proportional to the amount of equipment ruined or data lost.
Even though you were fortunate not to lose any data, you gained a lot of experience!
Re: I Accidentally Deleted 7TB of Videos Before Going to Production
#50"What does this teach us? Well, it teaches me to do more diverse tests when doing destructive operations. It also should probably teach something to Vimeo and to my contractor but I doubt it will (and yes, the upload for some reason is still manual to this day. Go figure!)" So you wrote bad code, didn't test it properly, ran it on production on the Friday before a release and are blaming Vimeo and [name redacted]? An…
I'd hire this guy if only being for this frank about his mistake. He owned it and that is what I would look for.
After deletion, what should he have done? Postpone the go-live? That's often not a a cost-effective option. As for a risk-analysis the worst what could happen was deletion of the remaining videos. I don't think that that makes big difference in this situation. And to do the right thing, you have to have the infrastructure in place, if you are in a hurry. I doubt that's the case for a 10 heads shop.