Live data from Hacker News

I Accidentally Deleted 7TB of Videos Before Going to Production

blog.thevinter.com

51–60 of 362 posts

Re: I Accidentally Deleted 7TB of Videos Before Going to Production

#51

Earlier quoted context omitted.

I'm baffled by this too. Unnecessary bridge burning I'd call it. It's not even necessary to the story.

Its explained on the first line: " I'm a Junior Developer with less than one year of actual experience. Some of the things that might seem obvious to some might not be so for me". I guess it applies to this, too, not just the technical aspects.

I might've missed it, but I don't think that line existed when this was first posted.

Re: I Accidentally Deleted 7TB of Videos Before Going to Production

#53
post #41
post #27

You can automate using puppeteer or selenium

The author used Playwright in the end to automate uploads. Using e2e tools for automating tasks is clever, I'm not sure I would've thought of it.

It's clever, but also brittle. And might have disastrous error conditions (like hitting "Delete" instead of "Continue" if the wrong UI part has focus).

Re: I Accidentally Deleted 7TB of Videos Before Going to Production

#54
post #4

> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…

I was involved with archiving of data that was legally required to be retained for PSD2 compliance. So it was pretty important that the data was correctly archived, but it was just as important that it was properly removed from other places due to data protection. This is basically the approach that was taken: log before and after every action exactly what data or files is being acted on and how. Don't actually do it…

Yes, I love the idea of the Plan Apply.

Re: I Accidentally Deleted 7TB of Videos Before Going to Production

#55
post #4

> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…

   SELECT COUNT(1) FROM table 
   -- UPDATE table SET col='val'
   WHERE 1=1

Re: I Accidentally Deleted 7TB of Videos Before Going to Production

#56

Earlier quoted context omitted.

Yes, I find command line tools that have a "--dry-run" flag to be very helpful. If the tool (or script or whatever) is performing some destructive or expensive change, then having the ability to ask "what do you think I want to do?" is great. It's like the difference between "do what I say" and "do what I mean"...

The rule we have is that anything that is not idempotent and not run as a matter of daily routine must dry-run by default, and not take action unless you pass --really. This has saved my bacon many times!

Deleting actually is idempotent. Doing it twice wont be different from doing it once.

Re: I Accidentally Deleted 7TB of Videos Before Going to Production

#57
post #4

> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…

SELECT COUNT(1) FROM table -- UPDATE table SET col='val' WHERE 1=1

    BEGIN TRANSACTION 
    UPDATE table SET col='val' WHERE 1=1
    ROLLBACK

Re: I Accidentally Deleted 7TB of Videos Before Going to Production

#59
post #4

> but at the time the code seemed completely correct to me It always does. > Well, it teaches me to do more diverse tests when doing destructive operations. Or add some logging and do a dry run and check the results, literally simple prints statements: print("-----") print("Downloading videos ids from url: {url}") print(list of ids) ... ... ... # delete() dangerous action commented out until I'm sure it's right print…

Experience is the best teacher™
Post reply on HN