Live data from Hacker News

Times I've Messed Up as a Developer

medium.com

111–120 of 137 posts

Re: Times I've Messed Up as a Developer

#111

Back in the nineties, at a previous employer, I wrote a code generator that read a kind of DSL from stdin and spat out C source to stdout. Having written the initial version and got it to compile, I thought I should test it before checking it into RCS for the first time. So I ran it with some simple input and piped the output to something like program.c. At that point I remembered that the code generator's main sourc…

I`ve done that on a national coding competition. Not fun at all, being timed and rewriting your program with itself at the last 30mins of the competition.

Re: Times I've Messed Up as a Developer

#112
I worked for a company once where a "consultant" was brought in to look at our "production efficiency". Some big wig came up with the idea that we should make sure we are using our production servers effectively. Not to many, not to few. We really only had 3 servers (Jerry, Kramer and Newman). Newman took a daily beating running some very large databases. It held firm though.

The "consultant" decided to move a database to its one machine. During this transition this person started a backup and then whacked the production database while the backup was still running because they felt the backup had "probably already backed up the database since it has been running for hours".

Dead in the water. Restored from a week old backup to the dismay of a lot of people working there whom had lost critical pieces of their work.

Re: Times I've Messed Up as a Developer

#113

Earlier quoted context omitted.

If you have never screwed up anything in production, no one at your org trusts you enough to give you production access. Which is ok and great for you, but at the end of the day SOMEONE has to have production access. You can write a devops template that wipes out data just as easy as you can sudo rm -rf /

I am much more terrified of Terraform than myself, especially `terraform destroy`. I mean, a human can only do so much so quickly but a script could kill everything semi-instantly. I look at `terraform plan -destroy` and it still terrifies to destroy an internal service.

We actually had a production outage due to a `terraform destroy` typed in production instead of dev.

Re: Times I've Messed Up as a Developer

#115
We had a good one recently where / got bind mounted to be used as a container rootfs in a quick and dirty test, then we decided to `rm -rf` the directory containing the bind mount. A clever way to get around pesky --no-preserve-root not being the default.

Re: Times I've Messed Up as a Developer

#116
post #94

Earlier quoted context omitted.

my go-to .bashrc hashes the hostname into a color and prints the prompt in that color. it has saved me from making a dumb mistake a few times.

Production prompts are red. Staging prompts are yellow. Development prompts are green.

Mine is exactly the same :-)

Re: Times I've Messed Up as a Developer

#117
post #110
post #83

Earlier quoted context omitted.

This alias seems to be set by default in many linux distributions these days (and has been that way for many years now). The problem is that you get used to this crutch and rely on it, until one day you're logged in someplace without the alias and end up deleting something you shouldn't. I'm not a fan and normally disable it. Instead, I prefer to be very careful and deliberate when using rm, especially with any wildc…

> This alias seems to be set by default in many linux distributions these days (and has been that way for many years now) Which Linux distributions do this? I have not seen this alias set by default on any distribution that I've used recently (Gentoo, Linux Mint, Debian, Ubuntu, Crunchbang, Arch).

I stand corrected, it doesn't seem to be the case anymore.

I can find old rpms and deb files with rm aliased in /etc/skel/.bashrc or /etc/skel/profile/aliases.sh, but the most recent I found is in a bash 3.2 package, so this practice seems to have stopped quite a few years ago.

Re: Times I've Messed Up as a Developer

#118
Another thing you can do to minimize the risk is limiting what your database user can do.

For instance, if you usually just need to view content. Don't let him truncate / delete / do other destructive operations.

Re: Times I've Messed Up as a Developer

#119
post #15
post #12

Earlier quoted context omitted.

Always write DELETEs as SELECTs, and once you run the SELECT and it returns the rows you want, then swap out the SELECT * with a DELETE.

That's good advice in general, not just for SQL. For example, I often write my 'rm' commands as 'ls' at first and then swap out the command once I know it gets the right thing.

I count to five before hitting enter. I have stopped on 3 or 4 a number of times. 5 seconds is frequently enough time for your brain to catch up with your fingers. And nobody is going to notice it took you 15 seconds to do something instead of 10.

Re: Times I've Messed Up as a Developer

#120

Fortune 500 company, Junior dev. Tried to log into our staging database, which has the same username, but different password than production. Tried 3 times, got the locked out error. Got frustrated, said I'd fix it after lunch, went out to eat, came back about 3 hours later after a doctor's appointment. My boss was waiting for me at the department door and I got sent back to his office for a stern talking to. I had l…

What a sadistic and irresponsible boss.
Post reply on HN