Live data from Hacker News

The anatomy of a 2AM mental breakdown

zarar.dev

51–60 of 286 posts

Re: The anatomy of a 2AM mental breakdown

#51
post #30

Earlier quoted context omitted.

This is the entire risk: being fired and financially in trouble. I spent 5 years eliminating that risk. Now I don't give a single fuck. There is no fear. They get better, more rational work and I have security.

How did you get to this point? I have $2M+ net worth, and while I live somewhere rent controlled, $2M isn't that much in the Bay Area. I'm still scared to lose my job.

It's not net worth. It's how long you can abide. If you have $2M net worth and $1.99 is tied up in a way you can't spend on bills, that don't mean nothin'. If things got bad, I guess you could move. You could probably buy a house now in fly-over country and just retire, living on interest from conservative investments.

Re: The anatomy of a 2AM mental breakdown

#52

Working as a SRE for a year in a large global company broke me out of this "panic" mode described in this post. To a business, every problem seems like a world-ending event. It's very easy to give in to panic in those situations. However, in reality, it's rarely that bad, and even if it is, you'll probably survive without harm. The key in these situations, and what I try to do (totally relate to breaking out in a swe…

For me, in situations like this, the fear and anxiety comes from the part of my brain screaming, "but what if I can NEVER fix this?" Which is of course completely nonsense. There is ALWAYS a solution somewhere, even if it's VERY hard to find, or even if it's not the one you want.

My first serious job as an adult was avionics repair. I knew how to do SOME component-level troubleshooting but I wasn't very good at it and always got lost in schematics pretty quickly. When I ran across a broken device that had a problem I couldn't solve within 30 minutes of investigation, my first instinct was to say, "welp, this is beyond my abilities" and ship the card or box off as permanently broken, or try to pawn it off to a more senior tech in the shop.

Fortunately, I had a mentor who disabused me of that tendency. He taught me that EVERY fault has a cause. Every time you think you've exhausted all options, what it really means is that you've only exhausted all CONVENIENT options and you just have to suck it up and get cracking on the harder paths.

Re: The anatomy of a 2AM mental breakdown

#53
post #30

Earlier quoted context omitted.

This is the entire risk: being fired and financially in trouble. I spent 5 years eliminating that risk. Now I don't give a single fuck. There is no fear. They get better, more rational work and I have security.

How did you get to this point? I have $2M+ net worth, and while I live somewhere rent controlled, $2M isn't that much in the Bay Area. I'm still scared to lose my job.

[deleted]

Re: The anatomy of a 2AM mental breakdown

#54

Working as a SRE for a year in a large global company broke me out of this "panic" mode described in this post. To a business, every problem seems like a world-ending event. It's very easy to give in to panic in those situations. However, in reality, it's rarely that bad, and even if it is, you'll probably survive without harm. The key in these situations, and what I try to do (totally relate to breaking out in a swe…

Bear in mind that when something "breaks" a business can freak out, but they're not freaking out about lots of other arguably more important problems:

- The software they purchased doesn't do anything because it was never staffed or configured correctly.

- Employees as a whole lose thousands of hours a year to terrible UX or meaningless requirements.

- Some capability doesn't actually do anything, but no one cares enough to notice.

- Useless meetings which waste collective employee hours daily.

- Capability serves no purpose, but just exists to meet audit requirements.

- Executives enrich themselves and waste company money constantly.

- Etc.

Downtime doesn't seem any worse than these problems, but it gets far more attention and panic. It feels a bit like terrorism vs. heart disease. It's important not to sweat it, because a company does not care about your sleep or mental health. It will push you as far as it can. I'm not calling companies malevolent, but they're a bit like a bully in that regard: they push you as much as you yield.

Re: The anatomy of a 2AM mental breakdown

#55
post #30

Earlier quoted context omitted.

This is the entire risk: being fired and financially in trouble. I spent 5 years eliminating that risk. Now I don't give a single fuck. There is no fear. They get better, more rational work and I have security.

How did you get to this point? I have $2M+ net worth, and while I live somewhere rent controlled, $2M isn't that much in the Bay Area. I'm still scared to lose my job.

At 2M net worth, the only thing that is making you fearful is your own poor assessment of risk/regret in life.

You are more likely to regret taking large debts or missing out on life milestones than on losing the job or going hungry.

Re: The anatomy of a 2AM mental breakdown

#56
post #2

[flagged]

Thank you for this. The article started interesting, but then devolved in boring all caps recap of panic thoughts which killed its momentum. I was still interested in the origin of the bug though.

You can always skip to the end if it bores you. I liked it.

Re: The anatomy of a 2AM mental breakdown

#57

Looks like the bug was in a monkey-patched `window.fetch` https://github.com/PostHog/posthog-js/blob/759829c67fcb8720f... The biggest lesson here is, if you're writing a popular library that monkey-patches global functions, it needs to be really well tested. There's a difference between "I'll throw posthog calls in a try/catch just in case" and "With posthog I literally can't make fetch() calls with POST"

Thank you for pointing this out. I did not read the post in detail but I was wondering how could a monitoring library cause the entire application to go down. At worst, I thought, it should have failed to process the monitoring events, assuming that it was integrated in a reasonable way.

PostHog patching a very important global function is a feature that should be well-documented so that the people who are using it are aware of it and can be reasonably expected to have it in mind when debugging these seemingly unexplainable issues.

Re: The anatomy of a 2AM mental breakdown

#58
post #34

> This is no good. Let me just try reverting to a version from a month ago. Nothing. Three months ago? Nothing. Still failing. A year ago? Zilch. Reverting your own code, but still using a broken PostHog update from that same day? For me, the lesson is to make sure that I can revert everything, including dependencies.

It seems that PostHog just always loads the latest version of this piece of itself: https://github.com/PostHog/posthog/issues/24471#issuecomment... Though you can opt to bundle it yourself: https://github.com/PostHog/posthog/issues/24471#issuecomment...

> I definitely want to figure out in detail what happened here so I can add a test to prevent a similar change in future!

Whoa! Good idea!

Could have been worse. At least the change didn't expose a hidden exploit.

Re: The anatomy of a 2AM mental breakdown

#59

Working as a SRE for a year in a large global company broke me out of this "panic" mode described in this post. To a business, every problem seems like a world-ending event. It's very easy to give in to panic in those situations. However, in reality, it's rarely that bad, and even if it is, you'll probably survive without harm. The key in these situations, and what I try to do (totally relate to breaking out in a swe…

Bear in mind that when something "breaks" a business can freak out, but they're not freaking out about lots of other arguably more important problems: - The software they purchased doesn't do anything because it was never staffed or configured correctly. - Employees as a whole lose thousands of hours a year to terrible UX or meaningless requirements. - Some capability doesn't actually do anything, but no one cares en…

> It feels a bit like terrorism vs. heart disease.

Fantastic analogy! One is scary but practically non-existent, and the other will bring early death to many people you know.

Post reply on HN