Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The anatomy of a 2AM mental breakdown

zarar.dev

Post reply on HN
Print view View original

151–160 of 286 posts

Re: The anatomy of a 2AM mental breakdown

#151

Post by x0x0 »

bluepnume wrote: post #36

Looks like the bug was in a monkey-patched `window.fetch` https://github.com/PostHog/posthog-js/blob/759829c67fcb8720f... The biggest lesson here is, if you're writing a popular library that monkey-patches global functions, it needs to be really well tested. There's a difference between "I'll throw posthog calls in a try/catch just in case" and "With posthog I literally can't make fetch() calls with POST"

it's common with all these analytics toolkits. When you're monkeying with such a core api, I have no idea how you can actually test everything.

eg heap analytics still (as of this month) bad touches something inside hotwire and randomly entirely breaks hotwire causing every click to do a full page load. ime, it affects 30-60% of page loads. You can fix it (but thanks for 50+ hours of debugging) but making heap load after all hotwire js.

Re: The anatomy of a 2AM mental breakdown

#152

Post by mlyle »

vlunkr wrote: post #98
everdrive wrote: post #54

Earlier quoted context omitted.

Bear in mind that when something "breaks" a business can freak out, but they're not freaking out about lots of other arguably more important problems: - The software they purchased doesn't do anything because it was never staffed or configured correctly. - Employees as a whole lose thousands of hours a year to terrible UX or meaningless requirements. - Some capability doesn't actually do anything, but no one cares en…

The zero-downtime culture is pretty out of control. Your flight can be delayed for hours, your roads can be closed for weeks, your internet can be out for hours because someone cut a line, that's all ok, but someone can't get into your website for 3 minutes to check the status of their order and we all must lose our minds and publicly apologize and explain in detail what happened and why it will never happen again.

You know, I think if there was a little more tendency to explain, and trust in those explanations, people would like airlines, local transportation departments, etc, a whole lot more.

Without information, it is easy to assume incompetence, or worse, just not caring.

Re: The anatomy of a 2AM mental breakdown

#153

Post by DylanSp »

sb8244 wrote: post #135
JohnMakin wrote: post #13

Working as a SRE for a year in a large global company broke me out of this "panic" mode described in this post. To a business, every problem seems like a world-ending event. It's very easy to give in to panic in those situations. However, in reality, it's rarely that bad, and even if it is, you'll probably survive without harm. The key in these situations, and what I try to do (totally relate to breaking out in a swe…

Some of the worst mistakes that I saw were from over-reaction in an active incident. One of my programming mantras is "no black magic." If I don't understand why something works, then it's not done. I take this same approach to an incident. If someone can't coherently identify why their suggestion will have an impact, I don't think they should do it. Now there may come a time that you need to just pull the trigger on…

To quote Gene Kranz during the start of problems on Apollo 13: "Let's not make things worse by guessing."

Re: The anatomy of a 2AM mental breakdown

#154

Post by digging »

Der_Einzige wrote: post #122
sillysaurusx wrote: post #61

For what it's worth, I'm not sure this is a mental breakdown, and it might give the wrong impression to people who do legitimately suffer breakdowns related to stress about tech. For me it's only happened once. It was an anxiety attack, and I'm very lucky my wife was there to talk me through it and help me understand what was happening. She's had them many times, but it was my first (and thankfully only). It turns ou…

One thing I did not expect about working in enterprise tech companies was the amount of coworkers who popped Xanax regularly. I've been an anxiety basket case my whole life and the idea that an addictive pill could take it all away terrifies me - I'd be hooked for life.

> I'd be hooked for life.

I mean, that's how a lot of prescriptions work anyway. I take daily medication to prevent anxiety and depression. It's not addictive, but I try pretty hard to never miss a day, because anxiety and depression will get in the way of me living my life. Assuming Xanax is safe to take daily (I have no idea), the only difference is that a supply disruption would have side-effects. But if I run out of my medication, I'll get depressed within weeks, almost guaranteed, withdrawals or no.

Re: The anatomy of a 2AM mental breakdown

#155

Post by hn83746 »

zappchance wrote: post #72
mtnviewer wrote: post #46

Earlier quoted context omitted.

How did you get to this point? I have $2M+ net worth, and while I live somewhere rent controlled, $2M isn't that much in the Bay Area. I'm still scared to lose my job.

If you're scared about your financial situation with over a $2M net worth either you are prone to paranoia or you're living well above your means. I'm constantly surprised how many well-off people try to pass themselves off as "one of the poors". My family and friends do okay with a net worth way below that, in the Bay Area. I've worked minimum wage jobs here in the bay, and had co-workers actually struggling to make…

It makes especially... interesting reading to those of us from the global South. To make $2 mil at my current salary, I will have to work for about 100-150 years, saving every penny and spending nothing, and I make decent money by our standards. There's even no relatively safe way to make some smart investments (which I imagine is how most of this money was acquired) because of incessant economical crises and non-existence of startup culture.

Man, some people here are completely out of touch with the average Eathlican. I'll have to show this discussion to my friends.

With that kind of money you could just... you know, move pretty much anywhere on the planet and live comfortably for many years, until a good opportunity comes up. Depending on the place, it might last you the entire lifetime (see my salary above).

Re: The anatomy of a 2AM mental breakdown

#156

Post by supportengineer »

Muromec wrote: post #67
mtnviewer wrote: post #46

Earlier quoted context omitted.

How did you get to this point? I have $2M+ net worth, and while I live somewhere rent controlled, $2M isn't that much in the Bay Area. I'm still scared to lose my job.

Why do you even bother to work if you have 2M ?

Doesn't help you much if you are a family in Bay Area.

Re: The anatomy of a 2AM mental breakdown

#157

Post by prerok »

everdrive wrote: post #86
caseyy wrote: post #77

Earlier quoted context omitted.

> It feels a bit like terrorism [..] It’s important not to sweat it Hmm… interesting thought. It’s interesting how labeling something a terror attack makes the public lose critical thinking and act incoherently for a bit. It’s like a stunning shock response. The very same this author describes. Do you think some managers intentionally play up the severity of problems to push their teams into this panic mode? People i…

>Do you think some managers intentionally play up the severity of problems to push their teams into this panic mode? I'm sure some do. Much more often I observe that the managers themselves are just reacting to the incentive system in their environment -- and that those incentive systems are complex, not-widely-understood, and often built "accidentally." ie, companies are often accidentally incentivizing the wrong be…

I am almost convinced most middle management are like small babies. That's not meant derogatively: there's a problem, they have been notified, yet they cannot do anything about it. They need someone else to fix it. So, they cry loud and clear that there is a need to fix this.

It's difficult to understand sometimes, but they are pressured from above, and they don't have the means to fix it by themselves.

Re: The anatomy of a 2AM mental breakdown

#158

Post by supportengineer »

Der_Einzige wrote: post #122
sillysaurusx wrote: post #61

For what it's worth, I'm not sure this is a mental breakdown, and it might give the wrong impression to people who do legitimately suffer breakdowns related to stress about tech. For me it's only happened once. It was an anxiety attack, and I'm very lucky my wife was there to talk me through it and help me understand what was happening. She's had them many times, but it was my first (and thankfully only). It turns ou…

One thing I did not expect about working in enterprise tech companies was the amount of coworkers who popped Xanax regularly. I've been an anxiety basket case my whole life and the idea that an addictive pill could take it all away terrifies me - I'd be hooked for life.

It's hard to compete against the people who are "enhanced" with drugs.

Re: The anatomy of a 2AM mental breakdown

#159

Post by dennis_jeeves2 »

vvoruganti wrote: post #137

A mantra I heard recently that has been helping me with my own 3AM panics was "None of this matters and we're all gonna die". A bit Nihilist maybe, but has been helpful and just kind of removing the weight of the situation

Good thinking. Sometimes I have to remind my nerdy colleagues about what a true emergency is. A true emergency is when your granny is having a heart attack and has to be rushed to the hospital. When a big corporate website, that does millions of dollars of transactions is down at night it is NOT an emergency. They have to hire staff specifically for night shifts if they are so concerned about the money that they may lose.

Re: The anatomy of a 2AM mental breakdown

#160

Post by Alupis »

vlunkr wrote: post #148
Alupis wrote: post #113

Earlier quoted context omitted.

The obvious difference is no matter how long the county or city takes to reopen that road - you'll go right back to using it because you don't have an alternate choice. For a website - particularly ecommerce websites - you have many choices. Rarely is a product only sold on a particular website. Being down when someone is trying to place an order can and does result in losing the sale, and potentially the customer fo…

That really depends on what type of service we're talking about. Yeah, if you're doing really high volume sales on products that can be purchased elsewhere for close to the same price and shipping options, then uptime is critical. That's not the situation most of us are in.

> That's not the situation most of us are in.

From an ecommerce website's perspective - it is exactly the situation most of us are in.

Unless you are the manufacturer for your own goods, then you are competing on often-thin margins within a crowded industry with lots of competition where everyone more-or-less offers the same price.

In an age where more and more people don't even consider shopping anywhere but Amazon, being down exactly when that customer decided to grace your website is akin to giving yourself a nice, stinging papercut. Do this often enough and long enough, those papercuts will start to leave scars.

So ya, being down is unacceptable for most ecommerce websites. The website literally generates your revenue, and without it you receive no revenue.

Post reply on HN