Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Public CA is down

status.pki.goog

Post reply on HN
Print view View original

111–120 of 166 posts

Re: Google Public CA is down

#111

Post by 0_____0 »

PostOnce wrote: post #81
TMEHpodcast wrote: post #43

It is a well-known fact that the moment YouTube goes down, the collective productivity of Earth increases by approximately 4,000%, which is immediately squandered by everyone going to Hacker News to read comments about YouTube being down. I myself have taken to podcasts… an ancient medium in which people simply talk at you for ninety minutes without a single sponsorship for a mobile game, and this is considered a fai…

They've begun injecting obnoxious ads into the downloadable mp3s on a lot of podcasts I've found. Hyperlocal ads for tire shops and bakeries. I don't want to buy tires, I want to learn about ______. The ads don't even make sense because they're irrelevant.

VPN to Sweden to get the IP geolocated ads to retarget. The ads still exist but they're less obnoxious, and they're often in Swedish so you don't have to know what they're on about anyway.

Re: Google Public CA is down

#113

Post by [deleted] »

tzs wrote: post #79
msie wrote: post #56

Earlier quoted context omitted.

I was thinking about the time some software influencer said that if you are afraid to deploy on Friday then there's something wrong with you. Eff that! Murphy's Law! (allen holub - https://x.com/allenholub/status/1637111242610610182 )

I often deployed on Friday evening. Several factors contributed to this decision. 1. Sales volume was lowest on weekends so if something went wrong it would affect fewer customers. 2. If something went wrong and I needed to revert, nobody was at work on weekends so it would not disrupt coworkers. 3. I always made it so reverting would be easy. 4. Most of my weekends were just relaxing at home, mostly doing online stu…

[deleted]

Re: Google Public CA is down

#114

Post by GeekyBear »

Arainach wrote: post #64
bigbuppo wrote: post #45

Earlier quoted context omitted.

It's inevitable that one of the major cloud providers will irrecoverably delete all customer data with one single fat-fingered command. Though in google's case I'll also consider the prophecy to be fulfilled if they delete their own data. It will forever be known as The Great Oops.

It's not inevitable, it's essentially impossible. There are a few things that can cause tremendously widespread outages , essentially all of them network configuration changes. Actually deleting customer data is dramatically more difficult to the point of impossible - there are so many different services in so many different locations with so many layers of access control. There is no "one command" that can do such a…

Google accidentally deleted customer location history data from customer devices (after intentionally deleting it from Google servers) just last year.

If didn't back it up yourself, it is gone forever.

Re: Google Public CA is down

#116

Post by bigbuppo »

Arainach wrote: post #106
ocdtrekkie wrote: post #95

Earlier quoted context omitted.

It's pretty much half the puzzle actually. You contend there's no global rm rf for a global cloud provider, but clearly a missing parameter can rm rf a customer in an irrecoverable manner. The only half you're missing is... how every major cloud outage happens today... a bad configuration update. These companies have hundreds of thousands of servers, but they also use orchestration tools to distribute sets of changes…

The release process, monitoring checks, etc. for a customer's private cloud is generally significantly different from the release process for a global product. I'm not going to get any more specific for all the standard NDA reasons, but having worked for Google and Microsoft among others....no, the risk you describe doesn't translate from one to the other.

Do you not remember crowdstrike?

Re: Google Public CA is down

#117

Post by bigbuppo »

rossjudson wrote: post #110
Arainach wrote: post #64

Earlier quoted context omitted.

It's not inevitable, it's essentially impossible. There are a few things that can cause tremendously widespread outages , essentially all of them network configuration changes. Actually deleting customer data is dramatically more difficult to the point of impossible - there are so many different services in so many different locations with so many layers of access control. There is no "one command" that can do such a…

Delete a decryption key. Good luck! I'll see you at the end of time. Break your control plane, and you can't stop the propagation of poison. Propagate the wrong trust bundle... everywhere. Also, it's not about the delete command. It's about the automatic cleanup following behind it that shreds everything, or repurposes the storage.

Children of the kubernetic line.

Re: Google Public CA is down

#118

Post by altairprime »

altairprime wrote: post #38
jtokoph wrote: post #24

> A fix to resolve the issue will roll out in about 8 hours oof

That feeling when you have to suspend production service until the time lock safe can be opened.

That feeling when you finally get the timelock safe open and have to do certificate work that shatters YouTube’s connection to the account personalization systems.

Re: Google Public CA is down

#119

Post by marak830 »

0_____0 wrote: post #111
PostOnce wrote: post #81

Earlier quoted context omitted.

They've begun injecting obnoxious ads into the downloadable mp3s on a lot of podcasts I've found. Hyperlocal ads for tire shops and bakeries. I don't want to buy tires, I want to learn about ______. The ads don't even make sense because they're irrelevant.

VPN to Sweden to get the IP geolocated ads to retarget. The ads still exist but they're less obnoxious, and they're often in Swedish so you don't have to know what they're on about anyway.

Careful, I enjoyed this bonus (being in Japan and not being able to keep up with the ads)... so much so, that I started ignoring the Japanese. Including my wife. You can imagine how well that went.

Re: Google Public CA is down

#120

Post by blackoil »

moregrist wrote: post #99
PostOnce wrote: post #81

Earlier quoted context omitted.

They've begun injecting obnoxious ads into the downloadable mp3s on a lot of podcasts I've found. Hyperlocal ads for tire shops and bakeries. I don't want to buy tires, I want to learn about ______. The ads don't even make sense because they're irrelevant.

Welcome to radio 2.0. Give it another 10-20 years and your 2 hour podcasts will be 30 minutes of morning zoo DJ banter, 10 minutes of guests, and 1.5 hours of ads. We’ll have reached peak 90s all over again. With any luck we’ll avoid recreating the conditions for another Nickelback and can stay in the weird zone where Trip Hop and pop punk could chart at the same time.

> 2 hour podcasts

You have high hopes. Next YT tool will be to split anything long in 30s reels as brains will be completely incapable of focusing for longer.

Post reply on HN