Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Cloud networking issues in us-east1

status.cloud.google.com

Post reply on HN
Print view View original

281–290 of 341 posts

Re: Google Cloud networking issues in us-east1

#281

Post by joshuamorton »

praneshp wrote: post #268
bardworx wrote: post #213

Earlier quoted context omitted.

That’s...that’s some petty fucking shit. I didn’t go through your comments but considering your email is in your profile, someone really had to have a hard-on to report you for leaks. I would love to understand the though process of someone going out of their way to remove someone’s livelihood from them because of a comment on HN (when applied in a normal circumstance of adding additional information or correcting a…

> I would love to understand the though process of someone going out of their way to remove someone’s livelihood from them because of a comment on HN This is not so difficult though. You just need to adjust your starting point to someone who doesn't like boulos' first. That's not so difficult IMO, it's a large org and boulos' seems to be a fairly prolific commenter here.

It also could be someone will intentioned who believes boulos is sharing things he shouldn't be.

He certainly shares stuff I wouldn't be comfortable sharing, but then again he's a lot better connected and in the know than I am.

Re: Google Cloud networking issues in us-east1

#282

Post by titanomachy »

IX-103 wrote: post #248
kyrra wrote: post #197

Earlier quoted context omitted.

It's a fine line. We are not allowed to represent Google in any kind of public discussion. But we can talk about some things we do, as long as we state it's our own opinion and we don't represent Google's views.

And don't disclose material nonpublic information (since that would run afoul of insider trading laws). It's probably okay to say that we know the problem and here are the steps we're taking to mitigate it. It would not be okay to say something with large scale stock price implications for Google it another publicly traded corporation. For instance a Google employee shouldn't say something like "faulty solar panels f…

I'm not a lawyer, but if the information is false I don't think you could get dinged for insider trading. The legal approach that's used to prosecute insider trading is basically "theft of secrets".

Re: Google Cloud networking issues in us-east1

#283

Post by zaphirplane »

9nGQluzmnq3M wrote: post #277
akhilcacharya wrote: post #264

Earlier quoted context omitted.

It is - but all companies I’ve ever worked for are. I’m not convinced the letter of the policy is much different.

There are definitely major differences between the FAANGs: what was the last time you saw an Apple employee commenting on anything on HN?

Definitely not all FADANGs are the same. never seen a Disney employee comment ;) Or oracle in FADANGOs ;) just kidding

Re: Google Cloud networking issues in us-east1

#284

Post by xapata »

mehrdadn wrote: post #201

Does anybody else feel like there have been a lot of outages in recent months? And I don't mean Google -- I mean lots of others too (I seem to recall CloudFlare, Facebook, etc.)... are they really increasing or are we just hearing more about them? Seems a bit odd.

Tinfoil hat: Maybe someone practicing for an attack?

Re: Google Cloud networking issues in us-east1

#285

Post by kalleboo »

owenmarshall wrote: post #235
m0zg wrote: post #221

Earlier quoted context omitted.

I'll tell you more. Much of the world is run by "horribly out of date" systems that nobody has touched in years _because they work_. And it all works fine. No "cloud", no Rust or Go rewrites, no Haskell, no fancy javascript frameworks or anything like that. Just boring ol' files, boring relational DBs with boring schemas, constraints and stored procedures, boring old languages, boring old hardware, boring old operati…

I got an email yesterday that told me the boring old HPUX server (which was racked before my intern was even born) barfed all over its boring old 50-pin SCSI drive and ops went scrambling to find one in storage so the boring old Oracle DB that was responsible for production lines running could be recovered. Took us around an hour, cost us a boring 5 figures. Luckily our sysadmin knows how to hide “unused parts” for d…

There's boring and there's legacy. Legacy is when the hardware is unsupported and doesn't get software/security updates. You don't want to let it become legacy.

The counter-story to yours is running that database on MongoDB in the cloud on a cluster. Instead you'd be having crazy MongoDB issues, data inconsistencies, connectivity issues when the cloud is down, etc etc.

The solution is somewhere in the middle. You can have modern, supported hardware running a LTS Linux and that counts as boring.

Re: Google Cloud networking issues in us-east1

#286

Post by mrmattyboy »

mirceal wrote: post #96
mrmattyboy wrote: post #30

To whomever commented something like 'laughs in AWS' (comment was removed before I submitted the comment)... please don't... glass house and all that... but I also share the same glass house as you.. I don't want bad luck ... and it's only a fluke that this happened to google in eu-east1 and not AWS in X region and then you (and I) would be having a time of hell! :/

i don't quite follow your logic. something about glass houses and bad luck? the whole point when something like this happens is for you to ensure that a region going down will not impact you - not to laugh at people that use another cloud or to assume that X is better than Y. That being said, there have been several Google related failures lately that don't help building confidence in the GCP offering - if you're jus…

My point was that, there was a comment from someone saying 'laughing from aws' and I was trying to point out that each service is most likely (or should be considered to be) as fragile (in the relative sense) as each other. So just because google have gone down several times, doesn't mean that AWS won't have a line of outages next. Really, their services are much of a black hole to us.. we can't see _how_ they deploy their changes, what kind of reviewing they do etc. etc. Even down to how cleverly they have _actually_ architected their DCs.

So my point was to _not_ to laugh at those at google (or those using their services), because AWS might be next.

The whole 'I share the same glass house', was a sort of karma thing.. if someone who uses AWS is laughing at Google. If karma came round and took out AWS, not only would it affect the guy laughing at google, but I'd be the one affected as well as a multitude of other people... and the tables could be easily turned

Re: Google Cloud networking issues in us-east1

#287

Post by sneak »

lwb wrote: post #131
CaptainJustin wrote: post #101

Earlier quoted context omitted.

It's quite common in cloud solution design to design for failure. One of the common assumptions that we hold to is that one region may go down. Other examples: Assume an instance of an app can go down. Assume a VM can go down. Assume a DC can go down. This is not to excuse the downtime in any way.

Do people ever worry that an entire cloud provider may go down, or is that too unlikely of a case?

Billing issues can take down your entire account at a given cloud provider all at once.

Re: Google Cloud networking issues in us-east1

#288

Post by t0astbread »

m0zg wrote: post #221
Operyl wrote: post #218

Earlier quoted context omitted.

My horribly out of date system works, therefore I should never strive to improve it or god forbid update it (since that involves “fucking with it” in ways that can break it from version to version)? That gets you technical debt and that’s not fun.

I'll tell you more. Much of the world is run by "horribly out of date" systems that nobody has touched in years _because they work_. And it all works fine. No "cloud", no Rust or Go rewrites, no Haskell, no fancy javascript frameworks or anything like that. Just boring ol' files, boring relational DBs with boring schemas, constraints and stored procedures, boring old languages, boring old hardware, boring old operati…

> Don't screw with it and it will work for a decade.

Don't screw with it and it will have security issues after a few months?

Re: Google Cloud networking issues in us-east1

#289

Post by FrankPetrilli »

edwintorok wrote: post #198
boulos wrote: post #136

Disclosure: I work on Google Cloud (but I'm not in SRE, oncall, etc.). As the updates to [1] say, we're working to resolve a networking issue. The Region isn't (and wasn't) "down", but obviously network latency spiking up for external connectivity is bad. We are currently experiencing an issue with a subset of the fiber paths that supply the region. We're working on getting that restored. In the meantime, we've remov…

You seem to have 3 status messages on the dashboard at 14:31, 14:44 and 14:48 with exactly the same contents. Were those messages really posted 3 times, or did something go wrong and they got duplicated?

We're aware this happened - that posting is the responsibility of an adjacent team to my own, specifically the person right next to me. :)
Post reply on HN