Live data from Hacker News

Why I distrust Google Cloud more than than AWS or Azure

iasylum.net

131–140 of 251 posts

Re: Why I distrust Google Cloud more than than AWS or Azure

#131
post #83
post #41

Can HN please add a filter for these increasingly lame "Google cancels all the stuff" posts? Yes, Google has cancelled services, but they've all been free things that they had every right to decide would never increase revenue. Why should Google have to keep everything they ever built running for ever? If you pay for services from Google, then it's a completely different story. We've used Appengine for 12 years now,…

> If you pay for services from Google, then it's a completely different story. We've used Appengine for 12 years now, and every time they've decided to deprecate services, there's always plenty of notice, a superior replacement, and usually lower costs. Really? I've had the complete opposite experience on AppEngine as a paying customer. I was using Python2 AppEngine with ndb and the Users API. Cloud Datastore + ndb a…

Hi-5 fellow "got caught across the boundary" traveller. This happened to me when I moved jobs to be devops lead on a new product for a company - the google recommended contractor implemented regular AppEngine with ndb, and about 6 months later we were staring down the barrel of everything being deprecated (and had to do the same: add a redis instance to the stack) and then hope that the new AppEngine was going to be ready when we actually went live.

It eventually came together but we ended up having to do a whole lot of refactoring while we were on a tight launch schedule.

Re: Why I distrust Google Cloud more than than AWS or Azure

#132

While I'm not arguing the general point of the article, I will counter point one thing. > Will Google Cloud even exist a decade from now? This seems wildly speculative, and the likelihood of GCP, or its core offerings, not existing any time so soon is next to zero. Google has to royally fuck up for this to be the case, but even if it ends up being case, there will be a string of lawsuits lined up that will likely cos…

Keeping GCP running to satisfy their contractual obligations and continuing GCP as a product sold to the general public are very different things. Shutting down any enterprise product tends to involve ending sales long before you actually shut it down.

Re: Why I distrust Google Cloud more than than AWS or Azure

#133

Earlier quoted context omitted.

Why isn't the failing tests the problem of the person who changed that line? What you describe is very strange. If someone changes a shared module, and some tests fail in result, his changes simply should not be merged. > you will get a ticket to understand what has changed, make changes and fire up every tests Do you fire tests manually?

If you make changes to a large shared module, is it your responsibility to chase down each and every usage of it? For example if you are upgrading a dependency due to a somewhat breaking security issue such as Jackson 2.8->2.12

Tldr yes it's the responsibility of the person doing the update to inform and usually fix the majority of users. There's often a long tail though.

Re: Why I distrust Google Cloud more than than AWS or Azure

#134

[DISCLAMER: I used to work at Google in general, but not at Google Cloud] I'm not sure whether this has been discussed here before, but I'd love to take this forum to share an angle from the tech side of things: IMO, Google is _cursed_ to keep deprecating its products and services. It's cursed by Google's famous choice of mono-repo tech stack. It makes all the sense and has all the benefits. But at a cost: we had to…

I have also worked at Google (in an unrelated department) and completely disagree with this. Maintaining old code & services is a problem everywhere. Monorepo vs multirepo, monolith service vs microservices etc. all have nothing to do with it. There will always be a broken dependency, a service/API/library you rely on about to deprecate, new urgent security patches, an outage somewhere upstream or downstream which you have to investigate, an important customer hitting an edge case which was hidden for years. You will always need a dedicated team to support a live product regardless of how it is engineered.

The problem at Google was (and maybe still is) with lack of incentives at the product level to do any of this. You don't get a fat bonus and promotion for saying that you kept things working as they should, made an incremental update or fixed bugs. When your packet goes up to the committee (who don't know you and know nothing about your team or background), the only thing that works in your favor is a successful new product launch.

And as an engineer you still have multiple avenues to showcase your skills. That new product manager you just hired from Harvard Business School who is eager to climb the ladder does not. And due to the lack of a central cohesive product strategy, this PM also has complete control of your team's annual roadmap.

Re: Why I distrust Google Cloud more than than AWS or Azure

#135

[DISCLAMER: I used to work at Google in general, but not at Google Cloud] I'm not sure whether this has been discussed here before, but I'd love to take this forum to share an angle from the tech side of things: IMO, Google is _cursed_ to keep deprecating its products and services. It's cursed by Google's famous choice of mono-repo tech stack. It makes all the sense and has all the benefits. But at a cost: we had to…

How does multi-repo codebase solve that problem? You would still need to keep up with your infra at minimum unless you run everything yourself too. Now you have another problem... Imo mono repo has little to do with it and it’s more just an eng culture of shipping above all else (heavily influenced by their promo process)

I don't know exactly what's going on at Google, but the key feature request seems to be that one chunk of code be able to depend on a consistent version of the library interfaces. If it wasn't a mono-repo, you could specify a dependency as a particular version of the other repository. But if everything is in the same repository, and one directory of code depends on a past version of library code, then everything falls apart. Keeping code that doesn't work in the mono repo, with its test last failing, is worse than deleting the code at the point that the API change breaks the other chunk of code.

Re: Why I distrust Google Cloud more than than AWS or Azure

#136
I've been in GCP support for over 4 years. My opinions are my own. I try to stay as impartial as I can about my employer. I know there are a lot of valid criticisms to be made about GCP. As one of the people often bearing the brunt of the fallout whenever there is a painful outage or deprecation, I share some of them.

But it never gets easy to read posts like this. This one appears to be a collection of old hacker news posts. And I can't help but think about all the posts that are never written, submitted, or upvoted about every time someone had a good experience with support. No one talks about their GCE VMs with years of uptime.

I'll spend hours on video calls with customers, going through logs, packet captures, perf profiles, core dumps, reading their code, conducting tests. Unpacking the tangled web until the problem is obvious. It's always a good feeling when we get to the end, and you get to reveal it like the end of a mystery novel. For me, that's the good part. Sometimes it takes a couple of hours. Sometimes weeks. Months even. And then the customer goes on with their life, as they should.

That's how it always should work. But no one talks about when a process works the way it's supposed to. People want to read about failures. And trade their own analyses about why that failure happened and how Google is fundamentally broken for these N simple reasons.

I don't want to diminish the negative stories as they are about people who went through real pain. I also realize that I'm just one person, and I can only work with so many customers in my time here. I'm not sure where I'm going with this.

I guess what I'm trying to say is, keep an open mind. This is a highly competitive field. There are strong incentives for GCP to listen to its customers.

Re: Why I distrust Google Cloud more than than AWS or Azure

#137
post #52

Earlier quoted context omitted.

I feel you on Cloudformation :S I haven't used Kubernetes on either platform - so there may be more to that. One thing I really dislike about GCP is how expensive it is for personal or hobby use. I burned through $300 for a simple vm on GCP in a few weeks because their cheapest instances are so expensive.

How? Their single core instance is like $25/mo, it took me months to burn through it

GKE in particular has a very high overhead cost per cluster. That's the only reason I bothered learning kOps and Terraform.

Re: Why I distrust Google Cloud more than than AWS or Azure

#138
post #107

Earlier quoted context omitted.

I'm shocked that any cloud provider lets you use prepaid cards... all of the major providers have problems with crypto mining and abuse, so it's crazy that AWS would allow prepaid cards that might be laundered, etc. And IMO, if you're a real customer, all the providers are fairly forgiving, provided you can get in touch with a real human who works there.

Google is infamous for making it impossible to reach a real human. Their DNA is all about automating all the things.

Also, if you're paying GCP any reasonable amount of money, you have an account manager who will respond in < 24 hours to connect you with the right PM to deal with the issue. Google deals with lots of random humans, GCP mostly deals with actual businesses. As much as I hated the support offshoring Google (and sometimes GCP) did, most actual businesses could get a human fairly quickly.

Re: Why I distrust Google Cloud more than than AWS or Azure

#139

[DISCLAMER: I used to work at Google in general, but not at Google Cloud] I'm not sure whether this has been discussed here before, but I'd love to take this forum to share an angle from the tech side of things: IMO, Google is _cursed_ to keep deprecating its products and services. It's cursed by Google's famous choice of mono-repo tech stack. It makes all the sense and has all the benefits. But at a cost: we had to…

How does multi-repo codebase solve that problem? You would still need to keep up with your infra at minimum unless you run everything yourself too. Now you have another problem... Imo mono repo has little to do with it and it’s more just an eng culture of shipping above all else (heavily influenced by their promo process)

its easier to park a codebase and have it rot when its not in a monorepo where everything is assumed to be consistent/working all the time.

I'd have thought you could just pull maintenance-mode products out of the monorepo tree and stash them somewhere else. Let it rot by choice. Is basically what everyone else does, let you perform maintenance tasks on your schedule not other monorepo participants schedule.

Re: Why I distrust Google Cloud more than than AWS or Azure

#140

[DISCLAMER: I used to work at Google in general, but not at Google Cloud] I'm not sure whether this has been discussed here before, but I'd love to take this forum to share an angle from the tech side of things: IMO, Google is _cursed_ to keep deprecating its products and services. It's cursed by Google's famous choice of mono-repo tech stack. It makes all the sense and has all the benefits. But at a cost: we had to…

I work for Google for the 3rd time; 12+ years in total. It's first time when this issue is brought to my conscience. I think that the issue exists, but it's not due to the monorepo, it's due to the internal APIs changing.

I learned how to avoid the Google3 tax that you mentioned, when the old thing is deprecated, and the new one is not working yet.

Surprisingly, the answer for me was to embrace Google Cloud: its APIs have stability guarantees. My current project depends on Google Cloud Storage, Cloud Spanner, Cloud Firestore and very few internal technologies.

I believe that this is in general a trend at Google: increasing reliance on Google Cloud for new projects. In this sense, both internal and external developers are in the same boat.

As for the monorepo - it's a blessing, in my perspective. Less rotten code, much easier to contribute improvements across the stack.

Post reply on HN