Live data from Hacker News

Spotify moves its back end to Google Cloud

news.spotify.com

131–140 of 397 posts

Re: Spotify moves its back end to Google Cloud

#131

Earlier quoted context omitted.

> infinitely auto-scalable platform where we don't have to do any devops or carry a pager You never know what you don't measure and test

Are you implying that we don't measure and test? We do and I'm very happy with the results. =) StackDriver, Google Cloud console, Intercom, Mixpanel and Pingdom are all being used as well.

If that's the case then you might actually be "doing devops". If pingdom is notifying you via sms then you may also be carrying a pager.

Re: Spotify moves its back end to Google Cloud

#132
post #66

I'm curious regarding the economics of this but could anyone approximate the cost associated with this move or expenses going forward. It sounds like this would be a pretty big account for Google.

You can bet your bottom dollar Spotify are getting heavy, heavy discounts. Google are ramping up here in Stockholm, partly to get this account. This has been in the works since before last summer, so they've taken time before announcing it. The precedent for Spotify was switching from Cloudera to Hortonworks a couple of years ago. Rumours are that they got their licenses for close to nothing from Hortonworks, they we…

Just with respect to some details: Spotify changed from being a self-supporting (free) user of Cloudera's open source platform to being a paid support customer of Hortonworks. So, it was never actually a Cloudera customer (despite what was widely "reported").

Disclaimer: Cloudera emp here.

Re: Spotify moves its back end to Google Cloud

#133

Earlier quoted context omitted.

I thought this as well. They have. This is Androids answer to Apple Music. edit: This also seemingly posted from a shelf account. Front page on HN, account 530+ days old, 0 comments, 0 posts. Reads like an acquisition. I am not sure what to make of this as it could be a total coincidence that after a year and a half dmichel was really interested in spotify's backend migration, but either way I wouldn't be surprised i…

Just an employee of spotify excited about the transition (I usually just lurk)

hahaha awesome! Glad you spoke up, exciting news. Just started using spotify a few weeks ago, but it is an awesome product. Cheers.

Re: Spotify moves its back end to Google Cloud

#134

What Google Cloud needs is a better sales story. AWS consistently beats out Google in this respect. This story is welcomed and there should be more success stories published like this. My business is a heavy GC user (AppEngine, Datastore, CloudStorage, BQ, ComputeEngine, ManagedVM) and I couldn't imagine achieving what we have achieved in such a short amount of time on any other platform. We (myself and another guy)…

I'm on mobile so this initial response will be short:

AWS' core product is AWS. They do a damn fine job of providing this.

GC's core is "who knows what" - they do a damn poor job of even making an offering if switch thousands and thousands of servers to - let alone thousands and thousands of man hours.

Ironically, Google has massive domain knowledge of how to run infra... Yet clueless how to adopt and support and foster companies other than google.

It sucks.

But I've been in the business of fork lifting many people to aws, for all the reasons you'd expect:

Find me a single ops eng on the market who has done scale in GC? Nope.

EDIT: may I please have a rebuttal?

Re: Spotify moves its back end to Google Cloud

#135
post #61

Earlier quoted context omitted.

Seems to be a pattern: wimpy mobile endpoint devices drive everything to the cloud. If so this is perhaps a dated phenomenon. Moore's Law is still a thing and today's generation of mobile devices are getting fatter and fatter.

Moore's Law has not been improving batteries, which is the real problem. Mobile P2P nodes need to be awake most/all the time, which is a real battery killer. Modern devices have improved battery life by being asleep almost all the time.

Depends on what those p2p nodes are doing. We do p2p on mobile but it's to allow the mobile device to act as a client to access an IoT device, not to force the mobile device to act as a CDN node. Spotify was doing the latter, which is more problematic.

Moore's Law eventually will matter, since faster and more efficient chips equal the ability to do more with lower power consumption. Batteries aren't getting better fast enough, but power consumption per fixed unit of compute is falling like a rock. As a result "effective battery life" for a given fixed workload is subject to Moore's Law.

Re: Spotify moves its back end to Google Cloud

#136
post #108

Earlier quoted context omitted.

Google Cloud SQL is no replacement for Amazon RDS in my opinion. Cloud SQL instances run outside of your project's private network, so connections from Google Compute Engine have to be over a Public IP. This means accepting connections from any host (insecure) or whitelisting each Google Compute Engine VM's IP address (pain in the ass). I've resorted to running my own MySQL instance inside of Google Compute Engine an…

Look no further (for item 1 of 2 anyway) — http://googlecloudplatform.blogspot.co.nz/2015/12/the-next-g...

Can't emphasize enough how much I wish GCE had PostgreSQL support. Managing and maintaining our database is something I'd love to hand off to my cloud provider. MySQL simply doesn't cut it these days, and RDS has made me happy and lazy.

Re: Spotify moves its back end to Google Cloud

#137
post #93

Earlier quoted context omitted.

Anecdotal: AppEngine deprecated their Master/Slave datastore[0] in April 2012, and it was actually shutdown on July 6, 2015. So that's 3+ years to move to newer tech. [0] http://googleappengine.blogspot.com/2012/04/masterslave-data...

Also worth noting that Google built an automated tool to handle the M/S to HRD datastore migration for you. It was remarkably painless.

Painless eventually; the first versions didn't work at all, at least on the project I was involved with. Fortunately by the time you had to switch, it worked very well.

Re: Spotify moves its back end to Google Cloud

#138

Earlier quoted context omitted.

I survive on ~$35k post-tax in SF. Though I make more than that.

...and I'm assuming you've got some killer rent control, or some other non-scalable, non-replicable housing situation. That's neat for you, but it's meaningless to anybody else.

I signed the lease in the past two years. So rent control has kicked in once. I have a room in a shared apartment. There are plenty of open rooms on Craigslist for similar rents. I'm not saying life will be great but it's entirely possible to live in the city on $50k/yr.

Re: Spotify moves its back end to Google Cloud

#139
post #115

Earlier quoted context omitted.

Google Cloud interacts with the core of Google's business (web services that needs scalable data infrastructure) in a way Parse never did for Facebook.

Google's core business is ads. Parse makes a lot of sense strategically for facebook, a big moat that they have built is around being your identity on the internet. Parse's platform encourages app developers to use facebook login as a core component of their apps. So even despite being ancillary to their business, they shut it down. Building a technology company around an ecosystem is potentially a lot riskier than p…

Google's core business is providing web services and putting ads on them. That core business means Google focuses a lot on running those web services effectively, which is why they are now focusing hard on the business angle of running other people's web services.

Google spends $2+B per quarter on capital expenditures; where do you think that money is going if not datacenters?

Using Facebook as a default login was never core to Parse (e.g. PFUser: https://parse.com/docs/osx/api/Classes/PFUser.html) nor is Parse nearly on the same scale as Google's investment in its datacenters / web software efficiency.

Re: Spotify moves its back end to Google Cloud

#140

Earlier quoted context omitted.

Are you implying that we don't measure and test? We do and I'm very happy with the results. =) StackDriver, Google Cloud console, Intercom, Mixpanel and Pingdom are all being used as well.

If that's the case then you might actually be "doing devops". If pingdom is notifying you via sms then you may also be carrying a pager.

Touché -- That said, we don't have pinggom tied to SMS, just email. =) Pingom is just an external tool for us to graph uptime from other parts of the world.

If we're down there is plenty of other alerting to notify us (mostly through Slack). That said, there isn't much we can do other than wait for Google to fix it.

It is part of the trade off that we have to consider, I'm paying Google for devops instead of paying a team to do it for me. If you've ever had to hire a 24/7 support/devops staff, it is a much easier hiring proposition to just rely on Google to do that for us.

Post reply on HN