Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spotify moves its back end to Google Cloud

news.spotify.com

Post reply on HN
Print view View original

161–170 of 397 posts

Re: Spotify moves its back end to Google Cloud

#161

Post by TorKlingberg »

stickfigure wrote: post #110
kyrra wrote: post #93

Earlier quoted context omitted.

Anecdotal: AppEngine deprecated their Master/Slave datastore[0] in April 2012, and it was actually shutdown on July 6, 2015. So that's 3+ years to move to newer tech. [0] http://googleappengine.blogspot.com/2012/04/masterslave-data...

Also worth noting that Google built an automated tool to handle the M/S to HRD datastore migration for you. It was remarkably painless.

I was mildly annoyed that I had to spend an afternoon "migrating" my app from the M/S datastore, even though I never used it in the first place.

Re: Spotify moves its back end to Google Cloud

#162

Post by simonswords82 »

toomuchtodo wrote: post #8
simonswords82 wrote: post #6

Earlier quoted context omitted.

Hmmm...interesting point but Google already has an awful lot of music infrastructure in the form of Google Music and YouTube does it not? Would it really make sense for them to purchase Spotify too?

They'd be buying the users.

That's about the only real value. But then what? Do they run the two entities separately? Or merge Google's offering with Spotify and risk pissing customers off who are loyal to the Spotify brand and app?

Re: Spotify moves its back end to Google Cloud

#163

Post by shockzzz »

nrh wrote: post #150
shockzzz wrote: post #87

Earlier quoted context omitted.

Right, I don't think they have to directly attack anyone. It's just a matter of tradeoffs - here are the things the Google Cloud gets us, here's a specific example that hones it in, and here's why that's critical to our business. At the moment, the blog post just says, "Yay we moved to the cloud" and threw out some marketing buzzwords. It should be beyond a shadow of doubt that they got a lower price as a result. Tha…

Spotifier here. Frankly, price is not the biggest factor in a decision like this. If we were going for the lowest cost cloud option, it probably wouldn't be either AWS or Google - there are other providers who are hungrier for business that would be willing to do deep cuts at our scale. The way we think about this is that there are basically two classes of cloud services: commodities and differentiated services. Comm…

Hey nrh,

Nice! Did you have the data tooling built out before you went to Google Cloud? If you did, I could imagine the migration was pretty hard as well.

Also, all of those seem relatively possible with AWS Redshift, Kinesis, and Data Pipelines. I'm interested what Google Cloud had to offer, spec-wise.

Re: Spotify moves its back end to Google Cloud

#164

Post by noahl »

liveoneggs wrote: post #42
latchkey wrote: post #35

What Google Cloud needs is a better sales story. AWS consistently beats out Google in this respect. This story is welcomed and there should be more success stories published like this. My business is a heavy GC user (AppEngine, Datastore, CloudStorage, BQ, ComputeEngine, ManagedVM) and I couldn't imagine achieving what we have achieved in such a short amount of time on any other platform. We (myself and another guy)…

> infinitely auto-scalable platform where we don't have to do any devops or carry a pager You never know what you don't measure and test

> You never know what you don't measure and test

Funny you should mention that. :) I work at Google on PerfKit Benchmarker (https://github.com/GoogleCloudPlatform/PerfKitBenchmarker). Not only can you measure and test GCP (and other clouds), but we're trying to make it very easy for you to do so!

(PKB doesn't have a benchmark for App Engine yet, though. Sorry.)

Re: Spotify moves its back end to Google Cloud

#167

Post by boulos »

derEitel wrote: post #158

I would be interested to get some more technical info. OK so you use GC, but how? What products do you use, how do you build your scalable infrastructure so anywhere in the world I can have all the music in the world? I love reading those type of posts from Netflix and was hoping to find something similar here

As I mentioned in a top-level comment (https://news.ycombinator.com/item?id=11159840), they will be speaking at GCP Next (which you can livestream) and should have some more detail to share.

Re: Spotify moves its back end to Google Cloud

#168

Post by matt_wulfeck »

The GCE model is the future. Google knows this because they've already discovered it internally by running their own services. They know containers are he future and they built a lot of the LXC kernel updates that makes things like docker work.

Most people don't need the future yet. Really people are just trying to get out of the business of managing and scaling their hardware with as little work as possible. Containerization is much farther down the line.

So I posit the future GCE customers are not people "moving to the cloud" like AWS, but rather people "moving to the small, contained, service" -- indeed, it's current AWS customers.

Re: Spotify moves its back end to Google Cloud

#169

Post by jamesblonde »

nrh wrote: post #144
jamesblonde wrote: post #119

The big point of this move is the hit it will have on on-premises Hadoop offerings: Cloudera, Hortonworks, MapR. It's a massive vote of no-confidence in their offerings. Spotify are effectively ditching a 2000 server, 90 PB Hadoop cluster to go GCE. As Spotify say themselves, that's big news....

Spotifier here. This is an important point. I have nothing bad to say about Cloudera or HWX (disclaimer: we're an HWX customer - we've had a pretty good experience), but I don't really see a compelling reason at this stage to manage your own cluster(s) (HIPAA/regulatory constraints, maybe?) Getting shared-storage and indepedently operated/scaled compute clusters on top of that storage isn't easily achievable with the…

In fact, I don't think large orgs like you (Spotify) really want independently operated clusters. That prevents easy sharing of data, causing data silos to appear. You really want to have true multi-tenancy, which isn't in Hadoop yet. Hadoop has worked more on Kerberos support at the cost of features like easy-to-use access control - Apache Ranger or Sentry anybody!?!?

Re: Spotify moves its back end to Google Cloud

#170

Post by meestaahjoshee »

beatpanda wrote: post #117
dsacco wrote: post #80

Earlier quoted context omitted.

This may shock you, but yes, it's livable. Many people live on $50,000 or less everyday in San Francisco. It may not be a standard of living that we as engineers are familiar with, but it is not poverty either. It's not a salary you'd want as a household income with dependents, but for an individual it is fine.

You can't afford rent in San Francisco at $50,000 a year, period. The going rate for a single bedroom (usually with no tenant's rights) is somewhere around $1,200 a month, well over 30% of your monthly take-home at $50,000/year. If you're going to force people to move to the Bay Area to work for you, you need to pay them enough to afford it. Good luck hiring though.

just playing devil's advocate here but since when do you NEED to spend less than 30% of your take home on rent to survive?

There are people out there paying 50% or more of their take home on their rent/mortgage I imagine - it's just not really considered to be wise move in most cases. 1200/mo for a 1 bed doesn't even sound that bad for what's supposed to be the most expensive city to live in America. I know people in DC who spend 1800/mo on a studio and that's pretty average. You could easily share a room and cut your rent in half if you were smart/willing.

Post reply on HN