Live data from Hacker News

Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020

engineering.atspotify.com

131–135 of 135 posts

Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020

#131

Spotify is a company that feels like they want to be a "big tech company" when in reality they do not need to. All they need to do is provide a great service with as much music as possible.

They don’t add music. Artists do. And they get paid fuck all by Spotify.

Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020

#132
post #129

Earlier quoted context omitted.

Yes I guess my point is when using Kafka in combination with Kafka Streams and you produce things partitioned in a way that you need them for consumption then you do not need to do any shuffling in the instance where you want to join because data is already partitioned correctly.

You seem to know what you're talking about. Any recommendations on learning resources for this type of flow? Or really understanding which platform works for in each situation? I'm learning proper data flow in real time as I look to transition ETL of product data into Postgres to a more applicable system. Finding the right learning resources is difficult! Cheers.

https://dataintensive.net/

Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020

#133

My off-topic rant: I'd really wish Spotify would focus on improving the core player experience. It has barely seen any improvements in years. * Not overwrite/delete my listening history everytime I switch devices * Allow tabs, or some way to resume what I've been listening to in different contexts * Option to open only one instance, instead of having multiple instances that mess with each other * Playing local files…

Almost all of these strike me as only benefiting a very small sliver of users, like well under 1%. An infinitesimal portion of Spotify listeners even know what dbus is. How much engineer time is it worth to improve something like that?

Spotify is a company with over 7 billion USD dollar of revenue in 2020. Linux is about 2% of desktop users, or about 1% of users in total.

Nevertheless, they decided not to spend a dime to work on the Linux client ("Spotify for Linux is a labor of love") [1]. The Linux app is an electron app, so there's relatively little effort required to maintain it.

Instead, they paid 100 million USD for Joe Rogan, and added video features that nobody wanted and asked for.

Take a look at /r/JoeRogan [2], and see just the extent of how much reputation damage it caused.

Take a look at /r/Spotify [3], most of the complaints are about basic functionality not working.

MPRIS [4] is a dbus API standard for controlling music players. End-users don't know or care about APIs. End-users care about having music widgets, but you can't do that without it. End-users care about keyboard accessiblity when they need it, but you can't do it without those APIs. And Linux users specifically care about programmability.

Limiting the API means that not only do I have to deal with the buggy app, it's prevents anyone else from building something better.

It's been 10 years since the Linux client came out. One engineer could fix these issues in a couple of months, but it's just not their priority.

The only saving grace for Spotify is that all their competitors are worse.

1. https://www.spotify.com/us/download/linux/

2. https://www.reddit.com/r/JoeRogan/

3. https://www.reddit.com/r/Spotify/

4. https://wiki.archlinux.org/index.php/MPRIS

Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020

#134
post #14

Earlier quoted context omitted.

Because going through all that just to listen to music is bonkers

but also far less effort and money than buying physical copies of music

Oh I totally agree, for me this is a matter of an absolutely backwards industry. Very often I WANT to give my money but can't find where to buy a DRM-free track, preferably in flac. And registering for 8 different stores is not a valid solution.

Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020

#135

listenbrainz has gotten pretty good in the last while for keeping track of your music stats. Its got a nice weekly stats page etc. Its also not ad laden like last.fm

Glad that you like it! I'm part of the (small) team working on ListenBrainz. If you have any feedback or feature requests, I'd be happy to hear it.

My email is me [at] param.codes.

Post reply on HN