Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
engineering.atspotify.com
Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
1–10 of 135 posts
Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
#2Hi not sure if I am just completely off here but I am wondering how this relates or compares to processing things with Kafka and Kafka Streams?
If I am reading things correctly with Kafka the workflow equivalent to what's written in the article would be to have your producer produce via hash-based-round-robin (the default partitioning algorithm) based on the key you are interested in into some topic and then your consumer would just read it and your data would already be sorted for the given keys (because within a partition Kafka has sorting guarantees) and also be co-partitioned correctly if you need to read some other topic in with the same number of partitions and the same logical keys produced via the same algorithm. No?
Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
#3Yet, I, as a user still cannot see my playcounts via app or API.
Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
#4Yet, I, as a user still cannot see my playcounts via app or API.
this is a massive gripe of mine. I remember being able to toggle so many stats in itunes back in the day which gave me such great insight into my listening habits and other cool tidbits. Spotify, for all its ease of listening has removed some of the magic there
Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
#5And they’re still less useful than the data last.fm makes available to you.
Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
#6Yet, I, as a user still cannot see my playcounts via app or API.
[deleted]
Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
#7Yet, I, as a user still cannot see my playcounts via app or API.
That might be explained as guarding their information, yet I still have trouble believing that they are unable to distinguish two artists with the same name from each other. Each week, my Release Radar playlist is basically invaded by 0 listener rappers who "feat" defunct 70s rock bands who do not manage their artist pages... Response from Spotify: please report them individually, from the desktop app.
Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
#8When this report came out it was the straw that broke the camel's back for me in terms of my data privacy. Most people seem to have found Wrapped 2020 entertaining but I found it creepy.
I miss being able to do something simple like listen to music or watch a movie without all my actions being recorded and saved. So I'm back to buying physical media and DRM free downloads.
I'm convinced that it is now important to hold on to older appliances that work without internet access or data collection this plus right to repair gives me hope for the future.
Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
#9Yet, I, as a user still cannot see my playcounts via app or API.
This is one of the reasons why I still keep using lastfm. Stats in Spotify are next to useless and even then it would not track my local listening.
Re: Spotify Optimized the Largest Dataflow Job Ever for Wrapped 2020
#10Spotify is a company that feels like they want to be a "big tech company" when in reality they do not need to. All they need to do is provide a great service with as much music as possible.