How random is xkcd? (2015) • Hacker News Board

Re: How random is xkcd? (2015)

#111

Post by Dylan16807 » Thu, Dec 28, 2023, 7:09 PM UTC

pxc wrote: post #106

feoren wrote: post #95

Earlier quoted context omitted.

> presumably it was unintuitive because people imagined more complicated ways it might be working behind the scenes I think this is exactly it. And then Sid Meyer calls his players stupid and irrational for assuming the game had more depth than it actually had. For assuming a celebrated game designer would put even a modicum of thought into making a combat system that was balanced, made sense, and felt good. It's lik…

> I think this is exactly it. And then Sid Meyer calls his players stupid and irrational I think this is overstating what Sid Meier says in the talk. His original goal was to make his simple combat stat system clear to users by describing its odds as odds conventionally are described. > For assuming a celebrated game designer would put even a modicum of thought into making a combat system that was balanced, made sens…

> His original goal was to make his simple combat stat system clear to users by describing its odds as odds conventionally are described.

Except that odds values don't add on to each other. It sounds like the numbers only worked like odds in a single way, and not in other ways. The system was inherently contradictory, and confusion is not irrational in that situation.

And it's easy to clarify something as odds by making it two opposing percentages out of 100.

Re: How random is xkcd? (2015)

#112

Post by lostlogin » Thu, Dec 28, 2023, 7:21 PM UTC

avianlyric wrote: post #30

mattnewton wrote: post #3

Kind of an aside to the nerd sniping happening here, but I think the fact that people complain about the random button is a sign that the feature isn’t doing what those people really want, even if it is doing what is advertised. Those people _want_ a button biased to return novel ones they haven’t read either in that session or across all time somehow, likely because they are using it to discover new comics.

There’s a really good article from Spotify Engineering that looks at exactly how Spotify bridged this gap between “random” and the “random” people actually expect. https://engineering.atspotify.com/2014/02/how-to-shuffle-son... It a good read on understanding what people generally expect when they ask for a random stream of songs (or comics), and how you can meet that expectation by carefully engineering how you gene…

There was a dark time with Spotify where ‘random’ played me a couple of random tracks then Cyndi Lauper. Glad that came right.

Re: How random is xkcd? (2015)

#113

Post by lostlogin » Thu, Dec 28, 2023, 7:22 PM UTC

hgomersall wrote: post #102

bumby wrote: post #78

Earlier quoted context omitted.

I believe this issue is pre-dated even further, back when iPod users didn't like that occasionally a random shuffle would play the same song back-to-back. So the 'random' shuffle was made a bit less random.

It can be biased and still random.

Can you explain that?

Re: How random is xkcd? (2015)

#114

Post by xtagon » Thu, Dec 28, 2023, 7:28 PM UTC

Dylan16807 wrote: post #103

bumby wrote: post #78

Earlier quoted context omitted.

I believe this issue is pre-dated even further, back when iPod users didn't like that occasionally a random shuffle would play the same song back-to-back. So the 'random' shuffle was made a bit less random.

You don't need to make it less random to achieve that. A 100% random shuffle won't have that issue, you just have to make sure it's actually a shuffle and not something else.

Given a set of songs or albums {a, b, c}

Suppose shuffling results in ordered set [b, a, c]

Suppose user plays through the entire set. Now it's shuffled again before repating.

Suppose the new shuffle results in ordered set [c, a, b]

The user now hears c play after c

Re: How random is xkcd? (2015)

#115

Post by rspeele » Thu, Dec 28, 2023, 7:33 PM UTC

pests wrote: post #99

BillyTheMage wrote: post #59

Earlier quoted context omitted.

I think you're onto something here. Spotify already saves various metadata about the musical qualities of songs like how "upbeat" it is or how much it "slaps". I'm imagining some kind of K- farthest -neighbors algorithm could make it so that you're constantly being given songs that are as different as possible from the recent ones you've listened to. I dunno, I'm not very well versed in algorithms yet, maybe that wou…

I don't even understand the goal here. My playlists are built up around topics and themes - most songs in playlists I build are going to be not very different.

I think there's probably a different optimum shuffle experience for real playlists like you describe vs. "all my liked songs" where the latter is often what I'd put on in the car by default.

Re: How random is xkcd? (2015)

#116

Post by lelanthran » Thu, Dec 28, 2023, 7:35 PM UTC

BillyTheMage wrote: post #45

avianlyric wrote: post #30

Earlier quoted context omitted.

There’s a really good article from Spotify Engineering that looks at exactly how Spotify bridged this gap between “random” and the “random” people actually expect. https://engineering.atspotify.com/2014/02/how-to-shuffle-son... It a good read on understanding what people generally expect when they ask for a random stream of songs (or comics), and how you can meet that expectation by carefully engineering how you gene…

Anecdotally, Spotify shuffle is one of the worst shuffles I've ever used. Or at least it used to be, not sure about now since they added Smart Shuffle. At least used to, maybe still, it would play a lot of songs over and over, but never play others. Like it had maybe 100 songs out of 2000 playing regularly, over and over. This isn't just me, but all my friends too. We're all the time finding old songs we saved that h…

> This isn't just me, but all my friends too. We're all the time finding old songs we saved that have never once been played with shuffle, while it's played this one song 3 times in the same day.

That isn't "shuffle", that's "random".[1]

Normal people know what "shuffle" means. It's only recently (since the iPod?) that tech companies tried to blame the user when the user hit the "shuffle" button and got "random" instead.

[1] When you shuffle a deck of cards and then look at each one in turn, do you expect to see a King of Hearts three times in a row? Why on earth, then, is it acceptable to blame the user when they shuffle a list of songs and then get some songs player twice and some not at all.

Re: How random is xkcd? (2015)

#117

Post by Semaphor » Thu, Dec 28, 2023, 7:38 PM UTC

Semaphor wrote: post #73

GuB-42 wrote: post #70

Earlier quoted context omitted.

Just for curiosity sake, I looked at the standard RNG in C# and it is indeed flawed, but probably not enough to be seen unless you are doing formal tests. In [1] it says that numbers have a 0.5034 probability of being odd due to rounding errors in the algorithm that picks a number in a range, which is unacceptable for a simulation, but may actually be better than a real coin toss [2]. The raw RNG is also flawed but n…

Okay, now you made me want actually look into that ;) I should be able to recreate the old code (it was years ago, but I still have the script) and make a comment if I end up actually finding statistical anomalies.

It will have to stay a mystery, looking at distribution, both algorithms looked the same.

Re: How random is xkcd? (2015)

#118

Post by rspeele » Thu, Dec 28, 2023, 7:49 PM UTC

BillyTheMage wrote: post #45

avianlyric wrote: post #30

Earlier quoted context omitted.

There’s a really good article from Spotify Engineering that looks at exactly how Spotify bridged this gap between “random” and the “random” people actually expect. https://engineering.atspotify.com/2014/02/how-to-shuffle-son... It a good read on understanding what people generally expect when they ask for a random stream of songs (or comics), and how you can meet that expectation by carefully engineering how you gene…

Anecdotally, Spotify shuffle is one of the worst shuffles I've ever used. Or at least it used to be, not sure about now since they added Smart Shuffle. At least used to, maybe still, it would play a lot of songs over and over, but never play others. Like it had maybe 100 songs out of 2000 playing regularly, over and over. This isn't just me, but all my friends too. We're all the time finding old songs we saved that h…

Yeah, Spotify's shuffle sucks. I have the same experience with it picking "favorite" songs and choosing to replay them many times while others in the list go unplayed forever. Maybe it believes those songs are my favorites because of how many times they've been played, in a form of circular reasoning.

I think I would be happy with a weighted random selection. Take 2 (or K) random numbers in range [0.0,1.0] and multiply them together to get X. X will favor the low end of the [0.0,1.0] range. Now sort the playlist from least recently played to most recently played. Next song = playlist[round(X * (playlist.Length - 1))].

Something like that. Note that "least recently played" would be a persisted attribute, not something that gets discarded in each shuffle "session". And not even local to this playlist -- just literally what is the last DateTime at which I played that song from my Spotify. It's possible to get a repeat but rare. When I hear a song I enjoy I know I could hear it again soon -- I'm not going to have to work through my entire library to hear it again -- but the odds favor hearing older stuff.

You could also remove the last N songs from the candidate list completely if you wanted to guarantee never repeating super recent tracks. Increasing the K factor would increase the favoritism of long-unplayed songs.

Re: How random is xkcd? (2015)

#119

Post by Dylan16807 » Thu, Dec 28, 2023, 7:50 PM UTC

xtagon wrote: post #114

Dylan16807 wrote: post #103

Earlier quoted context omitted.

You don't need to make it less random to achieve that. A 100% random shuffle won't have that issue, you just have to make sure it's actually a shuffle and not something else.

Given a set of songs or albums {a, b, c} Suppose shuffling results in ordered set [b, a, c] Suppose user plays through the entire set. Now it's shuffled again before repating. Suppose the new shuffle results in ordered set [c, a, b] The user now hears c play after c

I'm going to ignore the other ways to handle that and just point out that that is multiple shuffles and won't be a problem if the user initiates all shuffles.

Re: How random is xkcd? (2015)

#120

Post by PaulDavisThe1st » Thu, Dec 28, 2023, 7:58 PM UTC

BillyTheMage wrote: post #59

feoren wrote: post #53

Earlier quoted context omitted.

Just from reading the article you can see how bad it is, and it makes sense that it would be worse for larger playlists. They've completely discounted the fact that when you play large playlists, you don't listen to the whole thing. The problem with the dithering comparison is that in dithering, you're looking at the whole image at the same time. But nobody listens to a long playlist all at once. First of all, the co…

I think you're onto something here. Spotify already saves various metadata about the musical qualities of songs like how "upbeat" it is or how much it "slaps". I'm imagining some kind of K- farthest -neighbors algorithm could make it so that you're constantly being given songs that are as different as possible from the recent ones you've listened to. I dunno, I'm not very well versed in algorithms yet, maybe that wou…

In the early 1980s I worked in the incoming stock dept of the worlds largest record store (we were physically separated from the actual store). We had an employee controlled music system, mostly playing mix tapes. One of the goals of many of us was to create a never-ending stream of "constantly being given songs that are as different as possible from the recent ones you've listened to"

We were young and not that well versed in the full range of musical expression (yet). Nevertheless, that didn't stop one of us (me? not sure) hitting it out of the park with a 3 part segue from "King of the Swingers" from The Jungle Book soundtrack to the Sex Pistols "Pretty Vacant" to one of the Bach preludes from the WTC. This sort of thing was routine on a daily basis for all of us, and we delighted in the best ones.