Live data from Hacker News

Show HN: Tired of the non-portability of my playlists, I wrote my own format

universalplaylist.stavros.io

41–50 of 104 posts

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#41
A side note: at least in python, I benchmarked MsgPack as 3 times faster than JSON, and a whopping 850 times faster to read than YAML. It seems unlikely that people will ever be editing this file by hand, so for unstructured data, especially where playlists can get very large, I suggest MsgPack over YAML.

BUT... you have a defined schema, so it's probably to your advantage to use a storage format with a defined schema: ProtoBuf or Thrift. That would mean somebody trying to use your code would already have generated objects in their language.

As for the hashing algorithm, this is a good use for MD5 -- cheap and fast. You're unlikely to be concerned about somebody actively trying to generate the same checksum for two music files. For non-security (integrity verification) purposes, MD5 is still very appropriate.

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#42
post #39

Directories of numbered symlinks, my friends.

Removal performance of large directories is very bad. Reading large directories can result in lots of seeks. Directories are not suited to this at scale, and symlink directories definitely aren't portable.

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#43

Earlier quoted context omitted.

Really? For me, editing JSON is always a chore, because you have to match the multiple nested brackets, otherwise your whole file is invalid and you get to have fun finding out what you did wrong. YAML, in comparison, can be visually inspected quickly. However, with all the apprehension in the thread, I'm rethinking this decision. Maybe TOML would be better?

Pretty printed json isn't too bad, but I intentionally never bulk edit it. This format shouldn't end up heavily nested anyway? Toml is nice for simple configs/Cargo.toml but I think everything I said basically applies to it as well. Toml has recent work on it in 2017 for some (but not broad) library support but what does it look like in 2024?

Yeah, good point (plus it's not fantastic for this project anyway). I'm going to think hard on this, I quite like YAML...

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#44

A side note: at least in python, I benchmarked MsgPack as 3 times faster than JSON, and a whopping 850 times faster to read than YAML. It seems unlikely that people will ever be editing this file by hand, so for unstructured data, especially where playlists can get very large, I suggest MsgPack over YAML. BUT... you have a defined schema, so it's probably to your advantage to use a storage format with a defined schem…

Thank you, I agree. If I'm going to use JSON, I might as well use ProtoBuf.

About MD5, I was worried about a case where a service that serves user-submitted files would be exploited by MD5 collisions, leading users to open files that might exploit decoder bugs to execute code. Far-fetched, I know, but the tradeoff didn't seem worth it. I'm not married to that decision, though.

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#45

Earlier quoted context omitted.

Pretty printed json isn't too bad, but I intentionally never bulk edit it. This format shouldn't end up heavily nested anyway? Toml is nice for simple configs/Cargo.toml but I think everything I said basically applies to it as well. Toml has recent work on it in 2017 for some (but not broad) library support but what does it look like in 2024?

Yeah, good point (plus it's not fantastic for this project anyway). I'm going to think hard on this, I quite like YAML...

Fwiw I like yaml as well but it's just got a few too many issues these days.

While you are thinking maybe look at this

https://stedolan.github.io/jq/

and how people could leverage it when using your format.

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#46

What's wrong with xspf? Couldn't you just extend that with more attributes (which legacy apps could ignore)? https://en.m.wikipedia.org/wiki/XML_Shareable_Playlist_Forma...

Nothing's wrong per se (and it has roughly the same goals as mine), it's just that it's pretty much the XML equivalent of M3U, and doesn't have any provisions for stronger identification (it relies on titles), at least as far as I know. I would really like the MBID to be the main way of identifying songs throughout the industry, and possibly the AcoustID fingerprint, as that's more specific. I think it would be fanta…

XSPF is a format to enable sharing, AKA universality. It does this by defining a list of metadata fields to be used for resolving each track in the local context of the listener.

The UPF "ids" field maps to the XSPF "identifier" field: http://xspf.org/xspf-v1.html#rfc.section.4.1.1.2.14.1.1.1.2

XSPF is not M3U in any way. From the spec: http://xspf.org/xspf-v1.html#rfc.section.3.4

3.4 Content resolver

On a surface level you can use XSPF like any other playlist format. Drop a bunch of filenames into an XSPF document, prepend "file://" to each, and you're ready to go. Under the surface there is much more.

The guiding design principle was to separate the functionality of a catalog of files from the functionality of a list of songs. Most software music players on the PC have some sort of cache for file information. This cache stores a list, or catalog, of available files and metadata from ID3 tags and other sources. XSPF is not a catalog format. XSPF exists only to say which songs to play. Almost everything in XSPF is for the purpose of answering the question which resource, rather than the question what is this resource.

If XSPF is not a catalog format, what is it? XSPF is an intermediate format. We expected a new kind of software called a content resolver to do the job of converting XSPF to a plain old list of files or URIs. A content resolver would be smart enough to keep your playlists from breaking when you move your media from /oggs to /music/ogg. It would be able to figure out that a playlist entry by the artist "Hank Williams" with the title "Your Cheating Heart" could be satisfied by the file /vorbis/hankwilliams/yourcheatingheart.ogg. It might even know how to query the iTunes music store or another online provider to locate and download a missing song.

The content resolver maintains the catalog of your songs in whatever format it prefers. It might use a flatfile, a file in the Berkeley DB format, or a SQL database. It might use only ID3 metadata, but it might also know how to query MusicBrainz or another metadata service.

All XSPF user agents are content resolvers, in that they have complete leeway to turn the contents of a track element into a specific set of bytes.

3.5 Fuzzy names

Any given track can be identified in a number of ways. We provided means for absolute identifiers like URIs, filesystem paths and secure hashes, but also for query-based identifiers — free text fields like artist and work title and numeric fields for song length, all of which together should be enough for a good content resolver to turn into files.

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#47

Earlier quoted context omitted.

Really? For me, editing JSON is always a chore, because you have to match the multiple nested brackets, otherwise your whole file is invalid and you get to have fun finding out what you did wrong. YAML, in comparison, can be visually inspected quickly. However, with all the apprehension in the thread, I'm rethinking this decision. Maybe TOML would be better?

Pretty printed json isn't too bad, but I intentionally never bulk edit it. This format shouldn't end up heavily nested anyway? Toml is nice for simple configs/Cargo.toml but I think everything I said basically applies to it as well. Toml has recent work on it in 2017 for some (but not broad) library support but what does it look like in 2024?

> I think everything I said basically applies to it as well.

TOML is a much simpler format than YAML, and is a dialect/formalisation of INI files which have yet to go out of style. Direct database storage seems like a red herring (why would you want to store a playlist as a json blob?), so does "trivial JS consumption of the file".

[0] the entire spec is 10 pages, and that's with lots of examples, YAML 1.2 is ~80 pages

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#48
post #29

Earlier quoted context omitted.

Yes, please! We have enough playlists formats to support already (and most of them are half-baked/half-broken already). There is no good reason to not reuse and extend xspf. (Last time I counted 16 major playlists formats in VLC...)

Isn't one good reason the fact that you avoid the whole confusion of "I imported my XSPF playlist with all my IDs but my player didn't find any songs!" "Oh, that's because your player only supports XSPF 1, not 1.1"?

There is a version 0 and a version 1. They are identical except for minutiae related to date formats.

In version 0 of XSPF, dates were specified as an ISO 8601 date. In version 1 dates were specified as xsd:dateTime. This is the same thing (with better documentation) for almost every date in history, and as there are no playlist creation dates that might be different, there are no real world playlists that would be incompatible.

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#49
post #29

Earlier quoted context omitted.

Yes, please! We have enough playlists formats to support already (and most of them are half-baked/half-broken already). There is no good reason to not reuse and extend xspf. (Last time I counted 16 major playlists formats in VLC...)

Isn't one good reason the fact that you avoid the whole confusion of "I imported my XSPF playlist with all my IDs but my player didn't find any songs!" "Oh, that's because your player only supports XSPF 1, not 1.1"?

I'm sure you are welcome to submit a PR.

Re: Show HN: Tired of the non-portability of my playlists, I wrote my own format

#50
post #15
post #2

My own format = https://xkcd.com/927/

I don't see why this is being downvoted. It's fairly apropos to the subject. And it was the first thing I thought of too when I read the title.

[dead]
Post reply on HN