Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

1.3B Worldcat scrape and data science mini-competition

annas-blog.org

Post reply on HN
Print view View original

91–95 of 95 posts

Re: 1.3B Worldcat scrape and data science mini-competition

#91

Post by dredmorbius »

gmcharlt wrote: post #90
dredmorbius wrote: post #85

Earlier quoted context omitted.

That seems to be just metadata about metadata. Where do you go to actually find the Classifications or Subject Headings themselves, in whatever format?

https://id.loc.gov/download/ offers bulk downloads of many of the vocabularies, including the subject headings. In addition, individual records of various types can be search for.

Thanks.

That seems to have Subject Headings, but not the Classification, unless I'm blind (a fairly high likelihood).

I've spent quite some time poking around the LoC website over the years without finding anything more accessible than the PDFs of the Library of Congress Classification. Are you aware of that being generally available?

Re: 1.3B Worldcat scrape and data science mini-competition

#92

Post by gmcharlt »

dredmorbius wrote: post #91
gmcharlt wrote: post #90

Earlier quoted context omitted.

https://id.loc.gov/download/ offers bulk downloads of many of the vocabularies, including the subject headings. In addition, individual records of various types can be search for.

Thanks. That seems to have Subject Headings, but not the Classification, unless I'm blind (a fairly high likelihood). I've spent quite some time poking around the LoC website over the years without finding anything more accessible than the PDFs of the Library of Congress Classification. Are you aware of that being generally available?

Near as I can tell, the best that's available is starting with the class(es) you want (e.g., https://id.loc.gov/authorities/classification/H.html for social sciences) and crawling recursively to grab whatever serialization of the RDF you want to consume. There's no SPARQL endpoint for id.loc.gov, alas, but it beats resorting to the PDFs.

Re: 1.3B Worldcat scrape and data science mini-competition

#93

Post by dredmorbius »

gmcharlt wrote: post #92
dredmorbius wrote: post #91

Earlier quoted context omitted.

Thanks. That seems to have Subject Headings, but not the Classification, unless I'm blind (a fairly high likelihood). I've spent quite some time poking around the LoC website over the years without finding anything more accessible than the PDFs of the Library of Congress Classification. Are you aware of that being generally available?

Near as I can tell, the best that's available is starting with the class(es) you want (e.g., https://id.loc.gov/authorities/classification/H.html for social sciences) and crawling recursively to grab whatever serialization of the RDF you want to consume. There's no SPARQL endpoint for id.loc.gov, alas, but it beats resorting to the PDFs.

Thanks.

Re: 1.3B Worldcat scrape and data science mini-competition

#94

Post by sevenseventen »

jszymborski wrote: post #17
sevenseventen wrote: post #12

Earlier quoted context omitted.

There are regional ISBN agencies. The US agency, Bowker, assigns ISBN prefixes by publisher, and publishers assign within their prefix as they please. They're supposed to use one ISBN per edition and format, but many publishers use ISBN as a kind of SKU so you can't 100% count on that. If that sounds sloppy...I went to publishing conferences fairly regularly from the late 90's into the teens, and I never saw a progra…

Sounds a bit like how DOIs are assigned.

or vice versa! Although DOI RA's are organized functionally rather than regionally. CrossRef -- for journal articles -- is by far the largest RA.

DOI's are far, far more centralized in that CrossRef issues many orders of magnitude more DOI's than all of the others.

The cite-and-be-cited-by use case for scholarly articles is way more compelling than any other use case devised for DOI's.

Re: 1.3B Worldcat scrape and data science mini-competition

#95

Post by sevenseventen »

amelius wrote: post #34
bambax wrote: post #18

Earlier quoted context omitted.

It's exactly like this. Publishers get a range, and do whatever they want with it. Also, some agencies sell the numbers range to publishers (I believe the US is in that case) and others give them away at no cost (France). As a result, some small publishers get ISBNs from more liberal agencies than their own country: one can never be 100% sure a French ISBN matches a French publisher for example. It's also possible so…

Why not use UUIDs? People never enter ISBNs manually, anyways. So it might as well be longer string. Or a QR code.

The point is not just to have universally unique identifiers, but to collect common metadata that's associated with the identifiers. Like this:

https://www.bowker.com/siteassets/files/pdf-files/datasubmis...

Since this is the second time I've mentioned Bowker, let me just say that I do not and have never worked for them or with them, although I did meet with their reps several times when I worked in a different part of digital publishing. It's just that they're inescapable when you're talking about ISBNs.

Post reply on HN