41–49 of 49 posts
Would it be possible to use a SQLite file instead of a PostgreSQL instance? Or do you rely on some specific PostgreSQL functionality?
But once the database refactor is done, I wouldn’t say no to a patch that made the service database agnostic.
Nice approach! Merging metadata from multiple sources is tricky, especially handling conflicts like titles and covers. Curious how you plan to handle scalability as your database grows—caching helps, but will the naive field strategies hold with thousands of books?
Merging on the fly kinda works for the future too, for when data change or for when the merging process changes.
No idea what the future will hold. The idea is to pre-warm the database after the schema has been refactored, and once we have thousands of books from that, I’ll know for sure what to do next.
TLDR, there is a lot of “think and learn” as I go here, haha.
Earlier quoted context omitted.
Hardcover and ISBNDB have a good amount of data, with Hardcover being excellent for getting good covers and genres. I'm hoping Goodreads and Anna's Archive will help fill in the gaps, especially since Anna's Archive have gigantic database dumps available[1]. [1]: https://todo.sr.ht/~pagina394/librario/12
You should also consider OpenLibrary and LibraryThing. Both of which have good coverage on WikiData which also aggregates identifiers. In fact, now that I think about it, you could also contribute your work to WikiData. I don't see ISBNdb ids on WikiData so you could write a script to make those contributions. Then anyone else using WikiData for this sort of thing can benefit from your work
I’d love to help improve other services. I plan on charging for Librario at some point, but I’ll offer a free version and offer free API keys for projects like Calibre and others.
At least that’s the plan.
Earlier quoted context omitted.
We don't support POST, PATCH, and whatnot yet so I didn't take that into account yet, but it's in the plans. Still need to figure out how this will work, though.
Since you support merging fields you likely would want to track provenance (including timestamp) on a per-field basis. Perhaps via an ID for the originating request. Although I would suggest that rather than merge (and discard) on initial lookup it might be better to remember each individual request. That way when you inevitably decide to fix or improve things later you could also regenerate all the existing records.…
Earlier quoted context omitted.
Since you support merging fields you likely would want to track provenance (including timestamp) on a per-field basis. Perhaps via an ID for the originating request. Although I would suggest that rather than merge (and discard) on initial lookup it might be better to remember each individual request. That way when you inevitably decide to fix or improve things later you could also regenerate all the existing records.…
I’ll definitely discuss this with Drew, as he’s the one working on the database refactor. Thank you for the feedback!
Are you able to pull upcoming titles? All I want is a weekly/monthly list of books by authors I've ready which are coming out, and I've not been able to find it or to build it.
I find WikiData to be perfect for aggregating identifiers. I mostly work with species names and it's perfect for getting the iNaturalist, GBIF, Open Tree of Life, Catalogue of Life, etc identities all in one query I haven't tried it for books. I imagine it's not sufficiently complete to serve as a backbone but a quick look at an example book gives me the ids for OpenLibrary, Librarything, Goodreads, Bing, and even ni…
Cracks me up that OP is trying Anna's Archive before Wikidata, NGL! Both great sources, though. I recently (a year ago... wow) dipped my toe into the world of library science through Wikidata, and was shocked at just how complex it is. OP's work looks really solid, but I hope they're aware of how mature the field is! For illustration, here are just the book-relevant ID sources I focused on from Wikidata: ARCHIVERS: L…
Unless you mean the fact you can find the identifier for a book in several different websites in there, in which case, I did find it.
Earlier quoted context omitted.
Cracks me up that OP is trying Anna's Archive before Wikidata, NGL! Both great sources, though. I recently (a year ago... wow) dipped my toe into the world of library science through Wikidata, and was shocked at just how complex it is. OP's work looks really solid, but I hope they're aware of how mature the field is! For illustration, here are just the book-relevant ID sources I focused on from Wikidata: ARCHIVERS: L…
Mind pointing me to an example book in Wikidata? I managed to find a few, but not if I search by ISBN, which makes it hard to find. Unless you mean the fact you can find the identifier for a book in several different websites in there, in which case, I did find it.
I believe there's also a REST API somewhere if you really hate SPARQL
Earlier quoted context omitted.
Mind pointing me to an example book in Wikidata? I managed to find a few, but not if I search by ISBN, which makes it hard to find. Unless you mean the fact you can find the identifier for a book in several different websites in there, in which case, I did find it.
you can search by ISBN or any other property of an item. You have to use a SPARQL query https://query.wikidata.org/ I believe there's also a REST API somewhere if you really hate SPARQL