51–60 of 65 posts
Related, I did some book detection of hackernews comments using chatgpt 3.5 at [1]. Didn't do any fancy visualisation like this, but just a table of the most recommended books. [1] https://blog.reyem.dev/post/extracting_hn_book_recommendatio...
Really cool to see my favs show up, but I honestly don't understand what we're actually looking at; the groupings seem very opaque beyond very general themes like sci-fi, startups, biographies, math, physics. In other words, what are the clustering shapes telling us? Can we dig in based on geography, publishing date, key terms or themes? Either way, I can't keep the site open for more than 30-40 seconds before it cra…
There's a sort of regular repeating confusion with embeddings that they're very well behaved in visual dimensions. IMHO it's a category error that results from tutorials using the king + female = queen example (which, funnily enough, wasn't even true for the original word2vec, if commentary I've read previously here is correct). Working with them a lot has me picture them more as "a multivariate function that outputs…
More clearly, symmetric retrieval is easier to visualize in a dimensionality reduced space than asymmetric retrieval.
I suspect that some form of multi vector document embedding would be more understandable in the reduced space than this single vector representation.
Tastefully made. I'm gonna go over it in my leisure time.
About your question for a reliable source to get book covers. I run this api that could possibly do this if you collect the Amazon asin numbers (or urls) for the books (that can also be done with the search api I host): https://docs.unwrangle.com/amazon-product-data-api/
If it seems useful, you can reach out to me and mention this chat. I'll be happy to offer free credits for your project.
I'd like to explore this more, but I'm getting THOUSANDS of errors: Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec. This crashes my browser in less than a minute.
Earlier quoted context omitted.
> Either way, I can't keep the site open for more than 30-40 seconds before it crashes. Yup, probably was about to happen to me too, had I not closed it. CPU fan almost launched off the troposphere about 30 seconds in. Probably a cluttered bunch of heavily unoptimized ReactJS modules in there (no offense to OP, I know it probably sped up development by 10x at least)
Nope, hug of death is seems: Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec. Ad infinitum for a list of a couple .js files with repeating names. Guess we'll have to come back in a day or two to experience it in it's full glory :).
Really cool to see my favs show up, but I honestly don't understand what we're actually looking at; the groupings seem very opaque beyond very general themes like sci-fi, startups, biographies, math, physics. In other words, what are the clustering shapes telling us? Can we dig in based on geography, publishing date, key terms or themes? Either way, I can't keep the site open for more than 30-40 seconds before it cra…
You've got the cluster semantics spot on, to be honest. Broad genres are grouped together, with a tendency for sub-genres to be grouped locally within those.
There is no interpretation of the overall shapes or the global structure, those are more a result of a particular UMAP run than inherent in the data.
Would love to provide different views on it and go more in depth next, thanks for the suggestion.
Did you reduce the dimension before applying HDBSCAN?
Really cool to see my favs show up, but I honestly don't understand what we're actually looking at; the groupings seem very opaque beyond very general themes like sci-fi, startups, biographies, math, physics. In other words, what are the clustering shapes telling us? Can we dig in based on geography, publishing date, key terms or themes? Either way, I can't keep the site open for more than 30-40 seconds before it cra…
The crash was indeed not intended - my mistake! Should be fixed now. You've got the cluster semantics spot on, to be honest. Broad genres are grouped together, with a tendency for sub-genres to be grouped locally within those. There is no interpretation of the overall shapes or the global structure, those are more a result of a particular UMAP run than inherent in the data. Would love to provide different views on it…