Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I used Claude Code to discover connections between 100 books

trails.pieterma.es

Post reply on HN
Print view View original

41–50 of 159 posts

Re: Show HN: I used Claude Code to discover connections between 100 books

#41

Post by throwawaySimon »

durch wrote: post #33
afro88 wrote: post #29

Earlier quoted context omitted.

LLMs are generators, and that was the correct way to view them at the start. Agents explore.

Generator vs. explorer is a useful distinction, but it's incomplete. Agents without a recognition loop are just generators with extra steps. What makes exploration valuable is the cycle: act, observe, recognize whether you're closer to what you wanted, then refine. Without that recognition ("closer" or "drifting"), you're exploring blind. Context is what lets the loop close. You need enough of it to judge the outcome…

Please stop.

Re: Show HN: I used Claude Code to discover connections between 100 books

#42

Post by mathgeek »

tolerance wrote: post #37

I don’t like this product as a service to readers (i.e., people who read as a cognitive/philosophical exploit) but I do think that somewhere embedded in its backend there are things of benefit. I think that this sucks the discreet joy out of reading and learning. Having the ways that the topics within a certain book can cross over in lead into another book of a different topic externalized is hollowing and I don’t fi…

> Having the ways that the topics within a certain book can cross over in lead into another book of a different topic externalized is hollowing and I don’t find it useful.

Intuitively, I agree. This feels like the different between being a creator (of your own thoughts as inspired by another person's) and a consumer (although in a somewhat educational sense). There would need to be a big advantage to being taught those initial thoughts, analogous to why we teach folks algebra/calculus via formulas rather than having every student figure out proofs for themselves.

Re: Show HN: I used Claude Code to discover connections between 100 books

#43

Post by durch »

throwawaySimon wrote: post #41
durch wrote: post #33

Earlier quoted context omitted.

Generator vs. explorer is a useful distinction, but it's incomplete. Agents without a recognition loop are just generators with extra steps. What makes exploration valuable is the cycle: act, observe, recognize whether you're closer to what you wanted, then refine. Without that recognition ("closer" or "drifting"), you're exploring blind. Context is what lets the loop close. You need enough of it to judge the outcome…

Please stop.

Something in there you'd like to discuss further, I've been thinking a lot about these ideas ever since LLMs came around, and I think these are many more of these discussion ahead of us...

Re: Show HN: I used Claude Code to discover connections between 100 books

#45

Post by throwawaySimon »

durch wrote: post #43
throwawaySimon wrote: post #41

Earlier quoted context omitted.

Please stop.

Something in there you'd like to discuss further, I've been thinking a lot about these ideas ever since LLMs came around, and I think these are many more of these discussion ahead of us...

Kind of tedious trying to have a discussion with someone who clearly generates their part.

Re: Show HN: I used Claude Code to discover connections between 100 books

#46

Post by pmaze »

timoth3y wrote: post #25

What meaningful connections did it uncover? You have an interesting idea here, but looking over the LLM output, it's not clear what these "connections" actually mean, or if they mean anything at all. Feeding a dataset into an LLM and getting it to output something is rather trivial. How is this particular output insightful or helpful? What specific connections gave you, the author, new insight into these works? You c…

The connections are meaningful to me in so far as they get me thinking about the topics, another lens to look at these books through. It's a fine balance between being trivial and being so out there that it seems arbitrary.

A trail that hits that balance well IMO is https://trails.pieterma.es/trail/pacemaker-principle/. I find the system theory topics the most interesting. In this one, I like how it pulled in a section from Kitchen Confidential in between oil trade bottlenecks and software team constraints to illustrate the general principle.

Re: Show HN: I used Claude Code to discover connections between 100 books

#48

Post by DyslexicAtheist »

rjh29 wrote: post #34
timoth3y wrote: post #25

What meaningful connections did it uncover? You have an interesting idea here, but looking over the LLM output, it's not clear what these "connections" actually mean, or if they mean anything at all. Feeding a dataset into an LLM and getting it to output something is rather trivial. How is this particular output insightful or helpful? What specific connections gave you, the author, new insight into these works? You c…

100 books is too small a datasize - particularly given it's a set of HN recommendations (i.e. a very narrow and specific subset of books). A larger set would probably draw more surprising and interesting groupings.

> 100 books is too small a datasize

this to me sounds off. I read the same 8, to 10 books over and over and with every read discover new things. the idea of more books being more useful stands against the same books on repeat. and while I'm not religious, how about dudes only reading 1 book (the Bible, or Koran), and claiming that they're getting all their wisdom from these for a 1000 years?

If I have a library of 100+ books and they are not enough then the quality of these books are the problem and not the number of books in the library?

Re: Show HN: I used Claude Code to discover connections between 100 books

#50

Post by johnwatson11218 »

I did something similar whereby I used pdfplumber to extract text from my pdf book collection. I dumped it into postgresql, then chunked the text into 100 char chunks w/ a 10 char overlap. These chunks were directly embedded into a 384D space using python sentence_transformers. Then I simply averaged all chunks for a doc and wrote that single vector back to postgresql. Then I used UMAP + HDBScan to perform dimensionality reduction and clustering. I ended up with a 2D data set that I can plot with plotly to see my clusters. It is very cool to play with this. It takes hours to import 100 pdf files but I can take one folder that contains a mix of programming titles, self-help, math, science fiction etc. After the fully automated analysis you can clearly see the different topic clusters.

I just spent time getting it all running on docker compose and moved my web ui from express js to flask. I want to get the code cleaned up and open source it at some point.

Post reply on HN