Recently I’ve been imaging a world where social media algorithms were tuned to help people instead of “driving engagement” with ever more outrage bait. Oh you’re watching clips about machining and by your data profile you’re an uneducated adult? Here are some trade school, financial assistance, and self help links to nudge you toward a better life! What a world that would be.
Building a Knowledge System That Enhances Rather Than Replaces Thought
11–20 of 54 posts
Re: Building a Knowledge System That Enhances Rather Than Replaces Thought
#12Recently I’ve been imaging a world where social media algorithms were tuned to help people instead of “driving engagement” with ever more outrage bait. Oh you’re watching clips about machining and by your data profile you’re an uneducated adult? Here are some trade school, financial assistance, and self help links to nudge you toward a better life! What a world that would be.
Isn't this what Tiktok is in China? It mostly promotes educational and science content.
Re: Building a Knowledge System That Enhances Rather Than Replaces Thought
#13it's hard to understand what problem this person wants to solve.
is a version controlled directory of Markdown files + grep enough? or a super trivial wiki Rails app that will turn WikiText into links that you could probably prototype in an hour or less? why or why not?
Re: Building a Knowledge System That Enhances Rather Than Replaces Thought
#14> When you're dealing with ten notes, it's trivial to dump them into an LLM and generate connections. With a hundred notes, it's still manageable. But what happens when you hit 10,000 notes? Or a million? We quickly run into the limitations of context windows and processing capacity. Vector search helps narrow things down, but it's prone to missing important connections that a human mind might naturally make.
I run into the same problem. I made a RAG system and imported 15 years of reddit and HN content (my own message logs) + 2 years of LLM chats, totaling about 80MB of text. I can use it to retrieve fragments but there is duplication, and almost all duplicate fragments have slightly different approaches. How do I merge all of them, and how do I get a deduplicated taxonomy? I got about 290K unique keywords extracted from the text, it doesn't fit into LLMs.
I am gravitating towards building graphs of ideas, and having a way to generate unique (non duplicate) new ideas while ingesting new text.
Re: Building a Knowledge System That Enhances Rather Than Replaces Thought
#15After trying out a bunch of digital zettelkasten tools, I just went back to paper.
This take may be a bit hot, but I actually think paper and pen is already the optimal maxima for thinking—not for retrieval, mind you, but for helping us produce new thoughts. Zettelkasten (at least the way Luhmann used it) is meant for this purpose—it is not a system for storing information and retrieving it but rather for supporting the creation of new connections.
The computer aided tools are suboptimal for this because they lack the good constrains of paper (severely limiting the search space) and the good features (seamless ability to incorporate a variety of representational modes, text, image, equation, with zero overhead, and the ability to organize things freely in space). As much as the digital knowledge base sounds good in theory, I don't think it will ever be as optimal for generation of thought. If all you want to do is summon existing information, digital tools are evidently superior. I personally think a hybrid system where one "thinks" on paper and "archives" digitally (after the thinking is done) might be best, but ultimately, we will be most productive with whatever system we actually enjoy using.
Re: Building a Knowledge System That Enhances Rather Than Replaces Thought
#16> When you're dealing with ten notes, it's trivial to dump them into an LLM and generate connections. With a hundred notes, it's still manageable. But what happens when you hit 10,000 notes? Or a million? We quickly run into the limitations of context windows and processing capacity. Vector search helps narrow things down, but it's prone to missing important connections that a human mind might naturally make. I run i…
Intriguing. I've been thinking of something similar. Do you have any notion of what constitutes an 'idea' in this context?
Re: Building a Knowledge System That Enhances Rather Than Replaces Thought
#17After trying out a bunch of digital zettelkasten tools, I just went back to paper. This take may be a bit hot, but I actually think paper and pen is already the optimal maxima for thinking —not for retrieval, mind you, but for helping us produce new thoughts . Zettelkasten (at least the way Luhmann used it) is meant for this purpose—it is not a system for storing information and retrieving it but rather for supportin…
Tried Inkscape? I was all paper for a long time but it's taken over as I've gotten used to it an the hot keys. I carry a little Wacom tablet that works good with it too, but seldom even get it out of my bag.
It does have a (configurable) fixed page size, no ability to multi-page which was kind of annoying but I started thinking of it as a feature as well.
Re: Building a Knowledge System That Enhances Rather Than Replaces Thought
#18Recently I’ve been imaging a world where social media algorithms were tuned to help people instead of “driving engagement” with ever more outrage bait. Oh you’re watching clips about machining and by your data profile you’re an uneducated adult? Here are some trade school, financial assistance, and self help links to nudge you toward a better life! What a world that would be.
Isn't this what Tiktok is in China? It mostly promotes educational and science content.
yes, for natives.
but it also ties in with censorship/propaganda, ie: "no such thing as homeless people in china"
If you're a westerner, it will show how bad US is, lusty images, drug use, gambling. etc.
It's a super double-edged sword. We could have it in the west but we'd need a government that is looking out for the interests of people, not israeli/multinationals/lobbies/industry etc.
Re: Building a Knowledge System That Enhances Rather Than Replaces Thought
#19it's hard to understand what problem this person wants to solve. is a version controlled directory of Markdown files + grep enough? or a super trivial wiki Rails app that will turn WikiText into links that you could probably prototype in an hour or less? why or why not?
What you're describing sounds an awful lot like a barebones obsidian vault.(https://obsdian.md)
I'm not familiar with the OP's notion of a "zettlegarden" but I get the impression that they want something a bit more than just markdown files. Perhaps the idea is to increase the notion of connection between captured notes, whether intentional created, or "serendipitous".
Re: Building a Knowledge System That Enhances Rather Than Replaces Thought
#20> When you're dealing with ten notes, it's trivial to dump them into an LLM and generate connections. With a hundred notes, it's still manageable. But what happens when you hit 10,000 notes? Or a million? We quickly run into the limitations of context windows and processing capacity. Vector search helps narrow things down, but it's prone to missing important connections that a human mind might naturally make. I run i…
Author here. Glad I'm not the only one with this problem!
What I'm experimenting with is building a graph out of "entities", which I'm calling people, places and things found in each card. The point I guess is to allow the LLMs to make connections between cards, but this won't be replacing user made connections.
This lets me pick up all cards that reference something like, I don't know, "factories" as a concept, which isn't something the user will necessarily want to directly connect.
Downside is that deduplicated taxonomy: I have, from a scan through my list, "factories", "Factory" and "Factories". My attempt to head this off is by vector searching based on the proposed entity keywords and having an LLM decide if they are the same or not. I think I need to work on my prompt for this though.