Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Your File System Is Already A Graph Database

rumproarious.com

Post reply on HN
Print view View original

81–85 of 85 posts

Re: Your File System Is Already A Graph Database

#81

Post by itake »

weitendorf wrote: post #41
itake wrote: post #10

I'm wonder though: 1. Why does AI need that folder structure? Why not a flat list of files and let the AI agent explore with BM25 / grep, etc. 2. pre-compute compression vs compute at query time. Kaparthy (and you) are recommending pre-compressing and sorting based on hard coded human abstraction opinions that may match how the data might be queried into human-friendly buckets and language. Why not just let the AI ca…

> Why does AI need that folder structure? Why not a flat list of files and let the AI agent explore with BM25 / grep, etc. Progressive disclosure, same reason you don't get assaulted with all the information a website has to offer at once, or given a sql console and told to figure it out, and instead see a portion of the information in a way that is supposed to naturally lead you to finding the next and next bits of…

Maybe it depends on the use case, but my opinion is, if you do need to apply compression, it should be done via a tool call real time instead of in a pipeline.

For example, if you’re trying to summarize the status of a project, instead of feeding an agent (in real time or via summarization pipeline), it’s better to write a script that summarizes the status of all of the jira tickets, instead of asking the agent to read all of the tickets to create a summary

Another small data point, I think people would prefer to ask questions of an AI model instead of reading the generated summaries.

Re: Your File System Is Already A Graph Database

#82

Post by inflam52 »

One of the benefits of graph databases is that you can measure strengths of connections and also infer connections that don’t actually exist (edge prediction) among many other path traversal techniques. It’s not always just about the connection itself. Many have these algorithms built in so you don’t have to reinvent it.

Re: Your File System Is Already A Graph Database

#83

Post by embedding-shape »

mswphd wrote: post #70
embedding-shape wrote: post #40

Earlier quoted context omitted.

Yeah, I suppose, but how do I get sufficiently high quality synthetic data without sending the original data to OpenAI/Anthropic, or by using local models when none of them seem strong enough to be able to generate that "sufficiently high quality synthetic data" in the first place?

you could do something like rent GPU time yourself, and use it to run a higher-quality local model (e.g. one of the Chinese "close to frontier" ones). Not guaranteed to preserve privacy of course, but it at least avoids directly sending the data to OpenAI/Anthropic.

I couldn't, as this is essentially handing over the most private I have, to 3rd parties. Don't really care about what country they're based in, it's not a possibility.
Post reply on HN