21–30 of 143 posts
Perhaps such products are in the works, but I would love a private company LLM that slurps all messages, docs, meetings, and becomes an oracle that can be relied upon. Perhaps it could even infer tacit knowledge even when it's not explicit in any docs, since it could abstract principles and concepts from the maze of data.
This looks like a really good use case for LLMs. When you're small and starting out in person, record everything the dev team does -- meetings, whiteboads, pairing sessions, etc. (yeah, creepy, but I'm in brainstorming mode) and then as you start to scale beyond the ability to "ask someone who knows" trail up an LLM to serve as the 24x7 lore master.
Meetings can be messy, full of ambiguity, and "unsolved" at then end. If you feed a number of these experiences into an LLM, would its knowledge base be _that great_ at helping teach somebody about the company holding said meetings?
I'd argue that in order for your LLM suggestion to be effective, the input to it would need to be human-written content. I.E: the knowledge that the OP is discussing. In that case, the human effort is still required up front.
Perhaps such products are in the works, but I would love a private company LLM that slurps all messages, docs, meetings, and becomes an oracle that can be relied upon. Perhaps it could even infer tacit knowledge even when it's not explicit in any docs, since it could abstract principles and concepts from the maze of data.
Earlier quoted context omitted.
As someone who has spent far too much time diving into old Slack threads to unearth what happened in the past, I can see incredible value in a capability like this. On the other hand, I think many companies would hesitate to create lasting records of written communications that go beyond the company’s retention policy. While it’d be extremely useful, I think it would also increase potential liability that is currentl…
An excellent point around data retention and training data. It's not like you can scrub out bits of the neural network weights. I guess such a product would have to factor this in.
Granted you'd have to test it without recording that, and you'd never really know for sure if you'd fully scrubbed it, only that you hadn't found a trace of it with your subsequent queries. ...now that I say that I'm sure there's a film about roughly this, delving into an AI trying to retrieve something hidden or something? The Wargames sequel perhaps?
At a previous company where I had a lot of this "tacit knowledge", I had made a policy for myself. Whenever somebody asked me a question over IM or e-mail, I would look in the wiki. If the answer was not in the wiki, I would write a new wiki article or update an existing article with the answer, and I would respond with a link to the article, and a statement like "if this doesn't answer your question let me know what…
One example of tacit knowledge is knowing the Dell T420s Tower Servers have very poor cooling for the raid controllers and often, this can cook them over time, requiring replacement or, at least repasting the heatsink.
Now imagine 100s of those little tidbits. Useful, but hard to categorize.
Earlier quoted context omitted.
Such an oracle would be a devastating thing to lose to industrial espionage.
Indeed it would. But then the source training data would be bad to lose too. That already exists. I understand it would be a higher level of damage to lose the model of course, but the benefits outweigh the risks, and these could be managed via a very security-hardened product.
Even exfiltration of API access for a few well-prompted queries could be really bad.
Tacit knowledge is inevitable... but it's also different from what this article is about. Tacit knowledge is the kind of knowledge (and skill!) that cannot be fully explained or taught in words—think mechanical skills you can only pick up through physical practice or very context-specific expertise you only get through experience. In programming good taste is crucial and entirely tacit: we can try to distill taste in…
"If I walk into a bookstore and you help me find the book I'm looking for, then You've robbed me of the opportunity to find all the books I wasn't looking for" -- paraphrasing Neil Tyson
Earlier quoted context omitted.
Such an oracle would be a devastating thing to lose to industrial espionage.
Indeed it would. But then the source training data would be bad to lose too. That already exists. I understand it would be a higher level of damage to lose the model of course, but the benefits outweigh the risks, and these could be managed via a very security-hardened product.