Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tacit Knowledge Is Dangerous

er4hn.info

Post reply on HN
Print view View original

31–40 of 143 posts

Re: Tacit Knowledge Is Dangerous

#31

Post by ericbarrett »

plugin-baby wrote: post #30
jmfldn wrote: post #15

Earlier quoted context omitted.

Indeed it would. But then the source training data would be bad to lose too. That already exists. I understand it would be a higher level of damage to lose the model of course, but the benefits outweigh the risks, and these could be managed via a very security-hardened product.

Presumably threat actors can already feed stolen document dumps to LLMs to quickly find interesting data.

OP mentioned meeting notes (like otter.ai) and other ephemera. I think that really changes the risk profile.

Re: Tacit Knowledge Is Dangerous

#32

Post by jsty »

sonicanatidae wrote: post #26
hoherd wrote: post #17

At a previous company where I had a lot of this "tacit knowledge", I had made a policy for myself. Whenever somebody asked me a question over IM or e-mail, I would look in the wiki. If the answer was not in the wiki, I would write a new wiki article or update an existing article with the answer, and I would respond with a link to the article, and a statement like "if this doesn't answer your question let me know what…

How do you keep your wiki from evolving into a kludge of random-ish info? One example of tacit knowledge is knowing the Dell T420s Tower Servers have very poor cooling for the raid controllers and often, this can cook them over time, requiring replacement or, at least repasting the heatsink. Now imagine 100s of those little tidbits. Useful, but hard to categorize.

Use an internal Q&A / Quora-style tool with good search. Trying to keep little snippets of info like that under useful MECE categories only ever ends in pain & frequent refactoring.

Re: Tacit Knowledge Is Dangerous

#33

Post by lucidguppy »

sonicanatidae wrote: post #26
hoherd wrote: post #17

At a previous company where I had a lot of this "tacit knowledge", I had made a policy for myself. Whenever somebody asked me a question over IM or e-mail, I would look in the wiki. If the answer was not in the wiki, I would write a new wiki article or update an existing article with the answer, and I would respond with a link to the article, and a statement like "if this doesn't answer your question let me know what…

How do you keep your wiki from evolving into a kludge of random-ish info? One example of tacit knowledge is knowing the Dell T420s Tower Servers have very poor cooling for the raid controllers and often, this can cook them over time, requiring replacement or, at least repasting the heatsink. Now imagine 100s of those little tidbits. Useful, but hard to categorize.

I don't want to get too much on the ai hype train - but it seems like this would be perfect for machine learning, trained on the wiki-snippets of the company and emails (initiator would have to set a flag saying this is going to be training data).

Re: Tacit Knowledge Is Dangerous

#34

Post by peteradio »

haltist wrote: post #19
MarkusQ wrote: post #3

This looks like a really good use case for LLMs. When you're small and starting out in person, record everything the dev team does -- meetings, whiteboads, pairing sessions, etc. (yeah, creepy, but I'm in brainstorming mode) and then as you start to scale beyond the ability to "ask someone who knows" trail up an LLM to serve as the 24x7 lore master.

This is very much like my panoptic computronium cathedral™. I only need $80B to build it to achieve AGI. The AGI requires 24/7 surveillance for making optimal decisions because it needs access to as much information as possible. The second stage will require worshipers to ingest nanosensors so that even their bowel movements can be tracked and analyzed by the AGI. This is all for the benefit of those who are under co…

Surely I wouldn't need to ingest anything to monitor my bowels, mightnt I simply insert a device up my ass?

Re: Tacit Knowledge Is Dangerous

#35

Post by jmfldn »

OJFord wrote: post #24
jmfldn wrote: post #16

Earlier quoted context omitted.

An excellent point around data retention and training data. It's not like you can scrub out bits of the neural network weights. I guess such a product would have to factor this in.

You kind of can right, isn't that essentially fine-tuning or reinforcement learning? I.e. if you told it to 'forget about the July 2019 fraud we did' to adjust responses going forwards accordingly? Granted you'd have to test it without recording that, and you'd never really know for sure if you'd fully scrubbed it, only that you hadn't found a trace of it with your subsequent queries. ...now that I say that I'm sure…

Perhaps? I'm an LLM / NN layperson so I know basically zero. In my mind, the data is all smashed into a trillion numbers so there's no notion way of removing it as such.

Your point about fine-tuning it out is interesting though. As you say though, you'd have to be really sure you'd really erased it or made it "unreachable" .

Re: Tacit Knowledge Is Dangerous

#36

Post by jmfldn »

ericbarrett wrote: post #27
jmfldn wrote: post #15

Earlier quoted context omitted.

Indeed it would. But then the source training data would be bad to lose too. That already exists. I understand it would be a higher level of damage to lose the model of course, but the benefits outweigh the risks, and these could be managed via a very security-hardened product.

I think being able to steal a language model that can instruct you on things like the notes of every meeting, employee relationships, back-channel drama, security discussions, etc. and how these things relate to corporate and technical assets, all contained in a single artifact/tarball, really ups the ante for what kind of "security hardening" you'd need for this. Even exfiltration of API access for a few well-prompt…

Yep for sure. Loads to consider here! My initial comment was just a brainstorm /bluesky thinking kind of idea. Clearly the implications are incredibly deep for security, data retention and a host of other concerns.

Re: Tacit Knowledge Is Dangerous

#37

Post by antoineMoPa »

One easy and very useful habit we have where I work is to post all questions (even the stupidest questions) in public slack channels. That's also where most technical conversations happen. This helps us to hide less information in private chats.

Re: Tacit Knowledge Is Dangerous

#38

Post by rout39574 »

sonicanatidae wrote: post #26
hoherd wrote: post #17

At a previous company where I had a lot of this "tacit knowledge", I had made a policy for myself. Whenever somebody asked me a question over IM or e-mail, I would look in the wiki. If the answer was not in the wiki, I would write a new wiki article or update an existing article with the answer, and I would respond with a link to the article, and a statement like "if this doesn't answer your question let me know what…

How do you keep your wiki from evolving into a kludge of random-ish info? One example of tacit knowledge is knowing the Dell T420s Tower Servers have very poor cooling for the raid controllers and often, this can cook them over time, requiring replacement or, at least repasting the heatsink. Now imagine 100s of those little tidbits. Useful, but hard to categorize.

That's what a wiki is good for: collecting snippets of hard to categorize information. If you've got a culture of recording those notes, then you can develop full text search habits.

A kluge of randomish info by people who make occasional attempts to organize and garden... sucks compared to a beautifully manicured knowledge base. But it's fantastic compared to a blank wall and a shrug.

Re: Tacit Knowledge Is Dangerous

#40

Post by ycombinete »

pavel_lishin wrote: post #20
MarkusQ wrote: post #3

This looks like a really good use case for LLMs. When you're small and starting out in person, record everything the dev team does -- meetings, whiteboads, pairing sessions, etc. (yeah, creepy, but I'm in brainstorming mode) and then as you start to scale beyond the ability to "ask someone who knows" trail up an LLM to serve as the 24x7 lore master.

A loremaster who will lie when they don't know the answer instead of saying, "I don't know".

At which point it becomes the Dungeon Master
Post reply on HN