Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Privacy Filter

openai.com

Post reply on HN
Print view View original

71–78 of 78 posts

Re: OpenAI Privacy Filter

#71

Post by fzxu22 »

blfr wrote: post #51
fzxu22 wrote: post #37

Working on this: https://github.com/KevinXuxuxu/anon_proxy , a sort of anonymization proxy to use with LLM providers. It does model (OpenAI privacy filter) + regex PII detection, and replaces them back-and-forth for API requests and responses. With locally hosted detection model, no PII leaves your local environment. I find it very useful especially when you're working on sensitive documents (legal, tax, immigration…

This is very cool because it allows you to use any model. Obviously, it still lets the model and its operator see the entire context of the conversation. I quite like Moxie's Confer[1] approach to just encrypt the whole thing in such a way that no one except the end-user sees the plaintext. [1] https://confer.to/

Thanks for the comment! And yes, redaction based measures will always face the trade-off between privacy vs intellegence you get out of LLM. e.g. provider will inevitably know you're in some sorts of legal/tax issue even without any PII. And for some case the intellegence you want will depend on LLM knowing some detail (e.g. your AGI when doing tax preparation).

On the other hand Moxie's Confer is really interesting! On first glance I thought it's using homomorphic encryption but it turns out to be based on hardware isolation. TIL +1

Re: OpenAI Privacy Filter

#74

Post by traceroute66 »

CityOfThrowaway wrote: post #61
traceroute66 wrote: post #34

Earlier quoted context omitted.

> Think, ingesting call transcripts where those calls may include credit card numbers or private data. The call transcripts are very useful for various things, but for obvious reasons we don't want to ingest the PII. Credit card numbers are deterministic. A five year old could write a script to strip out credit card numbers. As for other PII ? You're seriously expecting an LLM to find every instance of every random p…

I think this is a bit dramatic of a comment. Credit card numbers relayed over the phone are not deterministic... "four three uh let's see sorry my vision is bad six eight..." Easy versions of problems are easy. But reality is messy. And no, neither I nor anybody else is expecting a 50B parameter model to find every instance. But finding 90% or 95% or 99% is pretty good, and sufficiently good for many use cases.

> Credit card numbers relayed over the phone are not deterministic...

I don't know the last time you relayed card details over the phone, but the last 100 times I did it, the agent did one of two things:

    (a) Said "Please wait while I turn off recording"; or
    (b) Transferred the call to an automated system that read the card details via the phone keypad input and then took back control of the call afterwards.
Relaying card details over the phone is a problem that has been comprehensively solved. You don't need an LLM for it !

> But finding 90% or 95% or 99% is pretty good

I would humbly suggest that you are over-estimating the capabilities of an LLM. ;)

Re: OpenAI Privacy Filter

#75

Post by [deleted] »

CMay wrote: post #65

Just so people are clear, these types of models are almost universally naive and basic. If all you have is a single generic neutral message, "Hi, this is Bob.", it will be sufficient in most cases. If you have a pile of data, I am not aware of any PII redaction tool that has factored in all of the risks to identity leakage. The problem is when companies use things like this and somehow believe they are anonymizing th…

[deleted]

Re: OpenAI Privacy Filter

#77

Post by hanneshapke »

The privacy filter is great, but it leaves a gap for non-technical users. Therefore, Dataiku's open source team at 575 Lab has built the Kiji Privacy Proxy. It is a complete app (MacOS/Linux/Chrome extension) that uses a fine-tuned DeBERTa model. It acts as a forward proxy, but we also have an experimental transparent proxy setup.

It detects 20+ entities, not just masks them, but also converts them back on the return trip.

The project and the entire ml stack are open source and Apache 2.0 (dataset and model on Huggingface, label review, and ml pipeline setup).

Repo: https://github.com/dataiku/kiji-proxy Demo: https://youtu.be/txzzY5bU2Ig

Post reply on HN