Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Cape API – Keep your sensitive data private while using GPT-4

capeprivacy.com

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#11

Post by canadiantim »

gavinuhma wrote: post #5
moffkalast wrote: post #2

Well let me ask the obvious question: Won't this redact data that is obviously crucial to getting the given task done? Let's say in case of financial statements, it if can't read credit card numbers and names, then it can't tell you which days some credit card was used and by who. Maybe that's not the typical use case, but I would imagine it being very annoying, given the already high typical LLM failure rate.

It's a great question. Redaction limits the LLMs ability to draw on the underlying training data on the subject. This can work to the developers benefit in many cases, like asking questions about your own provided context. Many developers have gotten away from relying on LLMs for facts, toward providing LLMs with facts and having those facts repurposed. For example, if you ask an LLM about a famous person, like Wayne…

that's actually pretty brilliant. I can imagine this also being useful for adding a chatbot for a website's content and really trying to limit the responses to only the content from the website as much as possible.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#12

Post by garciasn »

zb3 wrote: post #4

So now instead of sending the data to OpenAI we send it to Cape? I know that you "promise to keep it secure", but I can only trust you, right? Something like this should IMO be done on-premise

Yes, this has been my #1 issue with all the VC-backed startup dollars flowing lately. They are all 100% reliant on OpenAI and are just shuttling private information and pretending OpenAI's terms are good enough protection.

So far, most we have spoken to are literally SHOCKED that we require SOC3 (one company even told me they'd never even heard of SOC3) and everything needs to be hashed before it goes out and be mapped on our end back to actual. They think we're being too cautious and are really trying to get to sale without understanding that it's literally NOT something we can do and NO ONE else should be doing it either.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#13

Post by stevelacy »

zb3 wrote: post #4

So now instead of sending the data to OpenAI we send it to Cape? I know that you "promise to keep it secure", but I can only trust you, right? Something like this should IMO be done on-premise

They are using AWS Nitro instances for their enclaves. These can absolutely be run on-prem with self-hosted licensed software to perform the computational redacting.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#14

Post by gavinuhma »

sullivanmatt wrote: post #7

If I were a user or integrator, how do I know that the de-identification step is actually working? Is there a way to test (and/or continue testing) your regex patterns or whatever mechanism used continues to accurately strip my sensitive information before it goes to OpenAI?

Good question, some developers implement a manual approval step, so you can review the redacted prompt before you submit it rather than making it automatic. It depends on their product requirements.

Re mechanism, the redactions themselves are powered by a language model.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#15

Post by dbesemer »

gavinuhma wrote: post #9
zb3 wrote: post #4

So now instead of sending the data to OpenAI we send it to Cape? I know that you "promise to keep it secure", but I can only trust you, right? Something like this should IMO be done on-premise

Yep! The more you can do locally the better. An entirely local LLM is the best for data privacy and security. Any time data leaves it poses some risk. The de-identification itself requires a complex language model, which has its own complexity and costs to operate. At Cape we're going as far as we can to offer a secure API that's self-serve and easy to use to make these feature accessible to developers, but it does r…

I will add that running your own private LLM is complicated and costly; and that private LLM (at this point) will not be as capable as GPT-4. So while running a private LLM will certainly be the right solution for some, Cape's offering makes improved privacy available to many.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#16

Post by gavinuhma »

garciasn wrote: post #12
zb3 wrote: post #4

So now instead of sending the data to OpenAI we send it to Cape? I know that you "promise to keep it secure", but I can only trust you, right? Something like this should IMO be done on-premise

Yes, this has been my #1 issue with all the VC-backed startup dollars flowing lately. They are all 100% reliant on OpenAI and are just shuttling private information and pretending OpenAI's terms are good enough protection. So far, most we have spoken to are literally SHOCKED that we require SOC3 (one company even told me they'd never even heard of SOC3) and everything needs to be hashed before it goes out and be mapp…

Good points. I think the rabbit hole of OpenAI sub-processors is not commonly understood.

The humans at TaskUS are moderating prompts, and then you have Azure, CloudFlare, and Snowflake as sub-processors, each with their own list of sub-processors and on and on.

https://platform.openai.com/subprocessors

Data breaches can happen, so any data that you throw over the wall to OpenAI you must be willing to accept that it could become public.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#17

Post by gavinuhma »

survirtual wrote: post #10
zb3 wrote: post #4

So now instead of sending the data to OpenAI we send it to Cape? I know that you "promise to keep it secure", but I can only trust you, right? Something like this should IMO be done on-premise

Right... I want less parties involved with secure data, not more. This should be an on-prem solution with no external network access and no direct calls to OpenAI. A call is made to this service to obfuscate, then another call to OpenAI, all managed by a coordinating mechanism that is opensource / trusted. Better yet, maybe LLMs should be required to have weights released considering they are trained on the collectiv…

Entirely local and 0 sub-processors is the ideal! I hope we are trending that way as an industry

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#18

Post by luke-stanley »

I checked out Github for more info as suggested, and it seems the main ingredient, https://github.com/capeprivacy/private-ai is forked from udacity/private-ai. Hmmm. I was expecting to find a clever and useful repo to nicely identify and strips out personal info, that's not what it is.

I do think stripping and adding personal info back only when needed is in principle a good idea for some situations. But I have big doubts at the injection of another party into the mix.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#19

Post by moffkalast »

gavinuhma wrote: post #5
moffkalast wrote: post #2

Well let me ask the obvious question: Won't this redact data that is obviously crucial to getting the given task done? Let's say in case of financial statements, it if can't read credit card numbers and names, then it can't tell you which days some credit card was used and by who. Maybe that's not the typical use case, but I would imagine it being very annoying, given the already high typical LLM failure rate.

It's a great question. Redaction limits the LLMs ability to draw on the underlying training data on the subject. This can work to the developers benefit in many cases, like asking questions about your own provided context. Many developers have gotten away from relying on LLMs for facts, toward providing LLMs with facts and having those facts repurposed. For example, if you ask an LLM about a famous person, like Wayne…

Damn, that is actually a really cool approach.

I suppose most LLMs are not smart enough to make the connection and can be probably told to avoid doing it, but I would imagine that it's not impossible for it to figure out that Name1 is likely Wayne Gretzky from context?

Edit: Yep, it's definitely a problem unless the facts are also anonymized I guess: https://chat.openai.com/share/84dbe124-dca7-46e3-be73-79b194...

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#20

Post by bendecoste »

luke-stanley wrote: post #18

I checked out Github for more info as suggested, and it seems the main ingredient, https://github.com/capeprivacy/private-ai is forked from udacity/private-ai. Hmmm. I was expecting to find a clever and useful repo to nicely identify and strips out personal info, that's not what it is. I do think stripping and adding personal info back only when needed is in principle a good idea for some situations. But I have big d…

Thanks for checking it out! That repo is not related to this project, did you see it on the main list on https://github.com/capeprivacy or somewhere else? We will try to avoid the confusion in the future.
Post reply on HN