Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Cape API – Keep your sensitive data private while using GPT-4

capeprivacy.com

Post reply on HN
Print view View original

21–30 of 30 posts

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#21

Post by gavinuhma »

luke-stanley wrote: post #18

I checked out Github for more info as suggested, and it seems the main ingredient, https://github.com/capeprivacy/private-ai is forked from udacity/private-ai. Hmmm. I was expecting to find a clever and useful repo to nicely identify and strips out personal info, that's not what it is. I do think stripping and adding personal info back only when needed is in principle a good idea for some situations. But I have big d…

Yikes, we'll have to remove that. It's a really old course on privacy-preserving machine learning from 4 years ago and has nothing to do with this product despite the generic name.

Please see https://api.capeprivacy.com/v1/docs#/ for more info.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#22

Post by luke-stanley »

gavinuhma wrote: post #21
luke-stanley wrote: post #18

I checked out Github for more info as suggested, and it seems the main ingredient, https://github.com/capeprivacy/private-ai is forked from udacity/private-ai. Hmmm. I was expecting to find a clever and useful repo to nicely identify and strips out personal info, that's not what it is. I do think stripping and adding personal info back only when needed is in principle a good idea for some situations. But I have big d…

Yikes, we'll have to remove that. It's a really old course on privacy-preserving machine learning from 4 years ago and has nothing to do with this product despite the generic name. Please see https://api.capeprivacy.com/v1/docs#/ for more info.

I can confirm it was forked in the Capeprivacy GitHub repos list. The name is the same as the PII remover mentioned in the link, which I wanted to see how it worked!

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#23

Post by gavinuhma »

moffkalast wrote: post #19
gavinuhma wrote: post #5

Earlier quoted context omitted.

It's a great question. Redaction limits the LLMs ability to draw on the underlying training data on the subject. This can work to the developers benefit in many cases, like asking questions about your own provided context. Many developers have gotten away from relying on LLMs for facts, toward providing LLMs with facts and having those facts repurposed. For example, if you ask an LLM about a famous person, like Wayne…

Damn, that is actually a really cool approach. I suppose most LLMs are not smart enough to make the connection and can be probably told to avoid doing it, but I would imagine that it's not impossible for it to figure out that Name1 is likely Wayne Gretzky from context? Edit: Yep, it's definitely a problem unless the facts are also anonymized I guess: https://chat.openai.com/share/84dbe124-dca7-46e3-be73-79b194...

Exactly. A super famous person like Wayne Gretzky is really hard to protect.

For fun, you can try to tease ChatGPT with information like. "Who is [Name1]?", it won't know, but then add "[Name1] is considered the greatest [Occupation1] in the history of the [Organization1]". Greatest is now a big clue. Add "[Name1] has the most points in history". Points is a big clue, it's some kind of game or sport.. etc. It will eventually figure it out, but I've seen it guess wrong with like Michael Jordan instead.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#25

Post by gavinuhma »

moffkalast wrote: post #19
gavinuhma wrote: post #5

Earlier quoted context omitted.

It's a great question. Redaction limits the LLMs ability to draw on the underlying training data on the subject. This can work to the developers benefit in many cases, like asking questions about your own provided context. Many developers have gotten away from relying on LLMs for facts, toward providing LLMs with facts and having those facts repurposed. For example, if you ask an LLM about a famous person, like Wayne…

Damn, that is actually a really cool approach. I suppose most LLMs are not smart enough to make the connection and can be probably told to avoid doing it, but I would imagine that it's not impossible for it to figure out that Name1 is likely Wayne Gretzky from context? Edit: Yep, it's definitely a problem unless the facts are also anonymized I guess: https://chat.openai.com/share/84dbe124-dca7-46e3-be73-79b194...

I redacted the full wikipedia paragraph with the API. Like, the nickname "The Great One" is a pretty major tell!

[NAME_GIVEN_1] [NAME_FAMILY_1] CC ([NAME_GIVEN_2] [NAME_FAMILY_2]; born [DOB_1]) is a [ORIGIN_1] [OCCUPATION_1] and [OCCUPATION_2]. He played 20 seasons in the [ORGANIZATION_1] ([ORGANIZATION_2]) for four teams from [DATE_INTERVAL_1] to [DATE_INTERVAL_2]. Nicknamed \"the Great One\",[1] he has been called the greatest [OCCUPATION_3] ever by many [OCCUPATION_4], [OCCUPATION_5], The Hockey News, and by the [ORGANIZATION_2] itself,[2] based on extensive surveys of [OCCUPATION_6], [OCCUPATION_7], [OCCUPATION_8] and [OCCUPATION_9].[3] [NAME_FAMILY_1] is the leading goal scorer, assist producer and point scorer in [ORGANIZATION_2] history,[4] and has more career assists than any other [OCCUPATION_10] has total points. He is the only [ORGANIZATION_2] [OCCUPATION_10] to total over 200 points in one season, a feat he accomplished four times. In addition, [NAME_FAMILY_1] tallied over 100 points in 15 professional seasons, 13 of them consecutive. At the time of his retirement in [DATE_INTERVAL_2], he held 61 [ORGANIZATION_2] records: 40 regular season records, 15 playoff records, and 6 All-Star records.[2]

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#26

Post by moffkalast »

gavinuhma wrote: post #25
moffkalast wrote: post #19

Earlier quoted context omitted.

Damn, that is actually a really cool approach. I suppose most LLMs are not smart enough to make the connection and can be probably told to avoid doing it, but I would imagine that it's not impossible for it to figure out that Name1 is likely Wayne Gretzky from context? Edit: Yep, it's definitely a problem unless the facts are also anonymized I guess: https://chat.openai.com/share/84dbe124-dca7-46e3-be73-79b194...

I redacted the full wikipedia paragraph with the API. Like, the nickname "The Great One" is a pretty major tell! [NAME_GIVEN_1] [NAME_FAMILY_1] CC ([NAME_GIVEN_2] [NAME_FAMILY_2]; born [DOB_1]) is a [ORIGIN_1] [OCCUPATION_1] and [OCCUPATION_2]. He played 20 seasons in the [ORGANIZATION_1] ([ORGANIZATION_2]) for four teams from [DATE_INTERVAL_1] to [DATE_INTERVAL_2]. Nicknamed \"the Great One\",[1] he has been called…

> Based on the information provided, NAME_GIVEN_1 NAME_FAMILY_1, also known as NAME_GIVEN_2 NAME_FAMILY_2, played in the ORGANIZATION_1, which is also referred to as ORGANIZATION_2. He played for four teams within this organization over the course of 20 seasons, from DATE_INTERVAL_1 to DATE_INTERVAL_2.

Hey that's actually pretty good.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#27

Post by gavinuhma »

moffkalast wrote: post #26
gavinuhma wrote: post #25

Earlier quoted context omitted.

I redacted the full wikipedia paragraph with the API. Like, the nickname "The Great One" is a pretty major tell! [NAME_GIVEN_1] [NAME_FAMILY_1] CC ([NAME_GIVEN_2] [NAME_FAMILY_2]; born [DOB_1]) is a [ORIGIN_1] [OCCUPATION_1] and [OCCUPATION_2]. He played 20 seasons in the [ORGANIZATION_1] ([ORGANIZATION_2]) for four teams from [DATE_INTERVAL_1] to [DATE_INTERVAL_2]. Nicknamed \"the Great One\",[1] he has been called…

> Based on the information provided, NAME_GIVEN_1 NAME_FAMILY_1, also known as NAME_GIVEN_2 NAME_FAMILY_2, played in the ORGANIZATION_1, which is also referred to as ORGANIZATION_2. He played for four teams within this organization over the course of 20 seasons, from DATE_INTERVAL_1 to DATE_INTERVAL_2. Hey that's actually pretty good.

You can use CapeChat UI to mess around with it: https://chat.capeprivacy.com/

Or you can also create a free API key here: https://app.capeprivacy.com/api-keys to use the interactive API directly: https://api.capeprivacy.com/v1/docs#/Privacy/DeidentifyText

Click "Authorize" on the top right to add the key, and then click "Try it out" on any of the endpoints.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#28

Post by ebg1223 »

Cool concept! I do have a concern about the healthcare aspects of the product as advertised. Do you provide a signed BAA for healthcare organizations? Without one, all the healthcare use cases listed on the site are basically a non-starter. Having said that, if you are using AWS as described, you already have a BAA with them, and providing one to clients should not be a huge deal.

Re: Show HN: Cape API – Keep your sensitive data private while using GPT-4

#30

Post by gavinuhma »

noqcks wrote: post #29

How do you deal with data persistence for storing the documents/vectorDB inside a Nitro Enclave? I would assume that you as the SaaS vendor are unable to decrypt the sensitive documents inside the enclave or see a users chat history?

Good question! Uploaded documents get converted to embeddings within the Nitro Enclave (NE), and then the embeddings are encrypted with a key that only the NE has access to.

When the search endpoint is called the encrypted embeddings are pulled into the NE and decrypted. They are then loaded into a vector db in-memory and the search is executed all within the NE. This adds some latency but it’s more secure because embeddings are only accessible to the enclave.

In the case of chat history it is never stored by the API. The developer can develop their own client side. With CapeChat we keep chat history local on the device.

Post reply on HN