Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Tinfoil (YC X25): Verifiable Privacy for Cloud AI

news.ycombinator.com

Post reply on HN
Print view View original

101–106 of 106 posts

Re: Launch HN: Tinfoil (YC X25): Verifiable Privacy for Cloud AI

#101

Post by genewitch »

FrasiertheLion wrote: post #49
offmycloud wrote: post #48

Earlier quoted context omitted.

It looks like it might be the blur effect in a VM with no Firefox video acceleration. Also, email to support@tinfoil.sh (from "contact" link) just bounced back to me.

Ah we don't have support@tinfoil.sh set up yet. Can you try contact@tinfoil.sh?

Set up *@ and sort it later. ask an intern to monitor that box after lunch for a while. Catchall. You probably know this, but for anyone else thinking of doing email for their business.

For example if you do tools or RAG you probably ought have abuse@ as well, even though only 4 people will think to email that.

Re: Launch HN: Tinfoil (YC X25): Verifiable Privacy for Cloud AI

#102

Post by FrasiertheLion »

perching_aix wrote: post #100
natesales wrote: post #29

Earlier quoted context omitted.

We have to trust the hardware manufacturer (Intel/AMD/NVIDIA) designed their chips to execute the instructions we inspect, so we're assuming trust in vendor silicon either way. The real benefit of confidential computing is to extend that trust to the source code too (the inference server, OS, firmware). Maybe one day we’ll have truly open hardware ;)

Isn't this not the case for FHE? (I understand that FHE is not practically viable as you guys mention in the OP.)

Yeah not the case for FHE. But yes, not practically viable. We would be happy to switch as soon as it is.

Re: Launch HN: Tinfoil (YC X25): Verifiable Privacy for Cloud AI

#103

Post by interleave »

Technically my wife would be a perfect customer because we literally just prototyped your solution at home. But I'm confused.

For context:

My wife does leadership coaching and recently used vanilla GPT-4o via ChatGPT to summarize a transcript of an hour-long conversation.

Then, last weekend we thought... "Hey, let's test local LLMs for more privacy control. The open source models must be pretty good in 2025."

So I installed Ollama + Open WebUI plus the models on a 128GB MacBook Pro.

I am genuinely dumbfounded about the actual results we got today of comparing ChatGPT/GPT-4o vs. Llama4, Llama3.3, Llama3.2, DeepSeekR1 and Gemma.

In short: Compared to our reference GPT-4o output, none (as in NONE, zero, zilch, nil) of the above-mentioned open source models were able to create even a basic summary based on the exact same prompt + text.

The open source summaries were offensively bad. It felt like reading the most bland, generic and idiotic SEO slop I've read since I last used Google. None of the obvious topics were part of the summary. Just blah. I tested this with 5 models to boot!

I'm not an OpenAI fan per se, but if this is truly OS/SOTA then, we shouldn't even mention Llama4 or the others in the same breath as the newer OpenAI models.

What do you think?

Re: Launch HN: Tinfoil (YC X25): Verifiable Privacy for Cloud AI

#104

Post by FrasiertheLion »

interleave wrote: post #103

Technically my wife would be a perfect customer because we literally just prototyped your solution at home. But I'm confused. For context: My wife does leadership coaching and recently used vanilla GPT-4o via ChatGPT to summarize a transcript of an hour-long conversation. Then, last weekend we thought... "Hey, let's test local LLMs for more privacy control. The open source models must be pretty good in 2025." So I in…

Ollama does heavily quantize models and has a very short context window by default, but this has not been my experience with unquantized, full context versions of Llama3.3 70B and particularly, Deepseek R1, and that is reflected in the benchmarks. For instance I used Deepseek R1 671B as my daily driver for several months, and it was at par with o1 and unquestionably better than GPT-4o (o3 is certainly better than all but typically we've seen opensource models catch up within 6-9 months).

Please shoot me an email at tanya@tinfoil.sh, would love to work through your use cases.

Re: Launch HN: Tinfoil (YC X25): Verifiable Privacy for Cloud AI

#105

Post by FrasiertheLion »

genewitch wrote: post #101
FrasiertheLion wrote: post #49

Earlier quoted context omitted.

Ah we don't have support@tinfoil.sh set up yet. Can you try contact@tinfoil.sh?

Set up *@ and sort it later. ask an intern to monitor that box after lunch for a while. Catchall. You probably know this, but for anyone else thinking of doing email for their business. For example if you do tools or RAG you probably ought have abuse@ as well, even though only 4 people will think to email that.

Ha we didn't think of that, thanks for the tip!

Re: Launch HN: Tinfoil (YC X25): Verifiable Privacy for Cloud AI

#106

Post by interleave »

FrasiertheLion wrote: post #104
interleave wrote: post #103

Technically my wife would be a perfect customer because we literally just prototyped your solution at home. But I'm confused. For context: My wife does leadership coaching and recently used vanilla GPT-4o via ChatGPT to summarize a transcript of an hour-long conversation. Then, last weekend we thought... "Hey, let's test local LLMs for more privacy control. The open source models must be pretty good in 2025." So I in…

Ollama does heavily quantize models and has a very short context window by default, but this has not been my experience with unquantized, full context versions of Llama3.3 70B and particularly, Deepseek R1, and that is reflected in the benchmarks. For instance I used Deepseek R1 671B as my daily driver for several months, and it was at par with o1 and unquestionably better than GPT-4o (o3 is certainly better than all…

Hey Tanya! Thank you for helping me understand the results better.

I just posted the results of another basic interview analysis (4o vs. Llama4) here: https://x.com/SpringStreetNYC/status/1923774145633849780

To your point: Do I understand correctly that, for example, by running the default model of Llama4 via ollama, the context window is very short even when the model's context is, like 10M. In order to "unlock" the full context version, I need to get the unquantized version.

For reference, here's what `ollama show llama4` returns: - parameters 108.6B # llama4:scount - context length 10485760 # 10M - embedding length 5120 - quantization Q4_K_M

Post reply on HN