Ask HN: How do you use LLMs for private discussions?

#1

Post by emerongi » Sun, Jul 12, 2026, 10:17 PM UTC

I sometimes have things to discuss with LLMs, which are more private than usual. I was thinking how to do it in a way that doesn't reveal my identity.

  1. Use Tor to access the provider?
  2. Create a random account?
  3. Use some form of untraceable payment (which one?)
  4. Scrub all information provided to the LLM from personally identifiable information?

It seems like a lot of effort. So is running a local LLM, for which I don't even have the hardware. How do you do it?

Re: Ask HN: How do you use LLMs for private discussions?

#2

Post by blinded » Sun, Jul 12, 2026, 10:17 PM UTC

I spin up a gpu instance in a cloud, run my model via vllm, connect to it via an ssh tunnel. done.

Re: Ask HN: How do you use LLMs for private discussions?

#3

Post by downbad_ » Sun, Jul 12, 2026, 10:23 PM UTC

Can't you do it logged out?

Re: Ask HN: How do you use LLMs for private discussions?

#4

Post by stuxnet79 » Sun, Jul 12, 2026, 11:16 PM UTC

blinded wrote: post #2

I spin up a gpu instance in a cloud, run my model via vllm, connect to it via an ssh tunnel. done.

Can you elaborate on the first step? Which cloud and which service? What's the cost outlay if you are just having a convo and not doing anything 'agentic'?

Re: Ask HN: How do you use LLMs for private discussions?

#5

Post by blinded » Sun, Jul 12, 2026, 11:28 PM UTC

stuxnet79 wrote: post #4

blinded wrote: post #2

I spin up a gpu instance in a cloud, run my model via vllm, connect to it via an ssh tunnel. done.

Can you elaborate on the first step? Which cloud and which service? What's the cost outlay if you are just having a convo and not doing anything 'agentic'?

Hey sure.

It depends, but usually spin up an h100 on lambda.ai or coreweave. They have capacity and their UIs/APIs are nice. I spin it up for an hour or two, believe it was 6~ dollars an hour.

Once the gpu instance is up, you need to run vllm and a model, ie https://docs.lambda.ai/education/large-language-models/deplo....

Then you can connect your pi.dev, openwebui, etc etc to vllm and interact with it like normal.

Re: Ask HN: How do you use LLMs for private discussions?

#6

Post by hbcondo714 » Sun, Jul 12, 2026, 11:40 PM UTC

Maybe try Ollama Cloud, Prompt or response data is never logged or trained on:

https://ollama.com/pricing

Re: Ask HN: How do you use LLMs for private discussions?

#7

Post by Havoc » Sun, Jul 12, 2026, 11:46 PM UTC

With a local one

Re: Ask HN: How do you use LLMs for private discussions?

#8

Post by 7402 » Mon, Jul 13, 2026, 12:06 AM UTC

With llama.cpp, Intel i9-13900KS CPU, 96 GB RAM, RTX 4070 running locally.

The models I'm using right now with that are:

  gpt-oss-120b-F16.gguf
  Qwen_Qwen3.5-27B-Q4_K_M.gguf
  Qwen3.6-35B-A3B-UD-Q5_K_XL.gguf
  gemma-4-31B-it-UD-Q6_K_XL.gguf

Re: Ask HN: How do you use LLMs for private discussions?

#9

Post by ahdgs » Mon, Jul 13, 2026, 12:20 AM UTC

i've been using OpenAI's os PII-masking model, works decently and lightweight enough to run virtually anywhere https://huggingface.co/openai/privacy-filter

Re: Ask HN: How do you use LLMs for private discussions?

#10

Post by RobotCaleb » Mon, Jul 13, 2026, 1:45 AM UTC

I don't