I think the privacy story is super important, I like the use of local models as the fix here and understand that the output wouldn't be as good as frontier models. Perhaps look into venice for more private inference.
venice.ai? Ok nice I'll check it out. thanks. ya I added the openai compat endpoint, no brainer.