51–60 of 64 posts
Looks interesting! We're in the middle of building something similar right now for ourselves. We may look at this as an alternative for ourselves. By the way, I saw this after a quick glance poking through the code. This isn't encryption, it is hashing. Not sure where or how it is used but it is worth a rename at least: https://github.com/bricks-cloud/BricksLLM/blob/main/internal...
This reminds me of an idea I had for an OpenAI proxy that transparently handles batching of requests. The use case is that OpenAI has rate limits not only on tokens but also requests per minute. By batching multiple requests together you can avoid hitting the requests limit. This isn’t really feasible to implement if your app runs on lambda or edge functions, you’d need a persistent server. Here’s a diagram I drew of…
They’ve recently added this functionality to AWS Bedrock thankfully. Doesn’t support OpenAI models, but does support Anthropic. https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-be...
Earlier quoted context omitted.
> most folks get from Azure is compliance To be fair, this doesn't prevent problems. Paperwork doesn't plug any security gaps in any cloud provider.
When people say stuff like this on Hacker News, it makes me think even more they haven't done a lot of work with government, or at least not the parts of the government I'm familiar with. Obviously, there are a lot of governments out there. But the FedRamp private enclaves with IL5-certification for CUI handling offered by the major cloud providers are a hell of a lot more secure than OpenAI's servers, and for worklo…
I have done work with government and defence. Ad hominem stuff is really pointless.
Congratulations on shipping! We are currently evaluating replacing our homegrown version of an LLM proxy with this project: https://github.com/BerriAI/litellm Any comparison or contrast you would point out?
Have you taken a look at: https://www.getjavelin.io
Looks interesting! We're in the middle of building something similar right now for ourselves. We may look at this as an alternative for ourselves. By the way, I saw this after a quick glance poking through the code. This isn't encryption, it is hashing. Not sure where or how it is used but it is worth a rename at least: https://github.com/bricks-cloud/BricksLLM/blob/main/internal...
You might want to also look at: https://www.getjavelin.io
No you didn't. The whole point of using Azure OpenAI over plain OpenAI is the fact that your data doesn't get donated to OpenAI for training (this solves "compliance" for enterprise customers that need it), which you're not solving (because you obviously can't run the OpenAI models in your own data center).
OpenAI also does not use API data to train its models. Source: https://openai.com/enterprise-privacy
https://mitta.ai uses these API calls, has an Open Source code base for inspection, a strong user privacy policy, and doesn't store data in transit, other than documents that are uploaded. I'm working on TTLs for the files stored, so there is no data left behind (other than what might be stored by the user in the DB).
Congratulations on shipping! We are currently evaluating replacing our homegrown version of an LLM proxy with this project: https://github.com/BerriAI/litellm Any comparison or contrast you would point out?
Earlier quoted context omitted.
That language seems to be doing a lot of heavy lifting. A more straightforward phrasing would've been "we will never process or otherwise consume customer content once it has been delivered to you". As written, they could use it to train GPT-5 or what-have-you and refuse to share it with you, making that exempt as it is apart from "services". Or all manner of other shenanigans, if they have competent lawyers.
You can't really read the language at face value. 'Services' is defined term in the contract: > “Services” means any services for businesses and developers we make available for purchase or use, along with any of our associated software, tools, developer services, documentation, and websites, but excluding any Third Party Offering. Of course it's still language, and one can quibble with any language, but it's reasona…
At the end of the day you can’t blindly trust your data is safe, even with a solid contract (bad actors exist, after all). So whatever you do, do it at your own risk.