Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built an OSS alternative to Azure OpenAI services

github.com

Post reply on HN
Print view View original

11–20 of 64 posts

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#11

Post by happytiger »

danenania wrote: post #6
gigel82 wrote: post #2

No you didn't. The whole point of using Azure OpenAI over plain OpenAI is the fact that your data doesn't get donated to OpenAI for training (this solves "compliance" for enterprise customers that need it), which you're not solving (because you obviously can't run the OpenAI models in your own data center).

According to OpenAI, they don't use data sent to the API for training and delete it after 30 days. So your concern is misplaced unless you think OpenAI is outright lying about their policies.

Whoa there. That’s only half the story.

https://techcrunch.com/2023/03/01/addressing-criticism-opena...

They said only if they opt in not that they wouldn’t do it.

And they used to and could change those terms again after the market matures and developers have deeply integrated application investments.

Plenty of companies have followed this strategy in the past.

Let’s not oversimplify the situation as it was a recent change as well…

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#12

Post by ajcp »

gigel82 wrote: post #2

No you didn't. The whole point of using Azure OpenAI over plain OpenAI is the fact that your data doesn't get donated to OpenAI for training (this solves "compliance" for enterprise customers that need it), which you're not solving (because you obviously can't run the OpenAI models in your own data center).

Many here think that I'm afraid of my companies data being used to train models, but that is only half-correct.

The value proposition of Azure OpenAI to the enterprise is that it's bound by the Enterprise Agreement for not just data-use, but data access writ-large. I don't want my data to be accessible for *any reason* by a vendor that isn't covered explicitly in the EA.

While OpenAI says they delete your data after 30-days I have no contract or agreement in place that ensures that, or that within those 30-days they don't have carte blanche to my data for whatever else they may want to do with it.

The Azure OpenAI agreement *explicitly* states they only store my data for 30-days and can *only* access it if they suspect abuse of the service according to the ToC and our EA, which is no different than for other Azure services on my tenant. IIRC that's only after notifying me as well, and cannot exfiltrate that data even if they access it.

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#13

Post by kristjansson »

happytiger wrote: post #11
danenania wrote: post #6

Earlier quoted context omitted.

According to OpenAI, they don't use data sent to the API for training and delete it after 30 days. So your concern is misplaced unless you think OpenAI is outright lying about their policies.

Whoa there. That’s only half the story. https://techcrunch.com/2023/03/01/addressing-criticism-opena... They said only if they opt in not that they wouldn’t do it. And they used to and could change those terms again after the market matures and developers have deeply integrated application investments. Plenty of companies have followed this strategy in the past. Let’s not oversimplify the situation as it was a recent…

That's only for ChatGPT. The APIs are governed by [0]

> We will only use Customer Content as necessary to provide you with the Services, comply with applicable law, and enforce OpenAI Policies. We will not use Customer Content to develop or improve the Services.

[0]: https://openai.com/policies/business-terms

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#14

Post by kristjansson »

This looks like a useful value-add, but I'd hesitate to call it a replacement for Azure OpenAI. Governance and observability features of Azure OpenAI are really secondary to the stability and reliability guarantees of Azure owning and operating the model instead of OpenAI...

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#15

Post by gigel82 »

ajcp wrote: post #12
gigel82 wrote: post #2

No you didn't. The whole point of using Azure OpenAI over plain OpenAI is the fact that your data doesn't get donated to OpenAI for training (this solves "compliance" for enterprise customers that need it), which you're not solving (because you obviously can't run the OpenAI models in your own data center).

Many here think that I'm afraid of my companies data being used to train models, but that is only half-correct. The value proposition of Azure OpenAI to the enterprise is that it's bound by the Enterprise Agreement for not just data-use, but data access writ-large. I don't want my data to be accessible for *any reason* by a vendor that isn't covered explicitly in the EA. While OpenAI says they delete your data after…

I suspect a lot of folks never worked with Enterprise / Gov customers and don't understand the restrictions and compliance requirements (like data residence, access control, FedRAMP, TISAX, reliability SLAs, etc. which you get with Azure but not with some "move fast and break things" startup like OpenAI).

My comment is not dissing on the author, I'm just pointing out what most folks get from Azure is compliance (and maybe safety), and an OSS cannot solve that (unless they're running some other models in owned infrastructure or on-prem).

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#16

Post by danenania »

happytiger wrote: post #11
danenania wrote: post #6

Earlier quoted context omitted.

According to OpenAI, they don't use data sent to the API for training and delete it after 30 days. So your concern is misplaced unless you think OpenAI is outright lying about their policies.

Whoa there. That’s only half the story. https://techcrunch.com/2023/03/01/addressing-criticism-opena... They said only if they opt in not that they wouldn’t do it. And they used to and could change those terms again after the market matures and developers have deeply integrated application investments. Plenty of companies have followed this strategy in the past. Let’s not oversimplify the situation as it was a recent…

Microsoft can also change their terms so I don’t see much difference there.

I can understand preferring the Azure endpoints, but let’s be accurate about what the policies are.

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#17

Post by quickthrower2 »

gigel82 wrote: post #15
ajcp wrote: post #12

Earlier quoted context omitted.

Many here think that I'm afraid of my companies data being used to train models, but that is only half-correct. The value proposition of Azure OpenAI to the enterprise is that it's bound by the Enterprise Agreement for not just data-use, but data access writ-large. I don't want my data to be accessible for *any reason* by a vendor that isn't covered explicitly in the EA. While OpenAI says they delete your data after…

I suspect a lot of folks never worked with Enterprise / Gov customers and don't understand the restrictions and compliance requirements (like data residence, access control, FedRAMP, TISAX, reliability SLAs, etc. which you get with Azure but not with some "move fast and break things" startup like OpenAI). My comment is not dissing on the author, I'm just pointing out what most folks get from Azure is compliance (and…

Even just doing SOC2 we prefer it

Let alone if you are a SaaS your customers may demand it.

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#18

Post by 0xDEF »

kristjansson wrote: post #14

This looks like a useful value-add, but I'd hesitate to call it a replacement for Azure OpenAI. Governance and observability features of Azure OpenAI are really secondary to the stability and reliability guarantees of Azure owning and operating the model instead of OpenAI...

Azure OpenAI services also makes it possible to choose data centers outside the US.

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#19

Post by mdeeks »

Looks interesting! We're in the middle of building something similar right now for ourselves. We may look at this as an alternative for ourselves.

By the way, I saw this after a quick glance poking through the code. This isn't encryption, it is hashing. Not sure where or how it is used but it is worth a rename at least: https://github.com/bricks-cloud/BricksLLM/blob/main/internal...

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#20

Post by robertlagrant »

gigel82 wrote: post #15
ajcp wrote: post #12

Earlier quoted context omitted.

Many here think that I'm afraid of my companies data being used to train models, but that is only half-correct. The value proposition of Azure OpenAI to the enterprise is that it's bound by the Enterprise Agreement for not just data-use, but data access writ-large. I don't want my data to be accessible for *any reason* by a vendor that isn't covered explicitly in the EA. While OpenAI says they delete your data after…

I suspect a lot of folks never worked with Enterprise / Gov customers and don't understand the restrictions and compliance requirements (like data residence, access control, FedRAMP, TISAX, reliability SLAs, etc. which you get with Azure but not with some "move fast and break things" startup like OpenAI). My comment is not dissing on the author, I'm just pointing out what most folks get from Azure is compliance (and…

> most folks get from Azure is compliance

To be fair, this doesn't prevent problems. Paperwork doesn't plug any security gaps in any cloud provider.

Post reply on HN