Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built an OSS alternative to Azure OpenAI services

github.com

Post reply on HN
Print view View original

31–40 of 64 posts

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#32

Post by linkenaxx »

danenania wrote: post #16
happytiger wrote: post #11

Earlier quoted context omitted.

Whoa there. That’s only half the story. https://techcrunch.com/2023/03/01/addressing-criticism-opena... They said only if they opt in not that they wouldn’t do it. And they used to and could change those terms again after the market matures and developers have deeply integrated application investments. Plenty of companies have followed this strategy in the past. Let’s not oversimplify the situation as it was a recent…

Microsoft can also change their terms so I don’t see much difference there. I can understand preferring the Azure endpoints, but let’s be accurate about what the policies are.

+1 i dont think Azure provides that much more in terms of data privacy unless youre saying we should believe Azure's policies but not OpenAI's...

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#35

Post by 3abiton »

kristjansson wrote: post #14

This looks like a useful value-add, but I'd hesitate to call it a replacement for Azure OpenAI. Governance and observability features of Azure OpenAI are really secondary to the stability and reliability guarantees of Azure owning and operating the model instead of OpenAI...

But aeguably, being OSS, you can integrate additional OSS that performs thebafromentiones tasks of Obserbability. Just more hacking things together, but in theory should be doable.

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#36

Post by robertlagrant »

toomuchtodo wrote: post #23
robertlagrant wrote: post #20

Earlier quoted context omitted.

> most folks get from Azure is compliance To be fair, this doesn't prevent problems. Paperwork doesn't plug any security gaps in any cloud provider.

It potentially plugs the contractual and liability risks, which might be more important (talk to your legal and compliance folks). None of your data is going to launch nuclear missiles, if it leaks it would be unfortunate, but not as much as the litigation and regulatory costs you could potentially incur. Everyone gets popped eventually. It's your job to show you operated from a commercially reasonable security postu…

I'm not disagreeing, but I'm making a separate point. I'm familiar with CYA, and need to use it myself, but that doesn't affect my previous point.

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#37

Post by nonameiguess »

robertlagrant wrote: post #20
gigel82 wrote: post #15

Earlier quoted context omitted.

I suspect a lot of folks never worked with Enterprise / Gov customers and don't understand the restrictions and compliance requirements (like data residence, access control, FedRAMP, TISAX, reliability SLAs, etc. which you get with Azure but not with some "move fast and break things" startup like OpenAI). My comment is not dissing on the author, I'm just pointing out what most folks get from Azure is compliance (and…

> most folks get from Azure is compliance To be fair, this doesn't prevent problems. Paperwork doesn't plug any security gaps in any cloud provider.

When people say stuff like this on Hacker News, it makes me think even more they haven't done a lot of work with government, or at least not the parts of the government I'm familiar with. Obviously, there are a lot of governments out there. But the FedRamp private enclaves with IL5-certification for CUI handling offered by the major cloud providers are a hell of a lot more secure than OpenAI's servers, and for workloads that require it, the classified enclaves are probably close to impossible to breach if you're not Mossad. Data centers on military installations, no connection to the Internet, private DX hardware encrypted on the installation with point-to-point tunneling through national fiber backbone only, and if you get anywhere near the cables, men in black SUVs suddenly show up out of of nowhere to bring you in and figure out why. I'm not even just saying that as a hypothetical. I've literally seen it happen when AT&T dug too close to the wrong line they didn't even know about because it was used for a testing facility the Navy doesn't publicly acknowledge. And the data they really cared about didn't even use that. It was hand-carried by armed couriers who kept hard drives in Pelican cases.

They may be tedious as fuck to implement and make what should be simple work take forever, but there are plenty of compliance checklists out there that really do give you security.

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#38

Post by saberience »

This is a few basic tools put into an api, not a replacement for Azure OpenAI services. I know because I built similar tools to help me run the ChatGPT apis locally and it was a day or two of coding at max, even including calculating accurate token counts and cost.

Re: Show HN: I built an OSS alternative to Azure OpenAI services

#40

Post by cebert »

toomuchtodo wrote: post #23
robertlagrant wrote: post #20

Earlier quoted context omitted.

> most folks get from Azure is compliance To be fair, this doesn't prevent problems. Paperwork doesn't plug any security gaps in any cloud provider.

It potentially plugs the contractual and liability risks, which might be more important (talk to your legal and compliance folks). None of your data is going to launch nuclear missiles, if it leaks it would be unfortunate, but not as much as the litigation and regulatory costs you could potentially incur. Everyone gets popped eventually. It's your job to show you operated from a commercially reasonable security postu…

> (i report to a CISO, and we report to a board, thoughts and opinions are my own)

That sounds like an interesting role. How did you get there? Did you start as a security analyst and work your way up?

Post reply on HN