Live data from Hacker News

OpenAI frontier models and Codex are now available on AWS

openai.com

111–120 of 158 posts

Re: OpenAI frontier models and Codex are now available on AWS

#111

Earlier quoted context omitted.

Have you considered checking the actual AWS contract and the limited liability they explicitly stipulate in contracts and even linked docs from marketing materials? If you read the fine print, you'll notice something funny. You are largely responsible for data loss, SLA claims require you to present concrete evidence, and the remediation you accepted is usually credits for future spend on specifically the same produc…

They didn't mention anything about SLAs. This is about all the time, effort, paperwork and risk it takes to add yet another vendor. Having fewer vendors does actually reduce risk, as long as your chosen vendors are reasonably good. Though the bigger reason is certainly avoiding the additional bureaucracy, which is partly self-inflicted in larger companies but also not without merit.

Yeah, I understood the original point. And I'm tired of it.

I'm just tired of the 'everyone follows their immediate incentives while the system stays incoherent' as the de facto reality. I think shedding some light over the actual mechanics would maybe make someone consider 'perhaps we shouldn't allow our acquisition team just turn off their brain and choose the default to cover their bottoms; maybe vendors are worth more decision investment via actual thinking instead of performatively ending up on the default choice after a little ritualistic game of "eeny, meeny, miny, AWS"'.

I think it's worth pointing out that Jeff Bezos would fight this tooth and nail from happening in his companies. He popularised 'process as proxy'. Yet AWS as sold to external enterprises is the exact proxy Bezos warned against internally. Do what Bezos does, and even what Bezos preaches, just don't do by default what Bezos sells.

Re: OpenAI frontier models and Codex are now available on AWS

#112

Earlier quoted context omitted.

They didn't mention anything about SLAs. This is about all the time, effort, paperwork and risk it takes to add yet another vendor. Having fewer vendors does actually reduce risk, as long as your chosen vendors are reasonably good. Though the bigger reason is certainly avoiding the additional bureaucracy, which is partly self-inflicted in larger companies but also not without merit.

Yeah, I understood the original point. And I'm tired of it. I'm just tired of the 'everyone follows their immediate incentives while the system stays incoherent' as the de facto reality. I think shedding some light over the actual mechanics would maybe make someone consider 'perhaps we shouldn't allow our acquisition team just turn off their brain and choose the default to cover their bottoms; maybe vendors are worth…

Which vendor would you rather use in this context, with your sensitive customer data? -vendor A's list of sub-processors is a mile long and includes providers of questionable repute; -vendor B's list is short and includes AWS and GCP

Re: OpenAI frontier models and Codex are now available on AWS

#113
post #61

Every time somebody questions why you might "trust" AWS (or Azure or GCP or whatever), or why you'd pay this premium, I realize they are not accustomed to working in enterprise environments. In my case, I work at a large enterprise with strict data governance built into customer contracts, and (partly related, partly not) our own governance concerns. Using vendors where you not only have infosec permission, but they…

On top of this, there's a vast difference between "what do you mean that team spent $1000 on AI in their expense report, what did we get for that?" vs. "oh, the company-wide AWS bill went up by a few percent, let's look into that when we have time." The latter makes projects far more viable.

But note that this difference is the result of bad accounting.

Re: OpenAI frontier models and Codex are now available on AWS

#114
post #8

If you are wondering why anyone would spend more money to use these APIs through AWS instead of going direct: In some companies it’s nearly impossible to get new vendors approved. If the company has an AWS contract then you have to use what AWS offers.

Wait, is AWS just reselling access to some AI company's servers, or is AWS running the models on their own hardware?

AWS Bedrock is other companies’ models running on separate dedicated AWS hardware, metered through AWS billing. AWS owns and operates all of the infrastructure and the client interface; the model provider basically hands over the model and weights to AWS and AWS Bedrock take it from there.

So, as an example, if you use Codex through Bedrock, that’s a totally separate instance of Codex from anything you would be interfacing with if you directly used OpenAI’s API; if you use Codex via Bedrock, OpenAI never sees your data or prompts because they stay sandboxed in an ephemeral Bedrock instance. For many large enterprise deployments this hard boundary is a big big deal.

Over the past year, Claude being available via Bedrock and ChatGPT/Codex not being available via Bedrock has been a huge competitive advantage for Anthropic in the enterprise space.

Re: OpenAI frontier models and Codex are now available on AWS

#116

Earlier quoted context omitted.

A very interesting comment. Curious to understand how AI will continue to grow if this is the trend. Assuming most valuable data is behind such firewalls. And whatever is public has been harvested, trained on top of whatever has been acquired illegally (this is a grey area). Will it become a closed ecosystem without outside input?!

The pace of data creation is only increasing, and our capabilities of sharing and storing it is growing as well. Lots of this is out in the open, ready for anyone to crawl and scrape. There probably is a point of “peak data” where the amount of new data will start decreasing, but that’s likely a 22nd or 24rd century problem.

If anything, trend will go towards sharing data less. It will become more important to keep the knowhow and data to yourself so the companies will do that.

And individuals will loose motivation to share, because it wont be that pro-social activity anymore anyway.

Re: OpenAI frontier models and Codex are now available on AWS

#117
post #32
post #10

If you've used AI coding models in a large corporate setting, you'll know that a lot of big corporate deployments basically require using AWS Bedrock for two simple reasons: 1. Large companies tend to already have an existing relationship with AWS, which makes things way easier to go through vs. setting up a new vendor relationship 2. Large companies tend to have strong internal requirements about making sure that in…

To go a step further, the reason it's often impossible to add a new vendor if that you've signed a bunch of contracts with your customers saying you're not going to send their data to other vendors in all sorts of various flavors.

And the pain of the procurement process, specially when you follow a certification such as iso27001, soc2 or similar.

Re: OpenAI frontier models and Codex are now available on AWS

#118

Earlier quoted context omitted.

Have you considered checking the actual AWS contract and the limited liability they explicitly stipulate in contracts and even linked docs from marketing materials? If you read the fine print, you'll notice something funny. You are largely responsible for data loss, SLA claims require you to present concrete evidence, and the remediation you accepted is usually credits for future spend on specifically the same produc…

Nobody ever got fired for buying I̵B̵M̵ AWS. Most corporations already use AWS, used to its legal terms and accepted the risk. Any new provider will be scrutinised by legal more than an existing one.

Models on Bedrock can have different and additional terms and conditions, there's even variety within the same provider for some of them. The Anthropic ones certainly have their own EULA. It's a bit frustrating because ideally it should be a known legal status, but in fact it still needs legal review if you're doing anything interesting.

Re: OpenAI frontier models and Codex are now available on AWS

#119

Earlier quoted context omitted.

They didn't mention anything about SLAs. This is about all the time, effort, paperwork and risk it takes to add yet another vendor. Having fewer vendors does actually reduce risk, as long as your chosen vendors are reasonably good. Though the bigger reason is certainly avoiding the additional bureaucracy, which is partly self-inflicted in larger companies but also not without merit.

Yeah, I understood the original point. And I'm tired of it. I'm just tired of the 'everyone follows their immediate incentives while the system stays incoherent' as the de facto reality. I think shedding some light over the actual mechanics would maybe make someone consider 'perhaps we shouldn't allow our acquisition team just turn off their brain and choose the default to cover their bottoms; maybe vendors are worth…

The politics of multimillion dollar contracts for public clouds go far, far, far beyond the preferences of an acquisition team, or what the engineers may think.

Re: OpenAI frontier models and Codex are now available on AWS

#120

Earlier quoted context omitted.

Yeah, I understood the original point. And I'm tired of it. I'm just tired of the 'everyone follows their immediate incentives while the system stays incoherent' as the de facto reality. I think shedding some light over the actual mechanics would maybe make someone consider 'perhaps we shouldn't allow our acquisition team just turn off their brain and choose the default to cover their bottoms; maybe vendors are worth…

Which vendor would you rather use in this context, with your sensitive customer data? -vendor A's list of sub-processors is a mile long and includes providers of questionable repute; -vendor B's list is short and includes AWS and GCP

[deleted]
Post reply on HN