Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS S3 SDK breaks its compatible services

xuanwo.io

Post reply on HN
Print view View original

51–60 of 91 posts

Re: AWS S3 SDK breaks its compatible services

#51

Post by pradn »

arccy wrote: post #31
pradn wrote: post #27

Earlier quoted context omitted.

Well, you do have to worry about customers using old client libraries / SDKs, even if your whole backend has migrated to a new API. Many customers don't like to upgrade unless they need to. It can be significant toil for them. So, you do see some tail traffic in the wild that comes from SDKs released years ago. For a service as big as S3, I bet they get traffic from SDKs ever longer than that.

the server has to be compatible with old clients, but new clients don't have to be compatible with old servers, which is the case here

Ah, I see.

Re: AWS S3 SDK breaks its compatible services

#52

Post by r3trohack3r »

everfrustrated wrote: post #47

Time for the community to support a standards body effort to define S3 compatibility under its own brand and standard. Having a way for vendors to publish some Level of compatibility would be a great help. Eg Tier 1 might mean basic upload/download, Tier 2 might support storage classes and lifecycle policies etc. Right now is just a confusing mess of self-attestation.

I can't imagine there is a motion here that accomplishes what you're looking for unless Amazon S3 adopts the standard.

I might be wrong, but I'm betting all these 3rd party clients (including open source projects) choose to be S3 compatible because a majority of their addressable market is currently using the S3 API. "Switching over to our thing doesn't require any code refactoring, just update the url in your existing project and you're good to go."

Any standard that isn't the S3 compatible API would require adopters migrate their code off the S3 API.

Re: AWS S3 SDK breaks its compatible services

#53

Post by tabony »

js2 wrote: post #6

Okay, but I'd never expect an AWS SDK to remain backwards compatible with third-party services and would be leery about using an AWS SDK with anything but AWS. It's on the third parties to keep their S3-compatible API, well, compatible. On the client side, you just have to pin to an older version of the AWS SDK till whatever compatible service you're using updates, right? Also, this is the first I've heard of OpenDAL…

Agree too.

Just because you can get something to work doesn’t mean it’s supported. Using an Amazon S3 library on a non-Amazon service is “works but isn’t supported.”

Stick to only supported things if you want reliability.

Re: AWS S3 SDK breaks its compatible services

#54

Post by r3trohack3r »

kuschku wrote: post #46
femto113 wrote: post #32

> the AWS team has implemented it poorly by enforcing it This is whiny and just wrong. Best behavior by default is always the right choice for an SDK. Libraries/tools/clients/SDKs break backwards compatibility all the time. That's exactly what semver version pinning is for, and that's a fundamental feature of every dependency management system. AWS handled this exactly right IMO. Change was introduced in Python SDK v…

You mention semver, yet you also show that this API breaking change was introduced in a minor version. Not entirely sure that's how things work?

You're not wrong - the semver doesn't indicate a breaking API change. But, to be fair, this wasn't a breaking API change.

Any consumer of this software using it for its intended purpose (S3) didn't need to make any changes to their code when upgrading to this version. As an AWS customer, knowing that when I I upgrade to this version my app will continue working without any changes is exactly what this semver bump communicates to me.

I believe calling this a feature release is correct.

Re: AWS S3 SDK breaks its compatible services

#56

Post by r3trohack3r »

robocat wrote: post #23

Any third party is one update away from an external business shock should Amazon change their API. Setting up a business so that all your customers fail at the same moment is a poor business practice: nobody can support all their customers breaking at once. I'm guessing competitors compete on price, not reliability. Amazon has the incentive to break third parties, since their customers are likely to switch to Amazon.…

I think there is a strong incentive to support the S3 API for for customer. Not having to change any of your code other than the URL the SDK points too probably makes closing sales way easier.

But if your customer remains on the S3 SDK, the same reduced switching cost you enjoyed is now enjoyed by your competitors - and you have to eat the support cost when you stop being compatible with the S3 SDK (regardless of why you are no longer compatible).

Re: AWS S3 SDK breaks its compatible services

#57

Post by semiquaver »

merb wrote: post #41

actually the new default is sane. its WAY WAY WAY WAY better than before. especially for multi uploads. it's basically one of the features where gcloud had a insane edge. another thing was If-Match in cloud storage.

Just curious, what’s so much better about a different hash digest? Is CRC32 not fast enough?

Re: AWS S3 SDK breaks its compatible services

#58

Post by 0x457 »

r3trohack3r wrote: post #54
kuschku wrote: post #46

Earlier quoted context omitted.

You mention semver, yet you also show that this API breaking change was introduced in a minor version. Not entirely sure that's how things work?

You're not wrong - the semver doesn't indicate a breaking API change. But, to be fair, this wasn't a breaking API change. Any consumer of this software using it for its intended purpose (S3) didn't need to make any changes to their code when upgrading to this version. As an AWS customer, knowing that when I I upgrade to this version my app will continue working without any changes is exactly what this semver bump com…

While I agree that the author is just whining about this situation and that AWS did nothing wrong, I'd argue that a change in defaults is a breaking change.

Re: AWS S3 SDK breaks its compatible services

#60

Post by saghm »

js2 wrote: post #6

Okay, but I'd never expect an AWS SDK to remain backwards compatible with third-party services and would be leery about using an AWS SDK with anything but AWS. It's on the third parties to keep their S3-compatible API, well, compatible. On the client side, you just have to pin to an older version of the AWS SDK till whatever compatible service you're using updates, right? Also, this is the first I've heard of OpenDAL…

> Okay, but I'd never expect an AWS SDK to remain backwards compatible with third-party services and would be leery about using an AWS SDK with anything but AWS. It's on the third parties to keep their S3-compatible API, well, compatible.

Back when Microsoft started offering CosmosDB, I was working at MongoDB on the drivers team (which develops a slew of first-party client libraries for using MongoDB), and several of the more popular drivers got a huge influx of "bug" reports of users having issues with connecting to CosmosDB. Our official policy was that if a user reported a bug with a third-party database, we'd do a basic attempt to reproduce it with MongoDB, and if it actually turned out to be a bug that was with our code, it would show up there, and we'd fix it. Otherwise, we didn't spend any time trying to figure out what the issue with CosmosDB was. In terms of backwards compatibility, we spent enough time worrying about compatibility for arbitrary versions of our own client and server software for it to be worth spending any time thinking about how changes might impact third-party databases.

In the immediate week or two after CosmosDB came out, a few people tried out the drivers they worked on to see if we could spot any differences, and although at least for basic stuff it seemed to work fine, there were a couple small oddities with specific fields in the responses during the connection handshake and stuff like that, and I think as a joke someone made a silly patch to their driver that checked those fields and logged something cheeky, but management was pretty clear that they had zero interest in any sort of proactive approach like that; the stance was basically that drivers were intentionally licensed permissively and users were free to do anything they wanted with them, and it only became our business if they actually reached out to us in some way.

Post reply on HN