This bug hit us, and yes, I hadn't thought of just switching to opendal. That's indeed a great reminder.
AWS S3 SDK breaks its compatible services
71–80 of 91 posts
Re: AWS S3 SDK breaks its compatible services
#72Earlier quoted context omitted.
Agree too. Just because you can get something to work doesn’t mean it’s supported. Using an Amazon S3 library on a non-Amazon service is “works but isn’t supported.” Stick to only supported things if you want reliability.
a little less charitable is amazon is throwing its weight around to quash competition before they can get started; and shove tech debt onto third parties. I stopped giving amazon the benefit of the doubt about any aspect of their operations about 8 years ago.
It's one thing when the changes are obviously designed to damage competition, like Microsoft's embrace extend extinguish strategy, but in this case, the breaking changes seem to be pretty obviously motivated by a real need, and there isn't anything preventing so called "S3-compatible" storage services from implementing this new feature.
Re: AWS S3 SDK breaks its compatible services
#73I got bit by this a month ago. You can disable the new behavior by setting 2 environment variables export AWS_REQUEST_CHECKSUM_CALCULATION=when_required export AWS_RESPONSE_CHECKSUM_CALCULATION=when_required or adding the following 2 lines to a profile in ~/.aws/config request_checksum_calculation=when_required response_checksum_validation=when_required Or just pin your AWS SDK to version before the following. https:…
Oh, I just debugged this 4 hours today and now there is HN post and even better approach in comments….
Re: AWS S3 SDK breaks its compatible services
#74I'm confused. Why Cloudflare et al. won't release a package in pypi named boto-r2 or something, pinned to the last compatible version with the added benefit of setting the endpoint by default? It seems asinine to rely on Amazon as not to break things from time to time.
Re: AWS S3 SDK breaks its compatible services
#75Earlier quoted context omitted.
You're not wrong - the semver doesn't indicate a breaking API change. But, to be fair, this wasn't a breaking API change. Any consumer of this software using it for its intended purpose (S3) didn't need to make any changes to their code when upgrading to this version. As an AWS customer, knowing that when I I upgrade to this version my app will continue working without any changes is exactly what this semver bump com…
While I agree that the author is just whining about this situation and that AWS did nothing wrong, I'd argue that a change in defaults is a breaking change.
Re: AWS S3 SDK breaks its compatible services
#76Sometimes people forget that the S3 API is not an industry standard, but a proprietary inspectable interface the original author is at liberty to modify to their liking. And it does, indeed, have thorny edges like "which headers are expected", "which headers do you sign", what the semantics of the headers are and so forth.
It is also on the implementors of the "compatible" services to, for example, not require a header that can be assumed optional. If it is not "basic HTTP" (things like those checksums) - don't crash the PUT if you don't get the header unless you absolutely require that header. Postel's law and all.
The mention in the Tigris article is strange: is boto now doing a PUT without request content-length? That's not even valid HTTP 1.1
Re: AWS S3 SDK breaks its compatible services
#77Earlier quoted context omitted.
Here's a good discussion from here in 2021 about the fork that colors my perception: https://news.ycombinator.com/item?id=26780848
Yeah Amazon’s motivations aren’t great, but it’s occupying a space that opened up when Elastic changed the license on Elasticsearch. Nobody’s going to create another permissively licensed alternative to it just because they’re annoyed it’s an Amazon project.
Amazon, after making its pretty barebones fork, lobbed it off to the Linux Foundation. It ultimately feels lazy and self-serving in a way only a select few companies can pull off.
Re: AWS S3 SDK breaks its compatible services
#78Earlier quoted context omitted.
a little less charitable is amazon is throwing its weight around to quash competition before they can get started; and shove tech debt onto third parties. I stopped giving amazon the benefit of the doubt about any aspect of their operations about 8 years ago.
Less charitable or More cynical? How is Amazon supposed to track a 3rd party pulling their SDK and then reverse-engineering their own service side to work with the SDK? Assuming we're all okay with that premise to begin with, all sorts of other questions start popping up. Do these 3rd parties get veto power over a feature they can't support? Can they delay a launch if they need more time to make their reverse-enginee…
Requiring a header "just because you sniffed it to usually be there" is not Amazon being cynical, it's creatively-developing-overly-strict-checks. And it happens on the side of the S3-compatible service.
If your service no longer works with the AWS SDK because you crash at `headers["content-md5"]` just because "it seemed a good way to make things more correct" - it is on you to fix it, IMO.
Like, this changeset https://github.com/minio/minio/pull/20855/files#diff-be83836...
Why does Minio mandate the presence of Content-MD5? Is it in the docs somewhere for the S3 "protocol"? No, it's not. It's someone wanting to "be extra correct with validating user input" and thus creating a subtle extra restriction on the interface they do not control.
Re: AWS S3 SDK breaks its compatible services
#79Okay, but I'd never expect an AWS SDK to remain backwards compatible with third-party services and would be leery about using an AWS SDK with anything but AWS. It's on the third parties to keep their S3-compatible API, well, compatible. On the client side, you just have to pin to an older version of the AWS SDK till whatever compatible service you're using updates, right? Also, this is the first I've heard of OpenDAL…
It should be fairly easy to upgrade compatible APIs server side from reading the AWS docs. All that needs doing is to accept and ignore the new checksum header. I also expect that taking advantage of the checksum would be reasonable, a CRC32 isn't that hard. https://aws.amazon.com/blogs/aws/introducing-default-data-in...
Seconded - it's also unnecessarily creative overvalidation on the part of the devs at those joints.
Re: AWS S3 SDK breaks its compatible services
#80Treating a proprietary API as a standard is risky - this is a good example of why. From Amazon's point of view there's no reason to keep the S3 SDK backwards compatible with old versions of the S3 service, because they control the S3 service. Once this feature was rolled out in all regions, it was safe to update the SDK to expect it. Amazon may not be actively hostile to using their SDK with third party services, but…
Even more so: treating a proprietary API as a standard but _also_ adding your own checks on top which crash the interaction "because it seemed more correct to you". No, you are not guaranteed a CRC and you are not guaranteed a Content-MD5. Or - you may be getting them, but then do check whether they are in the signed headers of the request at least.