Live data from Hacker News

Amazon S3 Path Deprecation Plan – The Rest of the Story

aws.amazon.com

111–120 of 146 posts

Re: Amazon S3 Path Deprecation Plan – The Rest of the Story

#111
post #101
post #47

Earlier quoted context omitted.

I just now realised that if the web required two-way links there would be no way to put them into books! Some context: Back when the www was first released the most common criticism was the lack of back links. This is such a stupid and obvious deficiency that it really wasn't worth even looking into a system that was an obvious stillbirth. It wasn't just the "experts" saying this, but so many of them did because it w…

Your timeline’s all wrong. TCP/IP the Internet built on it were already well established by the time the web was born. There was nothing particularly special about HTTP or HTML, or even the concept of the web. What made it a success was the availability of a server reference architecture, and, more importantly, a browser. It was easy to try it out, see the value, and get up and running with your own server if you had…

The fact the tcp (and it’s own institutional infrastructure) were already established is what made the whole OSI network effort even more enjoyably absurd. It was the last gasp of Big IT trying to take over the crazies. Most amusingly to me, it seemed only to be discussed in enterprise contexts and Very Important IT Journals. Such people were officially committed to deployment, while their own people were busy getting stuff done. IIRC the first nail in the coffin was the US military ignoring the naked emperor and officially deciding to stick with TCP. But by that time most people with real work to be done had ignore the whole OSI effort.

> Discoverability was a problem in the early days.

Yeah, I remember being at a conference in which a smart person (actually a smart person, no snark) said that discoverability would be over, as indexing the web would require keeping a copy of everything which, of course, is completely impossible. And we all nodded, because indeed, that did make sense. And about six months later, when altavista launched, it seemed only to confirm this belief.

Re: Amazon S3 Path Deprecation Plan – The Rest of the Story

#112

Thank you for listening! The original plan was insane. The new one is sane. As I pointed out here https://twitter.com/dvassallo/status/1125549694778691584 thousands of printed books had references to V1 S3 URLs. Breaking them would have been a huge loss. Thank you!

If we're talking textbooks, well then. This is a textbook case for the 301 HTTP response code.

The old REST-style S3 URLs are specifically excluded from being able to redirect:

https://docs.aws.amazon.com/AmazonS3/latest/dev/how-to-page-...

You can create a new bucket or switch your existing one to "Static Website Hosting" mode to enable the ability to 301 your content going forward. But the URL for the "website" version isn't the same as the REST URL. And again, there's no way to redirect from the old naming scheme to the new one.

If you have content that you've ever linked with one of those URLs, it's stuck there forever.

Re: Amazon S3 Path Deprecation Plan – The Rest of the Story

#113

Earlier quoted context omitted.

If we're talking textbooks, well then. This is a textbook case for the 301 HTTP response code.

The old REST-style S3 URLs are specifically excluded from being able to redirect: https://docs.aws.amazon.com/AmazonS3/latest/dev/how-to-page-... You can create a new bucket or switch your existing one to "Static Website Hosting" mode to enable the ability to 301 your content going forward. But the URL for the "website" version isn't the same as the REST URL. And again, there's no way to redirect from the old naming…

> And again, there's no way to redirect from the old naming scheme to the new one.

For customers, no. For Amazon itself, yes. And I think that is what parent commenter meant. That Amazon should 301 all requests that are using old paths.

Re: Amazon S3 Path Deprecation Plan – The Rest of the Story

#114
post #111
post #101

Earlier quoted context omitted.

Your timeline’s all wrong. TCP/IP the Internet built on it were already well established by the time the web was born. There was nothing particularly special about HTTP or HTML, or even the concept of the web. What made it a success was the availability of a server reference architecture, and, more importantly, a browser. It was easy to try it out, see the value, and get up and running with your own server if you had…

The fact the tcp (and it’s own institutional infrastructure) were already established is what made the whole OSI network effort even more enjoyably absurd. It was the last gasp of Big IT trying to take over the crazies. Most amusingly to me, it seemed only to be discussed in enterprise contexts and Very Important IT Journals. Such people were officially committed to deployment, while their own people were busy gettin…

You both get so much of this story so utterly… /not even/ __quite__ wrong, but more importantly, leave so much detail out that, if I didn't presume better(which I do! Would seem rather paranoid if I didn't.), I'd suspect lying by omission. All of this, which includes the story to follow, makes me—and I don't say this for exaggeration purposes, it really does have an emotional impact—very sad, although it doesn't surprise me, barely anyone realizes the true technological horror lurking deep in the history of the 'Internet'.

Please, consider looking A BIT more at the history of TCP/IP:

http://rina.tssg.org/docs/DublinLostLayer140109.pdf (Slides!)

http://rina.tssg.org/docs/How_in_the_Heck_do_you_lose_a_laye... Day, John - How in the Heck Do You Lose a Layer!? (2012)

Abstract:

"Recently, Alex McKenzie published an anecdote in the IEEE Annals of the History of Computing on the creation of INWG 96, a proposal by IFIP WG6.1 for an international transport protocol. McKenzie concentrates on the differences between the proposals that lead to INWG 96. However, it is the similarities that are much more interesting. This has lead to some rather surprising insights into not only the subsequent course of events, but also the origins of many current problems, and where the solutions must be found. The results are more than a little surprising."

And here, a rather lengthy excerpt from later in the paper, as I suspect a lot of people might presume that the paper would go for some points it definitely DOESN'T go for:

"[…]

This does not mean that we should be doing OSI. Good grief, no. This only implies that the data OSI had to work with brought them to the same structure INWG had come to.[10] OSI would have brought along a different can of worms. OSI was the state of understanding in the early 80s. We have learned a lot more since.[11] There was much unnecessary complexity in OSI and recent insights allow considerable simplification over even current practice.

OSI also split the addresses from the error and flow control protocol. This creates other problems. But the Internet’s course is definitely curious. Everyone else came up with an internet architecture for an internet, except them. These were the people who were continually stressing that they were building an Internet.

Even more ironic is that the Internet ceased to be an Internet on the day most people would mark as the birth of the Internet, i.e. on the flag day January 1, 1983 when NCP was turned off and it became one large network.

It was well understood at the time that two levels of addressing were required. This had been realized in the ARPANET when the first host with redundant network connections was deployed in 1972. The INWG structure provided the perfect solution. It is clear why the Internet kept the name, but less clear why they dropped the Network Layer. Or perhaps more precisely, why they renamed the Network Layer, the Internet Layer. Did they think, just calling it something different made it different?

[…]

————

[…]

[10] There was little or no overlap between SC6/WG2 and INWG.

[11] And if the politics had not been so intense and research had continued to develop better understanding, we would have learned it a bit sooner.

[…]

[22] Someone will ask, What about IPv6? It does nothing for these problems but make them worse and the problem it does solve is not a problem.

[…]"

http://csr.bu.edu/rina/KoreaNamingFund100218.pdf more slides!

And much more, here:

http://rina.tssg.org/ (I find it rather very strange the RINA folks and the GNUnet folks seem to each pull their own thing instead of working together, it very much seems like a—hopefully NOT inevitable—repeat of the very thing John Day describes in the slides & articles above…)

——

Addendum #1:

See also, for a network security perspective:

http://www.toad.com/gnu/netcrypt.html

http://bitsavers.informatik.uni-stuttgart.de/pdf/bbn/imp/BBN... see also Appendix H here, starting on PDF page 180

———

Addendum #2:

Something in the back of my mind & the depths of my guts tells me I should link the following here, albeit I remain completely clueless as to why, or how it could seem relevant to—& topical for—any of the above, so, I'll just drop it here without explanation:

https://en.wikipedia.org/wiki/Managed_Trusted_Internet_Proto...

(Interesting standards compliance section there, by the way.)

Re: Amazon S3 Path Deprecation Plan – The Rest of the Story

#115
post #47

Earlier quoted context omitted.

I just now realised that if the web required two-way links there would be no way to put them into books! Some context: Back when the www was first released the most common criticism was the lack of back links. This is such a stupid and obvious deficiency that it really wasn't worth even looking into a system that was an obvious stillbirth. It wasn't just the "experts" saying this, but so many of them did because it w…

Pfff, TCP/IP will never succeed. It doesn't have enough layers! /s https://archive.org/details/elementsofnetwor00padl "The Book": The Elements of Networking Style: And Other Essays & Animadversions of the Art of Intercomputer Networking, by M. A. Padlipsky (1985) The World's Only Know Constructively Snotty Computer Science Book: historically, its polemics for TCP/IP and against the international standardsmongers' "OS…

You might want to read my comment in the other subthread over here:

https://news.ycombinator.com/item?id=19867467

Also, thanks a lot for the reference, strangely the first time I hear of this book! However, I have to most strongly disagree with the claim of "three layers is enough.", and you'll hopefully come to understand why, after checking the above comment. Which, by the way, DOESN'T speak in favor of OSI AT ALL - nor in favor of IP, for that matter. Two brief quotes from over there:

"[…]

This does not mean that we should be doing OSI. Good grief, no. …

[…]

————

[…]

[22] Someone will ask, What about IPv6? It does nothing for these problems but make them worse and the problem it does solve is not a problem.

[…]"

Re: Amazon S3 Path Deprecation Plan – The Rest of the Story

#116

Earlier quoted context omitted.

Storage durability has nothing to do with this. Changing how you access data saved in storage is a reasonable change to keep up with the evolution of networking technologies. The change here was deprecating an access pattern, not destroying data or anything remotely similar.

Tell that to the author of “The Peace Corps and Latin America”, who used S3 v1 URLs dozens of times in the book, with the assumption that they’d be available forever: https://books.google.com/books?id=Q312DwAAQBAJ&pg=PA135&dq=h... And thousands of other books like that.

We're arguing over semantics. Here's the exact quote from the page you linked:

> designed to provide 99.999999999% durability of objects over a given year. This durability level corresponds to an average annual expected loss of 0.000000001% of objects. For example, if you store 10,000,000 objects with Amazon S3, you can on average expect to incur a loss of a single object once every 10,000 years

They are referring to durability as corruption in the underlying data store due to known storage-technology risks. If they were to take into account all other risks, they would also have to include the risks of nuclear war, cyber warfare, the US government classifying your organization as a terrorist threat, etc etc.

Re: Amazon S3 Path Deprecation Plan – The Rest of the Story

#118
post #94

Earlier quoted context omitted.

Stupid question but can’t amazon simply do a redirection?

It'd break working paths, because bucket names can contain dots which work fine in "old style" paths but breaks using vhost style (a cert can only have a single wildcard, and the wildcard does not match dots, so " .s3.amazonraws.com" will not match "foo.bar.s3.amazonraws.com", and it's not possible to create a " .*.s3.amazonraws.com" cert).

  s3.amazonaws.com/abc-xyz -> abc-xyz.s3.amazonaws.com
  s3.amazonaws.com/abc-x.y.z -> only punycodeof(abc-x.y.z).encoded.s3.amazonaws.com

Re: Amazon S3 Path Deprecation Plan – The Rest of the Story

#119
post #81

It seems to me that adding a 301 redirect from the old URL to the new would not unresonably stress the resources of AWS? It seems perfectly resonable to update the library access, but breaking old URLs seems unessesary. They could even add a second of latency to incentivise people who can update their links.

Some HTTP clients don’t support redirects, or at least require an explicit configuration to enable them. So this would still be a breaking change for some applications.

Re: Amazon S3 Path Deprecation Plan – The Rest of the Story

#120
post #118

Earlier quoted context omitted.

It'd break working paths, because bucket names can contain dots which work fine in "old style" paths but breaks using vhost style (a cert can only have a single wildcard, and the wildcard does not match dots, so " .s3.amazonraws.com" will not match "foo.bar.s3.amazonraws.com", and it's not possible to create a " .*.s3.amazonraws.com" cert).

s3.amazonaws.com/abc-xyz -> abc-xyz.s3.amazonaws.com s3.amazonaws.com/abc-x.y.z -> only punycodeof(abc-x.y.z).encoded.s3.amazonaws.com

Punycode encodes ascii characters literally, so punycodeof(abc-x.y.z) is abc-x.y.z which fixes nothing whatsoever.
Post reply on HN