Live data from Hacker News

Show HN: Shortern URIs using Huffman Coding, not a database

urizip-dot-populace-soho.appspot.com

51–60 of 87 posts

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#51

I came up with a variant of this idea in an interview once. It did not go over too well. :)

What was the feedback the gave you?

I was asking for a function that converted a number (auto incremented id) to a string, such that all possible shortest urls would be iterated through.

E.g. given character set a-z, 0: a 1: b 26: aa 27: ab

etc.

I probably had no idea what the interviewee was suggesting and tried and failed to explain my question any better.

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#52
post #4
post #2

The url for this post is 45 characters: https://news.ycombinator.com/item?id=14245119 If you encode it with this shortener, you get this 35 character string: mNb:w9iIp7u8di:AKB2xrPUVYUFhfRUWHwA Assuming you were using this string as a key in a shortening service like this: https://short.url/mNb:w9iIp7u8di:AKB2xrPUVYUFhfRUWHwA ... you'd end up with a url longer than the original url! So it's not technically a url shor…

The idea is still interesting. I imagine using some dictionary based compression and crawled data to build a dictionary could get us somewhere.

Unfortunately the longest urls, the ones you want to shorten the most, are the worst candidates for compression (with long query strings full of things like uuids).

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#53

I came up with a variant of this idea in an interview once. It did not go over too well. :)

What was the feedback the gave you?

He just said "Yeah, ok, that could work, now do it with a database." :P

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#54
post #35

Earlier quoted context omitted.

Oh, but distributing links with tracking tokens attached is a really great way to mess with sites that track users: Well, we've had one user who really liked the new content. They viewed it 15,323 times... waaaaaait, damn it!

I habitually remove tracking and other non-essential cruft from shared URLs, for readability and for politeness to my correspondents. I don't think it's polite to associate their IP with my tracked activity.

Do you have any tools to automate that process?

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#56
post #35

Earlier quoted context omitted.

I habitually remove tracking and other non-essential cruft from shared URLs, for readability and for politeness to my correspondents. I don't think it's polite to associate their IP with my tracked activity.

Do you have any tools to automate that process?

Not really. I use the Uppity extension in Firefox.

So if Uppity shows me:

  http://fortune.com/2017/05/02/donald-trump-wall-street-elite/?xid=gn_editorspicks&google_editors_picks=true
  http://fortune.com/2017/05/02/donald-trump-wall-street-elite/
  http://fortune.com/2017/05/02/
  http://fortune.com/2017/05/
  http://fortune.com/2017/
  http://fortune.com/
  http://www.fortune.com/
Then I pick select the second one from the top, let it load to be sure I got a usable URL, then use that in my email. This is probably ideal, since anything more automated would probably get it wrong often enough that I'd go back to doing just this.

Firefox: Uppity https://addons.mozilla.org/en-US/firefox/addon/uppity/

Firefox: Navigate Up WE https://addons.mozilla.org/en-US/firefox/addon/navigate-up-w...

Chrome: Up https://chrome.google.com/webstore/detail/up/iohgglcbddjknne...

(And what is up with those chrome store URLs?)

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#57

I've got a suggestion for a better format called hurl, I'm working on the RFC as I write this. It's constant length , doesn't require database storage, avoids collisions and compresses longer urls far better than the huffman coding method. Here is the complete implementation (in go): func hurl(s string) string { sum := sha256.Sum256([]byte(s)) return base64.RawURLEncoding.EncodeToString(sum[:]) } https://play.golang.…

Do this with MD5 and then find the original link using bozocrack[0].

[0]: https://github.com/juuso/BozoCrack

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#59
post #4

Earlier quoted context omitted.

The idea is still interesting. I imagine using some dictionary based compression and crawled data to build a dictionary could get us somewhere.

Unfortunately the longest urls, the ones you want to shorten the most, are the worst candidates for compression (with long query strings full of things like uuids).

I have an idea to identify the uuid and other identifier type parts and encode them up as numbers. The file radix_coder.js is working toward this. So we cover a few formats like guid base64 digits base36, to get a little bit more gain.

But my initial experiments suggest it's just a little gain. Digits go to 68%, base36 to 90%, and then we also have to add in the prefix to indicate we are switching encodings.

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#60
post #8
post #2

The url for this post is 45 characters: https://news.ycombinator.com/item?id=14245119 If you encode it with this shortener, you get this 35 character string: mNb:w9iIp7u8di:AKB2xrPUVYUFhfRUWHwA Assuming you were using this string as a key in a shortening service like this: https://short.url/mNb:w9iIp7u8di:AKB2xrPUVYUFhfRUWHwA ... you'd end up with a url longer than the original url! So it's not technically a url shor…

Open standards for shortening URLs rather than private Hash Tables of random shortlinks is a great idea. Being open source and predictable it would not be necessary to specify any particular shortening service so that part of the URL is extraneous. Of course there would need to be some chars to refer to which particular open standard shortening encoder was used, still would be shorter than the oringal url for this po…

Yes that is actually a rather faithful rendering of the types of ideas and possible future paths or use cases I had in mind. Thank you for saying that.
Post reply on HN