Live data from Hacker News

Show HN: Shortern URIs using Huffman Coding, not a database

urizip-dot-populace-soho.appspot.com

41–50 of 87 posts

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#41
post #29

I tried: https://github.com/dosaygo-coder-0/urizip/commit/cb2bfd2e04e8b814943e42a5bf87eeff31a77126 Got kl9oo67XTkQCETxduLJSW5iUfSNh5pW6iDuhqKmzprOoJCaUYmMo5M7kiMaYxrBRjTQsWzqDks9BlBQWEQKsrKImrQU Which is longer. Also "OK" isn't a great commit message for almost every commit: https://github.com/dosaygo-coder-0/urizip/commits/master

It isn't great.... it's just ok

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#42

Often urls can be most effectively shortened manually. I wish everybody who sends me something like this: https://www.booking.com/hotel/fr/hotelwestminster.html?label=gen173nr-1FCAEoggJCAlhYSDNiBW5vcmVmaDuIAQGYATHCAQN4MTHIAQzYAQHoAQH4AQKSAgF5qAID;sid=9b4fe19e9de68a3cb71714046bf9d64a;checkin=2017-05-12;checkout=2017-05-13;ucfs=1;highlighted_blocks=5190001_91458119_0_2_0;all_sr_blocks=5190001_91458119_0_2_0;room1=A;hpo…

Amazon is another great example. https://www.amazon.com/Cat-Facts-Marcus-Schneck/dp/0880295589/ref=sr_1_1?ie=UTF8&qid=1493754913&sr=8-1&keywords=cat+facts can become https://amazon.com/dp/0880295589/

Amazon has their own link-shortener that you get by clicking "Share". Those links look like: http://a.co/a0GORjY

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#43
post #27

Earlier quoted context omitted.

I think you missed the part about it being serverless. I think the idea is to integrate the logic into the client.

But what's the benefit to turning a readable url into an unreadable one that's only about 3/4 the size?

Obviously, it's sending network packets with 90% of padding instead of just 80%.

Twitter has created some problems for the internet.

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#44
post #36

Earlier quoted context omitted.

This might be a Poe's Law situation, but how am I supposed to reverse an arbitrary SHA256 hash without having that information stored somewhere? e.g. how do you expect a client to turn "r0WGHoHvVmv4I51qW9FxCAIxX8NSfYlds1Pi-Of92ZI" into something useful?

Sorry should't post jokes like this on the internet - I've moved 'more seriously' to the end to make it clearer.

I wanted to be sure you weren't describing a DHT for url shorteners ... a distributed content-addressable hashtable of shortened urls

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#45
post #44

Earlier quoted context omitted.

Sorry should't post jokes like this on the internet - I've moved 'more seriously' to the end to make it clearer.

I wanted to be sure you weren't describing a DHT for url shorteners ... a distributed content-addressable hashtable of shortened urls

I suppose the nice bit of this (original) proposal is that it works without any kind of connection or storage. You could use something like a blockchain to store this kind of thing, but you're still reliant on that existing in the future. Nice idea tho. You can also use hashing if you're the origin server (the idea behind content addressable stuff), and if you go away the data goes away anyway so it doesn't matter...

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#46
If you use React with react-router[0] (or a comparable JS-based solution), you can just apply lz-string[1] to a stringified JS object and make the resulting string match a path. Then you need is some code to reverse the operation (lz-string -> JSON -> JS object). That's what I'm doing to create shareable state in an app I'm building for a research group.

Done naively, it will have the same problems others mentioned here: it barely shortens the URL. However, in the case of my specific app - a browser for data sets for single cell RNA sequencing - the state is largely dependent on the data set being viewed, and the data set is unchanging. What that means is that we can "externalise" the data being referred to, to pre-transform my JS object (which is an object tree) to nested arrays with integers (so a kind of "array tree") that use the data set as a look-up to reverse the operation. The latter is already a lot shorter when stringified, but also more compressible: the character set is limited to ten digits, commas and square brackets, and different values may be transformed into the same numbers, being distinguished by their position in the array tree.

To make this operation easier, I created a few helper functions to declare a "schema" that creates a recursive function for transforming the original JS object to said array of arrays, and vice-versa:

https://gist.github.com/JobLeonard/a47692a1f77bebc06c2518f32...

As you can see in the gist, that transformation shrunk a URL of 2466 characters down to a (admittedly still crazy) 638 characters.

I was thinking on putting it on NPM but it looked so specific in its applicability that I haven't bothered turning it into a proper package yet.

[0] https://github.com/ReactTraining/react-router

[1] https://github.com/pieroxy/lz-string

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#47
post #29

I tried: https://github.com/dosaygo-coder-0/urizip/commit/cb2bfd2e04e8b814943e42a5bf87eeff31a77126 Got kl9oo67XTkQCETxduLJSW5iUfSNh5pW6iDuhqKmzprOoJCaUYmMo5M7kiMaYxrBRjTQsWzqDks9BlBQWEQKsrKImrQU Which is longer. Also "OK" isn't a great commit message for almost every commit: https://github.com/dosaygo-coder-0/urizip/commits/master

https://github.com/dosaygo-coder-0/urizip/blob/master/build#...

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#48
post #9
post #4

Earlier quoted context omitted.

The idea is still interesting. I imagine using some dictionary based compression and crawled data to build a dictionary could get us somewhere.

That's basically what a Huffman code is. Once you have a big list of examples, it's just a way to encode it to take advantage of frequency of occurrence.

If it were to use a dictionary that defines a large subset of domain names and common strings found in URLs it could reduce the size somewhat more, but still seems too long to be viable. What might be more interesting is an Ethereum approach to URL shortening that is distributed amongst the Internet.

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#49
post #5
post #3

Neat but SMAZ has better performance :-) https://github.com/antirez/smaz/tree/master

Performance in terms of compression ratio or in terms of speed or in terms of both or in terms of something else like for example memory usage? Is SMAZ encoded data URL safe?

I meant compression ratio, but I overlooked the rather important fact that it's not URL safe

Re: Show HN: Shortern URIs using Huffman Coding, not a database

#50

Often urls can be most effectively shortened manually. I wish everybody who sends me something like this: https://www.booking.com/hotel/fr/hotelwestminster.html?label=gen173nr-1FCAEoggJCAlhYSDNiBW5vcmVmaDuIAQGYATHCAQN4MTHIAQzYAQHoAQH4AQKSAgF5qAID;sid=9b4fe19e9de68a3cb71714046bf9d64a;checkin=2017-05-12;checkout=2017-05-13;ucfs=1;highlighted_blocks=5190001_91458119_0_2_0;all_sr_blocks=5190001_91458119_0_2_0;room1=A;hpo…

Which is actually just https://www.reddit.com/68sgew

Which is really just

    https://redd.it/68sgew
Post reply on HN