11–20 of 48 posts
You can use 64bit floats (double) to store exact integers up to 53 bits.
This works in a lot of languages on non 64bit hardware, php for example.
As I have suggested in a post 9 months ago, and as I would have designed this system at any date since 2002 or so, the twit ID would be composed of userid bits + time bits. In the post below I suggested 32+32, but other divisions are acceptable depending on your "bot user" policy. Such an ID would at the same time be sufficient (up to 4G users, up to 5 tweets/sec/user AVERAGE, up until 2030). You can have two times as many users for just "2 tweets/sec average". Facebook only has 500M, so 4G should be sufficient.
Such a construct makes the entire system significantly simpler and more robust to "meaningful" failure. I haven't seen a single thing tweeter has done right in the technical sense.
They do deserve marketing and bizdev credit.
http://www.reddit.com/r/programming/comments/b2u6t/twitter_o...
When the first 32-bit troubles occurred, they should have just switched to a string-based ID, IMO.
That was my first thought... who on earth thought that a number (albeit 64-bit) would be enough for Twitter? Who even thought that a 32bit int would have been enough? I didn't know that Javascript couldn't handle numbers bigger than 53-bits, but honestly, these should have been strings from the beginning.
That's truly, truly dumb on twitter's behalf. They should have started with using the whole numeric range and/or switched to string ids long ago. As I have suggested in a post 9 months ago, and as I would have designed this system at any date since 2002 or so, the twit ID would be composed of userid bits + time bits. In the post below I suggested 32+32, but other divisions are acceptable depending on your "bot user"…
You would be changing behavior in a way that breaks apps. Twitter doesn't want to do that.
When the first 32-bit troubles occurred, they should have just switched to a string-based ID, IMO.
When the first 32-bit troubles occurred, they should have just switched to a string-based ID, IMO.
That was my first thought... who on earth thought that a number (albeit 64-bit) would be enough for Twitter? Who even thought that a 32bit int would have been enough? I didn't know that Javascript couldn't handle numbers bigger than 53-bits, but honestly, these should have been strings from the beginning.
This seems to be the relevant id generating code: https://github.com/twitter/snowflake/blob/master/src/main/sc...
When the first 32-bit troubles occurred, they should have just switched to a string-based ID, IMO.
I imagine that they did think of this, but a good string hash takes time. So switching to 64-bits takes much less time, and it would work until they finished snowflake.
My rule of thumb is to use strings for all IDs that belong to external systems. A bunch of people got burned in the same way a few years ago when Flickr photo IDs rolled past 32 bits.
http://www.ietf.org/rfc/rfc4122.txt
uuids are made just for this purpose, string based, never bigger than 40 characters (with dashes and curly braces).
Most products use uuids or Microsoft's name for them guids.
My rule of thumb is to use strings for all IDs that belong to external systems. A bunch of people got burned in the same way a few years ago when Flickr photo IDs rolled past 32 bits.
Earlier quoted context omitted.
That was my first thought... who on earth thought that a number (albeit 64-bit) would be enough for Twitter? Who even thought that a 32bit int would have been enough? I didn't know that Javascript couldn't handle numbers bigger than 53-bits, but honestly, these should have been strings from the beginning.
64 bits is enough to have everyone on Earth send over a billion Tweets and still have enough room to find a new solution. That sounds like more than enough to me.
IMO, it's not a bad idea on their part. A 32 bit UNIX timestamp * 2 ^ 32 + a 32 bit sequential id let's them track up to 4.2 billion tweets a second and should work just find up to the year 2106.
Edit: As to why it's a good idea, you can have different systems handing out ID's without stepping on each other’s toes or even talking to each other. The full ID is composed of a timestamp, a worker number, and a sequence number. Granted, I would probably put the sequence number ahead of the worker number so sub second tweets are better ordered vs. being ordered strictly based on the system that generated them.