Live data from Hacker News

Reddit's photo albums broke due to Integer overflow of Signed Int32

old.reddit.com

21–30 of 142 posts

Re: Reddit's photo albums broke due to Integer overflow of Signed Int32

#22
post #8

Earlier quoted context omitted.

Someone probably joked they would never reach that scale when they wrote that code

So you migrate to int64 and one day someone will wonder why the hell did we ever think no one would reach 2^64 rows in a database table. Or that 2^128 IP addresses would be enough for everyone.

Those numbers are much, much larger than 2^32.

No process that touches the real world gets into 2^64. No process at all gets into 2^128, at lwast not while we live around a single star.

Re: Reddit's photo albums broke due to Integer overflow of Signed Int32

#23

Earlier quoted context omitted.

Thinking "if that ever gets anywhere close to a problem we'll have vast resources and plenty of time to fix it" and then, I'm guessing, that person left a few months later and nobody owned that part of the code because it worked. Then 10 or so years went by... Whenever I write code like that which may break in say, 5 years, I'll sign it in the comments and put my personal email and phone number inviting future people…

To people reading this: please never ever do that, unless it was agreed upon with your client/employer beforehand (and compensated accordingly). If you know the code may break in 5 years, why don't you fix it now? Because your employer doesn't want to pay you to do that (probably for good reason, surely there's higher priorities). Then why would you do it for free years later?

No.

You can say, anticipate breaking changes that is calendared to happen as an eventually rollover or deprecation

Essentially things have to work one way now and there's solid reason to think it will no longer work that way later and there's no valid path to fix because exogenous conditions change.

This alternatively, can be done in bad faith to guarantee a future paycheck by intentionally placing timebombs in and then charging high rates to come back. There's incentives for abuse.

I'm making it clear and acting in good faith. I can give them 2 hours for free in 5 years, whatever. I'm a competent and responsible person and I get paid well by not being so cheap.

Re: Reddit's photo albums broke due to Integer overflow of Signed Int32

#24
post #3

A long time ago we discovered Twitter used a round robin of three servers for assigning IDs to tweets. We inferred the round robin was done by doing mod 3 of a signed int32, and because that space doesn't divide neatly by two it meant one of the three servers saw less load than the others and we could map ID assignment volume according to how often it overflowed and hence estimate total tweet volume for a given perio…

That's really neat, I'd love to hear more about it. Was this something you were actively trying to find out, or was it poking around until something caught your eye?

Not op, but at an ecommerce company I worked for we did similar things to track how well our competitors were doing relative to us so could be something like that.

Also collecting data like this can be useful if you want to beat markets.

Re: Reddit's photo albums broke due to Integer overflow of Signed Int32

#25
post #9
post #2

Two days ago, Reddit ids have finally incremented passed the 2,147,483,647, or the maximum range of a signed int32. It seems one of Reddit's subsystems, the one that serves its photo albums broke due to the integer overflow.

Strange thing is that photo albums re relatively new. Imgur was the go-to host for Reddit, and then they made their own uploader a looong time later. The "albums" functionality only came out in July of 2020, according to a Google search. Seems this was less likely a "someone else will deal with it" problem, and more of a development / QA testing problem.

For some reason most stuff still defaults to i32 and a lot of people use them for new code. At this point I'd not be against linters warning against using 32 bit ints unless you have a good reason.

Re: Reddit's photo albums broke due to Integer overflow of Signed Int32

#26
post #8

Earlier quoted context omitted.

Someone probably joked they would never reach that scale when they wrote that code

So you migrate to int64 and one day someone will wonder why the hell did we ever think no one would reach 2^64 rows in a database table. Or that 2^128 IP addresses would be enough for everyone.

“32-bit IPv4 addresses to 128-bit IPv6 addresses means going from 4.3 billion to 340,282,366,920,938,463,463,374,607,431,768,211,456”

Yes 128 bits is only 4x times the bit length, but the address space is exponentially bigger.

Some predict if we do run out, it might take ~100 years.

Re: Reddit's photo albums broke due to Integer overflow of Signed Int32

#27
Nostalgic flashback to Premier Manager games, where players stats decreased as they aged. When they went /below 0/ they flipped around to 127. So a good strategy was to scout out really bad players from lower leagues about to hit age 30+. And offer them very long contracts to prevent them retiring .. and give time for most of their stats to flip around, turning them into superstars.

Re: Reddit's photo albums broke due to Integer overflow of Signed Int32

#28

Earlier quoted context omitted.

Thinking "if that ever gets anywhere close to a problem we'll have vast resources and plenty of time to fix it" and then, I'm guessing, that person left a few months later and nobody owned that part of the code because it worked. Then 10 or so years went by... Whenever I write code like that which may break in say, 5 years, I'll sign it in the comments and put my personal email and phone number inviting future people…

To people reading this: please never ever do that, unless it was agreed upon with your client/employer beforehand (and compensated accordingly). If you know the code may break in 5 years, why don't you fix it now? Because your employer doesn't want to pay you to do that (probably for good reason, surely there's higher priorities). Then why would you do it for free years later?

I'm fairly certain most managers would argue that I'm wasting my time if I spend it chasing bugs that won't happen for several years.

Re: Reddit's photo albums broke due to Integer overflow of Signed Int32

#29

A classic case of "ids aren't numbers even if you choose to make them numeric"

ids being sortable has a lot of advantages over random guids.

This is really a false dichotomy you don't have to use guid/uuid. I'm saying even if you use sortable auto increment numbers, stop storing them like numbers.

Re: Reddit's photo albums broke due to Integer overflow of Signed Int32

#30

A classic case of "ids aren't numbers even if you choose to make them numeric"

ids being sortable has a lot of advantages over random guids.

Them being dense is advantageous too. Numbers are a very convenient format for encoding IDs, but that doesn't mean that IDs are numbers.
Post reply on HN