Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Idempotence: What is it and why should I care?

cloudingmine.com

Post reply on HN
Print view View original

21–30 of 77 posts

Re: Idempotence: What is it and why should I care?

#21

Post by tzs »

gotodengo wrote: post #7

Reminds me of the Spotify unicode username issue[1]. Where a function assumed to be idempotent (think tolower(username)) actually wasn't with certain unicode inputs. Allowing account takeovers. [1] https://labs.spotify.com/2013/06/18/creative-usernames/

spotify> For example it is hard to see the difference between Ω and Ω even though one is obviously a Greek letter and the other is a unit for electrical resistance and in unicode they indeed have different code points

This surprised me, because the correct Ohm symbol is in fact the Greek letter, so why does Unicode have a special code point for it?

Unicode also does this for Kelvin, where the correct symbol is a capital K but Unicode has a separate code point for it, and for ångström where the correct symbol is a capital A with a circle above it but Unicode gives it a separate code point.

They do not do this for Newtons (capital N), Joules (capital J), Watts (capital W), or anything else I can see where the standard symbol is an ordinary letter or group of letters.

In all three of these cases the Unicode Consortium recommends NOT using the separate code point.

So...what's special about Ohms, Kelvins, and ångström that (1) gives them their own place in Unicode, and (2) what is the point since we are not, according to the Unicode Consortium, supposed to use them?

Re: Idempotence: What is it and why should I care?

#22

Post by rectangletangle »

ageitgey wrote: post #18

This is vital if you are designing apis or clients that deal with charging a user money. It should be literally impossible for a user to accidentally get charged twice due to a flakey connection if you design correctly. The trick is to have the client generate a random 'idempotency key' (a uuid) to start each logical transaction and have the server use that id to prevent double charges of the same transaction. By alw…

This is solid advice. Another common trick is to disable the submit event when the submit button is clicked for the first time, preventing two requests from firing, when the user double clicks. Then re-enable the event, if the request fails. Ideally this is done in addition to server side nonce validation, and not as the only preventative measure, because browser differences, or network issues could cause a double request (more likely if the HTTP GET method is used, instead of a proper POST). Regardless server and client side techniques should be used together to create a better UX, similar to using both client and server side email format validation.

Re: Idempotence: What is it and why should I care?

#23

Post by gotodengo »

tzs wrote: post #21
gotodengo wrote: post #7

Reminds me of the Spotify unicode username issue[1]. Where a function assumed to be idempotent (think tolower(username)) actually wasn't with certain unicode inputs. Allowing account takeovers. [1] https://labs.spotify.com/2013/06/18/creative-usernames/

spotify> For example it is hard to see the difference between Ω and Ω even though one is obviously a Greek letter and the other is a unit for electrical resistance and in unicode they indeed have different code points This surprised me, because the correct Ohm symbol is in fact the Greek letter, so why does Unicode have a special code point for it? Unicode also does this for Kelvin, where the correct symbol is a capi…

At this point it's a backwards compatibility issue. Like you say for Ohm they now recommend using the omega symbol[1] but there's still code out there using the Ohm symbol.

Solving that wouldn't have helped in the Spotify case though since there's a ton of other edge cases like combining characters 'e' + ' ́' vs precomposed characters 'é' which still cause the need for an idempotent canonicalization of usernames.

Not to get too far from the topic at hand, but I came across the Spotify article earlier this week while looking to support Unicode usernames in an application. After consideration I've decided to just lock things down to ASCII for now. It's just too big a case to consider and there are bigger fish to fry.

[1] https://en.wikipedia.org/wiki/Ohm#Ohm_symbol

Re: Idempotence: What is it and why should I care?

#24

Post by [deleted] »

rectangletangle wrote: post #22
ageitgey wrote: post #18

This is vital if you are designing apis or clients that deal with charging a user money. It should be literally impossible for a user to accidentally get charged twice due to a flakey connection if you design correctly. The trick is to have the client generate a random 'idempotency key' (a uuid) to start each logical transaction and have the server use that id to prevent double charges of the same transaction. By alw…

This is solid advice. Another common trick is to disable the submit event when the submit button is clicked for the first time, preventing two requests from firing, when the user double clicks. Then re-enable the event, if the request fails. Ideally this is done in addition to server side nonce validation, and not as the only preventative measure, because browser differences, or network issues could cause a double re…

[deleted]

Re: Idempotence: What is it and why should I care?

#25

Post by rectangletangle »

idempotent wrote: post #4

A function f is idempotent if: f(f(x)) = f(x) The "absolute value" function is idempotent: abs(abs(-42)) = abs(-42) The "squared" function is not: sq(sq(7)) != sq(7)

One of my favorite parts of Python is the idempotent type casting functions.

Say you have a function that takes a list as input, but sometimes a tuple is passed. Wrapping the input like lvalues = list(values), ensures that lvalues will always be a list (or it throws a type error), so you won't get annoying attribute errors, when trying to access the list's methods. In the case of a list type(list([])).__name__ == 'list', as does type(list(list([]))).__name__, etc.

This can get a bit tricky with situations where type casting may be undesirable, say casting a string to a list. So judicious use is necessary.

Re: Idempotence: What is it and why should I care?

#26

Post by PinkMilkshake »

rectangletangle wrote: post #22
ageitgey wrote: post #18

This is vital if you are designing apis or clients that deal with charging a user money. It should be literally impossible for a user to accidentally get charged twice due to a flakey connection if you design correctly. The trick is to have the client generate a random 'idempotency key' (a uuid) to start each logical transaction and have the server use that id to prevent double charges of the same transaction. By alw…

This is solid advice. Another common trick is to disable the submit event when the submit button is clicked for the first time, preventing two requests from firing, when the user double clicks. Then re-enable the event, if the request fails. Ideally this is done in addition to server side nonce validation, and not as the only preventative measure, because browser differences, or network issues could cause a double re…

That's only a partial solution. It may be a user agent or a queuing system (or anything else sitting between a person clicking the button and where the final transaction is recorded) that attempts a retry.

Re: Idempotence: What is it and why should I care?

#27

Post by digitaLandscape »

rectangletangle wrote: post #22
ageitgey wrote: post #18

This is vital if you are designing apis or clients that deal with charging a user money. It should be literally impossible for a user to accidentally get charged twice due to a flakey connection if you design correctly. The trick is to have the client generate a random 'idempotency key' (a uuid) to start each logical transaction and have the server use that id to prevent double charges of the same transaction. By alw…

This is solid advice. Another common trick is to disable the submit event when the submit button is clicked for the first time, preventing two requests from firing, when the user double clicks. Then re-enable the event, if the request fails. Ideally this is done in addition to server side nonce validation, and not as the only preventative measure, because browser differences, or network issues could cause a double re…

I don't like this because in practice sites usually fail to re-enable the submit button if something goes wrong. Just let the user submit multiple requests if they want to retry, don't take that away from them, just make it harmless.

Re: Idempotence: What is it and why should I care?

#28

Post by joombaga »

jackthetab wrote: post #16
pmarreck wrote: post #5

One of the practical applications of this concept I've found is that I try to write idempotent database migrations (so rerunning the migration, which is a common necessity while you're developing it, but is also useful if problems occur, won't error). So in essence both the "up" and the "down" migrations are idempotent and warn if they are not (and why).

How do you do it without throwing an error? My philosophy has been to ignore the error if it's a re-application of the migration because the migration _is_ idempotent. So ALTER TABLE foo ADD COLUMN bar INT either succeeds (because there is no bar column) or it fails (because there is a bar column) with no harm done.

This would work if you're just ignoring that specific error, but it's better to guard for idempotence. Only apply that ALTER if COLUMN bar doesn't already exist.

Re: Idempotence: What is it and why should I care?

#29

Post by akira2501 »

tzs wrote: post #21
gotodengo wrote: post #7

Reminds me of the Spotify unicode username issue[1]. Where a function assumed to be idempotent (think tolower(username)) actually wasn't with certain unicode inputs. Allowing account takeovers. [1] https://labs.spotify.com/2013/06/18/creative-usernames/

spotify> For example it is hard to see the difference between Ω and Ω even though one is obviously a Greek letter and the other is a unit for electrical resistance and in unicode they indeed have different code points This surprised me, because the correct Ohm symbol is in fact the Greek letter, so why does Unicode have a special code point for it? Unicode also does this for Kelvin, where the correct symbol is a capi…

> So...what's special about Ohms, Kelvins, and ångström

Nothing other than misguided thinking in the early versions of the standard.

The other problems with these special symbols is that if you call tolower() or similar on them they'll return the "normal" character they're based off of. So toupper(tolower(char)) != char.

Post reply on HN