> To disable slow i18n routines and use traditional byte-based sort order, use export LC_ALL=C (in fact, consider putting this in your ~/.bashrc). Do not. Having a non-utf8 locale means you won't be able to handle utf-8 sanely ("that's why it's faster") and it will break at the most inexplicable times. Any non-latin1 character appearing in your prompt or command line with this will mess its spacing up for example. Do…
I have to reiterate this; don't do it -- sort order changes and you'll run into other mysterious issues. It's not worth the "performance improvement".
$ echo $'a\naa\nb\nå'| LC_ALL=C sort
a
aa
b
å
$ echo $'a\naa\nb\nå'| LC_ALL=nn_NO.UTF-8 sort
a
b
å
aa
Even worse, some characters that are byte-different collate the same: $ echo $'∨\n∧'|LC_ALL=C sort -u
∧
∨
$ echo $'∨\n∧'|sort -u
∨
Handling Unicode sanely requires understanding some Unicode.