The search for a faster CRC32
blog.fastmail.com
The search for a faster CRC32
1–10 of 53 posts
Re: The search for a faster CRC32
#2why not rip out CRC32 and put in xxhash?
Re: The search for a faster CRC32
#3why not rip out CRC32 and put in xxhash?
Probably don't want to go back and rewrite all their past data.
Re: The search for a faster CRC32
#4why not rip out CRC32 and put in xxhash?
Probably don't want to go back and rewrite all their past data.
Pretty much this. From the post:
"Because it produces different results, it would render all our stored checksums invalid, so we'd have to recalculate them all. That's a big job, so we'd need to do a lot of testing first to make sure that it's so much better that it's actually worth the effort."
Re: The search for a faster CRC32
#5why not rip out CRC32 and put in xxhash?
More about xxhash here, it's by Yann Collet, the author of LZ4:
http://fastcompression.blogspot.fr/2012/04/selecting-checksu...
Re: The search for a faster CRC32
#6Re: The search for a faster CRC32
#7Earlier quoted context omitted.
Probably don't want to go back and rewrite all their past data.
xxHash is at least an order of magnitude faster so it might be worth upgrading.
My quick tests here (using the same methods outlined in the post) suggests its around 30% faster on 64-byte buffers. If we're ever shopping for a new hash function entirely, I'll make sure its considered. Right now its not worth it because as noted in the post, we're not actually under any particular stress and changing hash functions is a big job.
Re: The search for a faster CRC32
#8why not rip out CRC32 and put in xxhash?
xxhash is great, I recommend it.
Re: The search for a faster CRC32
#9Hope they contribute the change back to Cyrus.
Re: The search for a faster CRC32
#10Earlier quoted context omitted.
xxHash is at least an order of magnitude faster so it might be worth upgrading.
My quick tests here (using the same methods outlined in the post) suggests its around 30% faster on 64-byte buffers. If we're ever shopping for a new hash function entirely, I'll make sure its considered. Right now its not worth it because as noted in the post, we're not actually under any particular stress and changing hash functions is a big job.
[deleted]