Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SHA-3 Buffer Overflow

mouha.be

Post reply on HN
Print view View original

151–160 of 186 posts

Re: SHA-3 Buffer Overflow

#151

Post by metadat »

metadat wrote: post #82
adrian_b wrote: post #80

Earlier quoted context omitted.

There is a good reason to use SHA-256 and not SHA-512. Many modern CPUs, e.g. all AMD Zen, most Intel Atom, Intel Ice Lake and newer, most 64-bit ARM, have hardware implementations of SHA-256, which are much faster than software computing SHA-512. Only some more recent 64-bit ARM CPUs also have hardware for SHA-512 and SHA-3. Whenever the speed matters, SHA-256 is the best choice, unless you choose different hash alg…

Interesting, in general I've noticed SHA-256 hashing is relatively CPU intensive and slow.. is the 2024 Xeon CPU in my server too old to include the hardware implementation?

*2014, yes it was a typo

Re: SHA-3 Buffer Overflow

#152

Post by metadat »

tedunangst wrote: post #103
metadat wrote: post #82

Earlier quoted context omitted.

Interesting, in general I've noticed SHA-256 hashing is relatively CPU intensive and slow.. is the 2024 Xeon CPU in my server too old to include the hardware implementation?

Is your software using SHA instructions?

I imagine the sha256sum program would use the best available technique.

Reasonable assumption?

Re: SHA-3 Buffer Overflow

#153

Post by manholio »

adev_ wrote: post #150
manholio wrote: post #148

Earlier quoted context omitted.

Surely "nails in the coffin of C/C++" does not imply an all out effort to rewrite all this existing and working software from scratch. The beast will die in the same sense Fortran or Cobol are dead: an outdated language that has no noticeable strengths in the modern world, used by a bunch of (difficult to hire) old timers to fix legacy systems.

> Surely "nails in the coffin of C/C++" does not imply an all out effort to rewrite all this existing and working software from scratch. Or you use the evolutionary approach: remove the unsafe part from C/C++ bit by bit. And do code migration (Carbon style) when you can do it. This is what some C and C++ committee members try to do and fortunately they are there. Because that is way more likely to happen in a reasona…

> remove the unsafe part from C/C++ bit by bit.

You cannot do that without breaking backward compatibility. If you maintain compatibility, what results is a sprawling meta-language with a large learning burden that has the theoretical ability to emulate a safe language, and which in practice will be used to mix both safe and unsafe idioms depending on the experience and discipline of the programmer, in an unholy mess that no static analyzer can prove as correct and no programmer can say they really fully understand.

Maybe I'm mistaken, I've last programmed in C++ a good number of years ago, but I have not encountered a (C++ programming paradigm, static linter) doublet that can prove data integrity and thread safety to an extent even close to the Rust compiler. And it would be useless for existing codebases anyhow.

Re: SHA-3 Buffer Overflow

#154

Post by adrian_b »

metadat wrote: post #152
tedunangst wrote: post #103

Earlier quoted context omitted.

Is your software using SHA instructions?

I imagine the sha256sum program would use the best available technique. Reasonable assumption?

Last time when I have checked what sha256sum (from coreutils) does was one year ago.

At that time, on x86 CPUs (I have not tested on ARM) neither sha256sum nor sha224sum nor sha1sum used the hardware instructions (and I have compiled them from sources, with the appropriate flags).

Because I have a Zen CPU (but that would also be true on Atom CPUs or Core CPUs since Ice Lake), I have to use "openssl dgst -r -sha256" instead of sha256sum and "openssl dgst -r -sha1" instead of sha1sum.

The openssl implementation uses the hardware instructions where available and in that case it is several times faster.

(The "-r" option flag in "openssl dgst" selects an output format that is more similar to the sha???sum tools, i.e. hash value followed by file name, but openssl still outputs an extra character, which must be accounted in scripts.)

Re: SHA-3 Buffer Overflow

#155

Post by kevincox »

GTP wrote: post #149
londons_explore wrote: post #141

Earlier quoted context omitted.

> two-sided negotiations of crypto algorithms This is a common problem, and massively increases attack surface, since if a vulnerability can be found in any algorithm, it can be exploited. Downgrade attacks are a varient of the same. Let me present an alternative: Each protocol has one specific encryption algorithm and set of parameters. However, that set isn't fixed, but defined based on the current date and time. F…

As you say, it could be that an algorithm gets broken before its "expiry date" or remains secure well past that (look e.g. at AES). There is a similar but better alternative, usually called "opnionated cryptography". A protocol specifies a single cryptographic primitive for each type needed (hash function, block cipher etc) so no negotiation is needed. If one of the primitives gets broken, a new version of the protoc…

It seems to me that it may be better in many cases to support exactly two configurations (or some other small fixed number). This way there is no need to wait until a new version is released and dropping the vulnerable one won't cause any downtime. Ideally both codepaths would be regularly exercised (maybe clients should pick randomly when both choices are available). Then as soon as a vulnerability is discovered (maybe even before it is publicly revealed) all clients and servers could immediately drop support for the one that was discovered vulnerable.

Then the release of a new configuration can be done slowly with reasonable caution to get back to the desired redundancy level.

Re: SHA-3 Buffer Overflow

#156

Post by kevincox »

thayne wrote: post #94
nequo wrote: post #56

Earlier quoted context omitted.

The purpose of running SHA is to determine whether the input can be trusted though. But it is true that relatively few files circulate at or above that size.

And even if they are, you probably aren't going to do a digest on the whole thing all at once.

IDK, I see a lot of "non-production" code that likes to read whole files into a in-memory buffer rather than bothering with streaming. With modern memory sizes it is completely possible that you could read a 4GiB file into memory without having issues.

Re: SHA-3 Buffer Overflow

#157

Post by aliqot »

nullc wrote: post #53
aliqot wrote: post #6

Interesting they both say "Official Sha3" and "by its designers", which as I remember it isn't that accurate. Keccak was chosen and then NIST added what is affectionately known as the 'mystery padding' before certification. What we know as official is not the way the designers submitted the proposal. This isn't an attempt at a scary accusation, but as a pedant, this got me. For those wondering, here is an explanation…

Who actually wrote the faulty code? I had the impression that the padding change was written by the keccak team at the request of NIST.

Keccak for xkcp and NIST for the padding, as far as I can remember. As I recall the padding was added first, and the explanation came later. There was great distrust at first during this time as a result, hence the name 'mystery padding' we used.

Re: SHA-3 Buffer Overflow

#158

Post by metadat »

adrian_b wrote: post #154
metadat wrote: post #152

Earlier quoted context omitted.

I imagine the sha256sum program would use the best available technique. Reasonable assumption?

Last time when I have checked what sha256sum (from coreutils) does was one year ago. At that time, on x86 CPUs (I have not tested on ARM) neither sha256sum nor sha224sum nor sha1sum used the hardware instructions (and I have compiled them from sources, with the appropriate flags). Because I have a Zen CPU (but that would also be true on Atom CPUs or Core CPUs since Ice Lake), I have to use "openssl dgst -r -sha256" i…

I also see similarly slow performance when using the Go built-in Sha256 hashing library.

Thanks for the info! Fortunately for my purposes of integrity checking, slow is okay.

Re: SHA-3 Buffer Overflow

#159

Post by remram »

GTP wrote: post #149
londons_explore wrote: post #141

Earlier quoted context omitted.

> two-sided negotiations of crypto algorithms This is a common problem, and massively increases attack surface, since if a vulnerability can be found in any algorithm, it can be exploited. Downgrade attacks are a varient of the same. Let me present an alternative: Each protocol has one specific encryption algorithm and set of parameters. However, that set isn't fixed, but defined based on the current date and time. F…

As you say, it could be that an algorithm gets broken before its "expiry date" or remains secure well past that (look e.g. at AES). There is a similar but better alternative, usually called "opnionated cryptography". A protocol specifies a single cryptographic primitive for each type needed (hash function, block cipher etc) so no negotiation is needed. If one of the primitives gets broken, a new version of the protoc…

And then you still have a roll-out period during which those two protocols are available, so the two parties still need to negotiate which version they want. It's not clear to me what the advantage over the current situation is?

Re: SHA-3 Buffer Overflow

#160

Post by adunsulag »

FYI in the healthcare space to meet ONC Cures Update regulations for bulk patient export, its a requirement to use SHA384 (RS384 or ES384). I would think most implementations will be safe due to the 4GB payload criteria, but if anyone is doing anything funky they'll need to look at this closely.
Post reply on HN