Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SHA-3 Buffer Overflow

mouha.be

Post reply on HN
Print view View original

31–40 of 186 posts

Re: SHA-3 Buffer Overflow

#31

Post by tptacek »

derefr wrote: post #27
tptacek wrote: post #23

Earlier quoted context omitted.

Easy enough to test, right? func main() { h := sha3.New224() buf := make([]byte, 4294967295) h.Write(buf) sum := h.Sum(nil) fmt.Printf("%x\n", sum) } Doesn't crash on my amd64 dev machine. Later I could have just looked at the code, too: the assembly you've linked to is just the Keccak permutation, not the entire Go hash; the buffer management is done in Go, not in assembly.

This is one of those cases where I'm actually more concerned that it doesn't crash. It's like seeing clearly-syntactically-invalid code that somehow compiles anyway — you wonder what semantics the compiler could have possibly ascribed to it. Presumably this isn't not-crashing just because the developers of the Golang stdlib somehow found+fixed this bug back in 2015 when this assembler file was baked. The error is in…

No, it causes the string

    c5bcc3bc73b5ef45e91d2d7c70b64f196fac08eee4e4acf6e6571ebe
No matter what Go is allocating under the hood, it has to be feeding exactly that many bytes to the SHA3 algorithm.

Re: SHA-3 Buffer Overflow

#32

Post by tptacek »

jbirer wrote: post #28
tptacek wrote: post #20

Earlier quoted context omitted.

You're responding to a post that links to the Keccak designers saying that the padding change isn't nefarious. You have to be able to do better than "NIST bad" in comments on threads like these.

Appeal to Authority is not a very good argument either.

It is in this case. Absent any argument other than "I don't trust NIST", the designers of Keccak can refute your comment by simply saying "no, this is fine", and that's what they did.

Re: SHA-3 Buffer Overflow

#33

Post by tptacek »

aliqot wrote: post #26
tptacek wrote: post #20

Earlier quoted context omitted.

You're responding to a post that links to the Keccak designers saying that the padding change isn't nefarious. You have to be able to do better than "NIST bad" in comments on threads like these.

From Daniel J. Bernstein: 2022.08.05: NSA, NIST, and post-quantum cryptography: Announcing my second lawsuit against the U.S. government. : https://blog.cr.yp.to/20220805-nsa.html https://twitter.com/hashbreaker/status/1555625577989541888

... is a FOIA lawsuit about the recent PQ contest they refereed that has literally nothing to do with SHA3.

Re: SHA-3 Buffer Overflow

#34

Post by kelnos »

jbirer wrote: post #28
tptacek wrote: post #20

Earlier quoted context omitted.

You're responding to a post that links to the Keccak designers saying that the padding change isn't nefarious. You have to be able to do better than "NIST bad" in comments on threads like these.

Appeal to Authority is not a very good argument either.

That's a bit of a stretch. I think the person who initially designed something signing off on a modification of that something is pretty compelling.

Re: SHA-3 Buffer Overflow

#35

Post by derefr »

tptacek wrote: post #31
derefr wrote: post #27

Earlier quoted context omitted.

This is one of those cases where I'm actually more concerned that it doesn't crash. It's like seeing clearly-syntactically-invalid code that somehow compiles anyway — you wonder what semantics the compiler could have possibly ascribed to it. Presumably this isn't not-crashing just because the developers of the Golang stdlib somehow found+fixed this bug back in 2015 when this assembler file was baked. The error is in…

No, it causes the string c5bcc3bc73b5ef45e91d2d7c70b64f196fac08eee4e4acf6e6571ebe No matter what Go is allocating under the hood, it has to be feeding exactly that many bytes to the SHA3 algorithm.

> it has to be feeding exactly that many bytes to the SHA3 algorithm

Well, yes, but this is supposed to be a buffer overflow — the algorithm itself is reading and/or writing past the end of the buffer it's been handed. My hypothesis was that Golang is allocating in such a way that reading/writing a single byte past the received slice bounds won't result in a protection fault in the way you'd expect if the allocation were exact.

Re: SHA-3 Buffer Overflow

#36

Post by retrocat »

tptacek wrote: post #23
derefr wrote: post #17

Earlier quoted context omitted.

The version in the Golang stdlib defaults to a pure-go implementation... unless you're compiling for amd64, in which case you get an assembler variant apparently directly derived from the XKCP package ( https://github.com/golang/crypto/blob/master/sha3/keccakf_am... ). Slightly concerning news for the (mostly-Golang-based) Ethereum ecosystem, which relies on SHA3-256 for pretty much everything...

Easy enough to test, right? func main() { h := sha3.New224() buf := make([]byte, 4294967295) h.Write(buf) sum := h.Sum(nil) fmt.Printf("%x\n", sum) } Doesn't crash on my amd64 dev machine. Later I could have just looked at the code, too: the assembly you've linked to is just the Keccak permutation, not the entire Go hash; the buffer management is done in Go, not in assembly.

Are you sure this is the same code that triggers the vulnerability? Is the double `update` in the post itself unimportant (i.e., first updating with a 1-byte string, then the 4,294,967,295-byte string)? As I don't think this does that.

Re: SHA-3 Buffer Overflow

#37

Post by [deleted] »

galangalalgol wrote: post #24
lmm wrote: post #21

Earlier quoted context omitted.

> So since no one has mentioned it yet, you could rewrite this stuff in rust as a poor man's substitute. It will catch some of the aame things, but ultimately there is no substitute for test coverage with sanitizers. That's backwards. You'll catch more cases with a Rust-style type system that naturally checks everything, than with sanitisers that can only check the paths that get executed in tests.

Rust isn't perfect. UB is a bug in rust, but it occasionally has bugs. Ideally you'd do rust and asan with good unit tests. And yes, if you only picked one, it should be rust, but don't just pick one. And just because you are using rust is no excuse to skip static analysis like prusti, coverage with gcov, llvm, or tarpaulin, and certainly not unit tests.

[deleted]

Re: SHA-3 Buffer Overflow

#39

Post by tptacek »

retrocat wrote: post #36
tptacek wrote: post #23

Earlier quoted context omitted.

Easy enough to test, right? func main() { h := sha3.New224() buf := make([]byte, 4294967295) h.Write(buf) sum := h.Sum(nil) fmt.Printf("%x\n", sum) } Doesn't crash on my amd64 dev machine. Later I could have just looked at the code, too: the assembly you've linked to is just the Keccak permutation, not the entire Go hash; the buffer management is done in Go, not in assembly.

Are you sure this is the same code that triggers the vulnerability? Is the double `update` in the post itself unimportant (i.e., first updating with a 1-byte string, then the 4,294,967,295-byte string)? As I don't think this does that.

Let's try! Gimme a sec. And: to be clear: I make no representation that this code proves anything, and people should of course take a hard look at the SHA3 amd64 assembly code. I'm just having a message board discussion.

One sec and I'll get you an answer to your thing about the extra write.

Re: SHA-3 Buffer Overflow

#40

Post by halffullbrain »

derefr wrote: post #35
tptacek wrote: post #31

Earlier quoted context omitted.

No, it causes the string c5bcc3bc73b5ef45e91d2d7c70b64f196fac08eee4e4acf6e6571ebe No matter what Go is allocating under the hood, it has to be feeding exactly that many bytes to the SHA3 algorithm.

> it has to be feeding exactly that many bytes to the SHA3 algorithm Well, yes, but this is supposed to be a buffer overflow — the algorithm itself is reading and/or writing past the end of the buffer it's been handed. My hypothesis was that Golang is allocating in such a way that reading/writing a single byte past the received slice bounds won't result in a protection fault in the way you'd expect if the allocation…

The buffer overflow is in the C version of the algorithm (and likely related to loop condition checks idiomatic to C's for-loops). The Go version is a fresh implementation, not a wrapping of the C version. I'm no Go programmer, but if I'm not mistaken, the Go implementation just eats little slices of the input buffer until no more buffer is left, leaving all the overflow-danger to the Go array implementation:

https://github.com/golang/crypto/blob/642fcc37f5043eadb2509c...

Possibly not as fast as C, but easier to reason about, I'd say.

Post reply on HN