Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A webshell and a normal file that have the same MD5

github.com

Post reply on HN
Print view View original

31–40 of 49 posts

Re: A webshell and a normal file that have the same MD5

#31

Post by adrian_b »

sltkr wrote: post #28
gruez wrote: post #26

Earlier quoted context omitted.

I only see new CPUs benchmarked, maybe that's because newer CPUs have SHA acceleration extensions? I'd expect SHA256 to be more complex and therefore be more computationally expensive.

Yes, SHA256 is faster than MD5 only if you have hardware accelleration. But SHA256 itself is pretty slow compared to the state of the art. For example, BLAKE3 is just as secure as SHA256 but an order of magnitude faster. Try this on your own system: $ head -c 1000000000 /dev/urandom > random-1gb $ time md5sum random-1gb ef72a3616aad5117ddf40a7d5f5d0162 random-1gb real 0m2.428s user 0m2.192s sys 0m0.202s $ time sha256…

Unlike SHA-256, BLAKE3 can be evaluated in parallel, so the speedup factor over SHA-256 depends on the number of available CPU cores.

While BLAKE3 can be many times faster than SHA-256, by consuming many times more power, the amount of work for computing a hash differs much less between the 2 hashes than the execution time on a multi-core CPU.

The speed difference quoted by you for a single thread is caused by your Skylake-based CPU, which does not have the SHA hardware instructions.

Moreover, even the programs that claim to use the SHA hardware instructions may have a speed several times lower than allowed by the hardware, because the more recent CPUs, e.g. from the last 4 years, have wider SHA instructions than the older CPUs, but the programs must have been compiled to support such CPUs, e.g. Zen 3 and newer or Alder Lake and newer.

Re: A webshell and a normal file that have the same MD5

#32

Post by bawolff »

lisper wrote: post #10
dsab wrote: post #4

It's a pity that there is no description of what it is supposed to be used for.

If you don't know, then you aren't the target audience. But there are two applications: the first is breaking in to a system under some very obscure set of circumstances that you are very unlikely to encounter in the real world. The second is to bump up your karma on HN.

> If you don't know, then you aren't the target audience.

If you do know, then you also know md5 being broken is really really old news.

Seriously. Cryptographers have been warning that md5 seems weak since 1996. There are probably people reading this thread who weren't even alive yet. (It got totally broken in 2004 but the warning signs were way earlier).

Re: A webshell and a normal file that have the same MD5

#33

Post by jgalt212 »

IshKebab wrote: post #17
chipsrafferty wrote: post #7

Earlier quoted context omitted.

Yes, but you'd need a situation where: 1. You can upload scripts that get scanned for malicious code 2. These scripts can be executed once deemed "safe" 3. The server is using MD5 hashes to determine if you uploaded the same file or if it should re-scan it 3. Is where the issue is. It should probably always re-scan it and it definitely should not be using MD5.

There's no need to rescan. You just need to use a secure hash.

Secure for now, rather. A solid game plan would be to have your code base set up to easily swap in a new hashing method when called for. I believe Django automatically promotes passwords stored with insecure hashes to secure ones the next time a user logs in.

Re: A webshell and a normal file that have the same MD5

#34

Post by ramses0 »

bawolff wrote: post #32
lisper wrote: post #10

Earlier quoted context omitted.

If you don't know, then you aren't the target audience. But there are two applications: the first is breaking in to a system under some very obscure set of circumstances that you are very unlikely to encounter in the real world. The second is to bump up your karma on HN.

> If you don't know, then you aren't the target audience. If you do know, then you also know md5 being broken is really really old news. Seriously. Cryptographers have been warning that md5 seems weak since 1996. There are probably people reading this thread who weren't even alive yet. (It got totally broken in 2004 but the warning signs were way earlier).

Someone with more karma motivation could post this as a top level story, but Plex offers to validate their Debian public key via MD5: https://support.plex.tv/articles/235974187-enable-repository...

Such security! Much wow!

Re: A webshell and a normal file that have the same MD5

#35

Post by amelius »

adrian_b wrote: post #31
sltkr wrote: post #28

Earlier quoted context omitted.

Yes, SHA256 is faster than MD5 only if you have hardware accelleration. But SHA256 itself is pretty slow compared to the state of the art. For example, BLAKE3 is just as secure as SHA256 but an order of magnitude faster. Try this on your own system: $ head -c 1000000000 /dev/urandom > random-1gb $ time md5sum random-1gb ef72a3616aad5117ddf40a7d5f5d0162 random-1gb real 0m2.428s user 0m2.192s sys 0m0.202s $ time sha256…

Unlike SHA-256, BLAKE3 can be evaluated in parallel, so the speedup factor over SHA-256 depends on the number of available CPU cores. While BLAKE3 can be many times faster than SHA-256, by consuming many times more power, the amount of work for computing a hash differs much less between the 2 hashes than the execution time on a multi-core CPU. The speed difference quoted by you for a single thread is caused by your S…

This makes me wonder how much security suffers if you split a file in N smaller files, compute a hash for each of them, then hash the concatenation of the hashes.

Re: A webshell and a normal file that have the same MD5

#36

Post by Blahagun »

sim7c00 wrote: post #21

honestly, normal.php is not a valid php file. i do understand that it might bypass some checks if say normal.php was somehow flagged as a valid / benign file but in all honesty that would be really bad sec product u wanna swap with something that more intelligently classifies files... additionally, most products these days also use sha1, sha2 and sometimes things like ssdeep to have multiple hash variants to check. t…

normal.php is a perfectly valid php file. Sure, it doesn't contain php code but that doesn't make it invalid php file. If it did have <?php somewhere and if the following wasn't a syntactically valid PHP code, then you could say it's not a valid php file.

Re: A webshell and a normal file that have the same MD5

#37

Post by slow_typist »

h33t-l4x0r wrote: post #24
slow_typist wrote: post #20

Earlier quoted context omitted.

Humans have to put the so called php-file on the server intentionally for any subsequent attack to work. But it is a binary file.

I imagine it's supposed to get onto the server by an exploited vulnerable image upload plugin

Maybe I don’t understand the scenario fully, but under your assumption there is no need to inject the malicious webshell later.

Re: A webshell and a normal file that have the same MD5

#38

Post by adrian_b »

amelius wrote: post #35
adrian_b wrote: post #31

Earlier quoted context omitted.

Unlike SHA-256, BLAKE3 can be evaluated in parallel, so the speedup factor over SHA-256 depends on the number of available CPU cores. While BLAKE3 can be many times faster than SHA-256, by consuming many times more power, the amount of work for computing a hash differs much less between the 2 hashes than the execution time on a multi-core CPU. The speed difference quoted by you for a single thread is caused by your S…

This makes me wonder how much security suffers if you split a file in N smaller files, compute a hash for each of them, then hash the concatenation of the hashes.

BLAKE3 and other parallelizable hashes do exactly this, but using a somewhat more complex algorithm, which ensures that the result is a secure hash.

Such an algorithm has been first published by Ralph Merkle, in 1979, but it has been improved later:

https://en.wikipedia.org/wiki/Merkle_tree

For security, it is necessary to use different hash functions at different levels in the hash tree, but this is trivially achieved by using the same hash function, but also hashing some extra distinguishing data besides the hashes from the previous level.

Re: A webshell and a normal file that have the same MD5

#39

Post by alkonaut »

lisper wrote: post #10
dsab wrote: post #4

It's a pity that there is no description of what it is supposed to be used for.

If you don't know, then you aren't the target audience. But there are two applications: the first is breaking in to a system under some very obscure set of circumstances that you are very unlikely to encounter in the real world. The second is to bump up your karma on HN.

> system under some very obscure set of circumstances that you are very unlikely to encounter in the real world.

Is there any way to use HN karma? Like, can I sell my account on some shady exchange like people sell big twitter accounts? And if I can, what's the going rate for internet points these days? Asking for an unscrupulous friend.

Re: A webshell and a normal file that have the same MD5

#40

Post by oconnor663 »

amelius wrote: post #35
adrian_b wrote: post #31

Earlier quoted context omitted.

Unlike SHA-256, BLAKE3 can be evaluated in parallel, so the speedup factor over SHA-256 depends on the number of available CPU cores. While BLAKE3 can be many times faster than SHA-256, by consuming many times more power, the amount of work for computing a hash differs much less between the 2 hashes than the execution time on a multi-core CPU. The speed difference quoted by you for a single thread is caused by your S…

This makes me wonder how much security suffers if you split a file in N smaller files, compute a hash for each of them, then hash the concatenation of the hashes.

It's "easy" to do it right but also very common to do it wrong: https://jacko.io/tree_hashing.html
Post reply on HN