Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When str.lower() is a security vulnerability in Python

sethmlarson.dev

Post reply on HN
Print view View original

51–60 of 85 posts

Re: When str.lower() is a security vulnerability in Python

#51

Post by raverbashing »

tptacek wrote: post #4
echoangle wrote: post #2

> This is why calling str.lower() represents a difference in the implementation and the specification, and therefore a vulnerability: I wish there was some explanation how this is a vulnerability and not just a bug generating erroneous data. Vulnerability for me sounds like there’s a reasonable way to create an exploit from the bug, and I don’t see one here as someone who’s not very familiar with the topic.

It creates a parser differential; two different components of the system can treat the same string as different hostnames. Things that have trusted hostnames, or privileged/admin hostnames that are screened out, or SSRF filters all depend on accurately comparing presented hostnames. This is pretty situational, though, isn't it? You still have to be dealing with IDN names.

From what I understand the vulnerability is not on lower() but on using different Unicode rules version for it

That sounds to me like a better explanation

Re: When str.lower() is a security vulnerability in Python

#52

Post by zrm »

inigyou wrote: post #39
tialaramex wrote: post #13

Earlier quoted context omitted.

It is situational, but it very much seems like a thing you'd squirrel away and bring out when you find a system where the differential is helpful. DNS names are a thing where Sales is going to tell the Engineer that they can't issue the customers randomized ASCII names like abxuewrf.my-thing.example because real customers want to write our-brand-name.my-thing.example instead - even though you already know bad guys wi…

Solution: customer emails you to request a name

How does the customer service rep tell that a name with some Unicode gubbins is an attack rather than a customer from Juárez or 서울? Having a busy hand copy and paste the attacker-provided string into the system doesn't get you out of it.

Re: When str.lower() is a security vulnerability in Python

#53

Post by quietbritishjim »

jwilk wrote: post #45
ummonk wrote: post #28

> The fix was to create new exceptions so that str.lower() would behave as if it was using Unicode 3.2.0 for only particular function. So, we go through each Unicode codepoint and record when the behavior of str.lower() is different when comparing the Unicode version shipped with Python and Unicode 3.2.0 This sounds like a really hacky solution compared to implementing a separate frozen Unicode 3.2.0 lower.

The first sentence sounds as if they modified the implementemention of str.lower(). That would be bonkers, but that's not what they did. https://github.com/python/cpython/commit/7e109d084d55e7eb The important part is: # B.3 is mostly Python's .lower, except for a number # of special cases, e.g. considering canonical forms. +# To enforce Unicode 3.2.0 behavior of .lower instead of +# whatever Unicode version is includ…

That fragment doesn't mean much in isolation. You've just said that they didn't modify str.lower (because "that would be bonkers") but you've posted a fragment which, for all we know, is part of the str.lower implementation.

Re: When str.lower() is a security vulnerability in Python

#54

Post by notpushkin »

inigyou wrote: post #41
cwillu wrote: post #21

Earlier quoted context omitted.

Yes, but github.cc or whatever would have sufficed for that; there's a reason the string they chose to register for prominent public use contains “usercontent”

literally github.io though

And before that one, user content was hosted on *.github.com! Wild times.

Re: When str.lower() is a security vulnerability in Python

#55

Post by lozenge »

inigyou wrote: post #40
xp84 wrote: post #15

Earlier quoted context omitted.

That's why for that type of 'semi-white-label' thing, since the main risk is one of impersonating the platform owner (like the billing.my-thing.example) or possibly lending the credibility of "our" brand to some rando UGC, I always push for the most boring and generic second-level domain, like if it's the travel business, 'travel-systems dot us' or in edtech, mylearningplatform dot net... Then push all customers who…

That's actually to prevent cookie stealing.

Yes, involving the Public Suffix List. It's quite ridiculous you need to register your domain on a list of you want to offer subdomains.

Re: When str.lower() is a security vulnerability in Python

#57

Post by ghusbands »

So someone used lower() from an unspecified version of Unicode when the standard was very specific about which to use. And they say "There's also a database of Unicode 3.2.0 data available on every version of Python (unicodedata.ucd_3_2_0) specifically for the StringPrep and IDNA algorithms", so the right version is available.

And then the fix is to hardcode a bunch of special cases which again depend on exactly which version of Unicode is in use, and so will break again in the same way in future, rather than just using the right version?

Re: When str.lower() is a security vulnerability in Python

#59

Post by amelius »

echoangle wrote: post #2

> This is why calling str.lower() represents a difference in the implementation and the specification, and therefore a vulnerability: I wish there was some explanation how this is a vulnerability and not just a bug generating erroneous data. Vulnerability for me sounds like there’s a reasonable way to create an exploit from the bug, and I don’t see one here as someone who’s not very familiar with the topic.

Fingerprinting comes to mind.

Re: When str.lower() is a security vulnerability in Python

#60

Post by philipwhiuk »

If this is a vulnerability every bug in every API is a vulnerability.

This is very spacebar-heating.

It's not a vulnerability, it's a bug. A system that used this bug in a way that relied on it to perform a security task would have a vulnerability.

We need to stop seeing library functions that are not themselves security systems as having vulnerabilities.

Post reply on HN