Be Careful with Python's New-Style String Format • Hacker News Board

Re: Be Careful with Python's New-Style String Format

#81

Post by antsar » Sat, Dec 31, 2016, 11:16 PM UTC

libeclipse wrote: post #51

I tend to use the old style ("%s" % "lel"). Just wondering, does this affect that too?

Attribute access doesn't seem to work with old-style formatting, so I would think it's safe.

    >>> class foo:
    ...     def __init__(self, **kwargs):
    ...             self.__dict__.update(kwargs)
    ...
    >>> my_foo = foo(secret="bazinga")
    >>>
    >>> "%(foo)s" % {'foo': my_foo}
    ''
    >>>
    >>> "%(foo.secret)s" % {'foo': my_foo}
    Traceback (most recent call last):
      File "", line 1, in 
    KeyError: 'foo.secret'

Re: Be Careful with Python's New-Style String Format

#82

Post by geofft » Sat, Dec 31, 2016, 11:23 PM UTC

yladiz wrote: post #76

So is the issue that it's a problem to let the format string be controlled arbitrarily? It's good to warn users around it because some may not know the dangers, but in general you don't trust user input, so I don't really see this as something you need to build something custom around, rather just be careful and follow good practices. It should be understood not to pass user data directly to internal code without san…

> It should be understood not to pass user data directly to internal code without sanitizing it.

But how do you sanitize it? If you hadn't read this article and were told to sanitize a format string, how would you implement that?

Unless you know exactly what the problem is, you're basically reduced to writing your own string formatter. (Though see 'anderskaseorg's example, it's a lot less code if you do.)

Re: Be Careful with Python's New-Style String Format

#83

Post by geofft » Sat, Dec 31, 2016, 11:25 PM UTC

gizmo686 wrote: post #66

detaro wrote: post #50

Earlier quoted context omitted.

> don't normally have side effects in non-pathological code so, "be careful with it if you can't make sure code isn't non-pathological or friendly", what the article says.

The example is not pathalogical. The property that the PEP states is that the expression should not have side effects, which is the case. Having pathological code in this context only elavates the severity from a pure data leak to partial code execution. Of course partial code execution in an unexpected context often leads to arbitrary code execution.

In many webapps, there's a very easy way to go from a data leak to arbitrary code execution. For instance, if you have signed cookies that are pickles of data, the cookie signing key lets you execute arbitrary code (because pickle deserialization can execute arbitrary code).

Re: Be Careful with Python's New-Style String Format

#84

Post by anamoulous » Sat, Dec 31, 2016, 11:26 PM UTC

goatlover wrote: post #53

anamoulous wrote: post #46

Earlier quoted context omitted.

What does Ruby have to do with any of this?

Like PHP and JS's template literals, it has full on string interpolation.

Arguably Ruby is worse because you can execute instance methods in interpolation, but I still don't see why that's germane to this discussion other than to pick a fight about languages.

Re: Be Careful with Python's New-Style String Format

#85

Post by progval » Sat, Dec 31, 2016, 11:30 PM UTC

Animats wrote: post #9

No, Rust does not have the ability to access any variable in the program via a format string. Rust has this: format!("{argument}", argument = "test"); // => "test" That's just named arguments to the format. Also, that's a macro; it's expanded at compile time. Python's approach is lame. It should have used something with a limited list of named arguments, or maybe a dict.

> It should have used something with a limited list of named arguments, or maybe a dict.

Python does have that feature, called Template Strings: https://docs.python.org/3/library/string.html#template-strin...

Re: Be Careful with Python's New-Style String Format

#86

Post by swift » Sat, Dec 31, 2016, 11:33 PM UTC

shakna wrote: post #79

kazinator wrote: post #36

Sane Lisp approach: provide easily analyzable target syntax. This is the TXR Lisp interactive listener of TXR 163. Use the :quit command or type Ctrl-D on empty line to exit. 1> (defvar foo 42) foo 2> `@(list foo) ... @foo` "42 ... 42" What is that backticked literal? Let's quote it: 3> '`@(list foo) ... @foo` `@(list foo) ... @foo` Hmm, prints back in same form. Probably syntactic sugar for a list; what is in the ca…

If you're LISP has first class environments, then this becomes much easier, and safer. (eval (read (get-some-user-string)) (let [ (post-title post-title) (post-date post-date) (post-author post-author) (fmt string-append) ] (null-environment))) eval is only executed in the context of some environment, and the only things it has access to are some strings, and a string concatenation procedure. It doesn't have access t…

Lua has this feature as well and it's fantastically useful for this kind of use-case, or more generally anywhere where you want to allow scripting.

Re: Be Careful with Python's New-Style String Format

#87

Post by shakna » Sat, Dec 31, 2016, 11:34 PM UTC

swift wrote: post #86

shakna wrote: post #79

Earlier quoted context omitted.

If you're LISP has first class environments, then this becomes much easier, and safer. (eval (read (get-some-user-string)) (let [ (post-title post-title) (post-date post-date) (post-author post-author) (fmt string-append) ] (null-environment))) eval is only executed in the context of some environment, and the only things it has access to are some strings, and a string concatenation procedure. It doesn't have access t…

Lua has this feature as well and it's fantastically useful for this kind of use-case, or more generally anywhere where you want to allow scripting.

The more I use Lua, the more it seems like the sanest of the scripting languages out there.

Re: Be Careful with Python's New-Style String Format

#88

Post by jessaustin » Sat, Dec 31, 2016, 11:37 PM UTC

nine_k wrote: post #73

icebraining wrote: post #24

Earlier quoted context omitted.

The author discovered this exploit in the template engine he wrote (Jinja2).

I'll consider switching to a different engine.

Perhaps one that hasn't had this exploit fixed yet?

Re: Be Careful with Python's New-Style String Format

#89

Post by tedunangst » Sat, Dec 31, 2016, 11:37 PM UTC

userbinator wrote: post #70

raquo wrote: post #65

Earlier quoted context omitted.

Yes, but it's not obvious how to sanitize input in this case, or that it even needs sanitizing. Formatting a string sounds pretty innocuous.

I think "it's not obvious how to sanitize input" is the main point here --- one advantage of the %-style format strings, they're easier to parse and escape.

But we don't want the string escaped. We want it interpreted.

Re: Be Careful with Python's New-Style String Format

#90

Post by tedunangst » Sat, Dec 31, 2016, 11:39 PM UTC

anderskaseorg wrote: post #39

The proposed idea of relying on undocumented internals and blacklisting attribute names to securely sandbox formatting strings is _really_ _dangerous_. Never do that in production code! Language expansions could render your sandbox unsafe at any time. You can write your own safe formatting engine in much less code. def safe_format(fmt, **args): return re.sub(r'\{([^{}]*)\}', lambda m: args.get(m.group(1)), fmt) safe_…

The first bell added is likely to be field access.