Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Be Careful with Python's New-Style String Format

lucumr.pocoo.org

Post reply on HN
Print view View original

131–140 of 155 posts

Re: Be Careful with Python's New-Style String Format

#131

Post by thinkpad20 »

jlarocco wrote: post #119
coldtea wrote: post #4

Err, why would you allow for the user to enter arbitrary format strings in the first place? Might as well write "be careful about eval of arbitrary user provided strings".

Yeah, that was my first thought, too, and it's not very fair to blame Python's new style string format. Treating user input as a format string has always been a bad idea in every language and library that use them.

Not necessarily true. Type systems can ensure safety without too much effort. For example in Haskell, you might have some sort of format function with the type signature

    -- Given appropriate Error and Context types 
    format :: Context -> Text -> Either Error Text
Which would be guaranteed to be safe (there is no way this function can execute arbitrary code or read from your file system, short of egregious abuses of unsafePerformIO in its implementation). Note that the signature I wrote above takes an explicit context (presumably some sort of mapping of available variables), which removes the need for supporting arbitrary attribute access in the first place, and is more explicit to boot.

There is no reason other languages or libraries could not implement a function with the same degree of safety, although their type systems might not be able to guarantee its safety. In fact I would wager that such pitfalls are only likely to be found in dynamic languages, which support runtime eval (which is, after all, the root of the described problem).

Re: Be Careful with Python's New-Style String Format

#132

Post by yawaramin »

tedunangst wrote: post #22
coldtea wrote: post #18

Earlier quoted context omitted.

Shouldn't that be based not on the built-in string capabilities, but on a template engine, that must also take care for much more narrow cases of abuse (e.g. SQL injection, or access to only of a restricted number of helper functions within the template)?

Many templates allow accessing fields. There's no risk of SQL injection, you're only echoing the output back to the user.

Didn't you just point out that some of those fields could be sensitive, like password info?

Re: Be Careful with Python's New-Style String Format

#133

Post by tzury »

If within the context that parse and execute user input, sensitive data is available, then what this has to do with New-Style String Format? I mean,

    {event.__init__.__globals__[CONFIG][SECRET_KEY]}
Shall not be available to any function call, whatsoever!

Re: Be Careful with Python's New-Style String Format

#134

Post by dozzie »

Franciscouzo wrote: post #61
bubblesocks wrote: post #54

Earlier quoted context omitted.

The nice thing about the new format is that you can reverse the order of the variables inside the string, which is nice when you're localizing strings into languages with a reversed sentence structures, like Japanese. I haven't done a ton of Python, but the old style won't let you do that, will it?

Reversing the order is not possible, but you can use a dict instead: >>> "%(foo)s" % {"foo": "bar"} 'bar'

Funny. I thought Python implemented all the quirks of the original printf() formatting, including using arguments at specific positions. Apparently it didn't (and arguably it's a good thing).

  $ printf '%2$s %2$s %1$s\n' aaa bbb
  bbb bbb aaa

Re: Be Careful with Python's New-Style String Format

#135

Post by shakna »

kazinator wrote: post #118
shakna wrote: post #79

Earlier quoted context omitted.

If you're LISP has first class environments, then this becomes much easier, and safer. (eval (read (get-some-user-string)) (let [ (post-title post-title) (post-date post-date) (post-author post-author) (fmt string-append) ] (null-environment))) eval is only executed in the context of some environment, and the only things it has access to are some strings, and a string concatenation procedure. It doesn't have access t…

My Lisp has search through a first-class environnment which falls back on a dynamic/global environment. A way could be provided to suppress the fallback, like a special environment node indicating "do not search for bindings past me, stop here". If we have a completely empty environment, then we cannot eval the (sys:quasi ...) form itself; the sys:quasi symbol itself has no binding. A nice idea might be to have a fil…

You don't want the sys:quasi form to evaluate: It can be used as an escape hatch. If you're binding to the null-environment, you can provide everything a user can use.

In my above example, a user can do things like:

(fmt post.title post.date)

(fmt post.title "@" post.date)

(fmt (fmt post.title "@" post.date) "by" post.author)

But they can't escape the sandbox and do anything else.

To make a list, they need access to list, or quote, otherwise they just have number and strings, and anything else you give them.

If you give them anything more than a null-environment, you begin to lose the safety of a safe evaluation, which might be something you want to do yourself, but not something that you want a user to have access to, or they'll find an escape hatch, like sys:quasi, and gain access to areas they shouldn't.

Which is as bad as any SQL injection, if not worse.

So, a safe eval, can be created, without the need for any further analysis, if eval is handed an environment, and only uses it.

If the eval then searches the global environment, then the eval is unsafe, and you should never hand it user input, unless you have first ensured it's safe... Which defeats all the niceness provided by first class environments.

Eval is only not evil, if the programmer controls all the bindings to eval's environment.

REPL's usually use an unsafe eval and environment, and that makes sense, as you want to be able to do anything you want in code.

But a templating language doesn't need, and shouldn't use, all the unsafeness of a Turing Complete language, that might have access to your system.

Re: Be Careful with Python's New-Style String Format

#137

Post by daveguy »

tedmiston wrote: post #95
geofft wrote: post #82

Earlier quoted context omitted.

> It should be understood not to pass user data directly to internal code without sanitizing it. But how do you sanitize it? If you hadn't read this article and were told to sanitize a format string, how would you implement that? Unless you know exactly what the problem is, you're basically reduced to writing your own string formatter. (Though see 'anderskaseorg's example, it's a lot less code if you do.)

I think asking how to sanitize it is really asking the wrong question. We shouldn't sanitize a format string from a user in the first place. We should get input and generate a format string rather than allow one to be supplied "raw". For example, using re.sub as another commenter pointed out. An even more Pythonic solution would be to use the built-in string.Template [1] which at a glance appears to be safe against t…

Since 2.4:

https://docs.python.org/2/library/string.html#template-strin...

Re: Be Careful with Python's New-Style String Format

#139

Post by azernik »

coldtea wrote: post #16
Animats wrote: post #11

Earlier quoted context omitted.

Internationalization, usually. Word order differs between languages, and this kind of format allows reordering the inserted values.

Yes, but that's meant for trusted translators of the UI, nor arbitrary users, right?

In practice, translators are often externally contracted; the amount of work required to translate any given language is not large enough to make these full-time positions. So only trusted so far.

Re: Be Careful with Python's New-Style String Format

#140

Post by smitherfield »

anamoulous wrote: post #84
goatlover wrote: post #53

Earlier quoted context omitted.

Like PHP and JS's template literals, it has full on string interpolation.

Arguably Ruby is worse because you can execute instance methods in interpolation, but I still don't see why that's germane to this discussion other than to pick a fight about languages.

Ruby string interpolation can only happen in literal source code, not in user input.

  2.4.0 :001 > a = 5
   => 5 
  2.4.0 :002 > gets.chomp
  #{a}
   => "\#{a}"
Post reply on HN