Be Careful with Python's New-Style String Format • Hacker News Board

Re: Be Careful with Python's New-Style String Format

#61

Post by Franciscouzo » Sat, Dec 31, 2016, 10:08 PM UTC

bubblesocks wrote: post #54

libeclipse wrote: post #51

I tend to use the old style ("%s" % "lel"). Just wondering, does this affect that too?

The nice thing about the new format is that you can reverse the order of the variables inside the string, which is nice when you're localizing strings into languages with a reversed sentence structures, like Japanese. I haven't done a ton of Python, but the old style won't let you do that, will it?

Reversing the order is not possible, but you can use a dict instead:

    >>> "%(foo)s" % {"foo": "bar"}
    'bar'

Re: Be Careful with Python's New-Style String Format

#62

Post by peller » Sat, Dec 31, 2016, 10:12 PM UTC

foolfoolz wrote: post #57

bubblesocks wrote: post #52

Earlier quoted context omitted.

Isn't ingesting user input directly considered a bad idea all around though?

ingesting raw user input is good emitting raw user input is bad

Isn't SQL injection caused by ingesting raw user input though?

Seems to me you always have to be careful with user-supplied data.

Re: Be Careful with Python's New-Style String Format

#63

Post by mistaken » Sat, Dec 31, 2016, 10:15 PM UTC

foolfoolz wrote: post #57

bubblesocks wrote: post #52

Earlier quoted context omitted.

Isn't ingesting user input directly considered a bad idea all around though?

ingesting raw user input is good emitting raw user input is bad

Even if you don't emit the input you can run commands on the server. You can open up a reverse shell or deduce data from timing based side-channels. IMHO working with raw user data is bad; it should be sanitized/canonicalized before doing anything.

Re: Be Careful with Python's New-Style String Format

#64

Post by nitely » Sat, Dec 31, 2016, 10:16 PM UTC

bubblesocks wrote: post #54

libeclipse wrote: post #51

I tend to use the old style ("%s" % "lel"). Just wondering, does this affect that too?

The nice thing about the new format is that you can reverse the order of the variables inside the string, which is nice when you're localizing strings into languages with a reversed sentence structures, like Japanese. I haven't done a ton of Python, but the old style won't let you do that, will it?

Yes, it will, just use named placeholders: "%(some)s" % {'some': 'helllo'}

Re: Be Careful with Python's New-Style String Format

#65

Post by raquo » Sat, Dec 31, 2016, 10:18 PM UTC

bubblesocks wrote: post #52

tedunangst wrote: post #19

Maybe a more fully worked example is needed. You're making a blog hosting service as a service service. Bloggers have different ideas about what page titles should be. Post Title Blog Name: Post Title Blog Name - Post Title Post Title - Blog Name Blog Name ----embdash---- Post Title ~~~ xXx Post Title xXx ~~~ It's a little overwhelming to put every possibility in a dropdown, so you allow the user to specify a format…

Isn't ingesting user input directly considered a bad idea all around though?

Yes, but it's not obvious how to sanitize input in this case, or that it even needs sanitizing. Formatting a string sounds pretty innocuous.

Re: Be Careful with Python's New-Style String Format

#66

Post by gizmo686 » Sat, Dec 31, 2016, 10:26 PM UTC

detaro wrote: post #50

gizmo686 wrote: post #17

Earlier quoted context omitted.

Neither does python. The problem being discussed here is that the format string can access and attribute of the object passed in. From PEP3101 [0]: " Unlike some other programming languages, you cannot embed arbitrary expressions in format strings. This is by design - the types of expressions that you can use is deliberately limited. Only two operators are supported: the '.' (getattr) operator, and the '[]' (getitem)…

> don't normally have side effects in non-pathological code so, "be careful with it if you can't make sure code isn't non-pathological or friendly", what the article says.

The example is not pathalogical. The property that the PEP states is that the expression should not have side effects, which is the case.

Having pathological code in this context only elavates the severity from a pure data leak to partial code execution. Of course partial code execution in an unexpected context often leads to arbitrary code execution.

Re: Be Careful with Python's New-Style String Format

#67

Post by meowface » Sat, Dec 31, 2016, 10:31 PM UTC

peller wrote: post #62

foolfoolz wrote: post #57

Earlier quoted context omitted.

ingesting raw user input is good emitting raw user input is bad

Isn't SQL injection caused by ingesting raw user input though? Seems to me you always have to be careful with user-supplied data.

"Ingesting raw user input is good if you only use it to interface with other systems that provide a way to separate data from instructions or at least escape strings."

Re: Be Careful with Python's New-Style String Format

#68

Post by foolfoolz » Sat, Dec 31, 2016, 10:37 PM UTC

peller wrote: post #62

foolfoolz wrote: post #57

Earlier quoted context omitted.

ingesting raw user input is good emitting raw user input is bad

Isn't SQL injection caused by ingesting raw user input though? Seems to me you always have to be careful with user-supplied data.

sql injection is commonly caused by combining your query with its related data parameters in unsafe ways. you are emitting raw user input you received to another program, the database, it's your responsibility to give this to the DB safely.

you still have to be careful, and when you follow all the right best practices you can safely ingest raw user input.

I've worked at a company that escaped user input before inserting into the DB. it's a horrible nightmare I don't think anyone should have to experience.

Re: Be Careful with Python's New-Style String Format

#69

Post by foolfoolz » Sat, Dec 31, 2016, 10:40 PM UTC

mistaken wrote: post #63

foolfoolz wrote: post #57

Earlier quoted context omitted.

ingesting raw user input is good emitting raw user input is bad

Even if you don't emit the input you can run commands on the server. You can open up a reverse shell or deduce data from timing based side-channels. IMHO working with raw user data is bad; it should be sanitized/canonicalized before doing anything.

if you're running commands on the server I would consider your program as emitting output directly to another program. it's up to you to make sure you call it correctly and not emit raw user data

Re: Be Careful with Python's New-Style String Format

#70

Post by userbinator » Sat, Dec 31, 2016, 10:41 PM UTC

raquo wrote: post #65

bubblesocks wrote: post #52

Earlier quoted context omitted.

Isn't ingesting user input directly considered a bad idea all around though?

Yes, but it's not obvious how to sanitize input in this case, or that it even needs sanitizing. Formatting a string sounds pretty innocuous.

I think "it's not obvious how to sanitize input" is the main point here --- one advantage of the %-style format strings, they're easier to parse and escape.