Live data from Hacker News

Using attrs for everything in Python

glyph.twistedmatrix.com

91–100 of 105 posts

Re: Using attrs for everything in Python

#91

How does this compare to schematics? It avoids inheritance, whereas in schematics and similar projects you'd have to inherit from their `Model` class (and thus acquire methods you might not want).

Schematics looks a bit like a less elegant version of schema. It does seem more focused on validation than attrs, I agree.

Schema [1] does not appear to have any support for object initialization.

[1] https://pypi.python.org/pypi/schema/0.6.2

Re: Using attrs for everything in Python

#92
post #84

Earlier quoted context omitted.

I know how decorators work and you might be missing the point I was making by a fair mile The following don't mutate the mental map of how the function works, its "this function is still what I typed": "add logging" "add tracing" "register functions/objects for various reasons" The following act as a filtering role, the inside isn't "changed", the mental map becomes "This function will operate within this feature in…

Just a data point: MacroPy uses the decorator syntax for "case" classes that produce objects with a similar behavior to "attr.s" generated objects https://github.com/lihaoyi/macropy

A few weeks ago, someone posted an article where the author showed how they could "inline" functions by playing with the AST. That was done to increase performance for the case of very simple functions along the lines of:

    def mul_by_pi(x):
        return x * 3.14
(I'm sure there are better examples, but that was the idea). I've seen production code where methods and functions are used to provide bitmasking with a class-dependent mask, for example (where the mask is a const, not an attribute), which would benefit from inlining.

From what I've seen, it seems MacroPy could do that, but it's not a direct example anywhere, and it seems they haven't tested it. Has it been done before?

My other problem is that the client is extremely averse of adding any new dependency (which is understandable), but I might be able to convince them if I can show a significant speedup on inlined code. The previous post I mentioned isn't a viable alternative because only "toy code" (as the author said) was provided, and in my team we aren't knowledgable enough in AST handling to feel comfortable hacking it together, but a more mature project like MacroPy definitely looks viable.

Re: Using attrs for everything in Python

#93
post #69

This is wonderful and cute as hell and please PLEASE don't ever use it in production code. ;-)

Why not? It's aggressively tested and benchmarked to ensure its correct and its performance impact is as close to zero as possible. (It's pretty close.)

tl;dr: I don't like metaprogramming.

----

Both of those things are very good, even great, but they don't speak to the issue: It's [bad] magic.

In this case, it does at module-load-time what should have been done previously at build-time. (And yes, I know Python normally doesn't have build-time the way e.g. C does.)

I'm working on a large production codebase right now where several of the cowboy-coders here have added all sorts of crazy metaprogramming and it's a PITA.

Let me tell you a story. We had a class decorator that introspected the attributes of the class and added "constants" to the enclosing module providing named string values for each of the class's attributes. It lets us access dict versions of these objects (they're data models) like foo_as_dict[foo_module.SOME_ATTR_NAME_BUT_IN_CAPS] ...

Good luck finding foo_module.SOME_ATTR_NAME_BUT_IN_CAPS in foo_module.py though, because it's not there. (and notice that if the model field name changes ALL of your uses of the "constant" have to be refactored to the new name too, so it's not really as helpful as it might seem in the first place.)

And now every new developer has to ask, "Where are these coming from?" and we get to point out the magic extra-clever decorator. (Try to imagine the horrible wonderful "voodoo" that decorator encompasses... Imagine the innocent newbie encountering it.)

We have a "service builder" thing that takes a bunch of arguments and build a web service object in memory at module-load time. No source for any of that machinery. A bug in the service gives you a traceback running through a bunch of weird generic meta-code.

Did I mention the guy who originally wrote it bailed to a new job a month after he finished it? No one else knows how it works. We can read the code and reverse engineer it, of curse, but that's kind of B.S., no? A junior dev could debug the real service code if it existed, but the meta-code that generates the services is another challenge (that they shouldn't have and the company shouldn't have to pay them to overcome.)

We had unittest that read a JSON file and built a test suite in memory to run a bunch of other tests. They finally let me re-write that to a build-time script that scans the exact same JSON files and emit plain-vanilla unittest modules, that then can be run as normal, and the dev/user can look at the actual code of the test, rather than the meta-code of the test-builder.

The same problem applies to this attr package.

If you're trying to trace into or debug code that uses attr you've got to know attr at least enough to be able to follow what it's doing. It's an in-joke. ;-)

(P.S. I'm a big fan dude. Nice to interact with you. All respect.)

Re: Using attrs for everything in Python

#94

Earlier quoted context omitted.

It adds magic. "Explicit is better than implicit" ~ https://en.wikipedia.org/wiki/Zen_of_Python

I feel like a person should be able to back their opinion up (for a specific case, not in general), instead of just mindlessly quoting a line from a bible.

See comment in this post in reply to Glyph. (And don't be a jerk.)

Re: Using attrs for everything in Python

#95

Earlier quoted context omitted.

It adds magic. "Explicit is better than implicit" ~ https://en.wikipedia.org/wiki/Zen_of_Python

Then let's get rid of the with semantics too, because they add magic. Compare: with open("file.txt") as somefile: for line in somefile: print line To the more explicit and clear: somefile = open("file.txt") line = somefile.readline() while line: print line line = somefile.readline() somefile.close() But I'm still using some magic, I should be more explicit: def explicit_readline(fd): buff = [] char = fd.read(1) while…

Oh c'mon Zoidberg, that's a strawman argument. "with" is a Python keyword and the statement syntax is in the grammar. I would expect any working Python developer to know what it is, and even understand how to write a context manager.

...wait wat? It's in the Standard Library? No, I just checked and it's not. Too bad, if it was "blessed" by inclusion in the library I would totally withdraw my objection on that grounds.

You had me going for a second there. ;-)

You make my point for me though when you say that you'd have to read up on it when you first encountered it. That's exactly my point:

The trade off is between one application of attr to the code base to save N time/effort for the current developer one time, versus every bit of lost time/effort of future devs taken to understand the attr package.

It's cute, but it's an in-joke. Don't put in-jokes in code you want other people to use, otherwise they have to "get the joke" before they can work with it. That's what I'm on about.

Re: Using attrs for everything in Python

#96

Earlier quoted context omitted.

It adds magic. "Explicit is better than implicit" ~ https://en.wikipedia.org/wiki/Zen_of_Python

Then let's get rid of the with semantics too, because they add magic. Compare: with open("file.txt") as somefile: for line in somefile: print line To the more explicit and clear: somefile = open("file.txt") line = somefile.readline() while line: print line line = somefile.readline() somefile.close() But I'm still using some magic, I should be more explicit: def explicit_readline(fd): buff = [] char = fd.read(1) while…

>Then let's get rid of the with semantics too, because they add magic.

With just runs the enter and exit method of a context manager. Nothing magical about it.

>My point was the "Explicit is better than implicit" means "be clear of your intent". And if I see someone using the attr module (which comes with the standard library), and I'm not familiar with it, I'll read into it.

You don't have to read into 'with' to make an intelligent guess at what the code does. It's meaningful even if you've never heard of context managers.

Unlike this.

Re: Using attrs for everything in Python

#97
post #39

The problem with constructs like this one is that it looks nice at first glance, but adds a cognitive load to an ecosystem that already has about "20 ways to do it" for every single problem. Python is beginning to rival C++ in that respect.

Please don't tell the C++ community that! We really don't want to encourage them. To someone who uses Python every day, the extra cognitive load of `attr` is more than made up for by all the boilerplate it eliminates. But for beginners, behind-the-scenes magic is a real turn off. I think that making `attr` an optional library is a fair compromise, but perhaps it would be a good idea to add a note to the library remin…

>To someone who uses Python every day, the extra cognitive load of `attr` is more than made up for by all the boilerplate it eliminates.

That's funny because the first thought I had when reading this was "this does not eliminate nearly enough boilerplate to justify using that much magic".

>But for beginners, behind-the-scenes magic is a real turn off.

It's a turn off for beginners and experts but a turn on for intermediates.

Re: Using attrs for everything in Python

#98

Earlier quoted context omitted.

Then let's get rid of the with semantics too, because they add magic. Compare: with open("file.txt") as somefile: for line in somefile: print line To the more explicit and clear: somefile = open("file.txt") line = somefile.readline() while line: print line line = somefile.readline() somefile.close() But I'm still using some magic, I should be more explicit: def explicit_readline(fd): buff = [] char = fd.read(1) while…

Oh c'mon Zoidberg, that's a strawman argument. "with" is a Python keyword and the statement syntax is in the grammar. I would expect any working Python developer to know what it is, and even understand how to write a context manager. ...wait wat? It's in the Standard Library? No, I just checked and it's not. Too bad, if it was "blessed" by inclusion in the library I would totally withdraw my objection on that grounds…

Weird, in Windows (Python 2.7.10) I have it without having ever installed it. Even pip uninstall didn't get it (just to make sure it wasn't installed from a dependency) but I can still import it.

On my Linux machine (Python 2.7.6) I can't import it without a previous install from pip install (that's reasonable).

Can't track down much about why this happens because googling "python attr" brings results about hasattr, getattr, etc, but few about the attr module.

Re: Using attrs for everything in Python

#99
post #69

Earlier quoted context omitted.

Why not? It's aggressively tested and benchmarked to ensure its correct and its performance impact is as close to zero as possible. (It's pretty close.)

tl;dr: I don't like metaprogramming. ---- Both of those things are very good, even great, but they don't speak to the issue: It's [bad] magic. In this case, it does at module-load-time what should have been done previously at build-time. (And yes, I know Python normally doesn't have build-time the way e.g. C does.) I'm working on a large production codebase right now where several of the cowboy-coders here have added…

The same problem does not apply.

I understand that spooky action-at-a-distance magic can really ruin a codebase's maintainability. But what you've developed here is not a judicious appreciation of its danger and its power, but a blanket aversion to both its risks and its benefits.

An apt metaphor would be, let's say: toast. If you try to make some toast, but accidentally burn your house down, it's understandable that you might want to have your bread un-toasted for a while. But it doesn't make sense to switch your diet to be entirely raw as a result, especially if you eat a lot of chicken.

In python, metaprogramming is like fire. People who haven't seen it before are fascinated before it, and try to touch it. This always goes badly. But that doesn't mean it's useless or we shouldn't have it. There are many things it's good for, if it's carefully and thoughtfully applied.

attrs goes out of its way to avoid any behind-the-scenes effects. It even generates Python code, rather than using setattr(), so that if any error happens in the constructor, you can step through it normally in the debugger, and see it in the traceback. Its semantics are clear and direct. It doesn't have these problems.

Re: Using attrs for everything in Python

#100
post #69

Earlier quoted context omitted.

Why not? It's aggressively tested and benchmarked to ensure its correct and its performance impact is as close to zero as possible. (It's pretty close.)

tl;dr: I don't like metaprogramming. ---- Both of those things are very good, even great, but they don't speak to the issue: It's [bad] magic. In this case, it does at module-load-time what should have been done previously at build-time. (And yes, I know Python normally doesn't have build-time the way e.g. C does.) I'm working on a large production codebase right now where several of the cowboy-coders here have added…

And, thanks for the P.S. Always nice to see people enjoy my work :)
Post reply on HN