Making Python's __init__ method magical • Hacker News Board

Re: Making Python's init method magical

#11

Post by stormbrew » Fri, Jan 03, 2014, 1:05 PM UTC

radiowave wrote: post #10

stormbrew wrote: post #7

Earlier quoted context omitted.

I do. __new__ is effectively an allocator, or an interface to it. It allocates an unconstructed object. Then the constructor, __init__ is called on that. Edited to add: Reading around, it appears that smalltalk has no separation between allocation and construction, having only the equivalent to `operator new`/`Class#new`/`obj.__new`, with initialization also happening in overloads of that. So some confusion may stem…

In Smalltalk, 'new' is a class method which allocates a new instance, but which then has no special access to the internals of that newly created instance. An instance method, 'initialize' is then almost always used to set the initial state (by convention - this is not an actual language feature). Most commonly, initialize is called by new, but this again is just a convention, and a fairly recent one.

Interesting. Yes, it is very similar in both ruby and python, except that the convention is stronger by virtue of being the default behaviour of #new or .__new__ respectively. The placement of the methods (new on the class object and init[ialize] on the instance) is identical.

I really should do something in smalltalk. I only know details about it from reading about it, really, I've never had an opportunity to use it directly, though I go out of my way to understand its influences on more contemporary languages.

Re: Making Python's init method magical

#12

Post by stormbrew » Fri, Jan 03, 2014, 1:13 PM UTC

reuven wrote: post #9

stormbrew wrote: post #7

Earlier quoted context omitted.

I do. __new__ is effectively an allocator, or an interface to it. It allocates an unconstructed object. Then the constructor, __init__ is called on that. Edited to add: Reading around, it appears that smalltalk has no separation between allocation and construction, having only the equivalent to `operator new`/`Class#new`/`obj.__new`, with initialization also happening in overloads of that. So some confusion may stem…

I'll have to check my terminology to understand this better. But I can assure you that most of the programmers who take my Python classes (who tend to come from Java, C#, and C++) are rather surprised that the constructor modifies an existing object, and doesn't actually create the object. That said, I'm totally willing to believe that neither they nor I understood the difference between an "allocator" and a "constru…

Well, they have a poor understanding[1] of the sequence of events in C++ if they think that the constructor allocates the object there (I won't speak too strongly to Java or C#, since I don't work in them if I don't have to, but I believe that they both work in a very similar way). This would in fact be impossible, since many constructors are usually called for any given object (and all constructors in the object's graph are always called, unlike the more traditional OO languages where the upcall is usually explicit). Also because where an object is allocated is a decision of the caller. It can be allocated with new on the heap, on the stack with a simple declaration, or at an arbitrary memory location with placement new.

In C++ the heap object initialization goes like this:

    Obj *obj = new Obj;

This calls either `Obj::operator new` or the globally scoped `::operator new`, which somehow obtains an area of memory at least sizeof(obj) bytes long and returns it. After that calls to the object's constructors are generated and called.

`operator new` is an allocator and the constructor initializes the raw bytes returned from it.

You can actually do the process explicitly in its entirety like so using placement new, which is a special form that calls the constructors on an arbitrary location:

    Obj *obj = ::operator new(sizeof(Obj);
    new(obj) Obj;

You can't call constructors directly (except with a C++0x feature that lets you call them from a same-level constructor) because of the issue of how they're ordered based on the object graph. They have a precondition of their parent class having been fully constructed.

[1] Note: I don't really mean this as a judgement, understanding of C++ internals are pretty poor in a lot of programmers and for pretty good reasons a lot of the time.

Re: Making Python's init method magical

#13

Post by lukashed » Fri, Jan 03, 2014, 1:19 PM UTC

Here's another magical version that uses a class decorator:

  class Autoinit(object):
      def __call__(self, cls):
          orig_init = cls.__init__
          arg_count = orig_init.func_code.co_argcount
          arg_vars = orig_init.func_code.co_varnames[1:1 + arg_count]
  
          class Wrapped(cls):
              def __init__(self, *args, **kwargs):
                  for pos, var in enumerate(arg_vars[:len(args)]):
                      setattr(self, var, args[pos])
                  for var in kwargs:
                      setattr(self, var, kwargs[var])
                  return orig_init(self, *args, **kwargs)
  
          return Wrapped

  
  if __name__ == '__main__':
      @Autoinit()
      class Test(object):
          def __init__(self, a, b):
              pass
  
      t = Test(1, b=2)
      print t.a, t.b

[Disclaimer: This is my first class decorator. Please correct me if I'm wrong.]

Re: Making Python's init method magical

#14

Post by stormbrew » Fri, Jan 03, 2014, 1:20 PM UTC

reuven wrote: post #8

stormbrew wrote: post #6

And here's a more magical version that wraps it with a decorator. If you're gonna do this I kind of like this way better, but I'm pretty sure your average python programmer finds it obscene: from pprint import pprint def default_init(init_func): arg_count = init_func.func_code.co_argcount arg_vars = init_func.func_code.co_varnames[1:1+arg_count] def wrap_init(self, *args, **kwargs): idx = 0 for arg in arg_vars: if ar…

Very nice; it does seems far more reasonable to implement this as a decorator than as an explicit function call __init__.

One thing I like a lot better about it in particular is that it doesn't reach into the stack frame, which strikes me as a very messy grab at internals people should never ever touch. The arity and name information from the compiled function object, however, has legitimate use in tooling and I suspect a lot of decorators do this sort of thing.

It's an interesting balance between one being more magical in its use of the system (inspecting live stack frames) and the other being more magical in appearance (having a function with just pass in it actually do something).

Re: Making Python's init method magical

#15

Post by smokey42 » Fri, Jan 03, 2014, 1:26 PM UTC

I remember seeing something like this in Matellis' Python Cookbook.

Re: Making Python's init method magical

#16

Post by mattdeboard » Fri, Jan 03, 2014, 1:37 PM UTC

Oh boy, so I started using Python first. I took "reflection everywhere" for granted so hard without even knowing it. I learned Clojure next, and while it had its rough points I never thought, "Man this is really hard!"

Then, I learned some Scala & Java. That's when I realized how for granted I took this thing called "reflection". Especially when it came to ORMs. Holy mackerel. I got over it but it is definitely a huge thing -- for better and worse -- that is so easy in Python that really spoils beginners.

Re: Making Python's init method magical

#17

Post by reuven » Fri, Jan 03, 2014, 2:03 PM UTC

stormbrew wrote: post #11

radiowave wrote: post #10

Earlier quoted context omitted.

In Smalltalk, 'new' is a class method which allocates a new instance, but which then has no special access to the internals of that newly created instance. An instance method, 'initialize' is then almost always used to set the initial state (by convention - this is not an actual language feature). Most commonly, initialize is called by new, but this again is just a convention, and a fairly recent one.

Interesting. Yes, it is very similar in both ruby and python, except that the convention is stronger by virtue of being the default behaviour of #new or .__new__ respectively. The placement of the methods (new on the class object and init[ialize] on the instance) is identical. I really should do something in smalltalk. I only know details about it from reading about it, really, I've never had an opportunity to use it…

My very limited experience working with the Smalltalk language and environment was quite negative, although I hear people sing its praises, so perhaps I just didn't know what I was doing.

That said, it seems to me that many Ruby developers (and to a more limited degree, Python developers) are learning Smalltalk nowadays, for the same reason as English speakers learn Latin: to understand the origins of the language, and thus get a deeper understanding of how it works.

Re: Making Python's init method magical

#18

Post by n0on3 » Fri, Jan 03, 2014, 2:23 PM UTC

stormbrew wrote: post #6

And here's a more magical version that wraps it with a decorator. If you're gonna do this I kind of like this way better, but I'm pretty sure your average python programmer finds it obscene: from pprint import pprint def default_init(init_func): arg_count = init_func.func_code.co_argcount arg_vars = init_func.func_code.co_varnames[1:1+arg_count] def wrap_init(self, *args, **kwargs): idx = 0 for arg in arg_vars: if ar…

nice but it doesn't work if not all kwargs are passed..

Re: Making Python's init method magical

#19

Post by radiowave » Fri, Jan 03, 2014, 2:28 PM UTC

reuven wrote: post #17

stormbrew wrote: post #11

Earlier quoted context omitted.

Interesting. Yes, it is very similar in both ruby and python, except that the convention is stronger by virtue of being the default behaviour of #new or .__new__ respectively. The placement of the methods (new on the class object and init[ialize] on the instance) is identical. I really should do something in smalltalk. I only know details about it from reading about it, really, I've never had an opportunity to use it…

My very limited experience working with the Smalltalk language and environment was quite negative, although I hear people sing its praises, so perhaps I just didn't know what I was doing. That said, it seems to me that many Ruby developers (and to a more limited degree, Python developers) are learning Smalltalk nowadays, for the same reason as English speakers learn Latin: to understand the origins of the language, a…

Yes, and while I'm generally of the view that Smalltalk has lessons to teach, of course they aren't all good ones - in particular I sometimes feel that the way construction and initialization is done in Smalltalk has tended to incentivise having getters and setters for every internal variable, and has perhaps contributed to the rather brittle style of program construction that is decried as part of "everything that's wrong with OOP".

Re: Making Python's init method magical

#20

Post by lmm » Fri, Jan 03, 2014, 2:29 PM UTC

This is pretty cool (reminds me of the scala approach to constructors). But please listen to the author's caveats, and never do something like this in a production system; there are few things worse than magic when you're debugging.