|  | Late initialization using __getattribute__ | | |
| | | bukzor |  |
|  Posted: Wed Sep 03, 2008 8:55 pm Post subject: Late initialization using __getattribute__ |  |
| |  | |
I want to make a MixIn class that waits to initialize its super-
classes until an attribute of the object is accessed. Not generally
useful, but desirable in my case. I've written this, and it works, but
would like to take any suggestions you guys have. I've commented out
the "delattr" call because it throws an AttributeError (although I
don't know why).
class LateInitMixIn(object):
def __init__(self):
print "LateInit initialization"
self.inited = False
def __getattribute__(self, attr):
print "Doing __getattribute__"
getattr = lambda attr:object.__getattribute__(self, attr)
if not getattr("inited"):
super(LateInitMixIn, self).__init__()
setattr(self, "inited", True)
#delattr(self, "__getattribute__")
return getattr(attr)
class Base(object):
def __init__(self):
print "Base initialization"
self.base = True
class LateInit(LateInitMixIn, Base): pass
def main():
S = LateInit()
print S
print
print "Should do Base init after now"
print S.base
print S.base
if __name__=="__main__": main()
This gives the following output:
LateInit initialization
<__main__.LateInit object at 0x2a960c1c50>
Should do Base init after now
Doing __getattribute__
Base initialization
True
Doing __getattribute__
True |
| |
| | | Bruno Desthuilliers | |
| | Posted: Wed Sep 03, 2008 8:55 pm Post subject: Re: Late initialization using __getattribute__ | |
| | | |
bukzor a écrit :
| Quote: | I want to make a MixIn class that waits to initialize its super-
classes until an attribute of the object is accessed. Not generally
useful, but desirable in my case. I've written this, and it works, but
would like to take any suggestions you guys have.
|
You shouldn't mess with __getattribute__ unless you really know what
you're doing and are ok to suffer the constant performance hit you'll
get. Remember that __getattribute__ actually *is* the implementation of
attribute lookup rules, and is called on each and every attribute
lookup. Your below snippet would be much better using __getattr__ (which
is only called as a last resort).
| Quote: | I've commented out
the "delattr" call because it throws an AttributeError (although I
don't know why).
|
__getattribute__ belongs to the class, not to the instance. delattr()
only removes instance attributes. You'd have to remove __getattribute__
from the LateInitMixIn class object itself, but then it would break the
whole thing.
| Quote: |
class LateInitMixIn(object):
def __init__(self):
print "LateInit initialization"
self.inited = False
def __getattribute__(self, attr):
print "Doing __getattribute__"
getattr = lambda attr:object.__getattribute__(self, attr)
if not getattr("inited"):
super(LateInitMixIn, self).__init__()
setattr(self, "inited", True)
#delattr(self, "__getattribute__")
return getattr(attr)
|
Here's another possible implementation (which doesn't solve all
problems, cf below) using __getattr__:
class LateInitMixin(object):
def __init__(self):
print "not yet"
self.__initialized = False
def __getattr__(self, name):
if self.__initialized:
raise AttributeError(
"object %s has no attribute '%s'" % (type(self), name)
)
super(LateInitMixin, self).__init__()
self.__initialized = True
return getattr(self, name)
class Base(object):
def __init__(self):
print "yet"
self.base = True
class LateInit(LateInitMixin, Base):
pass
def main():
print "shouldn't init"
S = LateInit()
print "should init"
print S.base
if __name__=="__main__":
main()
Ok, now, the other problem : what if Base.__init__ expects args ? |
| |
| | | Bruno Desthuilliers | |
| | Posted: Wed Sep 03, 2008 8:55 pm Post subject: Re: Late initialization using __getattribute__ | |
| | | |
bukzor a écrit :
(snip)
| Quote: | Thanks for the reply. Just to see it not work, I tried to remove
__getattribute__ from LateInitMixIn, but couldn't get it to work.
|
??? Sorry, I don't get what you mean...
| Quote: | My Base class is a C class (_mysql.connection from MySQLdb) that
sometimes segfaults if you try to use it before it's fully
initialized,
|
Then don't use it before it's fully initialized.
patient: "doctor, when I do this, it hurts"
doctor : "then don't do it"
!-)
More seriously, I have used MySQLdb for years on more than a dozen linux
distribs, and never had such a problem. Is this a known bug ? Or is
there something wrong with your setup ?
| Quote: | so unfortunately I think I need to use __getattribute__
to do this. I'm doing all this just to make the connection not
actually connect until used.
|
I may be dumb, but I don't get how this is supposed to solve your
problem. But anyway : there's a known design pattern for what you're
trying to do, that doesn't require mixins nor messing with
__getattribute__ (which, I repeat, is more often than not something you
*don't* want to do). The name of the design pattern is "proxy". I
strongly suggest that you 1/ try to cure the real problem instead of
hacking around and 2/ read about the proxy design pattern.
My 2 cents... |
| |
| | | bukzor | |
| | Posted: Wed Sep 03, 2008 9:46 pm Post subject: Re: Late initialization using __getattribute__ | |
| | | |
On Sep 3, 12:19 pm, Bruno Desthuilliers
<bdesth.quelquech...@free.quelquepart.fr> wrote:
| Quote: | bukzor a écrit :
I want to make a MixIn class that waits to initialize its super-
classes until an attribute of the object is accessed. Not generally
useful, but desirable in my case. I've written this, and it works, but
would like to take any suggestions you guys have.
You shouldn't mess with __getattribute__ unless you really know what
you're doing and are ok to suffer the constant performance hit you'll
get. Remember that __getattribute__ actually *is* the implementation of
attribute lookup rules, and is called on each and every attribute
lookup. Your below snippet would be much better using __getattr__ (which
is only called as a last resort).
I've commented out
the "delattr" call because it throws an AttributeError (although I
don't know why).
__getattribute__ belongs to the class, not to the instance. delattr()
only removes instance attributes. You'd have to remove __getattribute__
from the LateInitMixIn class object itself, but then it would break the
whole thing.
class LateInitMixIn(object):
def __init__(self):
print "LateInit initialization"
self.inited = False
def __getattribute__(self, attr):
print "Doing __getattribute__"
getattr = lambda attr:object.__getattribute__(self, attr)
if not getattr("inited"):
super(LateInitMixIn, self).__init__()
setattr(self, "inited", True)
#delattr(self, "__getattribute__")
return getattr(attr)
Here's another possible implementation (which doesn't solve all
problems, cf below) using __getattr__:
class LateInitMixin(object):
def __init__(self):
print "not yet"
self.__initialized = False
def __getattr__(self, name):
if self.__initialized:
raise AttributeError(
"object %s has no attribute '%s'" % (type(self), name)
)
super(LateInitMixin, self).__init__()
self.__initialized = True
return getattr(self, name)
class Base(object):
def __init__(self):
print "yet"
self.base = True
class LateInit(LateInitMixin, Base):
pass
def main():
print "shouldn't init"
S = LateInit()
print "should init"
print S.base
if __name__=="__main__":
main()
Ok, now, the other problem : what if Base.__init__ expects args ?
|
Thanks for the reply. Just to see it not work, I tried to remove
__getattribute__ from LateInitMixIn, but couldn't get it to work.
My Base class is a C class (_mysql.connection from MySQLdb) that
sometimes segfaults if you try to use it before it's fully
initialized, so unfortunately I think I need to use __getattribute__
to do this. I'm doing all this just to make the connection not
actually connect until used. |
| |
| | | bukzor | |
| | Posted: Wed Sep 03, 2008 10:26 pm Post subject: Re: Late initialization using __getattribute__ | |
| | | |
On Sep 3, 1:02 pm, Bruno Desthuilliers
<bdesth.quelquech...@free.quelquepart.fr> wrote:
| Quote: | bukzor a écrit :
(snip)
Thanks for the reply. Just to see it not work, I tried to remove
__getattribute__ from LateInitMixIn, but couldn't get it to work.
??? Sorry, I don't get what you mean...
|
Since you said __getattribute__ is an attribute of the class, I tried
to run (something to the effect of) delattr(self.__class__,
"__getattribute__"), but it threw an AttributeError.
| Quote: |
My Base class is a C class (_mysql.connection from MySQLdb) that
sometimes segfaults if you try to use it before it's fully
initialized,
... I have used MySQLdb for years on more than a dozen linux
distribs, and never had such a problem. Is this a known bug ? Or is
there something wrong with your setup ?
|
I'm trying to optimize my number of connections by not fully
initializing (read: not connecting) my connection until it's used in
some way. Of course the maintainer didn't envision this (mis)use, so
the object sometimes throws bad errors until it's fully initialized.
Of course the *correct* way to do this is to be more careful about
when I create connections, but I think I should be able to get this to
work, and it (would be) much easier to do it The Right Way once it
works.
| Quote: | so unfortunately I think I need to use __getattribute__
to do this. I'm doing all this just to make the connection not
actually connect until used.
I may be dumb, but I don't get how this is supposed to solve your
problem. But anyway : there's a known design pattern for what you're
trying to do, that doesn't require mixins nor messing with
__getattribute__ (which, I repeat, is more often than not something you
*don't* want to do). The name of the design pattern is "proxy". I
strongly suggest that you 1/ try to cure the real problem instead of
hacking around and 2/ read about the proxy design pattern.
My 2 cents...
|
I like the idea of mix-ins, but can't figure out how to make a proxy
work that way. For a long time I had a proxy class that added five or
six features on top of the MySQLdb package, but it wasn't configurable
enough, and I'm working on splitting each feature into its own MixIn
class.
As an aside, this is working for me pretty well. The "reconnect"
method (inheritied from a "Reconnectable" mixin) uses several of the
object's attributes, so I need to set _inited beforehand so that I
don't get into an infinite __getattribute__ loop. What I'd *really*
like to do is remove __getattribute__ from the object at that point.
def __getattribute__(self, attr):
"connect if it would otherwise cause an error."
getattr = lambda attr:object.__getattribute__(self, attr)
if not getattr("_inited"):
print "connecting."
setattr(self, "_inited", True)
getattr("reconnect")()
return getattr(attr)
Thanks for your help,
--Buck |
| |
| | | Michele Simionato | |
| | Posted: Thu Sep 04, 2008 3:26 am Post subject: Re: Late initialization using __getattribute__ | |
On Sep 4, 12:26 am, bukzor <workithar...@gmail.com> wrote:
| Quote: | I'm trying to optimize my number of connections by not fully
initializing (read: not connecting) my connection until it's used in
some way.
|
I had the same use case and I solved with a simple property. Here is
the code
I have for pymssql:
@property
def conn(self):
if self._conn is None:
self._conn = _mssql.connect(self.host, self.user,self.passwd)
self._conn.select_db(self.dbname)
return self._conn
The connection is really instantiate only when you call self.conn to
perform the query, not when you instantiate the class. |
| |
|
|
