Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Investigate pickling of singletons #55
Comments
|
Yep, our handling of Let's add something like this to the unit tests for from unpythonic import frozendict
import pickle
fd1 = frozendict({'a': 1, 'b': 2})
fd2 = frozendict({'c': 3, 'd': 4})
fd3 = frozendict()
s = pickle.dumps([fd1, fd2, fd3])
o = pickle.loads(s)
print(o) # [frozendict({}), frozendict({}), frozendict({})] # as of 0.14.2-preEDIT: 4e32777 now includes the new test case. The important part was to have both the empty |
|
Hacky handling of |
|
The empty The other three, namely But perhaps think a bit more before closing this. It's not that hard to fix those too. |
|
Fixed |
|
Fixed Note, pickling and unpickling |
|
Upon closer investigation, we can't prevent pickling in If Fortunately, no state is saved in the 5f87e18 adds a comment documenting this. I think this is now finally fully done. Closing. |
|
For documentation: as of v0.14.2,
These use cases split to three categories:
In many cases, for use case 1, a local If we don't need to support a docstring and arbitrary extra features, then a symbol, in the Lisp sense, could be another possibility. Python has Specifically,
The empty The last singleton, However, the implementation is slightly misleading in that See also #22. |
|
The empty (The current |
Currently (0.14.2-pre) the empty
frozendictis freshly created for each Python session.This can be an issue if
fd = frozendict(), we then picklefd, and later unpickle it in a new Python session. Then, in this new session,fd is frozendict()isFalse, because the singleton empty frozendict has been freshly created.This needs some thinking. It's a global, not an attribute, so it's not easily updated in a custom
__setstate__like ourconsdoes for thenilsingleton.Maybe
__getnewargs_ex__?Hmm, wait, I may be interpreting the docs incorrectly. Looking at Pickling Class Instances, it may actually happen that only one
frozendictinstance (which is supposed to be the empty one) is ever created upon unpickling. If so, this is a serious bug that needs to be fixed ASAP. (EDIT: yes, that's exactly what happened. Fixed.)We should also check how
nilbehaves. We can probably make the old and newid(nil)agree after unpickling, without the__setstate__hack currently incons, if we use the low-level approach to create the singleton: makeNil.__new__create one globalNilinstance and always return it. (Assuming that__new__is called at unpickling time even if no__getnewargs__is present. Need to test this.) (EDIT: Yes, seems to be so. Done. The hack is gone, and it works.)Keep in mind only the class name and the instance data are actually stored in the pickle file. When unpickling,
__init__is skipped, but__new__runs (by default with no args other than the mandatorycls).EDIT: At unpickle time, after
__new__, then__setstate__runs if defined; else the pickled state must be a dictionary, and it is dumped to__dict__.It seems this isn't documented explicitly, but from what I can gather, it seems that
__getnewargs__(and its_exvariant) will be called at pickling time. The return value will be stored, and later used when__new__is called to create the uninitialized object (for populating the state) at unpickling time. See Flyweighting in Python by Simon Engledew (2009). EDIT: Yes, that seems to be exactly how it works.Usually
__getnewargs__isn't needed nowadays, but in cases where the behavior of a custom__new__depends on the arguments passed to the constructor call, it is needed to support pickling properly.Edit 22 February 2020: Added some notes above based on investigation results.