[Python-Dev] PEP 574 -- Pickle protocol 5 with out-of-band data
Robert Collins
robertc at robertcollins.net
Wed Mar 28 21:40:17 EDT 2018
One question..
On Thu., 29 Mar. 2018, 07:42 Antoine Pitrou, <solipsis at pitrou.net> wrote:
> ...
>
=======
>
> Mutability
> ----------
>
> PEP 3118 buffers [#pep-3118]_ can be readonly or writable. Some objects,
> such as Numpy arrays, need to be backed by a mutable buffer for full
> operation. Pickle consumers that use the ``buffer_callback`` and
> ``buffers``
> arguments will have to be careful to recreate mutable buffers. When doing
> I/O, this implies using buffer-passing API variants such as ``readinto``
> (which are also often preferrable for performance).
>
> Data sharing
> ------------
>
> If you pickle and then unpickle an object in the same process, passing
> out-of-band buffer views, then the unpickled object may be backed by the
> same buffer as the original pickled object.
>
> For example, it might be reasonable to implement reduction of a Numpy array
> as follows (crucial metadata such as shapes is omitted for simplicity)::
>
> class ndarray:
>
> def __reduce_ex__(self, protocol):
> if protocol == 5:
> return numpy.frombuffer, (PickleBuffer(self), self.dtype)
> # Legacy code for earlier protocols omitted
>
> Then simply passing the PickleBuffer around from ``dumps`` to ``loads``
> will produce a new Numpy array sharing the same underlying memory as the
> original Numpy object (and, incidentally, keeping it alive)::
This seems incompatible with v4 semantics. There, a loads plus dumps
combination is approximately a deep copy. This isn't. Sometimes. Sometimes
it is.
Other than that way, I like it.
Rob
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <[Python-Dev] PEP 574 -- Pickle protocol 5 with out-of-band data
Next message (by thread): [Python-Dev] PEP 574 -- Pickle protocol 5 with out-of-band data
Messages sorted by:
[ date ]
[ thread ]
[ subject ]
[ author ]
More information about the Python-Dev
mailing list