python 3.x - Dill failed to load back numpy arrays after loading back numpy record arrays -


all:

i'm trying save data in python program dill binary file. program used both regular numpy array , numpy record array. i'm having problem on loading regular numpy arrays if saved both numpy record arrays , regular arrays. problem can demonstrated following simple code:

import dill pickle import numpy np  class testdill:    def __init__(self, data):        self.data =data  # numpy record array type persontype = [('name', 'a32'), ('age', '<i2'), ('salary', '<i4'), ('weight','<f8')]    person = np.zeros(2, dtype=persontype, order='f') d = np.random.randn(5) d1 = testdill(d)  d = np.random.randn(5) d2 = testdill(d)  d = np.random.randn(5) d3 = testdill(d)   f= open('testdill.dat', 'wb') pickle.dump(person, f)  pickle.dump(d1, f) pickle.dump(d2, f) pickle.dump(d3, f)  f.close()  f= open('testdill.dat', 'rb')  person2 = pickle.load(f) d11 = pickle.load(f) d21 = pickle.load(f) d31 = pickle.load(f)

the above code result key error on loading d11 object. don't know whether there bug in dill package or i'm not using dill correctly. appreciate suggestions , comments around problem. i'm using python 3.4 on 64-bit windows 7.

thanks,

john

i'm dill author. looks you've found bug. i'd appreciate if open issue on dill github page.

there workaround, , it's pickle structured array last. why work? um… don't know yet. (note: code simplified w/o changing test case)

import dill pickle import numpy np  class testdill(object):    def __init__(self, data):        self.data =data  persontype = [('name', 'a32'), ('age', '<i2')]    person = np.zeros(2, dtype=persontype, order='f') d = np.random.randn(5) d1 = testdill(d)  open('testdill.dat', 'wb') f:   pickle.dump(d1, f)   pickle.dump(person, f)   open('testdill.dat', 'rb') f:   d11 = pickle.load(f)   person2 = pickle.load(f)

update: dill bug has been fixed. code works recent version on github.


-

Comments

Post a Comment

Popular posts from this blog

serialization - Convert Any type in scala to Array[Byte] and back -

i have following question: i have variable value in program declared value. i want convert value byte array.. how can serialize byte array , back? found examples related other types such double or int, not any. this should need. it's pretty similar how 1 in java. import java.io.{bytearrayinputstream, bytearrayoutputstream, objectinputstream, objectoutputstream} object serialization extends app { def serialise(value: any): array[byte] = { val stream: bytearrayoutputstream = new bytearrayoutputstream() val oos = new objectoutputstream(stream) oos.writeobject(value) oos.close stream.tobytearray } def deserialise(bytes: array[byte]): = { val ois = new objectinputstream(new bytearrayinputstream(bytes)) val value = ois.readobject ois.close value } println(deserialise(serialise("my test"))) println(deserialise(serialise(list(1)))) println(deserialise(serialise(map(1 -> 2)))) println(deserialise(seri
Read more

matplotlib support failed in PyCharm on OSX -

any idea how fix ? how can debug might problem ? not seem : https://youtrack.jetbrains.com/issue/py-15520 i in console: python 2.6.9 (unknown, jul 14 2015, 19:46:31) [gcc 4.2.1 compatible apple llvm 6.0 (clang-600.0.39)] on darwin >>> import matplotlib.pyplot plt matplotlib support failed the backend : >>> import matplotlib >>> print(matplotlib.rcparams['backend']) macosx
Read more

python - Matplotlib: TypeError: 'AxesSubplot' object is not callable -

i new programming please pardon me if rookie mistake. i writing python script have defined functions create subplots of distributions. i have slider widget end user change value of variables(mu , sigma) , see effect on distribution. here's code import numpy np print np.__version__ import scipy print scipy.__version__ scipy.stats import norm, lognorm, uniform import matplotlib print matplotlib.__version__ import matplotlib.pyplot plt matplotlib.widgets import slider, button, radiobuttons, checkbuttons #####calculating mean , standard deviation##### global mu_a1 mu_a1 = 1 global mu_b1 mu_b1 = 10 global mu_c1 mu_c1 = -13 global sigma_a1 sigma_a1 = 0.14 global sigma_b1 sigma_b1 = 1.16 global sigma_c1 sigma_c1 = 2.87 mu_x01 = -11 sigma_x01 = 1.9 #####_____##### #####generating random data##### a1 = 0.75*mu_a1 + (1.25 - 0.75)*sigma_a1*np.random.sample(10000) b1 = 8*mu_b1 + (12 - 8)*sigma_b1*np.random.sample(10000) c1 = -12*mu_c1 + 2*sigma_c1*np.random.sample(10000)
Read more