Insecure string pickle

[aot]

I'm having trouble unpickling a file -- it gives me ValueError: insecure string pickle. The trouble is, it is nearly identical to another file that I was able to unpickle just fine. For the life of me I cannot see what is strange or different about the new file -- the only thing I can think is that it was pickled on a Windows machine, while the old one was pickled on a Mac.

And of course there are slight differences in what exactly was pickled. They each pickle the responses to various questionnaires, stored in a dictionary, and the new one had some new values that were stored (comments, nfaScore) while omitting an old one (email).

Anyway, I am attaching both files. If anyone has any clues I would be very grateful!

[jlm699]

This is strange. I'm getting

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Python25\lib\pickle.py", line 858, in load
    dispatch[key](self)
  File "C:\Python25\lib\pickle.py", line 1198, in load_setitem
    dict[key] = value
TypeError: 'str' object does not support item assignment

The object that you pickled, did it require any special classes?

Also, are you saying that you pickled one object on a Mac, and one on a Windows box, and are trying to unpickle both on the Windows box now?

[Ene Uran]

if you open the file with "r" you get:
'str' object does not support item assignment
if you open the file with "rb" you get:
Insecure string pickle

Somehow the new.txt file must be corrupted or used a strange dump() protocol.

[aot]

It was just a regular dictionary.

I pickled the new one on a Windows machine, the old one on a Mac. Tried to unpickle both on a Mac.

Note: I just managed to try the new one on a Windows machine, and it worked. WTF.

[jlm699]

Did the windows and mac have the same versions of Python installed? What version were you using by the way?

[Gribouillis]

For simple dictionaries, you can avoid using pickle and write to a file. Try this in a console

>>> from pickle import load
>>> D =  load(open("old.txt"))
>>> out = open("foo.txt", "w")
>>> out.write(repr(D))
>>> out.close()
>>> E = eval(open("foo.txt").read())
>>> D == E
True
>>>

My advice is: don't use pickle when you don't need it :)

[woooee]

Note: I just managed to try the new one on a Windows machine, and it worked. WTF.

It might have something to do with line endings. MS Windows uses decimal 13 & 10 and Linux (Mac) uses decimal 13 only. Since MS Windows pickle works on Windows and Mac's works on a Mac, it appears to be the way data is stored. If you store a normal text file on MS Windows and open it on a Mac you will see an odd character at the end, which is the additional decimal 10. There are several to convert files, depending on what you want to do. The quick solution, as stated above, is to use a normal file instead of pickle.

[Ene Uran]

woooee is correct, it is the line endings. You can look at the files with a hexeditor and find that Windows uses "\n\r" or hex '0d 0a', Linux uses "\n" or hex '0d' and the Mac uses "\r" or hex '0a'. So much for a standard!

Windows will accept the Linux ending but not the Mac ending.

[Gribouillis]

who needs a hex editor ?

>>> print repr(open("new.txt").read())

[woooee]

I convert to pdf whenever there is a lot of transferring to be done between different OS's just because it is a common standard and has converters to/from text for all of the machines.