I'm having trouble unpickling a file -- it gives me ValueError: insecure string pickle. The trouble is, it is nearly identical to another file that I was able to unpickle just fine. For the life of me I cannot see what is strange or different about the new file -- the only thing I can think is that it was pickled on a Windows machine, while the old one was pickled on a Mac.

And of course there are slight differences in what exactly was pickled. They each pickle the responses to various questionnaires, stored in a dictionary, and the new one had some new values that were stored (comments, nfaScore) while omitting an old one (email).

Anyway, I am attaching both files. If anyone has any clues I would be very grateful!

Attachments
(dp0
S'programTimeTaken'
p1
F824.43499994277954
sS'nfaScore'
p2
(lp3
I121
aF4.8399999999999999
aF1.5459624833740304
asS'timeTaken'
p4
F793.36099982261658
sS'aotScore'
p5
(lp6
I219
aF5.3414634146341466
aF1.5590021824481353
asS'aotScore2'
p7
(lp8
I219
aF5.3414634146341466
aF1.5590021824481353
asS'sdbScore'
p9
(lp10
I14
aF2.7999999999999998
aF1.6431676725154984
asS'maxScore'
p11
(lp12
I75
aF5.7692307692307692
aF1.7394369852458083
asS'id'
p13
S'038'
p14(dp0
S'programTimeTaken'
p1
F824.43499994277954
sS'nfaScore'
p2
(lp3
I121
aF4.8399999999999999
aF1.5459624833740304
asS'timeTaken'
p4
F793.36099982261658
sS'aotScore'
p5
(lp6
I219
aF5.3414634146341466
aF1.5590021824481353
asS'comments'
p7
S'\n'
p8
sS'sdbScore'
p9
(lp10
I14
aF2.7999999999999998
aF1.6431676725154984
asS'maxScore'
p11
(lp12
I75
aF5.7692307692307692
aF1.7394369852458083
asS'id'
p13
S'038'
p14
sS'qAnswers'
p15
(lp16
(lp17
I1
aI7
aa(lp18
I2
aI2
aa(lp19
I3
aI4
aa(lp20
I4
aI5
aa(lp21
I5
aI1
aa(lp22
I6
aI2
aa(lp23
I7
aI2
aa(lp24
I8
aI2
aa(lp25
I9
aI3
aa(lp26
I10
aI6
aa(lp27
I11
aI4
aa(lp28
I12
aI2
aa(lp29
I13
aI2
aa(lp30
I14
aI1
aa(lp31
I15
aI3
aa(lp32
I16
aI3
aa(lp33
I17
aI1
aa(lp34
I18
aI6
aa(lp35
I19
aI2
aa(lp36
I20
aI5
aa(lp37
I21
aI5
aa(lp38
I22
aI1
aa(lp39
I23
aI2
aa(lp40
I24
aI2
aa(lp41
I25
aI1
aa(lp42
I26
aI6
aa(lp43
I27
aI6
aa(lp44
I28
aI1
aa(lp45
I29
aI7
aa(lp46
I30
aI6
aa(lp47
I31
aI6
aa(lp48
I32
aI4
aa(lp49
I33
aI2
aa(lp50
I34
aI6
aa(lp51
I35
aI1
aa(lp52
I36
aI4
aa(lp53
I37
aI5
aa(lp54
I38
aI1
aa(lp55
I39
aI3
aa(lp56
I40
aI5
aa(lp57
I41
aI6
aa(lp58
I42
aI6
aa(lp59
I43
aI6
aa(lp60
I44
aI2
aa(lp61
I45
aI2
aa(lp62
I46
aI2
aa(lp63
I47
aI6
aa(lp64
I48
aI2
aa(lp65
I49
aI3
aa(lp66
I50
aI6
aa(lp67
I51
aI7
aa(lp68
I52
aI6
aa(lp69
I53
aI2
aa(lp70
I54
aI5
aa(lp71
I55
aI2
aa(lp72
I56
aI3
aa(lp73
I57
aI7
aa(lp74
I58
aI6
aa(lp75
I59
aI3
aa(lp76
I60
aI6
aa(lp77
I61
aI7
aa(lp78
I62
aI6
aa(lp79
I63
aI7
aa(lp80
I64
aI2
aa(lp81
I65
aI7
aa(lp82
I66
aI7
aa(lp83
I67
aI6
aa(lp84
I68
aI6
aa(lp85
I69
aI2
aa(lp86
I70
aI7
aa(lp87
I71
aI6
aa(lp88
I72
aI6
aa(lp89
I73
aI5
aa(lp90
I74
aI2
aa(lp91
I75
aI7
aa(lp92
I76
aI4
aa(lp93
I77
aI6
aa(lp94
I78
aI2
aa(lp95
I79
aI1
aa(lp96
I80
aI6
aa(lp97
I81
aI6
aa(lp98
I82
aI6
aa(lp99
I83
aI6
aa(lp100
I84
aI4
aa(lp101
I85
aI2
aa(lp102
I86
aI2
aa(lp103
I87
aI2
aa(lp104
I88
aI5
aa(lp105
I89
aI3
aa(lp106
I90
aI5
aa(lp107
I91
aI5
aa(lp108
I92
aI3
aa(lp109
I93
aI6
aa(lp110
I94
aI2
aa(lp111
I95
aI6
aa(lp112
I96
aI6
aa(lp113
I97
aI3
aa(lp114
I98
aI3
aa(lp115
I99
aI2
aa(lp116
I100
aI4
aa(lp117
I101
aI5
aa(lp118
I102
aI1
aa(lp119
I103
aI3
aasS'nfcScore'
p120
(lp121
I84
aF4.666666666666667
aF1.9097274212644622
as.
sS'qAnswers'
p15
(lp16
(lp17
I1
aI7
aa(lp18
I2
aI2
aa(lp19
I3
aI4
aa(lp20
I4
aI5
aa(lp21
I5
aI1
aa(lp22
I6
aI2
aa(lp23
I7
aI2
aa(lp24
I8
aI2
aa(lp25
I9
aI3
aa(lp26
I10
aI6
aa(lp27
I11
aI4
aa(lp28
I12
aI2
aa(lp29
I13
aI2
aa(lp30
I14
aI1
aa(lp31
I15
aI3
aa(lp32
I16
aI3
aa(lp33
I17
aI1
aa(lp34
I18
aI6
aa(lp35
I19
aI2
aa(lp36
I20
aI5
aa(lp37
I21
aI5
aa(lp38
I22
aI1
aa(lp39
I23
aI2
aa(lp40
I24
aI2
aa(lp41
I25
aI1
aa(lp42
I26
aI6
aa(lp43
I27
aI6
aa(lp44
I28
aI1
aa(lp45
I29
aI7
aa(lp46
I30
aI6
aa(lp47
I31
aI6
aa(lp48
I32
aI4
aa(lp49
I33
aI2
aa(lp50
I34
aI6
aa(lp51
I35
aI1
aa(lp52
I36
aI4
aa(lp53
I37
aI5
aa(lp54
I38
aI1
aa(lp55
I39
aI3
aa(lp56
I40
aI5
aa(lp57
I41
aI6
aa(lp58
I42
aI6
aa(lp59
I43
aI6
aa(lp60
I44
aI2
aa(lp61
I45
aI2
aa(lp62
I46
aI2
aa(lp63
I47
aI6
aa(lp64
I48
aI2
aa(lp65
I49
aI3
aa(lp66
I50
aI6
aa(lp67
I51
aI7
aa(lp68
I52
aI6
aa(lp69
I53
aI2
aa(lp70
I54
aI5
aa(lp71
I55
aI2
aa(lp72
I56
aI3
aa(lp73
I57
aI7
aa(lp74
I58
aI6
aa(lp75
I59
aI3
aa(lp76
I60
aI6
aa(lp77
I61
aI7
aa(lp78
I62
aI6
aa(lp79
I63
aI7
aa(lp80
I64
aI2
aa(lp81
I65
aI7
aa(lp82
I66
aI7
aa(lp83
I67
aI6
aa(lp84
I68
aI6
aa(lp85
I69
aI2
aa(lp86
I70
aI7
aa(lp87
I71
aI6
aa(lp88
I72
aI6
aa(lp89
I73
aI5
aa(lp90
I74
aI2
aa(lp91
I75
aI7
aa(lp92
I76
aI4
aa(lp93
I77
aI6
aa(lp94
I78
aI2
aa(lp95
I79
aI1
aa(lp96
I80
aI6
aa(lp97
I81
aI6
aa(lp98
I82
aI6
aa(lp99
I83
aI6
aa(lp100
I84
aI4
aa(lp101
I85
aI2
aa(lp102
I86
aI2
aa(lp103
I87
aI2
aa(lp104
I88
aI5
aa(lp105
I89
aI3
aa(lp106
I90
aI5
aa(lp107
I91
aI5
aa(lp108
I92
aI3
aa(lp109
I93
aI6
aa(lp110
I94
aI2
aa(lp111
I95
aI6
aa(lp112
I96
aI6
aa(lp113
I97
aI3
aa(lp114
I98
aI3
aa(lp115
I99
aI2
aa(lp116
I100
aI4
aa(lp117
I101
aI5
aa(lp118
I102
aI1
aa(lp119
I103
aI3
aasS'nfcScore'
p120
(lp121
I84
aF4.666666666666667
aF1.9097274212644622
as.
(dp0
S'programTimeTaken'
p1
F863.24152398109436
sS'timeTaken'
p2
F681.87854909896851
sS'aotScore'
p3
(lp4
I217
aF5.2926829268292686
aF2.0885868720144769
asS'sdbScore'
p5
(lp6
I22
aF4.4000000000000004
aF2.7928480087537881
asS'maxScore'
p7
(lp8
I48
aF3.6923076923076925
aF2.0568833780186058
asS'email'
p9
S'junk@junk.ac.uk'
p10
sS'qAnswers'
p11
(lp12
(lp13
I1
aI1
aa(lp14
I2
aI1
aa(lp15
I3
aI1
aa(lp16
I4
aI7
aa(lp17
I5
aI5
aa(lp18
I6
aI2
aa(lp19
I7
aI1
aa(lp20
I8
aI2
aa(lp21
I9
aI5
aa(lp22
I10
aI6
aa(lp23
I11
aI7
aa(lp24
I12
aI4
aa(lp25
I13
aI7
aa(lp26
I14
aI1
aa(lp27
I15
aI1
aa(lp28
I16
aI2
aa(lp29
I17
aI1
aa(lp30
I18
aI7
aa(lp31
I19
aI2
aa(lp32
I20
aI7
aa(lp33
I21
aI1
aa(lp34
I22
aI5
aa(lp35
I23
aI4
aa(lp36
I24
aI1
aa(lp37
I25
aI4
aa(lp38
I26
aI7
aa(lp39
I27
aI7
aa(lp40
I28
aI4
aa(lp41
I29
aI2
aa(lp42
I30
aI7
aa(lp43
I31
aI1
aa(lp44
I32
aI2
aa(lp45
I33
aI1
aa(lp46
I34
aI1
aa(lp47
I35
aI2
aa(lp48
I36
aI4
aa(lp49
I37
aI7
aa(lp50
I38
aI4
aa(lp51
I39
aI1
aa(lp52
I40
aI5
aa(lp53
I41
aI7
aa(lp54
I42
aI5
aa(lp55
I43
aI5
aa(lp56
I44
aI2
aa(lp57
I45
aI1
aa(lp58
I46
aI1
aa(lp59
I47
aI3
aa(lp60
I48
aI6
aa(lp61
I49
aI2
aa(lp62
I50
aI3
aa(lp63
I51
aI6
aa(lp64
I52
aI6
aa(lp65
I53
aI2
aa(lp66
I54
aI5
aa(lp67
I55
aI7
aa(lp68
I56
aI6
aa(lp69
I57
aI3
aa(lp70
I58
aI3
aa(lp71
I59
aI5
aa(lp72
I60
aI5
aa(lp73
I61
aI4
aa(lp74
I62
aI4
aa(lp75
I63
aI6
aa(lp76
I64
aI7
aa(lp77
I65
aI5
aa(lp78
I66
aI1
aa(lp79
I67
aI4
aa(lp80
I68
aI1
aa(lp81
I69
aI1
aa(lp82
I70
aI4
aa(lp83
I71
aI1
aa(lp84
I72
aI5
aa(lp85
I73
aI2
aa(lp86
I74
aI5
aa(lp87
I75
aI7
aa(lp88
I76
aI1
aa(lp89
I77
aI1
aasS'nfcScore'
p90
(lp91
I97
aF5.3888888888888893
aF1.289728146862918
as.

This is strange. I'm getting

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Python25\lib\pickle.py", line 858, in load
    dispatch[key](self)
  File "C:\Python25\lib\pickle.py", line 1198, in load_setitem
    dict[key] = value
TypeError: 'str' object does not support item assignment

The object that you pickled, did it require any special classes?

Also, are you saying that you pickled one object on a Mac, and one on a Windows box, and are trying to unpickle both on the Windows box now?

if you open the file with "r" you get:
'str' object does not support item assignment
if you open the file with "rb" you get:
Insecure string pickle

Somehow the new.txt file must be corrupted or used a strange dump() protocol.

It was just a regular dictionary.

I pickled the new one on a Windows machine, the old one on a Mac. Tried to unpickle both on a Mac.

Note: I just managed to try the new one on a Windows machine, and it worked. WTF.

Did the windows and mac have the same versions of Python installed? What version were you using by the way?

For simple dictionaries, you can avoid using pickle and write to a file. Try this in a console

>>> from pickle import load
>>> D =  load(open("old.txt"))
>>> out = open("foo.txt", "w")
>>> out.write(repr(D))
>>> out.close()
>>> E = eval(open("foo.txt").read())
>>> D == E
True
>>>

My advice is: don't use pickle when you don't need it :)

Note: I just managed to try the new one on a Windows machine, and it worked. WTF.

It might have something to do with line endings. MS Windows uses decimal 13 & 10 and Linux (Mac) uses decimal 13 only. Since MS Windows pickle works on Windows and Mac's works on a Mac, it appears to be the way data is stored. If you store a normal text file on MS Windows and open it on a Mac you will see an odd character at the end, which is the additional decimal 10. There are several to convert files, depending on what you want to do. The quick solution, as stated above, is to use a normal file instead of pickle.

woooee is correct, it is the line endings. You can look at the files with a hexeditor and find that Windows uses "\n\r" or hex '0d 0a', Linux uses "\n" or hex '0d' and the Mac uses "\r" or hex '0a'. So much for a standard!

Windows will accept the Linux ending but not the Mac ending.

I convert to pdf whenever there is a lot of transferring to be done between different OS's just because it is a common standard and has converters to/from text for all of the machines.

This article has been dead for over six months. Start a new discussion instead.