Loading from dump file into a dictionary

[karan_kanna]

Hi ,
i have a dump file . i want to load it into a dictionary.
i am using pickle load to load it as shown below:

pkl_file = open( r'D:\work\dumpfile.dat', 'rb+')
FinalTable = {}
FinalTable = pickle.load(pkl_file)

The problem is it is loading only one key and value from the dump file. Can anybody please tell me how to load the full dump file into dictionary.

Dump file values are like:
(dp0F1331114029.4375S'D:\\AAC\\TaskReport.xml'
(dp0F1331114029.46875S'D:\\AAC\\TaskReport.xml'

Bold ones are the keys and path is the value.

Thanks in advance

[Gribouillis]

It is possible that the (key, value) pairs where pickled one by one to the file using multiple calls to dump(). In this case, you can read them with multiple calls to load(). See this example

""" Example showing multiple calls of pickle.dump() to the same file.
    The objects stored in the file must be retrieved by multiple calls
    to pickle.load().
"""

import pickle

D = dict(enumerate("Hello this is some test data".split()))
print D

filename = "dummy.pkl"

print "starting dump()"
with open(filename, "wb") as ofh:
    for pair in D.items():
        pickle.dump(pair, ofh)
        
print "starting load()"
with open(filename, "rb") as ifh:
    try:
        while True:
            pair = pickle.load(ifh)
            print pair
    except EOFError:
        pass

""" my output -->
{0: 'Hello', 1: 'this', 2: 'is', 3: 'some', 4: 'test', 5: 'data'}
starting dump()
starting load()
(0, 'Hello')
(1, 'this')
(2, 'is')
(3, 'some')
(4, 'test')
(5, 'data')
"""

[karan_kanna]

Thanks Mate for your reply. I am already generating dump file from a dictionary running in a loop. The dumping is also done in loop process along with dictionary.The problem is that after the loop is completed the values are being dumped as separate dictionary key values like first line would have {key 1:value 1}
then again { key1: value1}

But , I want to load all those values in a single dictionary.That is why when i am loading it using pickle it is loading only the first dictionary key value.
I hope i am making myself clear.

It is possible that the (key, value) pairs where pickled one by one to the file using multiple calls to dump(). In this case, you can read them with multiple calls to load(). See this example

""" Example showing multiple calls of pickle.dump() to the same file.
    The objects stored in the file must be retrieved by multiple calls
    to pickle.load().
"""

import pickle

D = dict(enumerate("Hello this is some test data".split()))
print D

filename = "dummy.pkl"

print "starting dump()"
with open(filename, "wb") as ofh:
    for pair in D.items():
        pickle.dump(pair, ofh)
        
print "starting load()"
with open(filename, "rb") as ifh:
    try:
        while True:
            pair = pickle.load(ifh)
            print pair
    except EOFError:
        pass

""" my output -->
{0: 'Hello', 1: 'this', 2: 'is', 3: 'some', 4: 'test', 5: 'data'}
starting dump()
starting load()
(0, 'Hello')
(1, 'this')
(2, 'is')
(3, 'some')
(4, 'test')
(5, 'data')
"""

[Gribouillis]

No it is not clear. What is the output of running the following loop

import pickle
with open( r'D:\work\dumpfile.dat', 'rb') as pkl_file:
    try:
        while True:
            obj = pickle.load(pkl_file)
            print(obj)
    except EOFError:
        pass

?

[karan_kanna]

After trying your loop also, it is storing only the last value. I will try again to explain.
there is dictionary running in a loop whose key and values are changing every time in a loop and we are dumping those key and values in a .dat file. say for eg after the first run the dicionary KEY VALUE ARE '1: ABC'. These are dumped in a dat file. after the second loop key value are 2 : xyz. these are also dumped in the same file.
Now the dumped file has input like :
{1: ABC}
{2: XYZ}

Below is my original dump file:
(dp0
F1331114029.4375
S'D:\\3.aac_ab350f58389683366d9c11ecd2601c2a\\testplan.xml\\TaskReport.xml'
p1
s.(dp0
F1331114029.46875
S'D:\\3.aac_fb310a3df98d4f44ccee3d19aff24185\\testplan.xml\\TaskReport.xml'
p1
s.

now i want to load these different values in one dict. I have tried everything either it is loading the first value or the the last.

I hope i am clear now..

No it is not clear. What is the output of running the following loop

import pickle
with open( r'D:\work\dumpfile.dat', 'rb') as pkl_file:
    try:
        while True:
            obj = pickle.load(pkl_file)
            print(obj)
    except EOFError:
        pass

?

[karan_kanna]

Hey i have solved the problem using update with dictionary while loading the dump file.
Something like :
import pickle
Abc = {}
with open( r'D:\work\dumpfile.dat', 'rb') as pkl_file:
try:
while True:
obj = Abc.update.(pickle.load(pkl_file))
print(obj)
except EOFError:
pass

Hi ,
i have a dump file . i want to load it into a dictionary.
i am using pickle load to load it as shown below:

pkl_file = open( r'D:\work\dumpfile.dat', 'rb+')
FinalTable = {}
FinalTable = pickle.load(pkl_file)

The problem is it is loading only one key and value from the dump file. Can anybody please tell me how to load the full dump file into dictionary.

Dump file values are like:
(dp0F1331114029.4375S'D:\\AAC\\TaskReport.xml'
(dp0F1331114029.46875S'D:\\AAC\\TaskReport.xml'

Bold ones are the keys and path is the value.

Thanks in advance

[Gribouillis]

Hey i have solved the problem using update with dictionary while loading the dump file.

It's good. You could probably change your dump procedure to dump tuples like (1, 'ABC') instead of dictionaries with a single item like {1:'ABC'} . A tuple is a smaller object and your program is doing unnecessary work. You could then load the tuples with

...
        k, v = pickle.load(pkl_file)
        Abc[k] = v
...

Also learn to push the
[/b] button in this forum's post editor.[code]
button in this forum's post editor.