1,105,214 Community Members

multiprocessing blues in Windows7

Member Avatar
pyTony
pyMod
6,103 posts since Apr 2010
Reputation Points: 818 [?]
Q&As Helped to Solve: 1,056 [?]
Skill Endorsements: 42 [?]
Moderator
Featured
 
0
 

This example program of using the multiprocessing is crashing my i7 4 core windows7 64 bit computer (32 bit enthought Python 2.7), but is running fine in virtual machine Ubuntu with 4 cores assigned. Can anyone run it in their commputer in Windows7. My system memory (8 GB) is totally eaten by the program and mouse moves but I can not do anything with computer, even ctrl-alt-del to get to task manager.

# -*- coding: utf-8 -*-
from numpy import arange,sqrt, random, linalg
from multiprocessing import Pool

global counter
counter = 0
def cb(r):
    global counter
    print counter, r
    counter +=1

def det(M):
    return linalg.det(M)

po = Pool()
for i in xrange(1,300):
    j = random.normal(1,1,(100,100))
    po.apply_async(det,(j,),callback=cb)
po.close()
po.join()
print counter

http://pyinsci.blogspot.fi/2009/02/usage-pattern-for-multiprocessing.html

Member Avatar
pyTony
pyMod
6,103 posts since Apr 2010
Reputation Points: 818 [?]
Q&As Helped to Solve: 1,056 [?]
Skill Endorsements: 42 [?]
Moderator
Featured
 
2
 

I found out that in windows the creation of Pool must be guarded by

if __name__ == '__main__': 

as the module is imported by interpreter. So this works:

# -*- coding: utf-8 -*-
from numpy import arange,sqrt, random, linalg
from multiprocessing import Pool

global counter
counter = 0
def cb(r):
    global counter
    print counter, r
    counter +=1

def det(M):
    return linalg.det(M)

if __name__ == '__main__':
    po = Pool(2)
    for i in xrange(1,100):
        j = random.normal(1,1,(100,100))
        po.apply_async(det,(j,),callback=cb)
    po.close()
    po.join()
    print counter
Question Self-Answered as of 1 Year Ago
Member Avatar
vegaseat
DaniWeb's Hypocrite
6,984 posts since Oct 2004
Reputation Points: 1,544 [?]
Q&As Helped to Solve: 1,872 [?]
Skill Endorsements: 67 [?]
Moderator
 
0
 

Thanks for letting us know!

You
This question has already been solved: Start a new discussion instead
Post:
Start New Discussion
View similar articles that have also been tagged: