This is my first post, so I'll do my best...
I'm writing a Python app (using wxPython for the GUI) to copy large amounts of files. Each file is about 8-15 MB and there could be as many as 150,000 files. I'm currently using shutil (either
.copy(), .copy2() or .copyfiles() ) to manage the copies but I've noticed they are taking about twice as long as a normal finder copy (I'm working on a MAC). Is this just the Nature of shutil, or is the strange way i've set up my copy... maybe something else I'm missing?
Here's the code for the copy, it's a function that is calls itself to recursively copy subfolders and files. (I purposefully didn't use
def CopyDir(self, srcPath, dstPath, qItem, filesCopied=0): ''' Function to recursively copy all files and subfolders within a given srcPath to the dstPath. qItem is the reference to the queue object for feed back of the progress of the copy. ''' dirAndFileList = os.listdir(srcPath) dirList =  fileList =  for i in dirAndFileList: if os.path.isdir(os.path.join(srcPath, i)): dirList.append(i) elif os.path.isfile(os.path.join(srcPath, i)): fileList.append(i) dirList.sort() fileList.sort() for directory in dirList: if qItem.HasCancelled(): #If user has cancelled the copy, quit break os.mkdir(os.path.join(dstPath, directory)) newSrc = os.path.join(srcPath, directory) newDst = os.path.join(dstPath, directory) #Call itself to do move into the newly created subfolder and repeat... self.CopyDir(newSrc, newDst, qItem, filesCopied) for file in fileList: if qItem.HasCancelled(): #If user has cancelled the copy, quit break # Copy the file shutil.copyfile(os.path.join(srcPath, file), os.path.join(dstPath, file)) self.filesCopied += 1 #Feedback to the progress bar of the queue Item if self.filesCopied % 10 == 0: try: qItem.UpdateProgressBar(self.filesCopied) except: continue
Thanks in advance!