I'm a complete pyhthon n00b and I'm trying to execute an external command from within my python script. For this I use the subprocess module. Here is all the relevant code:
class PipeThread(threading.Thread): def __init__(self, fin): self.fin = fin self.sout = "" threading.Thread.__init__(self) def run(self): self.sout = self.fin.read() def getOutput(self): return self.sout
class ExecutionFailed(Exception): def __init__(self, value): self.parameter = value def __str__(self): return str(self.parameter)
def execute(command, name = None): try: if name == None: name = "'" + command + "'" cmd = subprocess.Popen(command, 0, None, subprocess.PIPE, subprocess.PIPE, subprocess.PIPE) cmd.stdin.close() outPipe, errPipe, retcode = PipeThread(cmd.stdout), PipeThread(cmd.stderr), cmd.wait() outPipe.start(), errPipe.start() out, err = outPipe.getOutput(), errPipe.getOutput() if retcode != 0 and err != '': raise ExecutionFailed, name + " exited with error code " + str(retcode) + " and errors:\n" + err elif retcode != 0: raise ExecutionFailed, name + " exited with error code " + str(retcode) + " and output:\n" + out elif err != '': return out + "\n" + name + " gave warnings:\n" + err else: return out except Exception, e: if isinstance(e, ExecutionFailed): raise else: raise ExecutionFailed, "Error while executing " + name + ":\n" + str(e)
def javac(out, root): return execute("javac -d" + " " + os.path.normpath(out) + " " + os.path.normpath(root), "Java Compiler")
So far I have only tested this in Windows.
javac is a console application and it seems that like all console applications it has a limited stdout (and stderr?) buffer size. Thus, if you don't empty its stdout stream fast enough it will appear to have hung. This is why I use the PipeThread class which is supposed to read data from the subprocess stdout and stderr streams and store them in a string, thus preventing a stall.
My problem is that it still won't work. If large amouts of data are sent to either stdout or stderr my script will hang. In my case this happens while
javac is running. If I close it with Ctrl+C I see that my script has indeed read a certain amount of output from the subprocess (always the same) but no more. So it seems to me that while
self.fin.read() does read data from
self.fin it doesn't actually empty it!
Can anyone explain this and tell me what the correct appoach is or if it's possible to change the default buffering behavior for windows applications?
Any help is greatly appreciated!!!