executing the multiple scripts in a py file

[prashanth s j]

Hi all,

Currently I execute multiple scripts in the following format:

  os.system(./script1)
  os.system(./script2)
  os.system(./script3)

But now I need to modify the code in such a way that script1, script2 etc return a success code(0) or failure code(1). But os.system can not return these values.
What I need to do so that I can write my scripts in the following fashion:

 Result = ./script1
 Result = ./script2 

and so on....
Your help would be very much appreciable.

Regards,
Prashanth

[Gribouillis]

I once wrote a small code snippet that does this and more: http://www.daniweb.com/code/snippet257449.html

[prashanth s j]

Hi thanks for the reply.
Based on your reply I wrote a script that contains the calling of the command "random" within the script itself. And it worked. What I did next was in place of the command "random" I gave ftp script as the argument. But because of network problems I could not test the FTP script.
So if I want to return the success or failure code from the FTP script, what changes I would be needed to make to the following part of the code? If I return some value from the FTP script, then will process.returncode take care of returning this return value from the FTP script?

#
self.failed = process.returncode
#
return self
#

#
@property
#
def returncode(self):
#
return self.failed
#

[TrustyTony]

Closing tag is

Hi thanks for the reply.
Based on your reply I wrote a script that contains the calling of the command "random" within the script itself. And it worked. What I did next was in place of the command "random" I gave ftp script as the argument. But because of network problems I could not test the FTP script.
So if I want to return the success or failure code from the FTP script, what changes I would be needed to make to the following part of the code? If I return some value from the FTP script, then will process.returncode take care of returning this return value from the FTP script?

self.failed = process.returncode
return self

@property
def returncode(self):
return self.failed

end quote.

[Gribouillis]

Hi thanks for the reply.
Based on your reply I wrote a script that contains the calling of the command "random" within the script itself. And it worked. What I did next was in place of the command "random" I gave ftp script as the argument. But because of network problems I could not test the FTP script.
So if I want to return the success or failure code from the FTP script, what changes I would be needed to make to the following part of the code? If I return some value from the FTP script, then will process.returncode take care of returning this return value from the FTP script?

self.failed = process.returncode
return self

@property
def returncode(self):
return self.failed

end quote.

If you are using the snippet's Command class, you don't need to change anything. Simply copy and paste the Command class to your program (after clicking 'Toggle Plain Text' in the post), then write

com = Command("./scrip1").run()
result = com.returncode
com = Command("./script2").run()
result = com.returncode

etc, or even simpler

result = Commmand("./script1").run().returncode

if you only need the return code in the Command object.

[prashanth s j]

Hi,
thanks very much for the reply.

Using your inputs I have been able to produce now some good scripts.
But still in some other kind of scripts I am facing issues.
I am running the scripts in this following fashion:

script1--->script2(has Command class)---spawn a process to run command-->command
Now the above works perfectly.
But the other scripts:
script1--->script2(has Command class)--spawn a process to run command--> command
this command further spawns processes on remote machines
command---use scp and do ssh on remote machines----->Two processes one on each remote machine.
Now on the same host the pipe created is responsible for sharing of the data between the parent and the child using the subprocess. But the pipe obvously doesnot work when the remote machines are involved. Can you please suggest a work around for this?
os.system obviously works. But it doesnot return any return value. If you could also suggest why os.system works it would be of immense value for me.

Regards,
Prashanth

[Gribouillis]

Hi,
thanks very much for the reply.

Using your inputs I have been able to produce now some good scripts.
But still in some other kind of scripts I am facing issues.
I am running the scripts in this following fashion:

script1--->script2(has Command class)---spawn a process to run command-->command
Now the above works perfectly.
But the other scripts:
script1--->script2(has Command class)--spawn a process to run command--> command
this command further spawns processes on remote machines
command---use scp and do ssh on remote machines----->Two processes one on each remote machine.
Now on the same host the pipe created is responsible for sharing of the data between the parent and the child using the subprocess. But the pipe obvously doesnot work when the remote machines are involved. Can you please suggest a work around for this?
os.system obviously works. But it doesnot return any return value. If you could also suggest why os.system works it would be of immense value for me.

Regards,
Prashanth

I don't understand which pipe you're talking about. The problem could be that the Command class communicates through the child process' stdin and stdout. I forgot to mention an alternative if you're only interested in the return code, try

import subprocess as sp
result = sp.call("./script2", shell=True)

[prashanth s j]

Hi Thanks again for the reply.
the solution helped me to resolve the issue for the scripts involving the usage of packet generation tools. But for scripts involving the code that does ssh to two different remote machines and run the scripts on those machines, the sp.call does not work. For example:

---has-->script2
SCRIPT1
----has--->script3
script2 and script3 are transferred to the remote machines remote2 and remote3 and they are executed on those machines by script1.
For the above scenario if i give script1 as the argument to sp.call, it is failing.
It says:

Traceback (most recent call last):
  File "./my_infra-tcp_test_suite-36.py", line 9, in <module>
    sys.exit(foo("/root/regression/tests/infra/tcp_test_suite/infra_nexus_correct_mul_tcp_and_udp_pns.py"))
  File "/root/regression/tests/infra/tcp_test_suite/no_test.py", line 17, in foo
    result = sp.call('python /root/regression/tests/infra/tcp_test_suite/infra_nexus_correct_mul_tcp_and_udp_pns.py')
  File "/usr/local/lib/python2.6/subprocess.py", line 470, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/local/lib/python2.6/subprocess.py", line 621, in __init__
    errread, errwrite)
  File "/usr/local/lib/python2.6/subprocess.py", line 1126, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

[Gribouillis]

I suggest you try these 2 ways to run the call

sp.call('python /root/regression/tests/infra/tcp_test_suite/infra_nexus_correct_mul_tcp_and_udp_pns.py',
  shell=True)

# or

sp.call(['python',
        '/root/regression/tests/infra/tcp_test_suite/infra_nexus_correct_mul_tcp_and_udp_pns.py'])

subprocess.call and subprocess.Popen behave have 2 syntaxes depending on their first argument being a string or a list of strings (see the documentation of module subprocess).

[prashanth s j]

Hi thanks for the response.
I would like to tell one thing.
U r great :)
Your solutions have helped me to provide solutions(to my test suite problems) !
For Now I am able to return the value by writing a value to a file from within a thread. when this thread terminates and the control returns to the process that spawned this thread, the process reads the file and then returns the value to its parent process using the exit(status).

Thanks and Regards,
Prashanth

[prashanth s j]

Hi,
I dint get why the error is "OSError: [Errno 2] No such file or directory"
Because for the child execs, it fetches the file name from the args[0] which is /root/regression/tests/infra/tcp_test_suite/infranexus_correct_mul_tcp_and_udp_pns.py
This file is present in the machine.

Regards,
Prashanth

[Gribouillis]

I think the problem is that (in unix), if you invoke subprocess.call or subprocess.Popen without the switch shell = True , the first argument must be either the name of the command to execute, without arguments, like sp.call("ls") or a list of arguments like sp.call(["python", "myfile.py"]) . With shell=True , the first argument may be a string containing command line arguments, like sp.call("python myfile.py", shell=True) .
In your case, the module probably looked for a command named "python myfile.py" , and since there is no file with this name, it raises OSError. Example:

>>> sp.call("ls -l")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib64/python2.6/subprocess.py", line 470, in call
    return Popen(*popenargs, **kwargs).wait()
  File "/usr/lib64/python2.6/subprocess.py", line 621, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.6/subprocess.py", line 1126, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
>>> sp.call("ls -l", shell=True)
total 116
-rw-r--r-- 1 eric eric  4085 2010-06-06 11:13 cube10.tm
0
>>> sp.call(["ls", "-l"])
total 116
-rw-r--r-- 1 eric eric  4085 2010-06-06 11:13 cube10.tm
0

Using a python tracing module, I obtained this

>>> sp.call("ls -l")
0. _execvpe('ls -l', ['ls -l']): # call, line 344 in os.py

which means that os._execvpe was called by the subprocess module with a (bad) file argument 'ls -l' .

[Gribouillis]

You can see the difference with the two other forms of call here

>>> sp.call("ls -l", shell=True)
0. _execvpe('/bin/sh', ['/bin/sh', '-c', 'ls -l']): # call, line 344 in os.py
>>> sp.call(["ls",  "-l"])
0. _execvpe('ls', ['ls', '-l']): # call, line 344 in os.py

[prashanth s j]

Excellent!
Thanks for your great help.
Your inputs have helped me to follow the python source code.