create functions in python

[sofia85]

I'm a beginner at python programming and now I want to try to write a function consisting of my already written programs. The goal is to get all probabilities when I'm calling on the function in the main program.

My already separately written programs looks something likes this and the goal of these programs is to extract information from different files until I reach the last file to be able to extract some values.

#program 1
f1=open('filename1', 'r')
newfile=('file_name1', 'w')

for lines in f.readlines():
	if 'number' in lines:
		do some stuff
		and save it to newfile.write()
f.close()
newfile.close()

#program 2
f2=open('filename1', 'r')  # open file created in newfile in program 1
newfile=('file_name2, 'w')

for lines in f2.readlines():
	f3=open('filename3', 'r')
	for data in f3.readlines():
		if lines.split('\t')[0] in datat:
			write to new_file	
	f3.close()
f2.close()
new_file.close()

# program 3
#here I'm extracting all values from a certain column from a file using 'filename2'

f5=open('file_name2', 'r')  # open file created in newfile in program 2
newfile=('value_file','w')

for lines in f5.readlines():
	f6= open('filename6', 'r')
	for data in 'filename6':
		if lines[column2] exist in data:
			then write all these values to a new_file
	f6.close()
f5.close()
new_file.close()

How do I create a function out of this to use in the main program. So I can just get all values automatically? I don't know where to start other than with def. Can one combine several programs into one fcn or do I need to create several functions? The important thing is program 3 where I'm getting all the values i'm interested in. But to get there I need to go through all these other files and search. Because then i'm gonna use these values in already written formula.

[ret801]

you could put them all inside of functions, did they need to be seperate programs?

[sofia85]

no they don't. I just want to be able to run them in another program all together. I just donät understand how to do it, because then I would have three fcns, but fcn 2 is build on fcn 1 and fcn three is build on fcn 2 as you can see in the code.

[Gribouillis]

You could define 3 functions which take 2 arguments, a file opened for reading and a file opened for writing. Then you could use StringIO objects instead of temporary files on the disk. StringIO are memory files. They can be used like opened files (for reading or writing or appending). Here is an example with 3 file transform functions and a main function which calls the 3 transform functions one after the other to chain their action, using StringIOs as intermediary files

try:
    from cStringIO import StringIO # python 2
except ImportError:
    from io import StringIO # python 3

def funcA(infile, outfile):
    for line in infile:
        outfile.write(line.upper())

def funcB(infile, outfile):
    for line in infile:
        outfile.write(line[:-1][::-1]+"\n")
        
def funcC(infile, outfile):
    for line in infile:
        outfile.write(line.replace('E', 'e'))

def main():
    with open("inputfile.txt", "r") as fh1:
        fh2 = StringIO()
        funcA(fh1, fh2)
    fh3 = StringIO()
    fh2.seek(0) # go to the beginning of file 2 because we're now going to read it.
    funcB(fh2, fh3)
    fh3.seek(0)
    with open("outputfile.txt", "w") as fh4:
        funcC(fh3, fh4)
        
if __name__ == "__main__":
    main()

Another good solution would be to use generator functions, but since you're new to python, you must learn ordinary functions before generators.

Notice that you can see the content of a StringIO by calling fh2.getvalue() for example, which returns a string that you could print or write in a file.

[woooee]

A basic principle is that you have bytes of data. In your case you want to modify and store the bytes several times. It does not matter how you store them. You can create files to store them, as in this program, or you can store them in a list or SQLite database and pass the container around. Also, in program 2, you open and read a file each time through the loop. Read it once only.

#program 2
f2=open('filename1', 'r')  # open file created in newfile in program 1
newfile=('file_name2', 'w')

f3_data=open(filename3. 'r').readlines()

for lines in f2.readlines():
#	f3=open('filename3', 'r')
	for data in f3_data:
		if lines.split('\t')[0] in datat:
			write to new_file	
#	f3.close()    ## open + readlines on one line does this for you

f2.close()
new_file.close()

[sofia85]

Thanks. But how does the result argument work? If I e.g. wanted to do something like this applied on my code (see code below). Where f2 does some stuff based in result in f1 and f3 does some stuff based on res2:

def f1():
    do something
    return res1
def f2(result1):
    do something 
    return res2
def f3(result2):
    do something 
    return res3

Also I was wondering how do you read in the data in to the file when using fcns? Do you use the open function? (like this: open('filename', 'r') and the same for the outfile but with 'w' instead of 'r'). Do I use it inside def or outside? I'm unsure about the syntax.

[Gribouillis]

Thanks. But how does the result argument work? If I e.g. wanted to do something like this applied on my code (see code below). Where f2 does some stuff based in result in f1 and f3 does some stuff based on res2:

def f1():
    do something
    return res1
def f2(result1):
    do something 
    return res2
def f3(result2):
    do something 
    return res3

Also I was wondering how do you read in the data in to the file when using fcns? Do you use the open function? (like this: open('filename', 'r') and the same for the outfile but with 'w' instead of 'r'). Do I use it inside def or outside? I'm unsure about the syntax.

The key questions are: what are the arguments and what does the functions return. In your code above you would compose the functions in an expression like result = f3(f2(f1())) . This works for example if f1() returns a string and f2() takes a string as argument and returns a string, and f3() takes a string as argument. There is no general rule as to where you open the files, inside or outside functions.
If you want f1 to take no argument and return a string, you could open an infile inside f1's body and create a StringIO for outfile and return the string obtained with the StringIO's getvalue() method for example.
The code I posted above should be very easy to use for you because you only need to write the code that you already wrote in your 3 scripts inside the body of funcA(), funcB(), funcC(), and you don't need to open or close files in the function bodies.

[woooee]

You would use readlines() to read the file into a list, process the list however many times you want, and then write the list to another file.

def f1(data_list):
    """ add one to even numbered elements
    """
    for ctr in range(1, len(data_list), 2):
        data_list[ctr] = int(data_list[ctr].strip()) + 1
    return data_list

def f2(data_list):
    """ subtract one from odd numbered elements
    """
    for ctr in range(0, len(data_list), 2):
        data_list[ctr] = int(data_list[ctr].strip()) - 1
    return data_list

"""
def f3(result2):
    do something 
    return res3
"""

#test_data = open(filename1, 'r').readlines()
## simulate open and readlines
test_data = [ '1\n', '2\n', '3\n', '4\n', '5\n', '6\n']
test_data = f1(test_data)
print "after f1", test_data
test_data = f2(test_data)
print "after f2", test_data

## write file here after all processing has been done

[sofia85]

I think I'm confusing myself in how to name everything in the functions and then be able to use the results in the 'main' program. I have done this so far in pseudocode.

f = open("filename", "r") 
for lines in f.readlines():
get name
	get nr
	f1 = fcn1(name,nr)
	f2 = fcn2(f1) 
	f3 = fcn3(f2)

def fcn1(a_name,a_nr): 
	f = open("filename", "r")
	for lines in f.readlines():
		if ('a_name=' in lines) and ('a_nr' in lines):
			search for it 
			return name, nr

def fcn2(f1): 
	f2 = open("filename", "r")
	for lines in f2.readlines():
                if a_nr in lines:
                        return lines

def fcn3(f2):
        if f2 exists: 
            f3=open(f2, 'r')   #consists of many txt files with m rows and n cols
            for data in f3.readlines():
                  if f2[1] in data:
                     return that column from data

I don't know how to name everything correct and then calling from the functions in the main program on top of the code I just wrote. Alos now I'm opening f two times.

[Gribouillis]

I think I'm confusing myself in how to name everything in the functions and then be able to use the results in the 'main' program. I have done this so far in pseudocode.

f = open("filename", "r") 
for lines in f.readlines():
get name
	get nr
	f1 = fcn1(name,nr)
	f2 = fcn2(f1) 
	f3 = fcn3(f2)

def fcn1(a_name,a_nr): 
	f = open("filename", "r")
	for lines in f.readlines():
		if ('a_name=' in lines) and ('a_nr' in lines):
			search for it 
			return name, nr

def fcn2(f1): 
	f2 = open("filename", "r")
	for lines in f2.readlines():
                if a_nr in lines:
                        return lines

def fcn3(f2):
        if f2 exists: 
            f3=open(f2, 'r')   #consists of many txt files with m rows and n cols
            for data in f3.readlines():
                  if f2[1] in data:
                     return that column from data

I don't know how to name everything correct and then calling from the functions in the main program on top of the code I just wrote. Alos now I'm opening f two times.

Perhaps you could read this tutorial page http://learnpythonthehardway.org/book/ex18.html and the subsequent pages first to understand how functions work.

[sofia85]

I read the tutorial. But I still don't know exactly what arguments my functions have, I know what they should return. For example if fcn1 returns nr and name and its name I want to use in fcn2. How do I accomplish that?

[Gribouillis]

I read the tutorial. But I still don't know exactly what arguments my functions have, I know what they should return. For example if fcn1 returns nr and name and its name I want to use in fcn2. How do I accomplish that?

if fnc1 returns 2 values and you only want to use one of them in the call to fcn2, there are different ways to do that

def fnc1():
    # some code here
    return nr, name

def fnc2(somestring):
    # some code

# first way
foo, bar = fnc1()
fnc2(bar)

# second way
fnc2(fnc1()[1])

# third way
result = fnc1()
fnc2(result[1])