Process text file with parameters

[ask2]

Hi

I am trying to process a large text file. I would like to include parameters at some places in the file. Run it through python and write out the "processed" file with the values of the parameters in the new file.

Example: original file:
a =1
b=2
c= param1
d=4

Run python with param1=3: Gives new file with
a=1
b=2
c=3
d=4

---
Is this possible? The original text file will be around 100Mb and have maybe 100 parameters.

Any ideas how to do this in python?

Best regards
Joakim

[slate]

If the input is a "text" file and the replacement values are encoded the same way and has "reasonably" short lines, then the following will suffice.

params={'param1':1, 'param2':2}
fi=open("a.txt")
fo=open("a_processed.txt","w")
for line in fi:
    for k,v in params.iteritems():
        fo.write(line.replace(k,str(v)))
fo.close()
fi.close()

You can also store the params in a separate file and load it before processing.

[slate]

For a more general approach see the KMP algo:
http://code.activestate.com/recipes/117214/

[woooee]

The general idea is to store the replacement values in a container like a dictionary, isolate the string that you want to test, and replace it if it is in the replacement dictionary. As stated above you could store them in a separate text file and read into a dictionary. The following code uses a list for input and output to simulate files, but gives the general idea.

class Param(object):
   def __init__(self):
      ##   parameter dictionary with the key=string to search for,
      ##   and value=replacement value
      self.params_dic={"param1":"3",
                       "param2":"2",
                       "param3":"1" } 


   def process_file(self, data_in):   
      output_list = []
      for rec in data_in:
         r_list = rec.split("=")
         test_s = r_list[1].strip()
         if test_s in self.params_dic:     ## found
            output_list.append("%s=%s\n" % \
                                        (r_list[0], self.params_dic[test_s]))
         else:                               ## not found
            output_list.append(rec)

      print "final output ="
      for rec in output_list:
         print rec,

P = Param()
input_file_list = ['a =1\n', 'b=2\n', 'c= param1\n', 'd=4\n', 'e=param3\n']
P.process_file(input_file_list)

[woooee]

Is this possible? The original text file will be around 100Mb and have maybe 100 parameters.

Were you able to do this with the suggestions posted?

[ask2]

Were you able to do this with the suggestions posted?

Hello

The suggested method works fine. I haven't been able to try it on a "production" size file yet. So I am not sure how long it will take. But on smaller files it works perfectly.

Thanks to all of you