Process text file with parameters

Please support our Python advertiser: Programming Forums - DaniWeb Sister Site
Reply

Join Date: Mar 2009
Posts: 2
Reputation: ask2 is an unknown quantity at this point 
Solved Threads: 0
ask2 ask2 is offline Offline
Newbie Poster

Process text file with parameters

 
0
  #1
Mar 16th, 2009
Hi

I am trying to process a large text file. I would like to include parameters at some places in the file. Run it through python and write out the "processed" file with the values of the parameters in the new file.

Example: original file:
a =1
b=2
c= param1
d=4

Run python with param1=3: Gives new file with
a=1
b=2
c=3
d=4

---
Is this possible? The original text file will be around 100Mb and have maybe 100 parameters.

Any ideas how to do this in python?

Best regards
Joakim
Reply With Quote Quick reply to this message  
Join Date: Jun 2008
Posts: 128
Reputation: slate is an unknown quantity at this point 
Solved Threads: 31
slate slate is offline Offline
Junior Poster

Re: Process text file with parameters

 
1
  #2
Mar 16th, 2009
If the input is a "text" file and the replacement values are encoded the same way and has "reasonably" short lines, then the following will suffice.

  1. params={'param1':1, 'param2':2}
  2. fi=open("a.txt")
  3. fo=open("a_processed.txt","w")
  4. for line in fi:
  5. for k,v in params.iteritems():
  6. fo.write(line.replace(k,str(v)))
  7. fo.close()
  8. fi.close()


You can also store the params in a separate file and load it before processing.
Reply With Quote Quick reply to this message  
Join Date: Jun 2008
Posts: 128
Reputation: slate is an unknown quantity at this point 
Solved Threads: 31
slate slate is offline Offline
Junior Poster

Re: Process text file with parameters

 
0
  #3
Mar 16th, 2009
For a more general approach see the KMP algo:
http://code.activestate.com/recipes/117214/
Reply With Quote Quick reply to this message  
Join Date: Dec 2006
Posts: 1,065
Reputation: woooee is a jewel in the rough woooee is a jewel in the rough woooee is a jewel in the rough 
Solved Threads: 299
woooee woooee is offline Offline
Veteran Poster

Re: Process text file with parameters

 
0
  #4
Mar 16th, 2009
The general idea is to store the replacement values in a container like a dictionary, isolate the string that you want to test, and replace it if it is in the replacement dictionary. As stated above you could store them in a separate text file and read into a dictionary. The following code uses a list for input and output to simulate files, but gives the general idea.
  1. class Param(object):
  2. def __init__(self):
  3. ## parameter dictionary with the key=string to search for,
  4. ## and value=replacement value
  5. self.params_dic={"param1":"3",
  6. "param2":"2",
  7. "param3":"1" }
  8.  
  9.  
  10. def process_file(self, data_in):
  11. output_list = []
  12. for rec in data_in:
  13. r_list = rec.split("=")
  14. test_s = r_list[1].strip()
  15. if test_s in self.params_dic: ## found
  16. output_list.append("%s=%s\n" % \
  17. (r_list[0], self.params_dic[test_s]))
  18. else: ## not found
  19. output_list.append(rec)
  20.  
  21. print "final output ="
  22. for rec in output_list:
  23. print rec,
  24.  
  25. P = Param()
  26. input_file_list = ['a =1\n', 'b=2\n', 'c= param1\n', 'd=4\n', 'e=param3\n']
  27. P.process_file(input_file_list)
Reply With Quote Quick reply to this message  
Join Date: Dec 2006
Posts: 1,065
Reputation: woooee is a jewel in the rough woooee is a jewel in the rough woooee is a jewel in the rough 
Solved Threads: 299
woooee woooee is offline Offline
Veteran Poster

Re: Process text file with parameters

 
0
  #5
Mar 18th, 2009
Is this possible? The original text file will be around 100Mb and have maybe 100 parameters.
Were you able to do this with the suggestions posted?
Reply With Quote Quick reply to this message  
Join Date: Mar 2009
Posts: 2
Reputation: ask2 is an unknown quantity at this point 
Solved Threads: 0
ask2 ask2 is offline Offline
Newbie Poster

Re: Process text file with parameters

 
0
  #6
Mar 18th, 2009
Originally Posted by woooee View Post
Were you able to do this with the suggestions posted?
Hello

The suggested method works fine. I haven't been able to try it on a "production" size file yet. So I am not sure how long it will take. But on smaller files it works perfectly.

Thanks to all of you
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:



Other Threads in the Python Forum


Views: 514 | Replies: 5
Thread Tools Search this Thread



Tag cloud for Python
About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC