Combining txt from 2 files

Please support our Python advertiser: Programming Forums - DaniWeb Sister Site
Thread Solved

•
•
Join Date: Nov 2008
Posts: 4
Reputation: grambo is an unknown quantity at this point 
Solved Threads: 0
grambo grambo is offline Offline
Newbie Poster

Combining txt from 2 files

 
0
  #1
Nov 10th, 2008
Alright this is my first program in python, so be gentle. I have 2 files I need to basically combine to create a record in mysql. Features.txt has around 1300 records and property.txt has about 25,000 records. I can read in property.txt fine and get my insert statements. My problem is trying to read in features.txt. I only need the lines that have field 1 = RES. I was thinking a dictionary ('1701' : 'Central Air', '1702' : 'Window Unit", etc) but can seem to quite get it.

python Syntax (Toggle Plain Text) 
  1. for featureline in ffeaturefile.readlines():
  2.         featureline = featureline.strip().split("|")
  3.         addlist += "'%s', '%s'," % (featureline[0], featureline[3])
  4.         resfeaturelist = dict(addlist)

Python Syntax (Toggle Plain Text) 
  1. features.txt
  2.  
  3. 1701|RES|HVAC|Central Air
  4. 1702|RES|HVAC|Window Unit
  5. 1703|RES|HVAC|Heat Pump - AC|
  6. 1704|RES|HVAC|No A/C
  7. 1705|RES|HVAC|Wall Unit - AC
  8. 1706|RES|HVAC|Multizone A/C
  9. 1708|RES|HVAC|Gas Hot Air
  10. 1701|CND|HVAC|Central Air
  11. ...
  12.  
  13. property.txt (there are a 105 fields so I shortened it here)
  14. 0.293|53548|47459||4|101|QUART|2|4||Matthews||1505,1510,1514||1701,1708,1722|
  15. ...

I'm inserting into a mysql database here is what I have right now:

Python Syntax (Toggle Plain Text) 
  1. Insert into openrealty_en_listingsdbelements (listingdbelements_field_name,listingdbelements_field_value,listingsdb_id,userdb_id) values ('HVAC','1701,1708','2','0');

but I need

Python Syntax (Toggle Plain Text) 
  1. Insert into openrealty_en_listingsdbelements (listingdbelements_field_name,listingdbelements_field_value,listingsdb_id,userdb_id) values ('HVAC','Central Air,Gas Hot Air','2','0');

python Syntax (Toggle Plain Text) 
  1. for number, resline in enumerate(fresfile):
  2.         resline = resline.strip().split("|")
  3.  
  4.         for count in [0, 21]:
  5.                 if (count == 21):
  6.                        #change values to text in features.txt
  7.                         resfields = resline[count].split(",")
  8.                         for i, item in enumerate(resfields):
  9.                                 if (featureline[0] == item):
  10.                                         print resfields[3]
  11.                 else:
  12.                         fsqlfile.write ("Insert into openrealty_en_listingsdbelements (listingdbelements_field_name,listingdbelements_field_value,listingsdb_id,userdb_id) values ('" + resheader[count] + "','" + resline[count] + "','" + str(number) + "','0');\n")

Also, once I get the features.txt into a dictionary how do I do the substitution?
Reply With Quote Quick reply to this message  
•
•
Join Date: Jul 2008
Posts: 1,062
Reputation: jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough 
Solved Threads: 267
Sponsor
jlm699's Avatar
jlm699 jlm699 is offline Offline
Knows where his Towel is

Re: Combining txt from 2 files

 
0
  #2
Nov 10th, 2008
•
•
•
•
Originally Posted by grambo View Post
python Syntax (Toggle Plain Text) 
  1. for featureline in ffeaturefile.readlines():
  2.         featureline = featureline.strip().split("|")
  3.         addlist += "'%s', '%s'," % (featureline[0], featureline[3])
  4.         resfeaturelist = dict(addlist)

python Syntax (Toggle Plain Text) 
  1. for number, resline in enumerate(fresfile):
  2.         resline = resline.strip().split("|")
  3.  
  4.         for count in [0, 21]:
  5.                 if (count == 21):
  6.                        #change values to text in features.txt
  7.                         resfields = resline[count].split(",")
  8.                         for i, item in enumerate(resfields):
  9.                                 if (featureline[0] == item):
  10.                                         print resfields[3]
  11.                 else:
  12.                         fsqlfile.write ("Insert into openrealty_en_listingsdbelements (listingdbelements_field_name,listingdbelements_field_value,listingsdb_id,userdb_id) values ('" + resheader[count] + "','" + resline[count] + "','" + str(number) + "','0');\n")

Also, once I get the features.txt into a dictionary how do I do the substitution?
I'm not quite sure what you mean by substituion.. do you mean use the value instead of the key? Also, I'm wary of how you are creating your dictionary.

I propose the following modification to the first bit of code:
python Syntax (Toggle Plain Text) 
  1. resfeaturelist = {}
  2. for featureline in ffeaturefile.readlines():
  3.         featureline = featureline.strip().split("|")
  4.         addlist += "'%s', '%s'," % (featureline[0], featureline[3])
  5.         resfeaturelist[ featureline[0] ] = featureline[3]
Now take a look at this example of working with a dictionary to see if it answers your question about "substitution":
python Syntax (Toggle Plain Text) 
  1. >>> dd = {} # Initialize the dictionary
  2. >>> dd[ '7801' ] = 'Heat Pump'
  3. >>> dd[ '7902' ] = 'Central A/C'
  4. >>> dd[ '8901' ] = 'Box Fan'
  5. >>> print dd[ '7801' ]
  6. Heat Pump
  7. >>> "INSERT x into y VALUES( '%s', '%s', '0' );" % ( dd['7801'], dd['8901'] )
  8. "INSERT x into y VALUES( 'Heat Pump', 'Box Fan', '0' );"
  9. >>> dd.get( '7901' )
  10. >>> dd.get( '7902' )
  11. 'Central A/C'
  12. >>>
As you can see there's two different methods to retrieving values from a dictionary. Using get() is safer since you can check if <value> == None: so that you don't get any code-stopping exceptions; however in your case you may want the code to stop.

Let me know whether this clears up your questions, and whether that first bit helped to alleviate your problems..
Last edited by jlm699; Nov 10th, 2008 at 10:47 am.
1. Use Code Tags.
2. Homework? Show Effort.
3. Keep discussions on the forum: no PMs
Reply With Quote Quick reply to this message  
•
•
Join Date: Nov 2008
Posts: 4
Reputation: grambo is an unknown quantity at this point 
Solved Threads: 0
grambo grambo is offline Offline
Newbie Poster

Re: Combining txt from 2 files

 
0
  #3
Nov 10th, 2008
That's perfect. I think I was originally just creating a list. I could find anything good for docs on dictionarys (or python for that matter) online. The dictionary is great, and I can see how to use it on one item. How can I do it for multiple items? Also, 1 record might have 1 and another might have 10.

my code right now is just

python Syntax (Toggle Plain Text) 
  1. if (count == 21):
  2.  resfields = resline[count].split(",")
  3.  print resfields

Which prints out:
['1701', '1708', '1722']
['1701', '1708']
['1701', '1708']
['1701', '1708', '1706']

which I need to transform into:
['Central Air', 'Gas Hot Air', 'Woodstove']
['Central Air', 'Gas Hot Air']
['Central Air', 'Gas Hot Air']
['Central Air', 'Gas Hot Air', 'Multizone A/C']

I'm sure it simple. I tried just
python Syntax (Toggle Plain Text) 
  1. if (count == 21):
  2.   resfields = resfeaturelist [resline[count].split(",")]
  3.   print resfields
Which just gives me an error.

Do you know of a good syntax reference?

Thanks!
Chris
Reply With Quote Quick reply to this message  
•
•
Join Date: Mar 2007
Posts: 110
Reputation: solsteel is on a distinguished road 
Solved Threads: 31
solsteel solsteel is offline Offline
Junior Poster

Re: Combining txt from 2 files

 
0
  #4
Nov 10th, 2008
If I understand you correctly, you want to substitute the "VALUE" of a residence feature which would be a "KEY" in a dictionary. For each different value, there will be a different key. A dictionary is perfect for this. Example:
Python Syntax (Toggle Plain Text) 
  1. # features = open('features.txt').read()
  2.  
  3. features = '''1701|RES|HVAC|Central Air
  4. 1702|RES|HVAC|Window Unit
  5. 1703|RES|HVAC|Heat Pump - AC|
  6. 1704|RES|HVAC|No A/C
  7. 1705|RES|HVAC|Wall Unit - AC
  8. 1706|RES|HVAC|Multizone A/C
  9. 1708|RES|HVAC|Gas Hot Air
  10. 1722|RES|HVAC|Woodstove
  11. 1701|CND|HVAC|Central Air'''
  12.  
  13. featuresDict = {}
  14. for item in features.split('\n'):
  15.     if 'RES' in item:
  16.         items = item.split('|')
  17.         featuresDict[items[0]] = items[3]
  18.  
  19. resfield = ['1701','1706','1722']
  20.  
  21. # as a function, retain original list
  22. def sub_features(s):
  23.     return [featuresDict[item] for item in s[:]]
  24.  
  25. print sub_features(resfield)
  26.  
  27. # OR
  28. # modify original list
  29. for i, item in enumerate(resfield):
  30.     resfield[i] = featuresDict[item]
  31.  
  32. print resfield
Both print:
>>> ['Central Air', 'Multizone A/C', 'Woodstove']
Reply With Quote Quick reply to this message  
•
•
Join Date: Nov 2008
Posts: 4
Reputation: grambo is an unknown quantity at this point 
Solved Threads: 0
grambo grambo is offline Offline
Newbie Poster

Re: Combining txt from 2 files

 
0
  #5
Nov 10th, 2008
That's what I needed. Thanks!!!!
Reply With Quote Quick reply to this message  
•
•
Join Date: Jul 2008
Posts: 1,062
Reputation: jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough jlm699 is a jewel in the rough 
Solved Threads: 267
Sponsor
jlm699's Avatar
jlm699 jlm699 is offline Offline
Knows where his Towel is

Re: Combining txt from 2 files

 
0
  #6
Nov 11th, 2008
•
•
•
•
Originally Posted by grambo View Post
Do you know of a good syntax reference?
Here's a good one: Python docs

Also, in the future it helps if you post the traceback of your error message instead of just saying "this gives me an error". It'll help the community debug your problem !
1. Use Code Tags.
2. Homework? Show Effort.
3. Keep discussions on the forum: no PMs
Reply With Quote Quick reply to this message  
Reply

Quick Reply to Thread
This thread has been marked solved.
Perhaps start a new thread instead?
•
•
•
•
•
•
•
•
Message:
•
•
•
•



Similar Threads
Other Threads in the Python Forum
Thread Tools Search this Thread



abrupt ansi anti approximation assignment avogadro backend beginner binary bluetooth calculator character cmd code customdialog decimals dictionaries dictionary directory drive dynamic error examples excel exe file float format function gnu graphics gui heads homework http ideas import input java launcher leftmouse line linux list lists logging loop module mouse number numbers output parsing path pointer port prime programming progressbar projects push py2exe pygame pyqt python random recursion schedule scrolledtext sqlite statistics stdout string strings sudokusolver sum table terminal text thread threading time tkinter tlapse tricks tuple tutorial twoup ubuntu unicode update urllib urllib2 variable ventrilo wikipedia windows write wxpython xlib
About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC