Combining txt from 2 files

[grambo]

Alright this is my first program in python, so be gentle. I have 2 files I need to basically combine to create a record in mysql. Features.txt has around 1300 records and property.txt has about 25,000 records. I can read in property.txt fine and get my insert statements. My problem is trying to read in features.txt. I only need the lines that have field 1 = RES. I was thinking a dictionary ('1701' : 'Central Air', '1702' : 'Window Unit", etc) but can seem to quite get it.

for featureline in ffeaturefile.readlines():
        featureline = featureline.strip().split("|")
        addlist += "'%s', '%s'," % (featureline[0], featureline[3])
        resfeaturelist = dict(addlist)

features.txt

1701|RES|HVAC|Central Air
1702|RES|HVAC|Window Unit
1703|RES|HVAC|Heat Pump - AC|
1704|RES|HVAC|No A/C
1705|RES|HVAC|Wall Unit - AC
1706|RES|HVAC|Multizone A/C
1708|RES|HVAC|Gas Hot Air
1701|CND|HVAC|Central Air
...

property.txt (there are a 105 fields so I shortened it here)
0.293|53548|47459||4|101|QUART|2|4||Matthews||1505,1510,1514||1701,1708,1722|
...

I'm inserting into a mysql database here is what I have right now:

Insert into openrealty_en_listingsdbelements (listingdbelements_field_name,listingdbelements_field_value,listingsdb_id,userdb_id) values ('HVAC','1701,1708','2','0');

but I need

Insert into openrealty_en_listingsdbelements (listingdbelements_field_name,listingdbelements_field_value,listingsdb_id,userdb_id) values ('HVAC','Central Air,Gas Hot Air','2','0');

for number, resline in enumerate(fresfile):
        resline = resline.strip().split("|")

        for count in [0, 21]:
                if (count == 21):
                       #change values to text in features.txt
                        resfields = resline[count].split(",")
                        for i, item in enumerate(resfields):
                                if (featureline[0] == item):
                                        print resfields[3]
                else:
                        fsqlfile.write ("Insert into openrealty_en_listingsdbelements (listingdbelements_field_name,listingdbelements_field_value,listingsdb_id,userdb_id) values ('" + resheader[count] + "','" + resline[count] + "','" + str(number) + "','0');\n")

Also, once I get the features.txt into a dictionary how do I do the substitution?

[jlm699]

for featureline in ffeaturefile.readlines():
        featureline = featureline.strip().split("|")
        addlist += "'%s', '%s'," % (featureline[0], featureline[3])
        resfeaturelist = dict(addlist)

for number, resline in enumerate(fresfile):
        resline = resline.strip().split("|")

        for count in [0, 21]:
                if (count == 21):
                       #change values to text in features.txt
                        resfields = resline[count].split(",")
                        for i, item in enumerate(resfields):
                                if (featureline[0] == item):
                                        print resfields[3]
                else:
                        fsqlfile.write ("Insert into openrealty_en_listingsdbelements (listingdbelements_field_name,listingdbelements_field_value,listingsdb_id,userdb_id) values ('" + resheader[count] + "','" + resline[count] + "','" + str(number) + "','0');\n")

Also, once I get the features.txt into a dictionary how do I do the substitution?

I'm not quite sure what you mean by substituion.. do you mean use the value instead of the key? Also, I'm wary of how you are creating your dictionary.

I propose the following modification to the first bit of code:

resfeaturelist = {}
for featureline in ffeaturefile.readlines():
        featureline = featureline.strip().split("|")
        addlist += "'%s', '%s'," % (featureline[0], featureline[3])
        resfeaturelist[ featureline[0] ] = featureline[3]

Now take a look at this example of working with a dictionary to see if it answers your question about "substitution":

>>> dd = {} # Initialize the dictionary
>>> dd[ '7801' ] = 'Heat Pump'
>>> dd[ '7902' ] = 'Central A/C'
>>> dd[ '8901' ] = 'Box Fan'
>>> print dd[ '7801' ]
Heat Pump
>>> "INSERT x into y VALUES( '%s', '%s', '0' );" % ( dd['7801'], dd['8901'] )
"INSERT x into y VALUES( 'Heat Pump', 'Box Fan', '0' );"
>>> dd.get( '7901' )
>>> dd.get( '7902' )
'Central A/C'
>>>

As you can see there's two different methods to retrieving values from a dictionary. Using get() is safer since you can check if <value> == None: so that you don't get any code-stopping exceptions; however in your case you may want the code to stop.

Let me know whether this clears up your questions, and whether that first bit helped to alleviate your problems..

[grambo]

That's perfect. I think I was originally just creating a list. I could find anything good for docs on dictionarys (or python for that matter) online. The dictionary is great, and I can see how to use it on one item. How can I do it for multiple items? Also, 1 record might have 1 and another might have 10.

my code right now is just

if (count == 21):
 resfields = resline[count].split(",")
 print resfields

Which prints out:
['1701', '1708', '1722']
['1701', '1708']
['1701', '1708']
['1701', '1708', '1706']

which I need to transform into:
['Central Air', 'Gas Hot Air', 'Woodstove']
['Central Air', 'Gas Hot Air']
['Central Air', 'Gas Hot Air']
['Central Air', 'Gas Hot Air', 'Multizone A/C']

I'm sure it simple. I tried just

if (count == 21):
  resfields = resfeaturelist [resline[count].split(",")]
  print resfields

Which just gives me an error.

Do you know of a good syntax reference?

Thanks!
Chris

[solsteel]

If I understand you correctly, you want to substitute the "VALUE" of a residence feature which would be a "KEY" in a dictionary. For each different value, there will be a different key. A dictionary is perfect for this. Example:

# features = open('features.txt').read()

features = '''1701|RES|HVAC|Central Air
1702|RES|HVAC|Window Unit
1703|RES|HVAC|Heat Pump - AC|
1704|RES|HVAC|No A/C
1705|RES|HVAC|Wall Unit - AC
1706|RES|HVAC|Multizone A/C
1708|RES|HVAC|Gas Hot Air
1722|RES|HVAC|Woodstove
1701|CND|HVAC|Central Air'''

featuresDict = {}
for item in features.split('\n'):
    if 'RES' in item:
        items = item.split('|')
        featuresDict[items[0]] = items[3]

resfield = ['1701','1706','1722']

# as a function, retain original list
def sub_features(s):
    return [featuresDict[item] for item in s[:]]

print sub_features(resfield)

# OR
# modify original list
for i, item in enumerate(resfield):
    resfield[i] = featuresDict[item]
    
print resfield

Both print:
>>> ['Central Air', 'Multizone A/C', 'Woodstove']

[grambo]

That's what I needed. Thanks!!!!

[jlm699]

Do you know of a good syntax reference?

Here's a good one: Python docs

Also, in the future it helps if you post the traceback of your error message instead of just saying "this gives me an error". It'll help the community debug your problem ! ;)