CSV DictWriter

[Jerix]

Hey, I'm trying to write a program right now with the following requirements:

1. Read a CSV file for field ID#.
2. Compare it to a second CSV file for the same ID #
3. If Csv1ID == Csv2ID, write the rows from both files to outputfile.

So basically I am merging the contents of 2 svc files based on an ID#. The problem I'm having is that I'm can't get the DictWriter to work. Here's my code so far:

import csv

fileIn = open('C:\DataFiles\Integrations\csvfiles\customer\c3.csv', 'r')
sffileIn = open('C:\DataFiles\Integrations\csvfiles\salesforce\sf2.csv', 'r')
fileOut = open('C:\DataFiles\Integrations\csvfiles\customer\Completedcsv\output.csv', 'w')

reader = csv.DictReader(fileIn, delimiter = ",")
sfreader = csv.DictReader(sffileIn, delimiter = ",")
output = csv.DictWriter(fileOut, fieldnames = reader.fieldnames, delimiter = ",", extrasaction = 'ignore', dialect = 'excel')

for row in reader:
    output.writerows(row)
    print row

Right now I'm just trying to test the writer with the bottom code, but I'm getting this garbled mess:

c,o,l,l,e,c,t,o,r,_,e,m,a,i,l c,o,l,l,e,c,t,o,r a,p,h,o,n,e c,o,l,l,e,c,t,o,r,_,f,a,x b,p,h,o,n,e b,e,m,a,i,l b,s,t,a,t,e a,a,t,t,e,n,t,i,o,n a,d,d,r,e,s,s,n,a,m,e a,s,t,a,t,e a,a,d,d,r,1 a,c,t,i,v,a,t,e,d a,c,i,t,y b,a,d,d,r,1 b,n,a,m,e c,u,s,t,o,m,e,r,i,d b,c,i,t,y i,n,v,o,i,c,e,t,e,r,m,s i,n,v,o,i,c,e,l,a,t,e,f,e,e,p,c,t a,e,m,a,i,l c,u,s,t,o,m,e,r,n,a,m,e b,a,d,d,r,2 a,z,i,p b,a,t,t,e,n,t,i,o,n s,u,b,m,i,t,d,a,t,e s,a,l,e,s,r,e,p a,a,d,d,r,2 b,z,i,p c,o,l,l,e,c,t,o,r,_,e,m,a,i,l c,o,l,l,e,c,t,o,r a,p,h,o,n,e c,o,l,l,e,c,t,o,r,_,f,a,x b,p,h,o,n,e b,e,m,a,i,l b,s,t,a,t,e a,a,t,t,e,n,t,i,o,n a,d,d,r,e,s,s,n,a,m,e a,s,t,a,t,e a,a,d,d,r,1 a,c,t,i,v,a,t,e,d a,c,i,t,y b,a,d,d,r,1 b,n,a,m,e c,u,s,t,o,m,e,r,i,d b,c,i,t,y i,n,v,o,i,c,e,t,e,r,m,s i,n,v,o,i,c,e,l,a,t,e,f,e,e,p,c,t a,e,m,a,i,l c,u,s,t,o,m,e,r,n,a,m,e b,a,d,d,r,2 a,z,i,p b,a,t,t,e,n,t,i,o,n s,u,b,m,i,t,d,a,t,e s,a,l,e,s,r,e,p a,a,d,d,r,2 b,z,i,p

Not only does it put the commas inbetween every letter of my fields, but it puts them in a completely random order that I can't figure out. It then just repeats this over and over for each row without writing the rows. Oddly, printing the rows has the proper values, but in a random order as well. I've tried playing with the parameters to no avail. Any idea on why it's doing this? Thanks.

[Jerix]

bump. Anyone?

[TrustyTony]

Difficult to help as you give only scrambled data.

I do not know if I recovered the data scrambled correctly. What you would like to ask?

>>> fileOut = open('output.csv', 'w')
>>> import csv
>>> for row in data.items():
	print row

	
('customername', 'bzip')
('aaddr2', 'bzip')
('invoicelatefeepct', 'bzip')
('aaddr1', 'bzip')
('aphone', 'bzip')
('bcity', 'bzip')
('acity', 'bzip')
('bname', 'bzip')
('bphone', 'bzip')
('submitdate', 'bzip')
('collector_email', 'bzip')
('azip', 'bzip')
('addressname', 'bzip')
('bstate', 'bzip')
>>> fileOut = open('output.csv', 'w')
>>> output = csv.DictWriter(fileOut, fieldnames = data.keys(), delimiter = ",", extrasaction = 'ignore', dialect = 'excel', lineterminator = '\n')
>>> output.writerows((dict(zip(data.keys(),data.keys())), data))
>>> fileOut.close()
>>> print(open('output.csv').read())
customername,aaddr2,invoicelatefeepct,aaddr1,aphone,bcity,acity,bname,bphone,submitdate,collector_email,azip,addressname,bstate
bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip

>>> output = csv.DictWriter(fileOut, fieldnames = sorted(data.keys()), delimiter = ",", extrasaction = 'ignore', dialect = 'excel', lineterminator = '\n')
>>> output.writerows((dict(zip(data.keys(),data.keys())), data))
>>> fileOut.close()
>>> print(open('output.csv').read())
aaddr1,aaddr2,acity,addressname,aphone,azip,bcity,bname,bphone,bstate,collector_email,customername,invoicelatefeepct,submitdate
bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip,bzip

>>>

[Jerix]

Here are the fields that get scrambled with my current code:

customerid,customername,salesrep,addressname,aattention,aaddr1,aaddr2,acity,astate,azip,aphone,aemail,bname,battention,baddr1,baddr2,bcity,bstate,bzip,bphone,bemail,invoiceterms,invoicelatefeepct,activated,submitdate,collector,collector_fax,collector_fax,collector_email

I'm not sure where I'm going wrong on the DicWriter, but when I got to print to my output file it takes the fields and gives out that garbled mess instead of printing out the rows properly. If I don't use the DictWriter and just do a straight print of the rows and field names, it will print the rows properly, but they're not in the order they should be. I'm not really sure why this is happening so I was wondering if maybe I did something wrong in setting up my code. Let me know if you need more info. Thanks.

[TrustyTony]

zip file attached (Advanced Editor -> Manage Attachments) with the data files in question would increase much your chance of getting help.