CSV Reader Recording Entries

[rhuffman8]

I am working with a CSV file with multiple rows, which I need to be able to access individual data elements from a row so I can assign them to variables and send them to the other program I am working with in a loop, going through each individual row.

I was previously working on a program originally written by another developer which used this

for i, record in enumerate(reader):
        if i > 1:  # dump first two
            blah1, blah2, blah3, blah4, blah5, blah6, blah7, \
                blah8, blah9, blah10, blah11, blah12, blah13, blah14, \
                blah14, blah15, blah16, blah17, blah18= record

to take the rows out and collect the individual data items from each. This all worked fine and I thought my understanding of the concept was enough that I could duplicate it in a similar program I am now writing from scratch.
My code for this section in the new program is as follows:

for i, record in enumerate(reader):
        if i > 0: #skip headers
            date, hk, ba, bsp, hgch, v995, v996, vct = record

When I run the code in cmd, I get an error message that reads

ValueError: unpack list of wrong size

I have looked into this briefly but haven't had much luck in applying solutions to my own problem.

If anyone can offer any insight to either reading csv files or the trouble with my code, that would be greatly appreciated.

[Gribouillis]

The error means that one of the records does not have length 8. The first solution is to print it to see what happens

from itertools import islice

for i, record in islice(enumerate(reader), 1, None):
    if len(record) != 8:
        raise ValueError, "Bad record at line %d : %s" % (i+1, str(record))
    else:
        date, hk, ba, bsp, hgch, v995, v996, vct = record

[woooee]

You should also be able to use the list as is, something like this:

for i, record in enumerate(reader):
    if (i > 0) and (len(record) > 7): #skip headers & record has all the fields
        print "date =", record[0]
        print "hk   =", record[1]
        print "ba   =", record[2]
        ## etc.

[rhuffman8]

I tried both solutions and didn't seem to have a problem with either one when running the commands in a command line but the variables still don't seem to be sending over properly. Perhaps the error is somewhere else in my code?
The function is supposed to take each line the the csv file and first decide if the first entry is a date and if the date is for the present month (these two functions are laid out elsewhere in the python file). Then, it checks if the date of the entry is within the last week. Depending on how far into the past week, it assigns the entries for the line to variables shared with another program and then should go through the same process for all the other rows in the csv file.
Here is the whole function definition:

def send_data():
    for i, record in enumerate(reader):
        if (i > 0) and (len(record) > 7): #skip headers
            date = record[0]
            hk = record[1]
            ba = record[2]
            bsp = record[3]
            hgch = record[4]
            v995 = record[5]
            v996 = record[6]
            vct = record[7]
            month, day, year = [int(x) for x in date.split('/')]
            evt_date = dt.datetime(year, month, day)
            if (event(date)) and (curr_month(date)):
                if evt_date.day >= curr.day - 6:
                    if evt_date.day == curr.day - 6:
                        svars.date1 = date
                        svars.hk1 = hk
                        svars.ba1 = ba
                        svars.bsp1 = bsp
                        svars.hgch1 = hgch
                        svars.v9951 = v995
                        svars.v9961 = v996
                        svars.vct1 = v995 + v996
                    elif evt_date.day == curr.day - 5:
                        svars.date2 = date
                        svars.hk2 = hk
                        svars.ba2 = ba
                        svars.bsp2 = bsp
                        svars.hgch2 = hgch
                        svars.v9952 = v995
                        svars.v9962 = v996
                        svars.vct2 = v995 + v996
                    elif evt_date.day == curr.day - 4:
                        svars.date3 = date
                        svars.hk3 = hk
                        svars.ba3 = ba
                        svars.bsp3 = bsp
                        svars.hgch3 = hgch
                        svars.v9953 = v995
                        svars.v9963 = v996
                        svars.vct3 = v995 + v996
                    elif evt_date.day == curr.day - 3:
                        svars.date4 = date
                        svars.hk4 = hk
                        svars.ba4 = ba
                        svars.bsp4 = bsp
                        svars.hgch4 = hgch
                        svars.v9954 = v995
                        svars.v9964 = v996
                        svars.vct4 = v995 + v996
                    elif evt_date.day == curr.day - 2:
                        svars.date5 = date
                        svars.hk5 = hk
                        svars.ba5 = ba
                        svars.bsp5 = bsp
                        svars.hgch5 = hgch
                        svars.v9955 = v995
                        svars.v9965 = v996
                        svars.vct5 = v995 + v996
                    elif evt_date.day == curr.day - 1:
                        svars.date6 = date
                        svars.hk6 = hk
                        svars.ba6 = ba
                        svars.bsp6 = bsp
                        svars.hgch6 = hgch
                        svars.v9956 = v995
                        svars.v9966 = v996
                        svars.vct6 = v995 + v996
                    elif evt_date.day == curr.day:
                        svars.date7 = date
                        svars.hk7 = hk
                        svars.ba7 = ba
                        svars.bsp7 = bsp
                        svars.hgch7 = hgch
                        svars.v9957 = v995
                        svars.v9967 = v996
                        svars.vct7 = v995 + v996						
                else:
                    pass
            elif not event(date):
                svars.hk_goal = hk
                svars.ba_goal = ba
                svars.bsp_goal = bsp
                svars.hgch_goal = hgch
                svars.v995_goal = v995
                svars.v996_goal = v996
                svars.vct_goal = vct
            else:
                pass

(if there is a more efficient/better way to do any of that, that would be great too...I'm still fairly new to python)

[Gribouillis]

I tried both solutions and didn't seem to have a problem with either one when running the commands in a command line but the variables still don't seem to be sending over properly. Perhaps the error is somewhere else in my code?
The function is supposed to take each line the the csv file and first decide if the first entry is a date and if the date is for the present month (these two functions are laid out elsewhere in the python file). Then, it checks if the date of the entry is within the last week. Depending on how far into the past week, it assigns the entries for the line to variables shared with another program and then should go through the same process for all the other rows in the csv file.
Here is the whole function definition:

def send_data():
    for i, record in enumerate(reader):
        if (i > 0) and (len(record) > 7): #skip headers
            date = record[0]
            hk = record[1]
            ba = record[2]
            bsp = record[3]
            hgch = record[4]
            v995 = record[5]
            v996 = record[6]
            vct = record[7]
            month, day, year = [int(x) for x in date.split('/')]
            evt_date = dt.datetime(year, month, day)
            if (event(date)) and (curr_month(date)):
                if evt_date.day >= curr.day - 6:
                    if evt_date.day == curr.day - 6:
                        svars.date1 = date
                        svars.hk1 = hk
                        svars.ba1 = ba
                        svars.bsp1 = bsp
                        svars.hgch1 = hgch
                        svars.v9951 = v995
                        svars.v9961 = v996
                        svars.vct1 = v995 + v996
                    elif evt_date.day == curr.day - 5:
                        svars.date2 = date
                        svars.hk2 = hk
                        svars.ba2 = ba
                        svars.bsp2 = bsp
                        svars.hgch2 = hgch
                        svars.v9952 = v995
                        svars.v9962 = v996
                        svars.vct2 = v995 + v996
                    elif evt_date.day == curr.day - 4:
                        svars.date3 = date
                        svars.hk3 = hk
                        svars.ba3 = ba
                        svars.bsp3 = bsp
                        svars.hgch3 = hgch
                        svars.v9953 = v995
                        svars.v9963 = v996
                        svars.vct3 = v995 + v996
                    elif evt_date.day == curr.day - 3:
                        svars.date4 = date
                        svars.hk4 = hk
                        svars.ba4 = ba
                        svars.bsp4 = bsp
                        svars.hgch4 = hgch
                        svars.v9954 = v995
                        svars.v9964 = v996
                        svars.vct4 = v995 + v996
                    elif evt_date.day == curr.day - 2:
                        svars.date5 = date
                        svars.hk5 = hk
                        svars.ba5 = ba
                        svars.bsp5 = bsp
                        svars.hgch5 = hgch
                        svars.v9955 = v995
                        svars.v9965 = v996
                        svars.vct5 = v995 + v996
                    elif evt_date.day == curr.day - 1:
                        svars.date6 = date
                        svars.hk6 = hk
                        svars.ba6 = ba
                        svars.bsp6 = bsp
                        svars.hgch6 = hgch
                        svars.v9956 = v995
                        svars.v9966 = v996
                        svars.vct6 = v995 + v996
                    elif evt_date.day == curr.day:
                        svars.date7 = date
                        svars.hk7 = hk
                        svars.ba7 = ba
                        svars.bsp7 = bsp
                        svars.hgch7 = hgch
                        svars.v9957 = v995
                        svars.v9967 = v996
                        svars.vct7 = v995 + v996						
                else:
                    pass
            elif not event(date):
                svars.hk_goal = hk
                svars.ba_goal = ba
                svars.bsp_goal = bsp
                svars.hgch_goal = hgch
                svars.v995_goal = v995
                svars.v996_goal = v996
                svars.vct_goal = vct
            else:
                pass

(if there is a more efficient/better way to do any of that, that would be great too...I'm still fairly new to python)

You can shorten the function like this

def set_fields(svars, suffix, **kwd):
    "helper function for send_data"
    for key, value in kwd.items():
        setattr(svars, key + suffix, value)

def send_data():
    for i, record in enumerate(reader):
        if (i > 0) and (len(record) > 7): #skip headers
            date, hk, ba, bsp, hgch, v995, v996, vct = record[:8]
            month, day, year = [int(x) for x in date.split('/')]
            evt_date = dt.datetime(year, month, day)
            if (event(date)) and (curr_month(date)):
                if evt_date.day >= curr.day - 6:
                    k = curr.day - evt_date.day
                    assert 0 <= k <= 6
                    set_fields(svars, str(7-k),
                        date = date, hk = hk, ba = ba, bsp = bsp, hgch = hgch,
                        v995 = v995, v996 = v996, vct = v995 + v996)
            elif not event(date):
                set_fields(svars, "_goal",
                    hk = hk, ba = ba, bsp = bsp, hgch = hgch,
                    v995 = v995, v996 = v996, vct = vct)

You say that the variables still dont send properly, but you don't describe where the problem is, so it's difficult to help ...

[rhuffman8]

Thank you for the suggestion on shortening the code.
I understand though. I'm having problems figuring out where the problem is exactly. I was just trying to see if anything specific stood out in the code that was obviously wrong. I will keep looking into it and if I find anything specific, I'll post that.

[rhuffman8]

So, after trying a few things, I'm starting to think there is something wrong with my command to open the csv file

reader = csv.reader(open('C:/Documents and Settings/VPC.VPC-SCALA.000/Desktop/RAH 7-11-11/RH/Projected Cash Goals Template.csv', 'rb'))

but from past experience, I cannot figure out what it would be. I commented out the rest of the program so I can go through line by line and figure out what's wrong and put in a statement to test whether things were working or not. The statement does not include the file opened so it should not really be affected but when the test statement is above the open command, everything works fine. When I move it below the open command, the test statement no longer works at all. I get no errors while compiling and when I run the same commands in a Python command line, I can get everything to work just fine. I'm running out of ideas and things to try.
(I'm working in Windows, by the way, in case that makes any difference to anything)

[woooee]

Are there any records in the file? Create a test file so you know what it contains and try it with the program. If the test file works, the problem is the csv file, not the program.

[rhuffman8]

Turned out there was a problem with the file location. Just a big ole' mistake on my part.