tabarray input error: incomplete records

[RogerI]

I have a problem reading in data from a CSV file using tabarray (see code below) because the CSV file has a incomplete fields in the last record of the file. As a consequence I get the following error:

File "build\bdist.win32\egg\tabular\io.py", line 170, in loadSVcols
assert (Lens == Lens[0]).all() , 'Not all records have same number of fields.'
AssertionError: Not all records have same number of fields

I need to find a way to read in the file while ignoring the last record but I am unable to figure out how to do this using the tabarray method of reading in the file - see below.

x = tb.tabarray(SVfile = 'myfile.txt',delimiter = ',', headerlines=2,usecols=[1,2,33,35,37], skiprows=324)

Please let me know how I can work around this problem?

Many thanks, Roger

[sneekula]

You might have to preprocess the file first.

[RogerI]

Thanks for the pointer - I took your advice and have gone about finding the solution below for anyone else who may find it useful.

fin = open( 'myfile.txt', "r" )
data_list = fin.readlines()
fin.close()
temp_data=data_list[0:len(data_list)-2]

# write the changed data (list) to a file
fout = open("temp_data.txt", "w")
fout.writelines(temp_data)
fout.close()

x = tb.tabarray(SVfile = 'temp_data.txt',delimiter = ',', headerlines=2,usecols=[1,2,33,35,37], skiprows=324)

Thanks, Roger