I have a problem reading in data from a CSV file using tabarray (see code below) because the CSV file has a incomplete fields in the last record of the file. As a consequence I get the following error:


File "build\bdist.win32\egg\tabular\io.py", line 170, in loadSVcols
assert (Lens == Lens[0]).all() , 'Not all records have same number of fields.'
AssertionError: Not all records have same number of fields

I need to find a way to read in the file while ignoring the last record but I am unable to figure out how to do this using the tabarray method of reading in the file - see below.

x = tb.tabarray(SVfile = 'myfile.txt',delimiter = ',', headerlines=2,usecols=[1,2,33,35,37], skiprows=324)

Please let me know how I can work around this problem?

Many thanks, Roger

You might have to preprocess the file first.

Thanks for the pointer - I took your advice and have gone about finding the solution below for anyone else who may find it useful.

fin = open( 'myfile.txt', "r" )
data_list = fin.readlines()
fin.close()
temp_data=data_list[0:len(data_list)-2]

# write the changed data (list) to a file
fout = open("temp_data.txt", "w")
fout.writelines(temp_data)
fout.close()

x = tb.tabarray(SVfile = 'temp_data.txt',delimiter = ',', headerlines=2,usecols=[1,2,33,35,37], skiprows=324)

Thanks, Roger

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.