Reading in from a file and rewriting certain columns to a new file

[newyawk]

Hi,

I have a file that has about 65000 lines and three columns. I need to read in the first 2 columns until line 112960. I then need to write the 12960 lines of 2 columns to a file.

This needs to be repeated for about 600 files...

I then need to graph the final file values against the index of the file (the file names go from xxx1.txt to xxx600.txt) as a line graph.

How can I go about doing this? I've been toying with one file, but it doesnt seem to work.

[Beat_Slayer]

How about you post some file and some code, so we can toy with also. ;)

Happy coding!

[newyawk]

This is what I have so far. It only reads in the figures and doesn't print to anything:

Sorry in the OP i mean line 12960. The number i typed is a typo!

import csv

def import_text(filename, separator):
    f=open('./xxx.txt','r')
    f.read()
    while len(f.readlines()) <12960:
        for line in csv.reader(open('./xxx.txt'), delimiter=separator, 
                               skipinitialspace=True):
            
                if line:
                    yield line
f=open('./panelVelocities0.vtk','r')
f.read()             
while len(f.readlines()) <12960:
    for data in import_text('xxx.txt', '/'):
        
            print (data)

[Beat_Slayer]

If you provide a sample file, it can get easier.

But this should do it, if youre csv files are not encoded, the module it's not needed.

f_in = open('xxx.txt').readlines()
f_output= open('output.txt', 'w')

for line in f_in:
    if not line.count('STOP'):
        firstdata, seconddata = line.split('/')[:2]
        f_output.write('%s, %s\n' % (firstdata, seconddata))
    else:
        print line

[newyawk]

Thanks but it doesnt work for my case. Here is the file I need to analyse:

And also, it should stop at line 12960 not 'stop' like i stipulated before. THanks

[newyawk]

Fixed the typo :)

[newyawk]

Oh actually there was a typo, so looks like it is now producing an output, however it reads past the POLYGONS text despite not being supposed to.

The line split should have been line.split (' ') rather than ('/') since the space is the separator.

[Beat_Slayer]

Like this?

f_in = open('xxx.txt').readlines()[:12960]
f_output= open('output.txt', 'w')

for line in f_in:
        firstdata, seconddata = line.split(' ')[:2]
        f_output.write('%s, %s\n' % (firstdata, seconddata))

[newyawk]

Thanks - yes that works better. The only thing is, if you run that script it loads up until point 12785 and not 12960. Any idea why?

And also what does the

'%s, %s\n' %

do?

Thanks for your help, appreciate it.

[Beat_Slayer]

Modulo description

[Beat_Slayer]

I believe this is what you want.

f_in = open('xxx.txt').readlines()[:12959]
f_output= open('output.txt', 'w')

for item in f_in[:5]:
    f_output.write(item)

for item in f_in[5:]:
        firstdata, seconddata = item.split(' ')[:2]
        f_output.write('%s, %s\n' % (firstdata, seconddata))

'%s %s\n' % (firstdata, seconddata)

The '%' places the variables, here firstdata and seconddata, formated as strings due to the 's', in the output string. The '\n' adds a linebreak.

[newyawk]

Thanks. I still can't get it to read to the end? I've now added some code to read in multiple files, but it doesn't seem to append to the file. I am missing some kind of append somewhere, how should I go about it?

import os, glob 
path = './' 
for dir, subdir, files in os.walk(path):
    for file in files:
        if glob.fnmatch.fnmatch(file,"xxx*.txt"): 


            f_input = open('./xxx0.txt').readlines()[:12959] 
f_output= open('output.txt', 'w')								
 
for line in f_input[5:]:
   
        firstdata, seconddata, thirddata = line.split(' ')[:3] 
        f_output.write('%s, %s, %s,\n' % (firstdata, seconddata, thirddata))

I changed it as I do need the 3 columns (stupid me).

[woooee]

I need to read in the first 2 columns until line 12960. I then need to write the 12960 lines of 2 columns to a file.

You can use enumerate to count the records as it appears that a file is shorter than you think or there are blank records.

import os, glob 
def process_file(fname, f_output):
    print "processing", fname
    f_input = open(fname, "r")
								
    for num, rec in enumerate(f_input):
        if (num > 5) and (num < 12961):
            firstdata, seconddata, thirddata = rec.split(' ')[:3]
            f_output.write('%s, %s, %s,\n' % (firstdata, seconddata, thirddata))

    f_input.close()


f_output= open('output.txt','w')
path = './' 
for dir, subdir, files in os.walk(path):
    for fname in files:
        if glob.fnmatch.fnmatch(file,"xxx*.txt"): 
            process_file(os.path.join(dir, fname), f_output)
f_output.close()