Hi,

I have a file that has about 65000 lines and three columns. I need to read in the first 2 columns until line 112960. I then need to write the 12960 lines of 2 columns to a file.

This needs to be repeated for about 600 files...

I then need to graph the final file values against the index of the file (the file names go from xxx1.txt to xxx600.txt) as a line graph.

How can I go about doing this? I've been toying with one file, but it doesnt seem to work.

Edited 6 Years Ago by newyawk: n/a

This is what I have so far. It only reads in the figures and doesn't print to anything:

Sorry in the OP i mean line 12960. The number i typed is a typo!

import csv

def import_text(filename, separator):
    f=open('./xxx.txt','r')
    f.read()
    while len(f.readlines()) <12960:
        for line in csv.reader(open('./xxx.txt'), delimiter=separator, 
                               skipinitialspace=True):
            
                if line:
                    yield line
f=open('./panelVelocities0.vtk','r')
f.read()             
while len(f.readlines()) <12960:
    for data in import_text('xxx.txt', '/'):
        
            print (data)

Edited 6 Years Ago by newyawk: n/a

If you provide a sample file, it can get easier.

But this should do it, if youre csv files are not encoded, the module it's not needed.

f_in = open('xxx.txt').readlines()
f_output= open('output.txt', 'w')

for line in f_in:
    if not line.count('STOP'):
        firstdata, seconddata = line.split('/')[:2]
        f_output.write('%s, %s\n' % (firstdata, seconddata))
    else:
        print line

Edited 6 Years Ago by Beat_Slayer: n/a

Thanks but it doesnt work for my case. Here is the file I need to analyse:

And also, it should stop at line 12960 not 'stop' like i stipulated before. THanks

Edited 6 Years Ago by newyawk: n/a

Attachments
# vtk DataFile Version 2.0 
file output from bem-oneSurface 
ASCII 
DATASET POLYDATA
POINTS 12954 float
-1.805265e-05 5.257664e-01 8.506007e-01 
-1.830841e-05 5.257844e-01 -8.506034e-01 
-1.659241e-05 -5.256944e-01 -8.506693e-01 
-1.677408e-05 -5.256938e-01 8.506575e-01 
8.505282e-01 9.059150e-05 5.257548e-01 
-8.505317e-01 9.129815e-05 5.257577e-01 
-8.505386e-01 1.019503e-04 -5.257452e-01 
8.505344e-01 1.019909e-04 -5.257421e-01 
5.258210e-01 8.504938e-01 -2.229533e-05 
5.259377e-01 -8.505653e-01 -1.725054e-05 
-5.259331e-01 -8.505799e-01 -1.712666e-05 
-5.258237e-01 8.505027e-01 -2.260152e-05 
-5.000253e-01 -3.089744e-01 8.088108e-01 
-4.999610e-01 3.090219e-01 8.087709e-01 
-6.983490e-06 1.057554e-04 9.997301e-01 
5.000171e-01 -3.089718e-01 8.088024e-01 
4.999585e-01 3.090131e-01 8.087640e-01 
5.000234e-01 -3.089570e-01 -8.088069e-01 
4.999558e-01 3.090191e-01 -8.087682e-01 
-6.951536e-06 1.168645e-04 -9.997343e-01 
-5.000318e-01 -3.089599e-01 -8.088161e-01 
-4.999586e-01 3.090274e-01 -8.087758e-01 
-8.088208e-01 5.001633e-01 3.091205e-01 
-3.090501e-01 8.087856e-01 4.999749e-01 
1.081561e-05 9.993485e-01 -1.866793e-05 
3.090671e-01 8.087742e-01 4.999667e-01 
8.088105e-01 5.001634e-01 3.091223e-01 
8.087758e-01 -4.998475e-01 3.089713e-01 
3.090031e-01 -8.087174e-01 4.999711e-01 
-3.903513e-06 -9.993746e-01 -2.037557e-06 
-3.089903e-01 -8.087293e-01 4.999806e-01 
-8.087795e-01 -4.998454e-01 3.089676e-01 
-8.088365e-01 5.001653e-01 -3.091039e-01 
-9.997343e-01 1.278491e-04 -2.323827e-06 
8.088259e-01 5.001663e-01 -3.091060e-01 
9.997299e-01 1.373686e-04 -2.323386e-06 
8.087887e-01 -4.998468e-01 -3.089555e-01 
-8.087922e-01 -4.998453e-01 -3.089517e-01 
-3.090643e-01 8.088167e-01 -4.999850e-01 
3.090741e-01 8.088058e-01 -4.999788e-01 
3.090104e-01 -8.087246e-01 -4.999743e-01 
-3.089970e-01 -8.087369e-01 -4.999837e-01 
-2.631499e-01 1.620107e-01 9.508449e-01 
-2.591210e-01 4.342621e-01 8.625237e-01 
-3.643831e-06 2.741785e-01 9.615335e-01 
-2.631903e-01 -1.618419e-01 9.508357e-01 
1.052168e-05 -2.740253e-01 9.615010e-01 
-2.590331e-01 -4.340657e-01 8.624527e-01 
-5.253198e-01 8.487359e-05 8.507528e-01 
-7.024912e-01 -1.600198e-01 6.931872e-01 
-7.024130e-01 1.601139e-01 6.931143e-01 
2.631436e-01 1.620024e-01 9.508500e-01 
2.591230e-01 4.342479e-01 8.625269e-01 
5.253093e-01 8.107691e-05 8.507492e-01 
7.024219e-01 1.601538e-01 6.931134e-01 
7.024873e-01 -1.600226e-01 6.931815e-01 
2.631759e-01 -1.618359e-01 9.508290e-01 
2.590255e-01 -4.340574e-01 8.624531e-01 
2.631235e-01 1.619581e-01 -9.507982e-01 
2.591265e-01 4.342569e-01 -8.625289e-01 
-3.619611e-06 2.741891e-01 -9.615360e-01 
2.631823e-01 -1.618260e-01 -9.508352e-01 
2.763655e-05 -2.739879e-01 -9.613662e-01 
2.590271e-01 -4.340501e-01 -8.624576e-01 
5.253178e-01 9.286789e-05 -8.507491e-01 
7.024982e-01 -1.600105e-01 -6.931725e-01 
7.024197e-01 1.601211e-01 -6.931030e-01 
-2.631581e-01 1.620182e-01 -9.508448e-01 
-2.591247e-01 4.342713e-01 -8.625264e-01 
-5.253287e-01 9.600335e-05 -8.507535e-01 
-7.024199e-01 1.601194e-01 -6.931103e-01 
-7.025025e-01 -1.600082e-01 -6.931787e-01 
-2.631966e-01 -1.618324e-01 -9.508426e-01 
-2.590346e-01 -4.340594e-01 -8.624572e-01 
-4.261309e-01 5.878199e-01 6.877248e-01 
-1.601343e-01 6.932172e-01 7.024688e-01 
-6.877498e-01 4.260859e-01 5.877128e-01 
-8.624622e-01 2.590779e-01 4.341579e-01 
-5.877030e-01 6.877647e-01 4.260575e-01 
-6.931999e-01 7.025127e-01 1.601119e-01 
-4.342718e-01 8.624788e-01 2.590407e-01 
-8.434483e-07 8.506669e-01 5.249842e-01 
1.601513e-01 6.931713e-01 7.024592e-01 
-1.618163e-01 9.505280e-01 2.630107e-01 
-2.742367e-01 9.612848e-01 -1.338514e-05 
1.618374e-01 9.505191e-01 2.630315e-01 
2.743161e-01 9.611932e-01 -1.638613e-05 
4.342642e-01 8.624520e-01 2.590594e-01 
4.261379e-01 5.878059e-01 6.877220e-01 
5.877126e-01 6.877551e-01 4.261164e-01 
6.931993e-01 7.024982e-01 1.601116e-01 
6.877532e-01 4.260887e-01 5.877129e-01 
8.624582e-01 2.590826e-01 4.341571e-01 
4.259763e-01 -5.876225e-01 6.877569e-01 
1.601304e-01 -6.931396e-01 7.026046e-01 
6.877575e-01 -4.260198e-01 5.877326e-01 
8.624975e-01 -2.589864e-01 4.342257e-01 
5.877308e-01 -6.876706e-01 4.259616e-01 
6.932995e-01 -7.024050e-01 1.601270e-01 
4.341911e-01 -8.624104e-01 2.590441e-01 
-8.427216e-06 -8.508251e-01 5.254660e-01 
-1.601223e-01 -6.931648e-01 7.026273e-01 
1.619416e-01 -9.507322e-01 2.631403e-01 
2.743293e-01 -9.615943e-01 -3.399512e-06 
-1.619381e-01 -9.507547e-01 2.631274e-01 
-2.743177e-01 -9.616339e-01 -3.115600e-06 
-4.341844e-01 -8.624206e-01 2.590370e-01 
-4.259771e-01 -5.876322e-01 6.877651e-01 
-5.877244e-01 -6.876760e-01 4.259614e-01 
-6.932177e-01 -7.023795e-01 1.601040e-01 
-6.877555e-01 -4.260171e-01 5.877235e-01 
-8.624999e-01 -2.589896e-01 4.342219e-01 
-9.508722e-01 2.632371e-01 1.619314e-01 
-9.613934e-01 4.656464e-05 2.740934e-01 
-9.508926e-01 2.632246e-01 -1.619252e-01 
-9.614279e-01 4.786521e-05 -2.740819e-01 
-8.624729e-01 2.590718e-01 -4.341523e-01 
-8.508368e-01 5.254337e-01 -3.342948e-06 
-6.932285e-01 7.025515e-01 -1.600940e-01 
9.508615e-01 2.632397e-01 1.619317e-01 
9.614133e-01 6.943600e-05 2.741131e-01 
8.508244e-01 5.254320e-01 -3.356140e-06 
6.932279e-01 7.025379e-01 -1.600923e-01 
9.508817e-01 2.632275e-01 -1.619256e-01 
8.624684e-01 2.590774e-01 -4.341515e-01 
9.614258e-01 5.446255e-05 -2.740808e-01 
9.508086e-01 -2.630329e-01 1.619948e-01 
9.508269e-01 -2.630137e-01 -1.619849e-01 
8.625032e-01 -2.589798e-01 -4.342116e-01 
8.508539e-01 -5.251915e-01 -8.088958e-06 
6.933253e-01 -7.024264e-01 -1.601178e-01 
-9.508085e-01 -2.630420e-01 1.619946e-01 
-8.508620e-01 -5.252011e-01 -7.874271e-06 
-6.933237e-01 -7.024395e-01 -1.601174e-01 
-9.507948e-01 -2.630029e-01 -1.619523e-01 
-8.625060e-01 -2.589835e-01 -4.342078e-01 
-6.877577e-01 4.260893e-01 -5.877119e-01 
-4.261579e-01 5.878712e-01 -6.877273e-01 
-1.601396e-01 6.932158e-01 -7.024944e-01 
-5.877288e-01 6.877770e-01 -4.261046e-01 
-4.342673e-01 8.625133e-01 -2.590353e-01 
-1.109214e-05 8.508174e-01 -5.251785e-01 
1.601529e-01 6.931762e-01 -7.024778e-01 
1.618254e-01 9.505403e-01 -2.630633e-01 
4.342664e-01 8.625047e-01 -2.590374e-01 
-1.618178e-01 9.505712e-01 -2.630733e-01 
4.261057e-01 5.878496e-01 -6.877244e-01 
6.877604e-01 4.260933e-01 -5.877116e-01 
5.877312e-01 6.877630e-01 -4.261062e-01 
4.259716e-01 -5.876146e-01 -6.877554e-01 
1.601313e-01 -6.931474e-01 -7.026179e-01 
5.877458e-01 -6.876708e-01 -4.259613e-01 
4.342113e-01 -8.624138e-01 -2.590417e-01 
6.877719e-01 -4.260101e-01 -5.877252e-01 
-8.063172e-06 -8.508368e-01 -5.254866e-01 
-1.601225e-01 -6.931732e-01 -7.026408e-01 
-1.619550e-01 -9.507600e-01 -2.631372e-01 
-4.342013e-01 -8.624176e-01 -2.590456e-01 
1.619598e-01 -9.507393e-01 -2.631482e-01 
-4.259723e-01 -5.876250e-01 -6.877635e-01 
-6.877653e-01 -4.259226e-01 -5.877082e-01 
-5.877392e-01 -6.876767e-01 -4.259613e-01 
-1.316860e-01 3.581664e-01 9.241248e-01 
-1.302018e-01 4.848709e-01 8.647370e-01 
-3.577250e-05 4.044391e-01 9.144622e-01 
-1.327549e-01 2.202993e-01 9.662213e-01 
-6.754532e-06 1.387483e-01 9.901784e-01 
-1.331270e-01 8.174850e-02 9.875358e-01 
-2.638628e-01 3.011932e-01 9.160234e-01 
-3.865915e-01 2.383717e-01 8.906329e-01 
-3.830917e-01 3.753922e-01 8.438795e-01 
-1.316643e-01 -3.579972e-01 9.240515e-01 
-3.400467e-05 -4.043242e-01 9.144800e-01 
-1.301771e-01 -4.847693e-01 8.647916e-01 
-2.639004e-01 -3.010912e-01 9.160039e-01 
-3.830396e-01 -3.753026e-01 8.438679e-01 
-3.866094e-01 -2.382595e-01 8.906468e-01 
-1.328561e-01 -2.201429e-01 9.662039e-01 
-1.331371e-01 -8.154563e-02 9.875491e-01 
-7.843883e-06 -1.385385e-01 9.901952e-01 
-7.109691e-01 9.970106e-05 7.027618e-01 
-7.843402e-01 -8.033547e-02 6.148567e-01 
-7.842835e-01 8.051366e-02 6.148033e-01 
-6.201472e-01 8.114848e-02 7.799945e-01 
-6.071168e-01 2.367639e-01 7.581786e-01 
-5.187194e-01 1.567334e-01 8.402047e-01 
-6.201877e-01 -8.094474e-02 7.800589e-01 
-5.188199e-01 -1.566702e-01 8.402591e-01 
-6.071792e-01 -2.366958e-01 7.582487e-01 
-3.995894e-01 -8.202886e-02 9.127933e-01 
-3.994741e-01 8.212215e-02 9.126927e-01 
-2.665204e-01 2.456292e-05 9.634940e-01 
1.316895e-01 3.581089e-01 9.241236e-01 
1.301882e-01 4.848369e-01 8.647289e-01 
2.638671e-01 3.011833e-01 9.160280e-01 
3.830920e-01 3.753763e-01 8.438857e-01 
3.865177e-01 2.382912e-01 8.906020e-01 
1.327465e-01 2.202934e-01 9.662251e-01 
1.331156e-01 8.174585e-02 9.875391e-01 
7.109632e-01 9.682861e-05 7.027599e-01 
7.842918e-01 8.054810e-02 6.148101e-01 
7.843331e-01 -8.033670e-02 6.148515e-01 
6.201798e-01 -8.094606e-02 7.800559e-01 
6.071728e-01 -2.366992e-01 7.582425e-01 
5.188106e-01 -1.566679e-01 8.402549e-01 
6.201401e-01 8.114257e-02 7.799919e-01 
5.187157e-01 1.567258e-01 8.402024e-01 
6.071154e-01 2.367635e-01 7.581720e-01 
1.316722e-01 -3.579212e-01 9.239976e-01 
1.301666e-01 -4.847392e-01 8.647803e-01 
1.328441e-01 -2.201373e-01 9.661924e-01 
1.331885e-01 -8.157858e-02 9.875034e-01 
2.638983e-01 -3.010839e-01 9.159951e-01 
3.865986e-01 -2.382495e-01 8.906425e-01 
3.830309e-01 -3.752896e-01 8.438661e-01 
3.995751e-01 -8.202840e-02 9.127892e-01 
2.665464e-01 4.382074e-05 9.635318e-01 
3.995014e-01 8.215541e-02 9.127141e-01 
1.316922e-01 3.581198e-01 -9.241251e-01 
1.301888e-01 4.848514e-01 -8.647316e-01 
-3.586892e-05 4.044526e-01 -9.144640e-01 
1.327506e-01 2.203033e-01 -9.662268e-01 
-6.624191e-06 1.387591e-01 -9.901813e-01 
1.331203e-01 8.175607e-02 -9.875415e-01 
2.638737e-01 3.011906e-01 -9.160277e-01 
3.865175e-01 2.382989e-01 -8.906063e-01 
3.830067e-01 3.753303e-01 -8.438674e-01 
1.316751e-01 -3.579164e-01 -9.240015e-01 
-3.366140e-05 -4.043209e-01 -9.144897e-01 
1.301593e-01 -4.846906e-01 -8.646878e-01 
2.639046e-01 -3.010740e-01 -9.160036e-01 
3.830376e-01 -3.752769e-01 -8.438740e-01 
3.866063e-01 -2.382377e-01 -8.906486e-01 
1.328481e-01 -2.201287e-01 -9.661999e-01 
1.331669e-01 -8.159819e-02 -9.874048e-01 
3.082743e-05 -1.385379e-01 -9.901268e-01 
7.109734e-01 1.089611e-04 -7.027513e-01 
7.843433e-01 -8.032544e-02 -6.148415e-01 
7.842992e-01 8.056036e-02 -6.147982e-01 
6.201475e-01 8.115271e-02 -7.799817e-01 
6.071214e-01 2.367711e-01 -7.581708e-01 
5.187178e-01 1.567316e-01 -8.402066e-01 
6.201914e-01 -8.093135e-02 -7.800511e-01 
5.188195e-01 -1.566542e-01 -8.402573e-01 
6.071860e-01 -2.366875e-01 -7.5

Oh actually there was a typo, so looks like it is now producing an output, however it reads past the POLYGONS text despite not being supposed to.

The line split should have been line.split (' ') rather than ('/') since the space is the separator.

Edited 6 Years Ago by newyawk: n/a

Like this?

f_in = open('xxx.txt').readlines()[:12960]
f_output= open('output.txt', 'w')

for line in f_in:
        firstdata, seconddata = line.split(' ')[:2]
        f_output.write('%s, %s\n' % (firstdata, seconddata))

Thanks - yes that works better. The only thing is, if you run that script it loads up until point 12785 and not 12960. Any idea why?

And also what does the

'%s, %s\n' %

do?

Thanks for your help, appreciate it.

Edited 6 Years Ago by newyawk: n/a

I believe this is what you want.

f_in = open('xxx.txt').readlines()[:12959]
f_output= open('output.txt', 'w')

for item in f_in[:5]:
    f_output.write(item)

for item in f_in[5:]:
        firstdata, seconddata = item.split(' ')[:2]
        f_output.write('%s, %s\n' % (firstdata, seconddata))
'%s %s\n' % (firstdata, seconddata)

The '%' places the variables, here firstdata and seconddata, formated as strings due to the 's', in the output string. The '\n' adds a linebreak.

Edited 6 Years Ago by Beat_Slayer: n/a

Thanks. I still can't get it to read to the end? I've now added some code to read in multiple files, but it doesn't seem to append to the file. I am missing some kind of append somewhere, how should I go about it?

import os, glob 
path = './' 
for dir, subdir, files in os.walk(path):
    for file in files:
        if glob.fnmatch.fnmatch(file,"xxx*.txt"): 


            f_input = open('./xxx0.txt').readlines()[:12959] 
f_output= open('output.txt', 'w')								
 
for line in f_input[5:]:
   
        firstdata, seconddata, thirddata = line.split(' ')[:3] 
        f_output.write('%s, %s, %s,\n' % (firstdata, seconddata, thirddata))

I changed it as I do need the 3 columns (stupid me).

Edited 6 Years Ago by newyawk: n/a

I need to read in the first 2 columns until line 12960. I then need to write the 12960 lines of 2 columns to a file.

You can use enumerate to count the records as it appears that a file is shorter than you think or there are blank records.

import os, glob 
def process_file(fname, f_output):
    print "processing", fname
    f_input = open(fname, "r")
								
    for num, rec in enumerate(f_input):
        if (num > 5) and (num < 12961):
            firstdata, seconddata, thirddata = rec.split(' ')[:3]
            f_output.write('%s, %s, %s,\n' % (firstdata, seconddata, thirddata))

    f_input.close()


f_output= open('output.txt','w')
path = './' 
for dir, subdir, files in os.walk(path):
    for fname in files:
        if glob.fnmatch.fnmatch(file,"xxx*.txt"): 
            process_file(os.path.join(dir, fname), f_output)
f_output.close()

Edited 6 Years Ago by woooee: n/a

This article has been dead for over six months. Start a new discussion instead.