Start New Discussion within our Software Development Community

My project needs to read records from a CSV file. The file contains 3 columns of data. Record is continuously being appended to the file by another application (about once every 6-8 seconds), and my application is to read the records and store the new ones.

I am currently opening the file, reading the entire contents row by row using a Input # statement, checking if the record has already been stored and storing the new ones. This is making the process unacceptably slow.

Is there a way by which I could skip to the position where I finished the last reading and continue from there ?

There are a couple of ways to solve this but first question is...
does the other program create the file if it is not there?

If so, you could wait until you have exclusive access to the file and once you do you could do a couple of different things. First you could name the file a different name and allow the other program to create a new file and continue on with its operations while you leisurly open the other file, read in its contents, and then delete it.

If the other program does not create the file if it is not there then, once again wait until you can get exclusive access to the file, rename it, then rename a blank file to replace it while you take your time once again.

Now, if you can never get exclusive access to the file, then you could keep track of how many bytes you have read in and each time you access the file (binary random access), you could jump to that position or you could remember how many lines you have read in and before you start comparing you could read in that many lines + or - 1 or exact, however you want. Then start comparing and adding.

Good Luck

This article has been dead for over six months. Start a new discussion instead.