Read a text file continually being updated by another process

[hhappak]

Hello,

I am facing a problem I cant seem to find an efficient solution for. My application is supposed to parse a text file while another process is continuously adding to it. I have tried scanner and inputstream but both seem to stop once they hit the end of the file at the moment they were initialized. Is there a way I can read the file in, so that it stops at the end of the file, then wait for a further update and then pull off the added line?

The text file can become very large and opening then closing it, then opening it again to start reading the updates is highly inefficient and I am searching for an alternative.

[~s.o.s~]

Try using the RandomAccessFile in read mode.

public class MainTest {
   
   public static void main(String args[]) throws Exception {
      RandomAccessFile in = new RandomAccessFile("file.txt", "r");
      String line;
      while(true) {
         if((line = in.readLine()) != null) {
            System.out.println(line);
         } else {
            Thread.sleep(2000); // poll the file every 2 seconds
         }
      }
   }
   
}

[hhappak]

Try using the RandomAccessFile in read mode.

public class MainTest {
   
   public static void main(String args[]) throws Exception {
      RandomAccessFile in = new RandomAccessFile("file.txt", "r");
      String line;
      while(true) {
         if((line = in.readLine()) != null) {
            System.out.println(line);
         } else {
            Thread.sleep(2000); // poll the file every 2 seconds
         }
      }
   }
   
}

Thank you for the quick reply,
I have tried this code, but once it is called, it pays no attention to any further updates to the file. Once it reaches the line==null, the null status never change and goes on in an infinite loop. Did not solve the problem :(

[BestJewSinceJC]

As for S.o.S's reply, you would expect it to infinitely loop, since you want to continuously check the file for updates. In other words it is supposed to infinitely loop so I don't see why you have an issue with that. Maybe I didn't understand the issue. Anyway, I'll give you another suggestion:

Create a File Object, storing the size of the file, and then continuously (like every 2s) create another File Object (based on the same, possibly updated file) and seeing if those File Objects have the same size. If they do have the same size, no changes were made, so do nothing and wait another couple seconds. If they do not have the same size, then seek to the location of the smaller size (this is the end of the file from before) and then read in the rest of the file and print it out. Now the "updated File Object" becomes the old file Object, so update your file size variable and update your old File Object variable, and continue the process again.

[hhappak]

As for S.o.S's reply, you would expect it to infinitely loop, since you want to continuously check the file for updates. In other words it is supposed to infinitely loop so I don't see why you have an issue with that. Maybe I didn't understand the issue. Anyway, I'll give you another suggestion:

Create a File Object, storing the size of the file, and then continuously (like every 2s) create another File Object (based on the same, possibly updated file) and seeing if those File Objects have the same size. If they do have the same size, no changes were made, so do nothing and wait another couple seconds. If they do not have the same size, then seek to the location of the smaller size (this is the end of the file from before) and then read in the rest of the file and print it out. Now the "updated File Object" becomes the old file Object, so update your file size variable and update your old File Object variable, and continue the process again.

Thanx alot, this did solve the problem. I even found a better way using File.lastModified() which returns the date/time it was last modified so it was better to use than measuring the size.

P.S--As for the previous solution suggested by u, I would expect it to go in an infinite loop, that wasnt the problem. But the problem was that it went on an infinite loop without ever reading the updates in the file. Basically it just went in circles and picked up nothing from the file once it finished reading it once.

[~s.o.s~]

> Did not solve the problem

You sure? The same piece of code works for me. BTW, what kind of process is writing to this file of yours? Another Java process? Which JDK version are you using? I used Notepad to update the file & the updates were reflected in my case.

[Ondřej_1]

after your program sleeps, close the RandomAccessFile with .close() method and run parent method recursively. duno if you can do that with main, but it works on normal method
this example even gets you on fresh line and reads it

private void readFile() throws FileNotFoundException, IOException, SQLException, InterruptedException{ 

        RandomAccessFile br = new RandomAccessFile(fileName, "r");
        String line;
        for(int i=0; i<lineNum; i++){
            br.readLine();
        }

        while (true){
            if((line = br.readLine()) != null){
                //do some stuff
            }else{
                System.out.println("waiting"+br.length());
                br.close();
                Thread.sleep(1000);
                readFile();        
            }
            lineNum++;
        }
    }

[JamesCherrill]

The text file can become very large and opening then closing it, then opening it again to start reading the updates is highly inefficient and I am searching for an alternative.

I guess you missed that? (not to mention this thread has been dead for 6 years)

[rproffitt]

Time passes and folk have written other things about this like http://stackoverflow.com/questions/27678477/read-from-a-file-while-another-process-is-writing-to-it-using-stdfstream