Ad:

Java Discussion Thread
Marked Solved
Views: 3900

Dec 6th, 2008

Reading in a text file -- efficiently

Expand Post »

I had a problem w/ efficiency - The problem was that using String concatenation was inefficient for large text files, when reading the whole file into the String. I've since thought about it, and I guess this is because Strings are immutable - so every time concatenation is done, a new String object has to be created and the underlying char array has to be copied? After some research, I came across StringBuilder and String___ (I forget the rest of the name of the class). My question is, how can a text file be read in as efficiently as possible? I don't really care what type its read into. Also, a lot of sources I found said StringBuilder was the best way, is that true?

Similar Threads

help in C++
delphi word in Pascal and Delphi

BestJewSinceJC

Reputation Points: 874

Solved Threads: 352

Posting Maven

Offline

2,758 posts
since Sep 2008

Permalink

Dec 6th, 2008

Re: Reading in a text file -- efficiently

I think StringBuffer might be the other String__ class you were referring to. Both the StringBuilder and StringBuffer class are better at concatenation than the base String class. StringBuilder will out-perform StringBuffer (although both are highly efficient at concatenation) but note that StringBuilder does not guarantee synchronisation, meaning that you should only use it if it is being accessed by a single Thread. Multi-Threaded applications need to use StringBuffer, or control the synchronisation themselves.

Last edited by darkagn; Dec 6th, 2008 at 7:05 pm.

darkagn

Reputation Points: 395

Solved Threads: 192

Veteran Poster

Offline

1,136 posts
since Aug 2007

Permalink

Dec 6th, 2008

Re: Reading in a text file -- efficiently

Thanks. The app I am currently working on is a single thread so I'll just use whichever looks easier, I guess.

BestJewSinceJC

Reputation Points: 874

Solved Threads: 352

Posting Maven

Offline

2,758 posts
since Sep 2008

Permalink

Dec 7th, 2008

Re: Reading in a text file -- efficiently

Terms like efficiency make little sense when used in an absolute context. For finding the most efficient way to solve a problem you first need to track down your applications' usage pattern i.e. what kind of file reading does your application need? Is the data read promptly consumed or is stored for future use? Profiling your application is by far the best approach to selecting an efficient solution.

BTW, StringBuilder & StringBuffer are mutable companion classes to the immutable class String ; one being non-thread safe and the other being thread safe respectively.

~s.o.s~

Super Moderator

Featured Poster

Reputation Points: 3233

Solved Threads: 719

Failure as a human

Offline

8,871 posts
since Jun 2006

Permalink

Dec 7th, 2008

Re: Reading in a text file -- efficiently

Well, if you're simply going to be reading the entire file into a single String without any kind of work on the lines read until after the entire file is read, I wouldn't use either of those, and wouldn't use a Reader at all. I would do it like this

Java Syntax (Toggle Plain Text)
File f = new File("filename");
FileInputStream fis = new FileInputStream(f);
byte[] b = new byte[f.length];
int read = 0;
while (read < b.length) {
  read += fis.read(b, read, b.length - read);
}
String text = new String(b);
File f = new File("filename");
FileInputStream fis = new FileInputStream(f);
byte[] b = new byte[f.length];
int read = 0;
while (read < b.length) {
  read += fis.read(b, read, b.length - read);
}
String text = new String(b);

This is, of course, missing all error handling. That's for you. This also assumes that encoding won't be problem.

You are not forced to use a Reader simply because what you're reading is a text file.

masijade

Moderator

Reputation Points: 1471

Solved Threads: 490

Industrious Poster

Offline

4,043 posts
since Feb 2006

Permalink

Dec 7th, 2008

Re: Reading in a text file -- efficiently

> This also assumes that encoding won't be problem.

If this is what he actually wants, encoding shouldn't be a problem as long as an appropriate character set is specified when creating the string.
String text = new String(bytes, "UTF-8"); .

> byte[] b = new byte[f.length]; .

Array bites? ;-)
byte[] bytes = new byte[f.length()];

~s.o.s~

Super Moderator

Featured Poster

Reputation Points: 3233

Solved Threads: 719

Failure as a human

Offline

8,871 posts
since Jun 2006

Permalink

Dec 8th, 2008

Re: Reading in a text file -- efficiently

Yeah, thats great - thanks guys. Leave the error handling to me. Lol.

BestJewSinceJC

Reputation Points: 874

Solved Threads: 352

Posting Maven

Offline

2,758 posts
since Sep 2008

Permalink

Dec 8th, 2008

Re: Reading in a text file -- efficiently

Click to Expand / Collapse Quote originally posted by ~s.o.s~ ...

> byte[] b = new byte[f.length]; .

Array bites? ;-)
byte[] bytes = new byte[f.length()];

masijade

Moderator

Reputation Points: 1471

Solved Threads: 490

Industrious Poster

Offline

4,043 posts
since Feb 2006

This thread is solved

Either the thread starter or a moderator has marked this thread as solved. You can most likely trust the responses and answers given. There is most likely no reason for any further responses to be posted here. If you have a related question, please start a new thread in this forum instead.

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.

Previous Thread in Java Forum Timeline: Modifying a file.

Next Thread in Java Forum Timeline: array index and returning

DaniWeb Message

Cancel Changes

User Name
Password