Java

tokenize an input text file..

Please support our Java advertiser: Programming Forums - DaniWeb Sister Site

•

Join Date: Apr 2008

Posts: 7

Reputation: minamoda is an unknown quantity at this point

Solved Threads: 0

minamoda

Offline

Newbie Poster

tokenize an input text file..

Apr 7th, 2008

Hi to every one,
I need some help about the program i am riting. I am a beginner in java and i cant split my input text file to its tokens.
the part of the input file is :

•

101 Car 06AB1212 100 Y A 3
102 Car 06BG0101 200 Y M 2
202 Motor 34BB123 60 Motor
203 Motor 34CC123 20 Scooter
301 Truck 78AA321 60 B 10
403 Bus 25TT432 200 D 45
11 BerilErtan B
12 BerraCan B
1002 Eti 1983
1003 Gezitur 1999

i wrote a program and the part for tokenize is:

 Help with Code Tags 
 Java Syntax (Toggle Plain Text) 
line= BbmReader.getFileReader("input.txt").readLine();
		StringTokenizer tokens = new StringTokenizer(line, " ");
		while( tokens.hasMoreTokens()&& BbmReader.getFileReader("input.txt").inputAvailable())
			System.out.println(tokens.nextToken());
line= BbmReader.getFileReader("input.txt").readLine();
		StringTokenizer tokens = new StringTokenizer(line, " ");
		while( tokens.hasMoreTokens()&& BbmReader.getFileReader("input.txt").inputAvailable())
			System.out.println(tokens.nextToken());

which only tokens one line not the others.

Please help me. Thank you all

•

Join Date: May 2007

Posts: 4,514

Reputation: Ezzaral has a brilliant future

Solved Threads: 523

Ezzaral

Offline

Industrious Poster

Re: tokenize an input text file..

Apr 7th, 2008

You need to have the read in a loop like so

 Help with Code Tags 
 Java Syntax (Toggle Plain Text) 
while ((line = reader.readLine()) != null){
    StringTokenizer tokenizer = new StringTokenizer(line," ");
    // do stuff
}
while ((line = reader.readLine()) != null){
    StringTokenizer tokenizer = new StringTokenizer(line," ");
    // do stuff
}

I have no idea where the "BbmReader" class comes from, so I can't speak to any specifics on that, but the general loop should work the same.

By the way, StringTokenizer is an old legacy class and you should generally use String.split() instead.

•

Join Date: Apr 2008

Posts: 7

Reputation: minamoda is an unknown quantity at this point

Solved Threads: 0

minamoda

Offline

Newbie Poster

Re: tokenize an input text file..

Apr 7th, 2008

"BbmReader" is a class in a library which we use in our school i have to use it and i think there isnt any problem with it but my program doesnt tokenize all the text file. only the first line.
And by the way, I HAVE TO use tokenizer because our teacher wants it.

Thank you so much for the help but still it doesnt do what i want.

•

Join Date: May 2007

Posts: 4,514

Reputation: Ezzaral has a brilliant future

Solved Threads: 523

Ezzaral

Offline

Industrious Poster

Re: tokenize an input text file..

Apr 7th, 2008

It certainly will if you use the loop as I indicated. Get the reader one time and then use that loop to read lines from it.

•

Join Date: Apr 2008

Posts: 7

Reputation: minamoda is an unknown quantity at this point

Solved Threads: 0

minamoda

Offline

Newbie Poster

Re: tokenize an input text file..

Apr 7th, 2008

You tell me that

 Help with Code Tags 
 Java Syntax (Toggle Plain Text) 
while ((line = reader.readLine()) != null){
    StringTokenizer tokenizer = new StringTokenizer(line," ");
    // do stuff
}while ((line = reader.readLine()) != null){
    StringTokenizer tokenizer = new StringTokenizer(line," ");
    // do stuff
}
while ((line = reader.readLine()) != null){
    StringTokenizer tokenizer = new StringTokenizer(line," ");
    // do stuff
}while ((line = reader.readLine()) != null){
    StringTokenizer tokenizer = new StringTokenizer(line," ");
    // do stuff
}

works i am confused now if "reader" in here is the same as file reader in my program or not.
i wrote

 Help with Code Tags 
 Java Syntax (Toggle Plain Text) 
line= BbmReader.getFileReader("input.txt").readLine();
while (line!= null){
		    StringTokenizer tokens = new StringTokenizer(line," ");
System.out.println(tokens.nextToken());}
line= BbmReader.getFileReader("input.txt").readLine();
while (line!= null){
		    StringTokenizer tokens = new StringTokenizer(line," ");
System.out.println(tokens.nextToken());}

but now it goes to an endless loop wich only writes the first token, "101"

•

Join Date: May 2007

Posts: 4,514

Reputation: Ezzaral has a brilliant future

Solved Threads: 523

Ezzaral

Offline

Industrious Poster

Re: tokenize an input text file..

Apr 8th, 2008

Of course it's endless - because you are not ever reading the next line. The loop that I wrote reads the line into "line" each iteration of the loop and checks it against null. Yours calls readLine() one time and keeps looping while "line" is not null - so the loop is either endless or never executes in the first place. If the syntax of the loop I wrote it confusing you, try it like this instead

 Help with Code Tags 
 java Syntax (Toggle Plain Text) 
// I can only assume this method returns a BufferedReader
BufferedReader reader = BbmReader.getFileReader("input.txt");
String line = reader.readLine();
while (line != null){
    StringTokenizer tokens = new StringTokenizer(line," ");
    System.out.println(tokens.nextToken());
    // you still need another loop here to read the rest of the tokens
    ...
    line = reader.readLine();
}
// I can only assume this method returns a BufferedReader
BufferedReader reader = BbmReader.getFileReader("input.txt");
String line = reader.readLine();
while (line != null){
    StringTokenizer tokens = new StringTokenizer(line," ");
    System.out.println(tokens.nextToken());
    // you still need another loop here to read the rest of the tokens
    ...
    line = reader.readLine();
}