Java

Java RSS

StringTokenizer problem

•

Join Date: Sep 2005

Posts: 42

Reputation: j1979c is an unknown quantity at this point

Solved Threads: 0

j1979c

Offline

Light Poster

StringTokenizer problem

Apr 24th, 2006

Hello,

I'm currently having some problems with the StringTokenizer from
java util.

I've declared a StringTokenizer like:

StringTokenizer token = new StringTokenizer(line,"|");

to token out a string read from a txt file such as:

Diablo|RPG|PG|PS2|20

and it works fine(detects all the "|" as delimeters)

The problem now is, if I have lines with spaces like:

C&C Generals|Real Time Strategy|PG|PS2|20

it detects the spaces as well and the tokenizing is out.

So, I'm wondering if anyone out there can help me solve this problem. I just want the tokenizer to recognize "|" as a delimiter and not anything else.

Thanks in advance.

•

Join Date: Aug 2005

Posts: 5,264

Reputation: iamthwee is a splendid one to behold

Solved Threads: 377

iamthwee

Offline

Posting Expert

Re: StringTokenizer problem

Apr 24th, 2006

How bout this?

Tokens.java

 Help with Code Tags 
 Java Syntax (Toggle Plain Text) 
import java.io.*; 
import java.util.*;
 
class Tokens 
{
 
     public static void main(String[] args)
     {
         int count=0;
         String array[]= new String[100];
          StringTokenizer st = new StringTokenizer("Diablo & generals|RPG|PG|PS2|20","|");
           while (st.hasMoreTokens()) 
           {
         //System.out.println(st.nextToken());
            array[count] =st.nextToken();
            count++;
           }
          System.out.print(array[0]); //change here to see different token
     }
}
import java.io.*; 
import java.util.*;

class Tokens 
{

     public static void main(String[] args)
     {
         int count=0;
         String array[]= new String[100];
          StringTokenizer st = new StringTokenizer("Diablo & generals|RPG|PG|PS2|20","|");
           while (st.hasMoreTokens()) 
           {
         //System.out.println(st.nextToken());
            array[count] =st.nextToken();
            count++;
           }
          System.out.print(array[0]); //change here to see different token
     }
}

http://img476.imageshack.us/img476/5171/cut20ln.png
Piworld ™
[Tis simple as Pie]

•

Join Date: Mar 2004

Posts: 763

Reputation: Phaelax is on a distinguished road

Solved Threads: 38

Phaelax

Offline

Master Poster

Re: StringTokenizer problem

Apr 24th, 2006

Help with Code Tags

(Toggle Plain Text)

String line = "C&C Generals|Real Time Strategy|PG|PS2|20";
String[] tokens = line.split("|");
for (int i=0;i<tokens.length;i++)
     System.out.println(tokens[i]);

•

Join Date: Sep 2005

Posts: 42

Reputation: j1979c is an unknown quantity at this point

Solved Threads: 0

j1979c

Offline

Light Poster

Re: StringTokenizer problem

Apr 25th, 2006

Yup.... I'm using the

line.split("|",5)

Found that in the Sun Java forum. I can forget bout StringTokenizer now,
cause the split method is definitely much more cooler...:mrgreen:

And the StringTokenizer is not recommended as of 5.0 I think...:-|

Well...forced to do my project in Java... if it were in Delphi I can finish it with 1/3 of the time I spent using Java.

Thanks for the replies guys...really helped

•

Join Date: Aug 2005

Posts: 5,264

Reputation: iamthwee is a splendid one to behold

Solved Threads: 377

iamthwee

Offline

Posting Expert

Re: StringTokenizer problem

Apr 25th, 2006

>And the StringTokenizer is not recommended...

What is all this hoo-ha, about using split instead of stringTokenizer. Can anyone explain what I should be using and why?

And what of the subsequent link?
http://www.stabile.org/blog/2005/11/...etc-technical/

http://img476.imageshack.us/img476/5171/cut20ln.png
Piworld ™
[Tis simple as Pie]

•

Join Date: Sep 2005

Posts: 42

Reputation: j1979c is an unknown quantity at this point

Solved Threads: 0

j1979c

Offline

Light Poster

Re: StringTokenizer problem

Apr 25th, 2006

It depends on the situation I guess.

For mine it works just fine. No performance drop compared to StringTokenizer, plus it works better... since there's no more need to create an array or arraylist to store the stuff and it ignores the spaces in between data...plus you get to specify how many tokens you want from each line as a second argument. Also you get to save quite a number of lines.

Was having problems with StringTokenizer cause it detects the spaces as delimeters as well though I specified that the delimeter is only "|".

As for the link, I think it's full of crap. Some people just can't switch to something new where they're already comfortable with the old one. Getting memory leaks in Java is quite impossible, cause that's what a garbage collector is for. You should see how memory leaks are like in Delphi...your com doesn't hang but it moves very very very slow till you reboot...if not it'll hang indefinitely. Looping a 100,000,000 times oh my... :cheesy:

•

Join Date: Mar 2004

Posts: 763

Reputation: Phaelax is on a distinguished road

Solved Threads: 38

Phaelax

Offline

Master Poster

Re: StringTokenizer problem

Apr 25th, 2006

That link makes some valid points about initialization costs. I find splitString easier to use though, and it does more than just search for delimiters, its parameter is actually a regular expression. The regex search could possibly slow the parsing down a bit, but I'm not sure by how much or how much it would even take to be noticable.

•

Join Date: Mar 2008

Posts: 1

Reputation: janarjan is an unknown quantity at this point

Solved Threads: 0

janarjan

Offline

Newbie Poster

Re: StringTokenizer problem

Mar 17th, 2008

Split is simple.
It follows the regular expression.
Anyways for the above code there is slight change

String line = "C&C Generals|Real Time Strategy|PG|PS2|20";
String[] tokens = line.split("\\|");
for (int i=0;i<tokens.length;i++)
System.out.println(tokens[i]);

janarjan