Ad:

Java Discussion Thread
Unsolved
Views: 5554

Apr 24th, 2006

StringTokenizer problem

Expand Post »

Hello,

I'm currently having some problems with the StringTokenizer from
java util.

I've declared a StringTokenizer like:

StringTokenizer token = new StringTokenizer(line,"|");

to token out a string read from a txt file such as:

Diablo|RPG|PG|PS2|20

and it works fine(detects all the "|" as delimeters)

The problem now is, if I have lines with spaces like:

C&C Generals|Real Time Strategy|PG|PS2|20

it detects the spaces as well and the tokenizing is out.

So, I'm wondering if anyone out there can help me solve this problem. I just want the tokenizer to recognize "|" as a delimiter and not anything else.

Thanks in advance.

Similar Threads

Best Way to Check for same Word in string? in Java
Txt Coordinates into chart... in Java
StringTokenizer in Java

j1979c

Reputation Points: 13

Solved Threads: 1

Light Poster

Offline

42 posts
since Sep 2005

Permalink

Apr 24th, 2006

Re: StringTokenizer problem

How bout this?

Tokens.java

Java Syntax (Toggle Plain Text)
import java.io.*; 
import java.util.*;
 
class Tokens 
{
 
     public static void main(String[] args)
     {
         int count=0;
         String array[]= new String[100];
          StringTokenizer st = new StringTokenizer("Diablo & generals|RPG|PG|PS2|20","|");
           while (st.hasMoreTokens()) 
           {
         //System.out.println(st.nextToken());
            array[count] =st.nextToken();
            count++;
           }
          System.out.print(array[0]); //change here to see different token
     }
}
import java.io.*; 
import java.util.*;

class Tokens 
{

     public static void main(String[] args)
     {
         int count=0;
         String array[]= new String[100];
          StringTokenizer st = new StringTokenizer("Diablo & generals|RPG|PG|PS2|20","|");
           while (st.hasMoreTokens()) 
           {
         //System.out.println(st.nextToken());
            array[count] =st.nextToken();
            count++;
           }
          System.out.print(array[0]); //change here to see different token
     }
}

http://img476.imageshack.us/img476/5171/cut20ln.png
Piworld ™
[Tis simple as Pie]

iamthwee

Featured Poster

Reputation Points: 1536

Solved Threads: 431

Posting Expert

Offline

5,865 posts
since Aug 2005

Permalink

Apr 24th, 2006

Re: StringTokenizer problem

(Toggle Plain Text)

String line = "C&C Generals|Real Time Strategy|PG|PS2|20";
String[] tokens = line.split("|");
for (int i=0;i<tokens.length;i++)
     System.out.println(tokens[i]);

Phaelax

Reputation Points: 92

Solved Threads: 51

Practically a Posting Shark

Offline

856 posts
since Mar 2004

Permalink

Apr 25th, 2006

Re: StringTokenizer problem

Yup.... I'm using the

line.split("|",5)

Found that in the Sun Java forum. I can forget bout StringTokenizer now,
cause the split method is definitely much more cooler...:mrgreen:

And the StringTokenizer is not recommended as of 5.0 I think...:-|

Well...forced to do my project in Java... if it were in Delphi I can finish it with 1/3 of the time I spent using Java.

Thanks for the replies guys...really helped

j1979c

Reputation Points: 13

Solved Threads: 1

Light Poster

Offline

42 posts
since Sep 2005

Permalink

Apr 25th, 2006

Re: StringTokenizer problem

>And the StringTokenizer is not recommended...

What is all this hoo-ha, about using split instead of stringTokenizer. Can anyone explain what I should be using and why?

And what of the subsequent link?
http://www.stabile.org/blog/2005/11/...etc-technical/

http://img476.imageshack.us/img476/5171/cut20ln.png
Piworld ™
[Tis simple as Pie]

iamthwee

Featured Poster

Reputation Points: 1536

Solved Threads: 431

Posting Expert

Offline

5,865 posts
since Aug 2005

Permalink

Apr 25th, 2006

Re: StringTokenizer problem

It depends on the situation I guess.

For mine it works just fine. No performance drop compared to StringTokenizer, plus it works better... since there's no more need to create an array or arraylist to store the stuff and it ignores the spaces in between data...plus you get to specify how many tokens you want from each line as a second argument. Also you get to save quite a number of lines.

Was having problems with StringTokenizer cause it detects the spaces as delimeters as well though I specified that the delimeter is only "|".

As for the link, I think it's full of crap. Some people just can't switch to something new where they're already comfortable with the old one. Getting memory leaks in Java is quite impossible, cause that's what a garbage collector is for. You should see how memory leaks are like in Delphi...your com doesn't hang but it moves very very very slow till you reboot...if not it'll hang indefinitely. Looping a 100,000,000 times oh my... :cheesy:

j1979c

Reputation Points: 13

Solved Threads: 1

Light Poster

Offline

42 posts
since Sep 2005

Permalink

Apr 25th, 2006

Re: StringTokenizer problem

That link makes some valid points about initialization costs. I find splitString easier to use though, and it does more than just search for delimiters, its parameter is actually a regular expression. The regex search could possibly slow the parsing down a bit, but I'm not sure by how much or how much it would even take to be noticable.

Phaelax

Reputation Points: 92

Solved Threads: 51

Practically a Posting Shark

Offline

856 posts
since Mar 2004

Permalink

Mar 17th, 2008

Re: StringTokenizer problem

Split is simple.
It follows the regular expression.
Anyways for the above code there is slight change

String line = "C&C Generals|Real Time Strategy|PG|PS2|20";
String[] tokens = line.split("\\|");
for (int i=0;i<tokens.length;i++)
System.out.println(tokens[i]);

janarjan

janarjan

Reputation Points: 10

Solved Threads: 0

Newbie Poster

Offline

1 posts
since Mar 2008

Permalink

Feb 28th, 2010