Tokenize
 

hello guyz,,who knows how to tokenize???i mean,,i am about to start to make "String Calculator" but i don't know how to tokenize,,,do you know any sites that could explain best about tokens??thank you ahead,,,,

Re: Tokenize
 

How do you need your tokens to be parsed?

An example...

public class TokenMaster{

        public static void main(String... args){

                String myReallyBIGString = "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n"

                                                                  + "ALPHABETICAL!\n"

                                                                  + "SOMETIMES I FEEL I'VE GOT TO 'AH-AH' RUN AWAY, I'VE GOT TO... "

                                                                  + "'AH-AH' GET AWAY!!!\n"

                                                                  + "RING RING RING RING RING RING RING... BANANA PHONE! 'doot, doot doot doot doot doot'"

                                                                  + "DING-DONG DING-DONG DING-DONG DING... BANANA PHONNNNNE!!!!!";



                String tokens[] = myReallyBIGString.split("\n"); // each token is determined by the newline character



                for(String element : tokens)

                        System.out.println(element); // printing the individual tokens generated from String.split("\n")



                System.out.println("\nTokens counted: " + tokens.length);

        }

}

Re: Tokenize
 

thank you very much,,,may i as,,how useful are tokens???

Re: Tokenize
 

you can also use String Tokenizer class.

Re: Tokenize
 

Or not, because StringTokenizer is considered an Obsolete class that receives no further support in future releases.

Furthermore, String.split() is better because it takes regular expressions into consideration. StringTokenizer does not.

Re: Tokenize
 

You may use the special PHP function strtok

<?php

$myText="online pharmacy web solution";

$strToken=strtok($myText," ");

while($strToken){

echo $strToken."<br>";

$strToken=strtok(" ");

}

?>
this my hel;p u

Re: Tokenize
 

Sure, maybe... if he was using PHP, but since this is the Java forum and his code is Java, your suggestion seems to fall a little bit short of helpful.

And this isn't a chat room, so you can leave the IM-speak at the door.