943,692 Members | Top Members by Rank

Ad:
  • Java Discussion Thread
  • Unsolved
  • Views: 961
  • Java RSS
Sep 2nd, 2008
0

Tokenize

Expand Post »
hello guyz,,who knows how to tokenize???i mean,,i am about to start to make "String Calculator" but i don't know how to tokenize,,,do you know any sites that could explain best about tokens??thank you ahead,,,,
Similar Threads
Reputation Points: 6
Solved Threads: 1
Light Poster
letlet_pogs is offline Offline
44 posts
since Aug 2008
Sep 2nd, 2008
0

Re: Tokenize

How do you need your tokens to be parsed?

An example...

java Syntax (Toggle Plain Text)
  1. public class TokenMaster{
  2. public static void main(String... args){
  3. String myReallyBIGString = "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n"
  4. + "ALPHABETICAL!\n"
  5. + "SOMETIMES I FEEL I'VE GOT TO 'AH-AH' RUN AWAY, I'VE GOT TO... "
  6. + "'AH-AH' GET AWAY!!!\n"
  7. + "RING RING RING RING RING RING RING... BANANA PHONE! 'doot, doot doot doot doot doot'"
  8. + "DING-DONG DING-DONG DING-DONG DING... BANANA PHONNNNNE!!!!!";
  9.  
  10. String tokens[] = myReallyBIGString.split("\n"); // each token is determined by the newline character
  11.  
  12. for(String element : tokens)
  13. System.out.println(element); // printing the individual tokens generated from String.split("\n")
  14.  
  15. System.out.println("\nTokens counted: " + tokens.length);
  16. }
  17. }
Last edited by Alex Edwards; Sep 2nd, 2008 at 3:14 am.
Reputation Points: 392
Solved Threads: 108
Posting Shark
Alex Edwards is offline Offline
971 posts
since Jun 2008
Sep 2nd, 2008
0

Re: Tokenize

thank you very much,,,may i as,,how useful are tokens???
Reputation Points: 6
Solved Threads: 1
Light Poster
letlet_pogs is offline Offline
44 posts
since Aug 2008
Sep 3rd, 2008
0

Re: Tokenize

you can also use String Tokenizer class.
Reputation Points: 31
Solved Threads: 5
Light Poster
yilmazhuseyin is offline Offline
48 posts
since Oct 2006
Sep 3rd, 2008
0

Re: Tokenize

you can also use String Tokenizer class.
Or not, because StringTokenizer is considered an Obsolete class that receives no further support in future releases.

Furthermore, String.split() is better because it takes regular expressions into consideration. StringTokenizer does not.
Last edited by Alex Edwards; Sep 3rd, 2008 at 11:32 am.
Reputation Points: 392
Solved Threads: 108
Posting Shark
Alex Edwards is offline Offline
971 posts
since Jun 2008
Sep 24th, 2008
0

Re: Tokenize

You may use the special PHP function strtok

<?php

$myText="online pharmacy web solution";

$strToken=strtok($myText," ");

while($strToken){

echo $strToken."<br>";

$strToken=strtok(" ");

}

?>
this my hel;p u
Reputation Points: 10
Solved Threads: 1
Newbie Poster
digitaldesperad is offline Offline
18 posts
since Sep 2008
Sep 24th, 2008
0

Re: Tokenize

this my hel;p u
Sure, maybe... if he was using PHP, but since this is the Java forum and his code is Java, your suggestion seems to fall a little bit short of helpful.

And this isn't a chat room, so you can leave the IM-speak at the door.
Moderator
Featured Poster
Reputation Points: 3239
Solved Threads: 838
Posting Genius
Ezzaral is offline Offline
6,757 posts
since May 2007

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in Java Forum Timeline: Runtime classname of null object
Next Thread in Java Forum Timeline: How to insert data read from text file into linked list?





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC