Tokenize

Reply

Join Date: Aug 2008
Posts: 29
Reputation: letlet_pogs has a little shameless behaviour in the past 
Solved Threads: 1
letlet_pogs letlet_pogs is offline Offline
Light Poster

Tokenize

 
0
  #1
Sep 2nd, 2008
hello guyz,,who knows how to tokenize???i mean,,i am about to start to make "String Calculator" but i don't know how to tokenize,,,do you know any sites that could explain best about tokens??thank you ahead,,,,
Reply With Quote Quick reply to this message  
Join Date: Jun 2008
Posts: 973
Reputation: Alex Edwards is a jewel in the rough Alex Edwards is a jewel in the rough Alex Edwards is a jewel in the rough Alex Edwards is a jewel in the rough 
Solved Threads: 107
Alex Edwards's Avatar
Alex Edwards Alex Edwards is offline Offline
Posting Shark

Re: Tokenize

 
0
  #2
Sep 2nd, 2008
How do you need your tokens to be parsed?

An example...

  1. public class TokenMaster{
  2. public static void main(String... args){
  3. String myReallyBIGString = "ABCDEFGHIJKLMNOPQRSTUVWXYZ\n"
  4. + "ALPHABETICAL!\n"
  5. + "SOMETIMES I FEEL I'VE GOT TO 'AH-AH' RUN AWAY, I'VE GOT TO... "
  6. + "'AH-AH' GET AWAY!!!\n"
  7. + "RING RING RING RING RING RING RING... BANANA PHONE! 'doot, doot doot doot doot doot'"
  8. + "DING-DONG DING-DONG DING-DONG DING... BANANA PHONNNNNE!!!!!";
  9.  
  10. String tokens[] = myReallyBIGString.split("\n"); // each token is determined by the newline character
  11.  
  12. for(String element : tokens)
  13. System.out.println(element); // printing the individual tokens generated from String.split("\n")
  14.  
  15. System.out.println("\nTokens counted: " + tokens.length);
  16. }
  17. }
Last edited by Alex Edwards; Sep 2nd, 2008 at 3:14 am.
Reply With Quote Quick reply to this message  
Join Date: Aug 2008
Posts: 29
Reputation: letlet_pogs has a little shameless behaviour in the past 
Solved Threads: 1
letlet_pogs letlet_pogs is offline Offline
Light Poster

Re: Tokenize

 
0
  #3
Sep 2nd, 2008
thank you very much,,,may i as,,how useful are tokens???
Reply With Quote Quick reply to this message  
Join Date: Oct 2006
Posts: 48
Reputation: yilmazhuseyin is an unknown quantity at this point 
Solved Threads: 5
yilmazhuseyin's Avatar
yilmazhuseyin yilmazhuseyin is offline Offline
Light Poster

Re: Tokenize

 
0
  #4
Sep 3rd, 2008
you can also use String Tokenizer class.
Reply With Quote Quick reply to this message  
Join Date: Jun 2008
Posts: 973
Reputation: Alex Edwards is a jewel in the rough Alex Edwards is a jewel in the rough Alex Edwards is a jewel in the rough Alex Edwards is a jewel in the rough 
Solved Threads: 107
Alex Edwards's Avatar
Alex Edwards Alex Edwards is offline Offline
Posting Shark

Re: Tokenize

 
0
  #5
Sep 3rd, 2008
Originally Posted by yilmazhuseyin View Post
you can also use String Tokenizer class.
Or not, because StringTokenizer is considered an Obsolete class that receives no further support in future releases.

Furthermore, String.split() is better because it takes regular expressions into consideration. StringTokenizer does not.
Last edited by Alex Edwards; Sep 3rd, 2008 at 11:32 am.
Reply With Quote Quick reply to this message  
Join Date: Sep 2008
Posts: 18
Reputation: digitaldesperad is an unknown quantity at this point 
Solved Threads: 1
digitaldesperad digitaldesperad is offline Offline
Newbie Poster

Re: Tokenize

 
0
  #6
Sep 24th, 2008
You may use the special PHP function strtok

<?php

$myText="online pharmacy web solution";

$strToken=strtok($myText," ");

while($strToken){

echo $strToken."<br>";

$strToken=strtok(" ");

}

?>
this my hel;p u
Reply With Quote Quick reply to this message  
Join Date: May 2007
Posts: 4,427
Reputation: Ezzaral has much to be proud of Ezzaral has much to be proud of Ezzaral has much to be proud of Ezzaral has much to be proud of Ezzaral has much to be proud of Ezzaral has much to be proud of Ezzaral has much to be proud of Ezzaral has much to be proud of Ezzaral has much to be proud of Ezzaral has much to be proud of 
Solved Threads: 507
Moderator
Featured Poster
Ezzaral's Avatar
Ezzaral Ezzaral is offline Offline
Industrious Poster

Re: Tokenize

 
0
  #7
Sep 24th, 2008
Originally Posted by digitaldesperad View Post
this my hel;p u
Sure, maybe... if he was using PHP, but since this is the Java forum and his code is Java, your suggestion seems to fall a little bit short of helpful.

And this isn't a chat room, so you can leave the IM-speak at the door.
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:


Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC