I am trying to use the stringtokenizer on input strings containing the " character, but stringtokenizer seems to treat that as the end of the string.

For instance, "a cat is /"not/" a dog"
should return the tokens a, cat, is, "not", a, dog
but instead it returns a, cat, is

What can I do so the tokenizer does not end on the first instance of the " character?

Thanks for the help

Update: Figured it out (stupid mistake on my part.)

Hi everyone,

If you still need help with the string tokenizer class you can read the below link


Here is a good example from kick java

import java.util.*;

class ParseString
  public static void main (String args[]) 
	String s = "first,second, third fourth,  fifth";
	StringTokenizer st;

	st = new StringTokenizer (s, ", ");

	while (st.hasMoreTokens ())
	  System.out.println (">>" + st.nextToken () + "<<");

The output is:


Yours Sincerely

Richard West

You have the escape sequence backwards..That shouldn't even compile.
Here is a small example of what you should have:

import java.util.*;

public class TestTokenizer
	public static void main(String[] args)
		String tokenizeThis = "a cat is \"not\" a dog";
		StringTokenizer st = new StringTokenizer(tokenizeThis);
		while (st.hasMoreTokens())