Break a string into tokens

[shaunmccann1]

Hi,

Ive got a similar problem, it seems a run of the mill thing to break a string into tokens yet its kept me head scratching for days, is there another option or is it a simple code tweaking problem.

Fairly new to C/C++ but ive been coding in C# for nearly 4 years and never had such a simple function issue.

Any advice guys would be great, oh and its my first post so play fair...

char *list[200];
int a;
char *allocate;
char phrase[500];

main() {

cin >> phrase;

allocate = strtok(phrase," ");
while (allocate != NULL)
{	
	for( a = 0; a < 10; a++);
	{
		list[a] = allocate;
	}
}

[mrnutty]

Just use the stringstream to break a string into tokens :

stringstream tokenBreaker;
string aLine = "Please break this into six tokens";
tokenBreaker << aLine; //insert string
string tokens[7]; //for now lets not use vectors and do it manually
int currentToken = 0;
while( tokenBreaker >> tokens[currentToken]  && currentToken != 7) 
       ++currentToken;

[shaunmccann1]

Thats brilliant thanks very much for your prompt help!

[shaunmccann1]

Ive hit a wall again, string to char conversion is fine.... but i am checking the value from tokens[] against a mysql row[].

Basic question then is at the moment I having to move a string array back to char for checking against the query. Cant stringstream convert accross to a char array, I only ask as ive tried so many workarounds and ultimately this is the resulting problem. If you can think of a similar function to aide me ill be willing to consider that too. ive already gone though strstr, match against in mysql, all just dont do it accurately enough.

Thanks in advance

[mitrmkar]

I'm suspecting that you are unnecessarily trying to convert data back and forth (being somewhat unfamiliar with c++). So, if you'd describe in detail what you need to do, someone here might suggest a simple solution.

[shaunmccann1]

ok, I need to simply take an input check said input against a database, and output value. Problem is the value can have variables so,

INPUT VARIABLE

so in this case input and variable will both need to be checked separately.

My original idea was to use strtok to separate each word and and then check each separately against the database.

[Lerner]

Is the input string a null terminated char array (C style string)? What type is the output to be? What type(s) do you want the token(s) to be?

[shaunmccann1]

input must be a string as it needs spaces and needs to be flexable, output will be run though system() so potentially char or string, and token needs to match up with mysql row.

Simply put the only prerequies actually has to be the mysql section everthing else may or may not be a string or char.

I would prefer it to be simple and just use one type but as its getting larger and larger I am starting to thing there is a function about that i dont no about that will make it much easier.

[Lerner]

Strings can be native null terminated char array or an object of any of a number of classes including the STL string class, the CString class, the String^ class, the TString class, etc. Which type of string are you refering to?

A string may have zero, one or more than one char.

What type is a mysql row?

When you separated the tokens, what type are you going to compare the token to?

What type is the output supposed to have?

[shaunmccann1]

ok, i think people are making this overlly complicated task it really is a input processor output sort of job.

as for mysqlrow i am not sure what it will accept, my current app works with a char which is converted from a string.

[mrnutty]

Post your code, we have nothing to work with except your unclear
wording.

What do you want to do ?
What did you do ?
What were the errors ?

[shaunmccann1]

ok my original code as posted at the beginning of the thread, the error was simply that for some reason it wouldn't allow strcat to use data from the array, no other problem and like i say i have a functioning version of this just without strtok instead i am using strstr which isnt the best solution.

if you need anymore than that just ask, thanks for your helps guys, i am on a steep learning curve so bare with me if you can and if you do think i am doing anything wrong feel free to say. only been coding in the language since December so any help is great.

[Lerner]

The biggest problem I see with the code in the first post is that in the while loop the value of allocate never varies. So, either the loop will never run, if allocate is NULL or it will never stop if allocate isn't NULL. To fix it I would use the while loop without the for loop if I didn't know how many tokens there will be or the for loop without the while loop if knew there would be exactly 10 tokens. Then a call to strtok() within the loop to change the value of allocate would be needed. I would also strcopy the value of each token to the array of tokens and not assign it.

char list[10][100];  //Max 10 tokens, Max 99 char each token
int a = 0; 
allocate = strtok(phrase," ");
while (allocate != NULL && a < 10)
{
     strcpy(list[a], allocate);
     allocate = strtok(NULL, " ");
     ++a;
}

[shaunmccann1]

ok that does look similar to my original plan but with some subtle difference. Again thanks very much I can see what i was missing now although the program was completing successfully which drew me to look online.

Ill give it a try in the morning and see what happens, fingers crossed and again thanks for all your time and assistance.

[Lerner]

I also note that since the >> operator is used to get input into the variable called phrase, there will be no spaces in phrase, since whitespaces, such as a space char, terminate input when the >> operator is used. If phrase is intended to be string with spaces in it so the space char can be used as a delimiter for strtok() to break phrase into tokens, then use of getline() rather than >> would be appropriate.

[shaunmccann1]

your a superstar thanks! just ran it for the first time and was able to open my garage door via the application! :)