Help with looping through a vector

[Deiwos]

Hi folks.

I'm having some trouble with a program I'm working on. What it is is a chatbot based on eliza. Part of the program tokenizes what ever the user inputs. The problem I'm having is that the tokenizeing function stores the input into a vector. What I need to do is loop through the vector looking for 2 words within it, and I haven't got a clue on how to do it.

If you need any more code from the program just ask.
Any help would be much appreciated.

void tokenize(const string& str, vector<string>& tokens, const string& delimiters )

string::size_type lastPos;
string::size_type pos;

	tokens.clear();		// Make sure the vector is empty to start with

	// Find the first "word" (token)
	lastPos = str.find_first_not_of(delimiters, 0);		// Skip delimiters at beginning.
	pos     = str.find_first_of(delimiters, lastPos);	// Find first "non-delimiter".

    while ( (pos != string::npos) || (lastPos != string::npos) )
    {
        // Found a token, add it to the vector.
        tokens.push_back(str.substr(lastPos, pos - lastPos));

        // Find the next word
        lastPos = str.find_first_not_of(delimiters, pos);		// Skip delimiters.
        pos		= str.find_first_of(delimiters, lastPos);		// Find next "non-delimiter"
    }

}

[Rajith Cherian]

As I understand your problem there are two ways you could proceed...

Scenario 1:

If you are going to lookup ( search ) for words within the vector i would recommend you to do a little change in your data structure. try using a map instead of a vector... If thats the case u r tokenize function will look like

void tokenize(const string& str, map<string , string>& tokens, const string& delimiters )

string::size_type lastPos;
string::size_type pos;

	tokens.clear();		// Make sure the vector is empty to start with

	// Find the first "word" (token)
	lastPos = str.find_first_not_of(delimiters, 0);		// Skip delimiters at beginning.
	pos     = str.find_first_of(delimiters, lastPos);	// Find first "non-delimiter".

    while ( (pos != string::npos) || (lastPos != string::npos) )
    {
        // Found a token, add it to the vector.
        tokens[str.substr(lastPos, pos - lastPos)] = str.substr(lastPos, pos - lastPos);

        // Find the next word
        lastPos = str.find_first_not_of(delimiters, pos);		// Skip delimiters.
        pos		= str.find_first_of(delimiters, lastPos);		// Find next "non-delimiter"
    }

}

In the above code the string is tokenized and stored in a map instead of a vector. The main advantage in a map is that lookup will be much faster.

// This is just a sample
// Declare a map and an iterator
map<string , string> tokens;
map<string,string>::iterator mItr;

//Call the tokeize function and fill the map with tokens  and then use the following piece of code

mItr = tokens.find( str );// Lookups in the map for the word
if ( mItr != tokens.end() ) // if the word is found then the iterator will be valid
      strTxt = mItr->first; // you can get the data using this code

Scenario 2:

If you want to still stick with vector then the following code will tell you whether the word is present in a vector or not.

BOOL IsPresentinVector( const string& str, vector<string> &tokens )
{
        if ( token.size() < 1 )
            return FALSE;          /// vector is empty

        for ( vector<string>::iterator vItr = tokens.begin() ; vItr != token.end() ; vItr++ )
        {
                if ( *vItr == str )
                     return TRUE;            // Word found
        }
        

         return FALSE;   //word not found

}

Use iterators for better performance