954,487 Members — Technology Publication meets Social Media
Username:
Password:
Lost login information?
Have something to say? Contribute New Article Reply to this Article

Splitting words from string into character array

So Im trying to write some code that takes a chunk of text, splits it up into words, then puts these words into an array. What I have is:

string buf;
stringstream ss(database); // database is my chunk of text

    
    while (ss >> buf) // Tokenize when whitespace met
    {
          ???
    }


So in that while loop I want to take the string "buf" and put it into an array. Thus the result is an array of strings containing every word in the paragraph. This is where I feel like an idiot.

I have tried a few ways using string streams and converting strings to char arrays etc, but nothing will work for me. I have a feeling that this has to do with multidimensional arrays and pointers, but my searches have been fruitless.

Thanks in advance!

hbweb500
Newbie Poster
6 posts since Feb 2005
Reputation Points: 10
Solved Threads: 0
 

My advice, use the vector container. One of the advantages of using C++.

bumsfeld
Nearly a Posting Virtuoso
1,445 posts since Jul 2005
Reputation Points: 404
Solved Threads: 184
 
#include <vector>
#include <boost/tokenizer.hpp>
#include <string>
#include <vector>
#include <iostream>
using namespace std;
using namespace boost;

int main()
{
  string str = "The boost tokenizer library provides a flexible "
                       "and easy to use way to break of a string or other "
                       "character sequence into a series of tokens.\n"
                       "Here is a simple example that will break up a "
                       "paragraph into words.\n";
  tokenizer<> toker( str );
  vector<string> words( toker.begin(), toker.end() ) ;
  cout << "there are " << words.size() << " words\n" ;
  cout << "words[6]: " << words[6] << '\n' ;
}
vijayan121
Posting Virtuoso
1,606 posts since Dec 2006
Reputation Points: 1,159
Solved Threads: 287
 

What also complicates matters is if your words are separated by more than one space, or even tabs.

In that case, it would probably be wiser to use regular expressions. The task becomes almost trivial.

iamthwee
Posting Expert
5,950 posts since Aug 2005
Reputation Points: 1,543
Solved Threads: 439
 

This is perfect, vijayan, thanks a bunch!

hbweb500
Newbie Poster
6 posts since Feb 2005
Reputation Points: 10
Solved Threads: 0
 

This question has already been solved

Post: Markdown Syntax: Formatting Help
You