String Tokenizer?

[ff4930]

Hello everyone,

I have a project to do which involves reading each line of text from an input file
which holds integers.

Each line represents a polynomial, and I have to rearrange it in canonical form which is sorted by powers.

I was wondering if there is a string tokenizer function for c++ such as the one from Java.

an example line would be
-1 0 5 1 20 3 -9 2 -2 1 1 2 -2 3
which would be
-x + 5x^1 + 20x^3 - 9x^2 - 2x + x^2 - 2x^3

Each integer is separated by a space.

string line;
	ifstream myfile("input.txt");
	if(myfile.is_open()){
		while(!myfile.eof()){
			getline(myfile, line);
			cout << line << endl;
		}//while
		myfile.close();		
	}//if
	else cout << "Unable to open file" << endl;

Is there a way to extract 2 digits at a time from the string "line"?

Thanks.

[Ancient Dragon]

If the file contains several lines as you described then after getline() you can use stringstream to divide it up into individual integers. Also note the difference in the while statement -- using eof() like that doesn't work.

#include <sstream>
<snip>
string line;
ifstream myfile("input.txt");
if(myfile.is_open()){
    while(getline(myfile, line) )
          cout << line << "\n";
          int value;
          stringstream str(line);
          while( str >> value)
              cout << value << "\n";             
    }//while
    myfile.close();		
}//if
    else cout << "Unable to open file" << endl;

>>I was wondering if there is a string tokenizer function for c++ such as the one from Java
I dont' think so.

[Alex Edwards]

That's the beauty of c++ though, learn how to custom build your own "helper" classes and you'll become more efficient at programming.

[Alex Edwards]

I created my own string class with a tokenizer method included, here's the code--

/**
   defined as header file myString.h

    a my_string class object takes 3 parameters--
    
    -a String literal
    -a character pointer
    -an object that inherits from the class ToString (a pure virtual class)
    */      
    class my_string
    {
          
          private:
                  char* theName;
                  
                  my_string()
                  {
                     //main initialization here
                  };
                  
          public:
                  my_string(char *name)
                  {
                      my_string::my_string();
                      theName = name;
                  };
                  
                  my_string(my_string *ms)
                  {
                       my_string::my_string();
                       theName = ms->getString();    
                  };
                  
                  my_string(ToString *ts)
                  {
                      my_string::my_string();
                      theName = ts->className();
                  };
                  
                  char *getString(){return theName;};
                  
                  friend std::ostream &operator<< (std::ostream &os, my_string &ms)
                  {
                      return os << ms.theName;    
                  };
                  
                  friend std::istream &operator>> (std::istream &is, my_string &ms)
                  {
                      return is >> ms.theName;
                  }
                  
                  void operator= (char *arg)
                  {
                       theName = arg;
                  };
                  
                  /**
                  Splits the string into several peices based on the token.
                  */
                  static my_string *tokenizer(char *value, char token){
                        int peices = 1;
                        
                        std::cout << "attempting to tokenize!" << std::endl;  
                            
                        for(int i = 0; value[i] != '\0'; i++)
                        {
                             if(value[i] == token)
                                 peices++;
                        }    
                        
                        my_string *strPtr;
                        strPtr = (my_string*)malloc((peices) * sizeof(my_string));
                        
                        int trailer = 0, current = 0, position = 0;
                        bool done = false;
                        
                        char *theString = 0;
                        
                        for(int i = 0; !done; i++)
                        {
                             if(value[i] == token || value[i] == '\0')
                             {
                                 current = i;
                                 
                                 theString = (char*)malloc(((current) - trailer) * sizeof(char));
                                   
                                 for(int j = (trailer); j <= (current); j++)
                                 {
                                      if(value[j] != token)   
                                          theString[(j) - trailer] = value[j];
                                      else theString[(j) - trailer] = '\0';
                                      std::cout << "j is: " << j << std::endl;
                                      std::cout << "trailer is: " << trailer << std::endl;
                                      std::cout << "this is theString[" << (j-trailer) << "]: " << theString[j-trailer] << std::endl;
                                      std::cout << "this is value[" << j << "]: " << value[j] << std::endl;                                     
                                 }
                                 
                                 strPtr[position] = theString;
                                 std::cout<< "loaded value " << strPtr[position] << std::endl;
                                 position++;
                                 trailer = current + 1;
                               //  delete theString;
                             }
                            
                             
                             if(value[i] == '\0')
                                 done = true;
                                 
                             current++;    
                                
                        }
                        return strPtr;
                  };
                  
                  /**
                  Takes a char* argument and copies the contents of the valueToCopy into the arg
                  */
                  static void copyString(char *&arg, const char *valueToCopy)
                  {
                       int amount = 0;
                       char *temp;
                       
                       for(amount; valueToCopy[amount] != '\0'; amount++)
                                   continue;
                       
                       temp = new char[amount];
                       
                       for(int i = 0; i < amount; i++)
                               temp[i] = valueToCopy[i];
                               
                       temp[amount] = '\0';
                       
                       arg = temp;
                  };
    
    };

and here's the example code--

#include <cstdlib>
#include <iostream>
#include "myString.h"

using namespace std;

int main(int argc, char *argv[])
{
    first_library::my_string ms = "my string object!";
    cout << ms << endl;
    
    first_library::my_string *trying = ms.tokenizer("letsztokenisedzthis!", 'z');
    
    cout << "\n\n" << endl;
    
    for(int i = 0; i < 3; i++)
      cout << trying[i] << endl;
    
    cin.get();
    return 0;
}

[ff4930]

I have used the stringstream to read each integer.

I have created a linked list and creating each node with a coefficent and exponent.

I sorted the polynominal in cannonical form/sorted.
I have to implement addition/subtraction/multiplication/division methods on the polynominals.

Anyone know a good way to do this?

I was thinking of going down the list since it is already ordered by exponents, keep adding the coefficents until the last one and just add the sum to it.
and delete the previous ones, so I will be left with a shorter linkedlist with the sum of polynominals.

http://www.daniweb.com/forums/post524949-6.html

I'm glad you decided to use stringstreams...so much easier.

[Duoas]

Heh, that was an interesting thread...

Yes, there is a simple, one-shot way to do it using the STL, but it is fairly advanced stuff (though I don't know why) :twisted:
Here's a little program to demonstrate it.

#include <algorithm>  // for copy()
#include <iostream>   // cin, cout
#include <iterator>   // all the funky iterators
#include <sstream>    // stringstream
#include <string>
#include <vector>

using namespace std;

//--------------------------------------
// This is the type of my polynomial in one variable.
//
struct term_t
  {
  int coefficient;  // (these could be doubles, you know)
  int exponent;
  };
typedef vector <term_t> polynomial1v_t;

//--------------------------------------
// Overload the input operator to read a term
//
istream& operator >> ( istream& ins, term_t& term )
  {
  ins >> term.coefficient >> term.exponent;
  return ins;
  }

//--------------------------------------
// Here is the one-line magic:
// This function reads a polynomial from a stream
// The stream is assumed to contain nothing but the poly's terms
//
polynomial1v_t read_polynomial1v( istream& ins )
  {
  polynomial1v_t result;
  copy(
    istream_iterator <term_t> ( ins ),
    istream_iterator <term_t> (),
    back_insert_iterator <polynomial1v_t> ( result )
    );
  return result;
  }

//--------------------------------------
int main()
  {
  string line;

  // get the terms from the user
  cout << "Please enter your polynomial terms> ";
  getline( cin, line );

  // convert them into a poly of one var
  istringstream iss( line );
  polynomial1v_t poly = read_polynomial1v( iss );

  // pretty-print the poly wanna cracker
  for (unsigned i = 0; i < poly.size(); i++)
    {
    if ((poly[ i ].coefficient >= 0) and (i > 0)) cout << '+';
    cout << poly[ i ].coefficient << "x^" << poly[ i ].exponent << ' ';
    }
  cout << endl;

  // the end ;-)
  return 0;
  }

Hope this helps.

(BTW, if you are working on homework, be sure to mention to your teacher that you got this iterator stuff from help on the web, or he'll just assume you got somebody else to do your entire homework for you... since AFAIK they don't teach this stuff in anything under 450-500 level courses.)