Determining the number of unique words in a .txt file
 

Hey I have to write a program that reads a text file that contains a list of words, 1 word per line. I have to store the unique words and count the occurrences of each unique word. When the file is completely read, I have to print the words and the number of occurrences to a text file. The output should be the words in alphabetical order along with the number of times they occur. Then print to the file some statistics:

I have to use character arrays instead of strings.
I must use the linear search (something that looks like this)

array.int search ( int array[], int number, int key )

{

        int pos = 0;

        

        while(pos < number && array[pos] != key)

                pos++;

        

        if( pos == number)

                pos = 0;

        else

                pos = 1;

        

        return pos;

}

to determine if a word is in the array. The array is an array of structures and that the key is a char array so the string comparison must be used. The search task should be a separate function.
The search must be a separate function that returns an integer values. I cant use a for loop and the function must have only one return statement.
I tried to start off by reading the files and storing the words

#include <iostream>

#include <fstream>

#include <cstdlib>

#include <string>

#include <iomanip>

using namespace std;



void displayFile( char []);



void main ()

{

        int const wordLength = 21;

        int const Num = 101;

        int const fileSize = 255;

        

        

        char filename[fileSize];

        ifstream inFile;

        

        

        struct

        {

                char word[wordLength];

                int count;

        } wordCount;

        

        wordCount array[Num];

        

    cout << "Please enter the name of thr file you wish to open: "<< endl;

        cin >> filename;

        

        displayFile (filename);

        

}



void displayFile (char fileName [] )

{

    ifstream infile;

        char array [101];

    char line [101];

        int ch;

    int i;

        infile.open (fileName);



    while ((ch = infile.peek()) != EOF) 

    {

        infile.getline (line, 101);

        line = array[i];

        i++;

    }

        

   for (int j = 0; j<101; j++)

        cout << array[j] << endl;

    infile.close ();

Any tips would be appreciated, and thanks in advance.

Re: Determining the number of unique words in a .txt file
 

I would first create a structure

struct words

{

    std::string word;

    int count;

};

Next an vector (array) of those structures

vector<words> wordArray;

Now, then the program reads a line, check wordArray if its already in the array. if it is then increment the count. If not, then add a new element to the vector. When done, you will have the information you need to display on the screen.

Another method is to use a <map>, but implementation is left for someone else.

Re: Determining the number of unique words in a .txt file
 

I think you should look at the below code more carefully and must understand the complexity and map datastructure,

if you've any confusion post, I will explain you in more detail then.

        fstream inFile("C:\\Laiq.txt");



        string line;

        std::map<string, int> mapWordCount;

        string word;

        std::string::size_type endWord;

        std::string::size_type startWord;

        while (std::getline (inFile,line)) {                                

                startWord = 0;

                while ((endWord = line.find_first_of(" ", startWord))!= line.npos) 

                {                                

                                word = line.substr (startWord, endWord - startWord);

                                mapWordCount[word]++;

                                startWord = endWord+1;

                }

        }

        inFile.close();





        for(map<string,int>::const_iterator iter = mapWordCount.begin(); 

                iter != mapWordCount.end(); ++iter) {

                        cout<< "The Frequency of word: "<< iter->first << " is : "<< iter->second <<endl;

        }

Re: Determining the number of unique words in a .txt file
 

You have two different problems:
1. Tokenize the input stream (extract words from the stream).
2. Build word dictionary.
The second one has a very simple solution: use std::<map> data structure as Laiq Ahmed mentioned above. However you need another code to process every word with the map-based word dictionary:

if the next word is found in the map then

    do notning or increment this word counter

else

    insert new word in the map

endif

There are lots of methods to solve the 1st problem. For example:

open file stream

create an empty map

loop // until eof

   skip non-letters

   clear word buffer // use std::string::clear()

   append letters to the word buffer

   process the word with the map // see #2

endloop

process a possible last word

traverse map (file word dictionary)

Summary:
- use std::ifstream
- use std::string for word buffer
- use std::map<std::string,int> for dictionary with counters
You have a good chance to write a simple and clear code after a proper functional decomposition of pseudocode snippets ;)...

Re: Determining the number of unique words in a .txt file
 

>>I have to use character arrays instead of strings.
I must use the linear search

That pretty much precludes use of a map. It also means you can't use STL strings within a struct, and may preclude use of a vector to hold the structs, too.

You could use a C style string with either static or dynamic memory within the struct. The static version will limit the maximum length of any possible string entered, but since these strings will be words, then it would be unlikely that any given word would have more than 20 or 30 letters per word. Using dynamic memory to store the string within the struct will minimize the memory wasted by using strings of length less than than maximum length in the static version, but will require you to manage your own memory, which is a bit of a hassle, though not an overwhelming task by any means.

The non-STL structures you could use to store the structs could be either an array or a list. If you don't know the maximum number of possible unique words you could encounter then using a list might be advantageous. You could make a guess as to the max number of words, but that isn't quite as predictable as the maximum size of each word. Linear searching is possible with either lists or arrays.

Sorting the structures by string in alphabetical order before printing/sending values to file could be done in either of several different ways. For beginners, bubble sorts with arrays and insertion sorts with lists seem pretty popular.

Re: Determining the number of unique words in a .txt file
 

Sorry, I have not noticed those absurd restruictions.
The only result of this idiotic methodology: a pupil sheds tears on DaniWeb then use copy/paste for extremelly ineffective codes in early 60-th style (or worse)...

Re: Determining the number of unique words in a .txt file
 

The maximum length of a word is 20, so the character array needs to be 21. The maximum numbers of words is 100, if there are more than 100 words I need to return a message saying something like "The stats did not use the words after the 100th word"

Anyways, I'm having a hard time even starting the program. I'm starting off by trying to read and store the words of the text file. My text book has code for reading line by line a text file and then displaying it. So I figured I should start with that. Heres the code-

#include <iostream>

#include <fstream>

#include <cstdlib>   // needed for exit()

#include <string>

using namespace std;



int main()

{

  string filename = "text.dat";  // put the filename up front

  string line;

  ifstream inFile;

  

  inFile.open(filename.c_str());



  if (inFile.fail())  // check for successful open

  {

    cout << "\nThe file was not successfully opened"

         << "\n Please check that the file currently exists."

         << endl;

    exit(1);

  }



  // read and display the file's contents

  while (getline(inFile,line))

    cout << line << endl;



  inFile.close(); 



  cin.ignore();  // this line is optional



  return 0;

}

Heres my code split into two functions, however it does not compile. I get an error at the while statement.-

#include <iostream>

#include <fstream>

#include <cstdlib>

#include <string>

#include <iomanip>

using namespace std;



void displayFile( char []);



void main ()

{

        int const wordLength = 21;

        int const Num = 101;

        int const fileSize = 255;

        char filename[fileSize];

        

        cout << "Please enter the name of the file you wish to open: "<< endl;

        cin.getline(filename,fileSize);

        

        displayFile (filename);

        cin.ignore();

}



void displayFile (char fileName[] )

{

    ifstream inFile;

        

    char line [101];

        

        inFile.open(fileName);



 while (getline(inFile, line))

    cout << line << endl;



        inFile.close(); 



        return 0;

}

My code is nearly identical except for the fact that I use character arrays instead of strings, so I'm not sure why its not compiling.
Also, is this a good way to start off the program? Or should I try something else.

Re: Determining the number of unique words in a .txt file
 

Ok I figured out I need to use "inFile.getline(line,length)" instead of "getline(inFile, line)". I created a struct like stated early. However I'm having trouble storing the words from the text file.

I use this function-

void displayFile (char fileName[], words array[] )

{

        int i = 0;

    ifstream inFile;

        

    char line [101];

        

        inFile.open(fileName);



 while (inFile.getline(line,101))

 {   

        cout << line << endl;

    array[i].word = line;

    i++;

 }   

        inFile.close(); 



}

Where array, is an array of structs made up of character array"word" and an integer "count". Any help on storing the words and the number of words in the array would be helpfull.

Re: Determining the number of unique words in a .txt file
 

Read-a-line-in-a-char-array solution has an obvious defect: you must define max line length a priori.
Look at the function which can get words from an input stream (not only from fstream) directly:

// #include <cctype> dependency

typedef unsigned char Uchar; // needed for isalpha()

/// Get or skip the next word from a stream. 

/// wbufsize is the max word length + 1 (for null char)

bool getWord(char* word, int wbufsize, std::istream& is)

{

    int i = 0, n = wbufsize - 1; // max letters

    bool fill = (word && n > 0); // word wanted

    char ch; // current char

    

    while (is.get(ch) && !isalpha(Uchar(ch)))

        ;   // skip delimiters

    if (is) // have a letter

    do {

        if (fill) { // have a room

            word[i] = ch; // append letter

            if (i >= n)   // no more room

                fill = false;

        }

        ++i; // letters counter

    } while (is.get(ch) && isalpha(Uchar(ch)));

    if (word)// have a buffer

        word[i] = '\0'; // end of word

    return i > 0; // have a word

}

Of course, it's possible to adopt this algorithm to extract words from a char array.

Re: Determining the number of unique words in a .txt file
 

I tried a different Approach to achieve the same

        char* arr = " Hallo WOrld";

        char* ptrFirst = arr;

        char* ptrIter = arr;

        int nWordCount = 0;



        // Base Condition.

        while (isspace (*ptrFirst))

                ++ptrFirst;



        ptrIter = ptrFirst;

        

        while (ptrFirst) {

                

                while (isalpha(*ptrIter)) {

                        ++ptrIter;

                }



                ++nWordCount;

                

                while (isspace (*ptrIter))

                        ++ptrIter;



                if (ptrFirst == ptrIter)

                        break;



                ptrFirst = ptrIter;

        }



        cout << nWordCount-1 <<endl;

Hope this approach will help you understand?