| | |
Determining the number of unique words in a .txt file
Please support our C++ advertiser: Intel Parallel Studio Home
![]() |
•
•
Join Date: Nov 2008
Posts: 13
Reputation:
Solved Threads: 0
Hey I have to write a program that reads a text file that contains a list of words, 1 word per line. I have to store the unique words and count the occurrences of each unique word. When the file is completely read, I have to print the words and the number of occurrences to a text file. The output should be the words in alphabetical order along with the number of times they occur. Then print to the file some statistics:
I have to use character arrays instead of strings.
I must use the linear search (something that looks like this) to determine if a word is in the array. The array is an array of structures and that the key is a char array so the string comparison must be used. The search task should be a separate function.
The search must be a separate function that returns an integer values. I cant use a for loop and the function must have only one return statement.
I tried to start off by reading the files and storing the words
Any tips would be appreciated, and thanks in advance.
I have to use character arrays instead of strings.
I must use the linear search (something that looks like this)
C++ Syntax (Toggle Plain Text)
array.int search ( int array[], int number, int key ) { int pos = 0; while(pos < number && array[pos] != key) pos++; if( pos == number) pos = 0; else pos = 1; return pos; }
The search must be a separate function that returns an integer values. I cant use a for loop and the function must have only one return statement.
I tried to start off by reading the files and storing the words
C++ Syntax (Toggle Plain Text)
#include <iostream> #include <fstream> #include <cstdlib> #include <string> #include <iomanip> using namespace std; void displayFile( char []); void main () { int const wordLength = 21; int const Num = 101; int const fileSize = 255; char filename[fileSize]; ifstream inFile; struct { char word[wordLength]; int count; } wordCount; wordCount array[Num]; cout << "Please enter the name of thr file you wish to open: "<< endl; cin >> filename; displayFile (filename); } void displayFile (char fileName [] ) { ifstream infile; char array [101]; char line [101]; int ch; int i; infile.open (fileName); while ((ch = infile.peek()) != EOF) { infile.getline (line, 101); line = array[i]; i++; } for (int j = 0; j<101; j++) cout << array[j] << endl; infile.close ();
Any tips would be appreciated, and thanks in advance.
I would first create a structure
Next an vector (array) of those structures
Now, then the program reads a line, check wordArray if its already in the array. if it is then increment the count. If not, then add a new element to the vector. When done, you will have the information you need to display on the screen.
Another method is to use a <map>, but implementation is left for someone else.
C++ Syntax (Toggle Plain Text)
struct words { std::string word; int count; };
Next an vector (array) of those structures
C++ Syntax (Toggle Plain Text)
vector<words> wordArray;
Now, then the program reads a line, check wordArray if its already in the array. if it is then increment the count. If not, then add a new element to the vector. When done, you will have the information you need to display on the screen.
Another method is to use a <map>, but implementation is left for someone else.
Don't PM me with questions -- you might get a nasty PM in response. If you have a question then post it in one of the forums.
•
•
Join Date: Jun 2006
Posts: 147
Reputation:
Solved Threads: 20
I think you should look at the below code more carefully and must understand the complexity and map datastructure,
if you've any confusion post, I will explain you in more detail then.
if you've any confusion post, I will explain you in more detail then.
cpp Syntax (Toggle Plain Text)
fstream inFile("C:\\Laiq.txt"); string line; std::map<string, int> mapWordCount; string word; std::string::size_type endWord; std::string::size_type startWord; while (std::getline (inFile,line)) { startWord = 0; while ((endWord = line.find_first_of(" ", startWord))!= line.npos) { word = line.substr (startWord, endWord - startWord); mapWordCount[word]++; startWord = endWord+1; } } inFile.close(); for(map<string,int>::const_iterator iter = mapWordCount.begin(); iter != mapWordCount.end(); ++iter) { cout<< "The Frequency of word: "<< iter->first << " is : "<< iter->second <<endl; }
Last edited by Laiq Ahmed; Dec 3rd, 2008 at 5:06 am. Reason: Changing Code Intendation
You have two different problems:
1. Tokenize the input stream (extract words from the stream).
2. Build word dictionary.
The second one has a very simple solution: use std::<map> data structure as Laiq Ahmed mentioned above. However you need another code to process every word with the map-based word dictionary:
There are lots of methods to solve the 1st problem. For example:
Summary:
- use std::ifstream
- use std::string for word buffer
- use std::map<std::string,int> for dictionary with counters
You have a good chance to write a simple and clear code after a proper functional decomposition of pseudocode snippets
...
1. Tokenize the input stream (extract words from the stream).
2. Build word dictionary.
The second one has a very simple solution: use std::<map> data structure as Laiq Ahmed mentioned above. However you need another code to process every word with the map-based word dictionary:
C++ Syntax (Toggle Plain Text)
if the next word is found in the map then do notning or increment this word counter else insert new word in the map endif
C++ Syntax (Toggle Plain Text)
open file stream create an empty map loop // until eof skip non-letters clear word buffer // use std::string::clear() append letters to the word buffer process the word with the map // see #2 endloop process a possible last word traverse map (file word dictionary)
- use std::ifstream
- use std::string for word buffer
- use std::map<std::string,int> for dictionary with counters
You have a good chance to write a simple and clear code after a proper functional decomposition of pseudocode snippets
... •
•
Join Date: Jul 2005
Posts: 1,676
Reputation:
Solved Threads: 262
>>I have to use character arrays instead of strings.
I must use the linear search
That pretty much precludes use of a map. It also means you can't use STL strings within a struct, and may preclude use of a vector to hold the structs, too.
You could use a C style string with either static or dynamic memory within the struct. The static version will limit the maximum length of any possible string entered, but since these strings will be words, then it would be unlikely that any given word would have more than 20 or 30 letters per word. Using dynamic memory to store the string within the struct will minimize the memory wasted by using strings of length less than than maximum length in the static version, but will require you to manage your own memory, which is a bit of a hassle, though not an overwhelming task by any means.
The non-STL structures you could use to store the structs could be either an array or a list. If you don't know the maximum number of possible unique words you could encounter then using a list might be advantageous. You could make a guess as to the max number of words, but that isn't quite as predictable as the maximum size of each word. Linear searching is possible with either lists or arrays.
Sorting the structures by string in alphabetical order before printing/sending values to file could be done in either of several different ways. For beginners, bubble sorts with arrays and insertion sorts with lists seem pretty popular.
I must use the linear search
That pretty much precludes use of a map. It also means you can't use STL strings within a struct, and may preclude use of a vector to hold the structs, too.
You could use a C style string with either static or dynamic memory within the struct. The static version will limit the maximum length of any possible string entered, but since these strings will be words, then it would be unlikely that any given word would have more than 20 or 30 letters per word. Using dynamic memory to store the string within the struct will minimize the memory wasted by using strings of length less than than maximum length in the static version, but will require you to manage your own memory, which is a bit of a hassle, though not an overwhelming task by any means.
The non-STL structures you could use to store the structs could be either an array or a list. If you don't know the maximum number of possible unique words you could encounter then using a list might be advantageous. You could make a guess as to the max number of words, but that isn't quite as predictable as the maximum size of each word. Linear searching is possible with either lists or arrays.
Sorting the structures by string in alphabetical order before printing/sending values to file could be done in either of several different ways. For beginners, bubble sorts with arrays and insertion sorts with lists seem pretty popular.
Klatu Barada Nikto
•
•
Join Date: Nov 2008
Posts: 13
Reputation:
Solved Threads: 0
The maximum length of a word is 20, so the character array needs to be 21. The maximum numbers of words is 100, if there are more than 100 words I need to return a message saying something like "The stats did not use the words after the 100th word"
Anyways, I'm having a hard time even starting the program. I'm starting off by trying to read and store the words of the text file. My text book has code for reading line by line a text file and then displaying it. So I figured I should start with that. Heres the code-
Heres my code split into two functions, however it does not compile. I get an error at the while statement.-
My code is nearly identical except for the fact that I use character arrays instead of strings, so I'm not sure why its not compiling.
Also, is this a good way to start off the program? Or should I try something else.
Anyways, I'm having a hard time even starting the program. I'm starting off by trying to read and store the words of the text file. My text book has code for reading line by line a text file and then displaying it. So I figured I should start with that. Heres the code-
C++ Syntax (Toggle Plain Text)
#include <iostream> #include <fstream> #include <cstdlib> // needed for exit() #include <string> using namespace std; int main() { string filename = "text.dat"; // put the filename up front string line; ifstream inFile; inFile.open(filename.c_str()); if (inFile.fail()) // check for successful open { cout << "\nThe file was not successfully opened" << "\n Please check that the file currently exists." << endl; exit(1); } // read and display the file's contents while (getline(inFile,line)) cout << line << endl; inFile.close(); cin.ignore(); // this line is optional return 0; }
Heres my code split into two functions, however it does not compile. I get an error at the while statement.-
C++ Syntax (Toggle Plain Text)
#include <iostream> #include <fstream> #include <cstdlib> #include <string> #include <iomanip> using namespace std; void displayFile( char []); void main () { int const wordLength = 21; int const Num = 101; int const fileSize = 255; char filename[fileSize]; cout << "Please enter the name of the file you wish to open: "<< endl; cin.getline(filename,fileSize); displayFile (filename); cin.ignore(); } void displayFile (char fileName[] ) { ifstream inFile; char line [101]; inFile.open(fileName); while (getline(inFile, line)) cout << line << endl; inFile.close(); return 0; }
Also, is this a good way to start off the program? Or should I try something else.
•
•
Join Date: Nov 2008
Posts: 13
Reputation:
Solved Threads: 0
Ok I figured out I need to use "inFile.getline(line,length)" instead of "getline(inFile, line)". I created a struct like stated early. However I'm having trouble storing the words from the text file.
I use this function-
Where array, is an array of structs made up of character array"word" and an integer "count". Any help on storing the words and the number of words in the array would be helpfull.
I use this function-
C++ Syntax (Toggle Plain Text)
void displayFile (char fileName[], words array[] ) { int i = 0; ifstream inFile; char line [101]; inFile.open(fileName); while (inFile.getline(line,101)) { cout << line << endl; array[i].word = line; i++; } inFile.close(); }
Read-a-line-in-a-char-array solution has an obvious defect: you must define max line length a priori.
Look at the function which can get words from an input stream (not only from fstream) directly:
Of course, it's possible to adopt this algorithm to extract words from a char array.
Look at the function which can get words from an input stream (not only from fstream) directly:
c++ Syntax (Toggle Plain Text)
// #include <cctype> dependency typedef unsigned char Uchar; // needed for isalpha() /// Get or skip the next word from a stream. /// wbufsize is the max word length + 1 (for null char) bool getWord(char* word, int wbufsize, std::istream& is) { int i = 0, n = wbufsize - 1; // max letters bool fill = (word && n > 0); // word wanted char ch; // current char while (is.get(ch) && !isalpha(Uchar(ch))) ; // skip delimiters if (is) // have a letter do { if (fill) { // have a room word[i] = ch; // append letter if (i >= n) // no more room fill = false; } ++i; // letters counter } while (is.get(ch) && isalpha(Uchar(ch))); if (word)// have a buffer word[i] = '\0'; // end of word return i > 0; // have a word }
•
•
Join Date: Jun 2006
Posts: 147
Reputation:
Solved Threads: 20
I tried a different Approach to achieve the same
Hope this approach will help you understand?
cpp Syntax (Toggle Plain Text)
char* arr = " Hallo WOrld"; char* ptrFirst = arr; char* ptrIter = arr; int nWordCount = 0; // Base Condition. while (isspace (*ptrFirst)) ++ptrFirst; ptrIter = ptrFirst; while (ptrFirst) { while (isalpha(*ptrIter)) { ++ptrIter; } ++nWordCount; while (isspace (*ptrIter)) ++ptrIter; if (ptrFirst == ptrIter) break; ptrFirst = ptrIter; } cout << nWordCount-1 <<endl;
Hope this approach will help you understand?
![]() |
Other Threads in the C++ Forum
- Previous Thread: Dynamic arrays in a class
- Next Thread: passing fstream to a function and searching for a string
| Thread Tools | Search this Thread |
api array based beginner binary bitmap c++ c/c++ calculator char char* class code coding compile compiler console conversion count database delete deploy developer directshow dll download dynamic dynamiccharacterarray email encryption error file forms fstream function functions game givemetehcodez google graph gui homeworkhelp homeworkhelper iamthwee ifstream input int java lib linkedlist linker list loop looping loops map math matrix memory multiple news node number numbertoword output parameter pointer problem program programming project proxy python random read recursion recursive reference rpg sorting string strings temperature template test text text-file tree unix url variable vector video visualstudio win32 windows winsock word wordfrequency wxwidgets






