Runtime error with file handling

[vavazoom]

#include <iostream>
#include <fstream>
using namespace std;

int main (int argc, char* argv[]) 
{
     int length;
     char * buffer;
     
     string searchWord = argv[2];
     string compareStr;
     int searchSize = searchWord.size();
     
     ifstream readFile;
     readFile.open (argv[1], ios::in );
     ofstream writeFile;
     writeFile.open("newData.txt", ios::out);
     
     readFile.seekg (0, ios::end);
     length = readFile.tellg();
     readFile.seekg (0, ios::beg);
     
     buffer = new char [length];
     
     while(!readFile.eof())
     {    
          readFile.getline(buffer, length);
               compareStr = "";
               for (int i = 0; i < searchSize; ++i)
               {
                    compareStr += buffer[i];
               }
               if (searchWord.compare(compareStr) == 0)
               {
                    writeFile << buffer << "\n";
               }
     }    
     readFile.close();
     writeFile.close();
     delete[] buffer;
     return 0;
}

I am running on Windows XP, and this program runs with test files (70 MB) just fine, but not with the much larger (1.5 GB) file. I get this error "This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information."

Any help would be appreciated! Thanks! :)

-MissVaVaZoom

[Unimportant]

if(file.is_open()) // is a good place to start

Mind you, you may just not have enough RAM, especially with windows XP, to deal with a heap allocation of 1.5GB

Programs such as 7zip, which unzip files of greater sizes in XP, do so with partial reading, intermediary files, or any number of 'workarounds', but your RAM is still a soft cap on memory allocation.

[vavazoom]

I have 2 gigs of RAM at the moment. For partial reading, do you recommend deleting the buffer each time after dumping it into the writeFile?

[vavazoom]

What I failed to mention was not only dumping the buffer, but also only allocating enough space for the length of each line in the readFile.

[Unimportant]

What you are doing appears to be a text search, correct me if I am mistaken.

Why not read in a specific size at a time, such as 100MB. Make sure that the final character of your read is a space, or some other delimiter, as the exact size doesn't matter that much (in this case). Then, when you finish parsing that 100MB, rather than even deleting your array at all, just start writing to array[0] again (no need to reallocate necessarily). You could also just make the array static size in the first place (100MB + 1000 bytes to finish a word), it's usually faster to use stack memory, especially with large amounts of allocation.

Hope that helps

[vavazoom]

Thanks, that helps a lot...will try it out!