Ad:

C++ Discussion Thread
Unsolved
Views: 8391

Sep 13th, 2006

using ifstream, counting elements of each line in a dat file

Expand Post »

So I have a question regarding ifstream and reading a file.

I have to read in this file, but some of the lines of the file will have missing sections of data. These lines I want to just throw away and then
continue reading the file on the next line until I've read in the whole thing

A small chunk of the data file (which I'm using for testing now...the actual file is like 1 million lines long) looks like so:

Year,Month,Day,Time(hhmmss.mm),Latitude,Longitude,Magnitude,Depth
1973,01,05,053105.80, 33.47, 140.87,4.5, 56
1973,01,05,114837.50, 33.16, 140.91,3.9, 33
1973,01,06,102116.30, 33.49, 140.85,4.2, 33
1973,01,06,112154.70, 33.27, 140.93,4.5, 46
1973,01,06,145552.80, 33.15, 140.71,4.7, 61
1973,01,09,022114.80, 37.81, 141.69,3.5, 59
1973,01,10,075113.60, 33.62, 140.51,4.1,103
1973,01,10,135240.60, 36.84, 137.36,4.4, 33
1973,01,12,074443.90, 33.11, 140.98,3.9, 20
1973,11,30,170127.70, 34.69, 138.57, ,223
1973,01,13,002147.50, 33.28, 141.06,3.9, 40
1973,01,13,231451.10, 33.33, 141.05,3.8, 33
1973,01,15,131515.80, 36.93, 141.93,4.1, 43
1973,01,15,170405.80, 33.37, 140.82,4.9, 58
1973,01,16,181727.80, 35.96, 139.49,4.1, 60
1973,01,16,201134.10, 37.80, 141.71,4.1, 58
1973,01,17,030734.50, 33.28, 140.61,4.0, 58

notice the one line ending with 223 that is missing a longitude value...that line should be thrown away when I read the whole file as it is an incomplete data record. I called the parameter for the file eqFile (it's earthquake data)

So the procedure I use to make this happen is like so:

C++ Syntax (Toggle Plain Text)
void foo(string &eqFile)
{
        ifstream inp;
        char thrw = ' ';
	float year,month,day,time,lat,lng,mag,depth;
 
inp.open(eqFile.c_str(), ifstream::in);
 
    //if the file is not found
    if(inp.fail())
    {
        cout<< "The file \"" << eqFile << "\" does not exist. Exiting... " << endl;
        exit(1);
    }
    //read from the file if it is found
    else
    {
		//strip off the first line. It is unncessary
		do
		{
			inp.get(thrw);
			cout << thrw;
		}
		while(thrw != '\n');
 
		while(!inp.eof())
        {
			if(inp >> year >> thrw >> month >> thrw >> day >> thrw >> time >>
				thrw >> lat >> thrw >> lng >> thrw >> mag >> thrw >> depth)
			{
				cout << year << "," << month << "," << day << "," << time << ",";
				cout << lat << "," << lng << "," << mag << "," << depth;
				cout << endl;
			}
        }
 
        inp.close(); //close off the file now that it's done being used
    }
}
void foo(string &eqFile)
{
        ifstream inp;
        char thrw = ' ';
	float year,month,day,time,lat,lng,mag,depth;

inp.open(eqFile.c_str(), ifstream::in);
         
    //if the file is not found
    if(inp.fail())
    {
        cout<< "The file \"" << eqFile << "\" does not exist. Exiting... " << endl;
        exit(1);
    }
    //read from the file if it is found
    else
    {
		//strip off the first line. It is unncessary
		do
		{
			inp.get(thrw);
			cout << thrw;
		}
		while(thrw != '\n');

		while(!inp.eof())
        {
			if(inp >> year >> thrw >> month >> thrw >> day >> thrw >> time >>
				thrw >> lat >> thrw >> lng >> thrw >> mag >> thrw >> depth)
			{
				cout << year << "," << month << "," << day << "," << time << ",";
				cout << lat << "," << lng << "," << mag << "," << depth;
				cout << endl;
			}
        }
		
        inp.close(); //close off the file now that it's done being used
    }
}

What this does however is never stops running once it reaches that line where the errored record exists. I need it to skip over the line somehow and continue reading as before

Can anyone tell me what I've got wrong here? There is something I don't get about when ifstream reading encounters an error in a line i suppose

Similar Threads

dat,txt file and csv file???? in C++
dat file txt file help???? in C
Read data from a two-way array from a .DAT file? in C++
Reading .dat file in Visual Basic 4 / 5 / 6
Reading in a .dat file from SSH shell command line in Java

kharri5

Reputation Points: 13

Solved Threads: 0

Junior Poster in Training

Offline

56 posts
since Jan 2005

Permalink

Sep 14th, 2006

Re: using ifstream, counting elements of each line in a dat file

I would use getline() instead of that loop, then parse the line just read

C++ Syntax (Toggle Plain Text)
std::strine line;
while( getline(inp,line) )
{
 
   // check line for correct number of commas
   // count the number of commas -- should be 8 
   // and make sure the last comma is not the last thing on the line
   // if there are not 8 commas or the last comma is the last one on the
   // line, then continue to the top of this loop
   //
   // line is probably ok to use.  So now break it up into individul fields.
   // use std::string's find() to locate the position of the next comma, then    
   // substr() method to extract the text.
   // 
   // or use istringstream() -- but I'm not familar with it so someone else
   // can probably give you good example if you need it.
}
std::strine line;
while( getline(inp,line) )
{

   // check line for correct number of commas
   // count the number of commas -- should be 8 
   // and make sure the last comma is not the last thing on the line
   // if there are not 8 commas or the last comma is the last one on the
   // line, then continue to the top of this loop
   //
   // line is probably ok to use.  So now break it up into individul fields.
   // use std::string's find() to locate the position of the next comma, then    
   // substr() method to extract the text.
   // 
   // or use istringstream() -- but I'm not familar with it so someone else
   // can probably give you good example if you need it.
}

Ancient Dragon

Sponsor

Team Colleague

Featured Poster

Reputation Points: 5608

Solved Threads: 2282

Retired and Enjoying Life

Offline

21,950 posts
since Aug 2005

Permalink

Sep 14th, 2006

Re: using ifstream, counting elements of each line in a dat file

Click to Expand / Collapse Quote originally posted by Ancient Dragon ...

I would use getline() instead of that loop, then parse the line just read
C++ Syntax (Toggle Plain Text)
std::strine line;
while( getline(inp,line) )
{
 
   // check line for correct number of commas
   // count the number of commas -- should be 8 
   // and make sure the last comma is not the last thing on the line
   // if there are not 8 commas or the last comma is the last one on the
   // line, then continue to the top of this loop
   //
   // line is probably ok to use.  So now break it up into individul fields.
   // use std::string's find() to locate the position of the next comma, then    
   // substr() method to extract the text.
   // 
   // or use istringstream() -- but I'm not familar with it so someone else
   // can probably give you good example if you need it.
}
std::strine line; while( getline(inp,line) ) { // check line for correct number of commas // count the number of commas -- should be 8 // and make sure the last comma is not the last thing on the line // if there are not 8 commas or the last comma is the last one on the // line, then continue to the top of this loop // // line is probably ok to use. So now break it up into individul fields. // use std::string's find() to locate the position of the next comma, then // substr() method to extract the text. // // or use istringstream() -- but I'm not familar with it so someone else // can probably give you good example if you need it. }

Thanks for the suggestions Ancient Dragon, but...

The above code is nice, but I'm looking for a way to make it work without having to use getline. Getline seems like it is much more work than I should have to do. I feel like a one or two liner could solve my problem, but just don't know enough abotu the dynamics of using inp >> some_var and what happens when it doesn't find some_var. Any tips on making it work the way I have it currently.

kharri5

Reputation Points: 13

Solved Threads: 0

Junior Poster in Training

Offline

56 posts
since Jan 2005

Permalink

Sep 14th, 2006

Re: using ifstream, counting elements of each line in a dat file

There is no quick and easy solution to your problem. As you have already found out just reading the data into integers doesn't solve your problem because the program doesn't distinguish one line from another. Only getline() can do that for you. Once the line is read, the line must be parsed to see if it contains any errors (or missing elements). Again, ifstream's simple extraction operator >> will not do that for you.

Last edited by Ancient Dragon; Sep 14th, 2006 at 10:30 am.

Ancient Dragon

Sponsor

Team Colleague

Featured Poster

Reputation Points: 5608

Solved Threads: 2282

Retired and Enjoying Life

Offline

21,950 posts
since Aug 2005

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.

Previous Thread in C++ Forum Timeline: character count

Next Thread in C++ Forum Timeline: I got a Link error 1181 with fresh code from source safe.

DaniWeb Message

Cancel Changes

User Name
Password