CSV file

Closed Thread

Join Date: Nov 2005
Posts: 29
Reputation: Aldin is an unknown quantity at this point 
Solved Threads: 0
Aldin Aldin is offline Offline
Light Poster

CSV file

 
0
  #1
Dec 14th, 2005
Hi
I am having a file called Log.csv which contains 196 columns 9 of them empty and 1447 rows, I start reading the file but as I think I am facing a problem of getting the whole data, actually it gives me empty screen when print out on the screen.

I need some help in reading the csv file and put it in array because later on I have to get the date (from the first column) and min,max for some other columns.
I am really gratefull if anyone can help me.

I am going to paste 1 line from my Log.csv and then paste my code.

11/10/2004,00:00:45,190,0,13.94,346.81,201.11,18.46,32,3,47,6, ,127,230,228,228,348,128,64,64,8,6,6,50,50,50,34,34,35,6,6,6,0, ,63,230,228,230,344,66,66,66,8,6,7,50,50,50,35,36,35,6,6,6,0, ,49.41,0.00,-0.30,30.91,00,63,6,4,0, ,49.67,0.00,0.06,30.91,00,63,6,4,0, ,49.67,0.00,0.06,30.91,00,63,6,4,0, ,49.67,0.00,-0.30,30.91,00,63,6,4,0, ,49.67,0.00,-0.30,32.87,00,63,6,4,0, ,49.67,0.00,-0.30,32.87,00,63,6,4,0, ,49.94,0.00,-0.30,30.91,00,63,6,4,0, ,49.41,0.00,-0.30,30.91,00,63,6,4,0, ,49.67,0.00,-0.30,30.91,00,63,6,4,0, ,49.41,0.00,-0.30,32.87,00,63,6,4,0, ,49.67,0.00,-0.30,30.91,00,63,6,4,0, ,49.67,0.00,-0.30,32.87,00,63,6,4,0, ,49.41,0.00,0.06,32.87,00,63,6,4,0, ,49.67,0.00,0.06,30.91,00,63,6,4,0, ,




My code is

#include <iostream>
#include <fstream>
using namespace std;

float matrix_points[1447][196];
int main()
{
std::ifstream input_file("Log.csv");
int row(0), col(0);
char buffer[256];
char line[255];

input_file.clear();
input_file.seekg(0);

/* brings all the data in .csv file and put them in an array*/
while(!(input_file.eof()))
{
for ( col=0; col<196; col++)
{

if (matrix_points[row][col] != matrix_points[row][195])
{
input_file.getline(line, sizeof(buffer), ',');
matrix_points[row][col] = atof(line);
}

else
{
input_file.getline(line, sizeof(buffer), '\n');
matrix_points[row][12] = atof(line);

}
}

row++;
}
/* for loop to print data on screen*/
for (int i=0; i< row ; i++)
for (int j=0; j< col ; j++)
{

if (matrix_points[i][j] != matrix_points[i][195])
cout << matrix_points[i][j] << " " ;
else
cout << matrix_points[i][195]<< endl;
}


cout << row <<endl;
cout << "\n\n";
// system("PAUSE");
return 0;
}
Quick reply to this message  
Join Date: Jul 2005
Posts: 1,673
Reputation: Lerner is a name known to all Lerner is a name known to all Lerner is a name known to all Lerner is a name known to all Lerner is a name known to all Lerner is a name known to all 
Solved Threads: 261
Lerner Lerner is offline Offline
Posting Virtuoso

Re: CSV file

 
0
  #2
Dec 14th, 2005
As a new member to the board it will behoove you to read the instructions on how to post code to maintain the indentation protocol you wrote the code with. To do that, enclose any posted code in code tags, that is [ code ] before the code and [ /code] after the code (with out the space between the square brackets and the contents between the brackets.

As a new programmer, I encourage you to try to do things modularly. In this case I'd try to read in just the first item from the file. When I could do that, I'd try reading in the first line of the file. When I could do that, I'd try reading in the entire file.

When you get to the stage where you are reading the whole file, note that using the return of eof() as a conditional in a control statement is not the best idea---it frequently leads to undefined behaviour such as inserting the last data read in twice. You can search the board for excruciating detail if you want.

Instead, since each line has exactly 196 fields and there are exactly 1447 lines, I'd use nested for loops rather than a while loop/for loop combination use input_file.getline(line, 256, ','); as the conditional (why bother calling sizeof() each time??). Then within the loop I'd do something like this:
  1. for(row...
  2. for(col...
  3. input_file.getline(line, 256, ','); //(why bother calling sizeof() each time??)
  4. matrix_points[row][col] = atof(line);
Now if I didn't know there were exactly 1447 lines, then a conditional loop would be reasonable.
  1. row = -1;
  2. while (input_file.getline(line, 256, ',')) //if this works there are 195 more to go on this line
  3. matrix_points[++row][0] = atof(line);//start a new row and
  4. //place the sentinel value at col == 0;
  5. for(col = 1; ... //the rest go from 1 to < 196
  6. matrix_points[row][col] = atof(line);

In addition I'm not sure what atof() will do with a string that consists of all whitespace, so I would develop a protocol to identify a "blank" column, and not call atof() if line consisted of a single space.
Quick reply to this message  
Join Date: Nov 2005
Posts: 29
Reputation: Aldin is an unknown quantity at this point 
Solved Threads: 0
Aldin Aldin is offline Offline
Light Poster

Re: CSV file

 
0
  #3
Dec 15th, 2005
Thank you Lerner for your reply, I did many things starting with reading the first item and the I cut a bit of the Log.csv file and it was working perfect with only 10 column and 1447 row, and it enabled me to get the min and max for one of the column.

  1. #include <iostream>
  2. #include <cstdlib>
  3. #include <fstream>
  4. using namespace std;
  5.  
  6. float matrix_points[10000][10];
  7. int main()
  8. {
  9. std::ifstream input_file("test.csv");
  10. int counter(0);
  11. char buffer[256];
  12. char line[255];
  13.  
  14. input_file.clear();
  15. input_file.seekg(0);
  16.  
  17. while(!(input_file.eof()))
  18. {
  19. input_file.getline(line, sizeof(buffer), ',');
  20. matrix_points[counter][0] = atoi(line);
  21. input_file.getline(line, sizeof(buffer), ',');
  22. matrix_points[counter][1] = atoi(line);
  23. input_file.getline(line, sizeof(buffer), ',');
  24. matrix_points[counter][2] = atof(line);
  25. input_file.getline(line, sizeof(buffer), ',');
  26. matrix_points[counter][3] = atof(line);
  27. input_file.getline(line, sizeof(buffer), ',');
  28. matrix_points[counter][4] = atof(line);
  29. input_file.getline(line, sizeof(buffer), ',');
  30. matrix_points[counter][5] = atof(line);
  31. input_file.getline(line, sizeof(buffer), ',');
  32. matrix_points[counter][6] = atoi(line);
  33. input_file.getline(line, sizeof(buffer), ',');
  34. matrix_points[counter][7] = atoi(line);
  35. input_file.getline(line, sizeof(buffer), ',');
  36. matrix_points[counter][8] = atoi(line);
  37. input_file.getline(line, sizeof(buffer), '\n');
  38. matrix_points[counter][9] = atoi(line);
  39. counter++;
  40. }
  41.  
  42. for (int i=0; i<counter; i++){
  43. cout << matrix_points[i][0] << " " << matrix_points[i][1] << " " << matrix_points[i][2] << " "
  44. << matrix_points[i][3] << " " << matrix_points[i][4] << " " << matrix_points[i][5] << " "
  45. << matrix_points[i][6] << " " << matrix_points[i][7] << " " << matrix_points[i][8] << " "
  46. << matrix_points[i][9] << endl;
  47. }
  48. cout << counter<<endl;
  49.  
  50. cout << "\n\n";
  51. // system("PAUSE");
  52. return 0;
  53. }

but all the problem started when I start increasing the numbers of the column I want to read, yesterday code start only giving me a one vertical line of 0s. this why I seek anyone help. I attach the Log.csv in a zip file
Attached Files
File Type: zip Log.zip (82.6 KB, 16 views)
Quick reply to this message  
Join Date: May 2004
Posts: 178
Reputation: jim mcnamara is on a distinguished road 
Solved Threads: 10
jim mcnamara jim mcnamara is offline Offline
Junior Poster

Re: CSV file

 
0
  #4
Dec 15th, 2005
This is a simple-minded C version. You can use strtok in C++ as well.

  1. #include <stdio.h>
  2. #include <stdlib.h>
  3. #include <string.h>
  4. #define MAXFLDS 200 /* maximum possible number of fields */
  5. #define MAXFLDSIZE 32 /* longest possible field + 1 = 31 byte field */
  6.  
  7. void parse( char *record, char *delim, char arr[][MAXFLDSIZE],int *fldcnt)
  8. {
  9. char*p=strtok(record,delim);
  10. int fld=0;
  11.  
  12. while(*p)
  13. {
  14. strcpy(arr[fld],p);
  15. fld++;
  16. p=strtok('\0',delim);
  17. }
  18. *fldcnt=fld;
  19. }
  20.  
  21. int main(int argc, char *argv[])
  22. {
  23. char tmp[1024]={0x0};
  24. int fldcnt=0;
  25. char arr[MAXFLDS][MAXFLDSIZE]={0x0};
  26. int recordcnt=0;
  27. FILE *in=fopen(argv[1],"r"); /* open file on command line */
  28.  
  29. if(in==NULL)
  30. {
  31. perror("File open error");
  32. exit(EXIT_FAILURE);
  33. }
  34. while(fgets(tmp,sizeof(tmp),in)!=0) /* read a record */
  35. {
  36. int i=0;
  37. recordcnt++;
  38. printf("Record number: %d\n",recordcnt);
  39. parse(tmp,",",arr,&fldcnt); /* whack record into fields */
  40. for(i=0;i<fldcnt;i++)
  41. { /* print each field */
  42. printf("\tField number: %3d==%s\n",i,arr[i]);
  43. }
  44. }
  45. fclose(in);
  46. return 0;
  47. }
Quick reply to this message  
Join Date: Nov 2005
Posts: 29
Reputation: Aldin is an unknown quantity at this point 
Solved Threads: 0
Aldin Aldin is offline Offline
Light Poster

Re: CSV file

 
0
  #5
Dec 19th, 2005
Hi everyone
I am absolutely stacked in the above csv file, please help:cry:
Quick reply to this message  
Join Date: Aug 2005
Posts: 15,343
Reputation: Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute 
Solved Threads: 1460
Team Colleague
Featured Poster
Ancient Dragon's Avatar
Ancient Dragon Ancient Dragon is offline Offline
Still Learning

Re: CSV file

 
0
  #6
Dec 19th, 2005
here is a c++ version that puts the strings into a 2d vector
  1. #pragma warning(disable: 4786) // VC++ 6.0 disable warning about debug line too long
  2. #include <iostream>
  3. #include <fstream>
  4. #include <vector>
  5. #include <string>
  6. using namespace std;
  7. typedef vector<string> LINE;
  8.  
  9. int main()
  10. {
  11. string line;
  12. int pos;
  13. vector<LINE> array;
  14.  
  15. ifstream in("log.csv");
  16. if(!in.is_open())
  17. {
  18. cout << "Failed to open file" << endl;
  19. return 1;
  20. }
  21. while( getline(in,line) )
  22. {
  23. LINE ln;
  24. while( (pos = line.find(',')) >= 0)
  25. {
  26. string field = line.substr(0,pos);
  27. line = line.substr(pos+1);
  28. ln.push_back(field);
  29. }
  30. array.push_back(ln);
  31. }
  32. return 0;
  33. }
Quick reply to this message  
Join Date: Nov 2005
Posts: 29
Reputation: Aldin is an unknown quantity at this point 
Solved Threads: 0
Aldin Aldin is offline Offline
Light Poster

Re: CSV file

 
0
  #7
Dec 19th, 2005
Hi
I do not know what to say Ancient Dragons but the code you sent giving me black screen that saying (press any key to continue) :rolleyes:
Quick reply to this message  
Join Date: Aug 2005
Posts: 15,343
Reputation: Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute 
Solved Threads: 1460
Team Colleague
Featured Poster
Ancient Dragon's Avatar
Ancient Dragon Ancient Dragon is offline Offline
Still Learning

Re: CSV file

 
0
  #8
Dec 19th, 2005
look at the code -- it doesn't print anything on the screen. All it does is read the csv file and put the strings into an array. What you do with it after that is up to you because I have no idea what you want (my crystal ball isn't working today).
Quick reply to this message  
Join Date: Nov 2005
Posts: 29
Reputation: Aldin is an unknown quantity at this point 
Solved Threads: 0
Aldin Aldin is offline Offline
Light Poster

Re: CSV file

 
0
  #9
Dec 19th, 2005
:o
Hi again
I know you do not have your krystal ball with you, but if you read my first email you would know that I would like to get the date from the first column and min and max for other columns, can you give me some of your time and give me some hint on your code to make it easier for me.
Quick reply to this message  
Join Date: Aug 2005
Posts: 15,343
Reputation: Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute Ancient Dragon has a reputation beyond repute 
Solved Threads: 1460
Team Colleague
Featured Poster
Ancient Dragon's Avatar
Ancient Dragon Ancient Dragon is offline Offline
Still Learning

Re: CSV file

 
0
  #10
Dec 19th, 2005
vectors are similar to normal arrays that relieve you with all the hassel and mistakes of memory allocations. The 2d vector is a vector of vectors. The inner-most vector contains the strings for each line, and the outer vector is the array of each line. So to get column 0 of row 0
  1. vector<LINE> array;
  2.  
  3. // get row 0
  4. LINE& row = array[0];
  5. // get column 0
  6. string& s = row[0];

The first row (from your data file) contains column names, so the above will get the name in column 0. There are other tricks to get a column based on its name instead of a column number -- if you ever add or subtract columns in the data file, using column names instead of numbers makes your program more portable, but slightly slower to process.
  1. string GetColData(int row_number,string ColName)
  2. {
  3. LINE& columnNames = array[0];
  4. // find the collumn with the above column name
  5. for(int col = 0; col < columnNames.size(); col++)
  6. {
  7. if( ColName == ColumnNames[i] )
  8. {
  9. // get data for this column
  10. LINE& row = array[row_number];
  11. return row[col];
  12. }
  13. return "";
  14. }
Quick reply to this message  
Closed Thread

This thread is more than three months old.
Perhaps start a new thread instead?
Message:


Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC