| | |
improve performance of the following io codes
![]() |
•
•
Join Date: Oct 2005
Posts: 22
Reputation:
Solved Threads: 0
can anyone offer performance tips to improve the running time ?
this function opens a file (a 7000 row by 30 col), and stores each elements in a matrix data . current running time is 4 sec, and i desperately need to minimize the running time as i need to iterate thru
thousands of such files please help, thanks
this function opens a file (a 7000 row by 30 col), and stores each elements in a matrix data . current running time is 4 sec, and i desperately need to minimize the running time as i need to iterate thru
thousands of such files please help, thanks
C Syntax (Toggle Plain Text)
void iocsv(vector<vector<double> >& data, string path) { string s; ifstream inFile; inFile.open(path.c_str()); if (inFile) { while (getline(inFile, s)) { vector<double> col; tokenizer<escaped_list_separator<char> > tok(s); for (tokenizer<escaped_list_separator<char> >::iterator beg=tok.begin(); beg!=tok.end(); ++beg) { istringstream price; price.str(*beg); double x; price >> x; col.push_back(x); } data.push_back(col); } } else { cerr << "Warning: cannot open file " << path << endl; cerr << "Program terminating ......" << endl; } inFile.close(); }
•
•
•
•
can anyone offer performance tips to improve the running time ?
this function opens a file (a 7000 row by 30 col), and stores each elements in a matrix data . current running time is 4 sec, and i desperately need to minimize the running time as i need to iterate thru
thousands of such files please help, thanks
C Syntax (Toggle Plain Text)
void iocsv(vector<vector<double> >& data, string path) { string s; ifstream inFile; inFile.open(path.c_str()); if (inFile) { while (getline(inFile, s)) { vector<double> col; tokenizer<escaped_list_separator<char> > tok(s); for (tokenizer<escaped_list_separator<char> >::iterator beg=tok.begin(); beg!=tok.end(); ++beg) { istringstream price; price.str(*beg); double x; price >> x; col.push_back(x); } data.push_back(col); } } else { cerr << "Warning: cannot open file " << path << endl; cerr << "Program terminating ......" << endl; } inFile.close(); }
*Voted best profile in the world*
How long does an empty loop take?
Separate the "time to read the file" from the time to "tokenise the file".
It it takes <1 second, then there might be something you can do.
If it takes >3 seconds, then all your tokenising/vector stuff is not the problem.
> and i desperately need to minimize the running time as i need to iterate thru
> thousands of such files
Or just not worry about it and let the program run overnight, and it will all be done by morning anyway. If that can be done, it certainly isn't worth spending more than a day trying to make it vastly more efficient.
By your measure, it's about 900 files per hour.
C Syntax (Toggle Plain Text)
while (getline(inFile, s)) { }
It it takes <1 second, then there might be something you can do.
If it takes >3 seconds, then all your tokenising/vector stuff is not the problem.
> and i desperately need to minimize the running time as i need to iterate thru
> thousands of such files
Or just not worry about it and let the program run overnight, and it will all be done by morning anyway. If that can be done, it certainly isn't worth spending more than a day trying to make it vastly more efficient.
By your measure, it's about 900 files per hour.
Last edited by Salem; Oct 18th, 2006 at 4:45 pm.
![]() |
Similar Threads
- C++ Performance Tips (C++)
- Clean Your Prefetch to Improve Performance (Windows tips 'n' tweaks)
- Improve performance ISAPI (C++)
- Soundcard improves performance? (PCI and Add-In Cards)
- Modify Settings to Improve Performance (Windows tips 'n' tweaks)
Other Threads in the C Forum
- Previous Thread: draw diagnola ellipse
- Next Thread: representing a quadratic equation
| Thread Tools | Search this Thread |
#include adobe ansi api array asterisks binarysearch changingto char character cm copyimagefile cprogramme creafecopyofanytypeoffileinc createcopyoffile csyntax database directory dynamic execv feet fgets file fork forloop frequency function getlasterror givemetehcodez global grade graphics gtkgcurlcompiling hacking hardware highest histogram i/o include incrementoperators infiniteloop input interest kernel keyboard kilometer license linked linkedlist linux linuxsegmentationfault list locate logical_drives looping loopinsideloop. lowest match matrix meter microsoft motherboard mqqueue mysql number odf opensource owf pattern pdf performance pointer posix probleminc process program programming radix recursion recv repetition research reversing scanf segmentationfault sequential shape socket socketprograming standard string systemcall threads turboc unix user voidmain() wab windows.h windowsapi






