What is the most efficient (fastest) way to load data from a file on HDD to RAM? (which would allow to only load a limited section of that file - eg. load only half of the file etc.)

4 Years
Discussion Span
Last Post by mike_2000_17

For windows. I mean in terms of what function to use. Are there more efficient ways to read? (especially if you'll be reading long sequences?)


Standard file-streams are buffered for efficiency. They read data in advance from the file and temporarily store it in a buffer which is emptied as you extract data from it. So, just use the std::ifstream, which has an underlying std::filebuf accessible through rdbuf(). Normally, the buffering is set to be a reasonably good trade-off between not loading too much file data into memory and having enough in-memory data to deliver it in a timely fashion to the program reading it. However, if you want to manually adjust the size of the buffer, you can use the following code:

const std::size_t buf_size = 32768;   // choose some buffer size.
char* my_buffer = new char[buf_size]; // create a buffer of that size.
std::ifstream in_file("C:\\Some\\Path\\To\\file.txt");
in_file.rdbuf()->pubsetbuf(my_buffer, buf_size);   // give the buffer to the filebuf.

// .. read data from 'in_file'

delete[] my_buffer;

Edited by mike_2000_17: typo

This topic has been dead for over six months. Start a new discussion instead.
Have something to contribute to this discussion? Please be thoughtful, detailed and courteous, and be sure to adhere to our posting rules.