Member Avatar for doublebond

Hi Guys,

I want to read a huge file which contains address bits, data bits and instruction bits. Please let me know how to do this.

Cheers,
Bond

  • 2 Contributors
  • 3 Replies
  • 471 Views
  • 7 Hours Discussion Span
  • Latest Post Latest Post by doublebond

Dani AI

Generated

Practical, evergreen guidance for processing very large trace files (address/data/instruction bits): first determine the on-disk format (text lines vs fixed-size binary records) and whether the file size exceeds available RAM. If the trace will not fit comfortably in memory, avoid loading it whole — either stream it (parse record-by-record) or use an OS memory map so the kernel pages data on demand. (man7.org)

For line-oriented text traces, prefer a line-at-a-time loop and a parser that works on a single line (tokenize or use fast numeric parsers like std::from_chars). Avoid control with eof() — loop on the read operation itself so failed reads don’t produce spurious records. Example pattern for text input:

std::ifstream in("trace.txt");
std::string line;
while (std::getline(in, line)) {
    // parse address/data/instr from 'line' (use substr, std::from_chars, string_view)
}

References on correct line reading and the eof pitfall. (en.cppreference.com)

For fixed-size or binary traces, read large blocks (e.g. 256KiB–4MiB) and scan the buffer rather than calling per-byte I/O. Check the number of bytes actually read (gcount) after each read and process only those bytes. Disable unnecessary iostream synchronization and avoid per-character string growth for speed. Example block-read loop:

std::ifstream in("trace.bin", std::ios::binary);
std::vector<char> buf(1 << 20); // 1 MiB
while (in.read(buf.data(), buf.size()) || in.gcount()) {
    std::size_t n = static_cast<std::size_t>(in.gcount());
    // process buf.data()..buf.data()+n
}

See the std::istream::read/gcount details and the sync_with_stdio performance option.

Memory-mapping (mmap on POSIX, MapViewOfFile on Windows) simplifies random access and can be fastest for very large read-only analyses, but watch portability and address-space limits (32-bit processes) and map permissions. If the thread’s sample code is doing character-by-character reads and using eof() as the loop guard (as shown earlier), that pattern commonly produces off-by-one or duplicated-last-record bugs — switch to the above idioms and parse deterministically. (man7.org)

Recommended Answers

All 3 Replies

Member Avatar for doublebond 
#include <fstream>
 #include <iostream>
 #include <cstdlib>
 using namespace std;
 using std::ifstream;
 using std::ofstream;
 using std::cout;
 int main( )
 {
         ifstream fin;
         ofstream fout;
         fin.open("trace.txt");

         if (fin.fail( ))
         {
                 cout << "Input file opening failed.\n";
                 exit(1);
         }

         fout.open("stats.txt");

         if (fout.fail( ))
        {
         cout << "Output file opening failed.\n";
         exit(1);
        }
int next;
         int n = 1;

         fin.get(next);
         fout << n << " ";
         while (! fin.eof( ))
         {
                fout << next;

         if (next == 1)
         {
                 n++;
                 fout << n << ' ';
         }

         fin.get(next);
 }

 fin.close( );
 fout.close( );
 return 0;
 }
Member Avatar for Tom Gunn

How huge is huge? What problem are you having?

Member Avatar for doublebond

How huge is huge? What problem are you having?

The thing is that--- I am trying take a trace of the traffic flowing in the bus which connects the memory components, i am not yet done with pulling the trace out. This trace contains address bits, data bits and instruction bits. The problem with me is like, i want read the trace.txt to analyze it. This trace.txt is expected to be very huge as we running large application programs on the machine.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.