Problems reading a Binary File more than 2.0 GB with VC++

[Georges5]

Hi all,

I have written a C++ code that reads data (as double) from a huge binary file with size 2.5 GB, then process the data and write it to another binary file.

What is really strange, the code works fine and smooth in Borland C Builder 6, but since we ported to VC++ 2008 the code starts writing rubbish data to the output file after the offset reaches 2^32 - 1 (which is 2 G.B = 2147483647).
I am reading the binary file as double values so I am reading 8 bytes at a time, so when I read data at address "2147483640" the value read is correct, then when I want to get the data at the next address, which is at "2147483648" it reads rubbish data. and I note that after this address all data written to the output file is the same.

here is the code on how I read and write the files:

double getValue ( int org, int dst, int attrNr )
{
   if( org >= total || dst >= total)
   {
      cout << "ERR:  Origin-destination pair (";
      cout << org << "," << dst << ") not within OD range (" << nrOfOD << ")." << endl;
      abort();
   }
  
   double tmpDbl;
 
   unsigned int position = (attrNr*total*total + org*total + dst)*sizeof(double); // this is where I calculate the offset of the value in the binary file.

  // fseek32 is a customized fseek function to reset the offset incase it exceeds 2^3 -1 

   fseek32( filepointer , positie , SEEK_SET  ); //Note that filepointer is pointer to the Large Binary file initialized in the class contructor
 
   int result = fread(&tmpDbl, sizeof(double), 1, filepointer);

    if ( result != 1 )
   {
      cout<<"at address: "<<position <<endl;
      cout << "ERR: Access of attribute nr. " << attrNr << " of OD-pair (";
      cout << org << "," << dst << ") in matrix " << " failed.";
	  
      abort();
   }
    return tmpDbl;
}

int fseek32( FILE * stream, unsigned int offset, int origin )
{
   int retVal;
   if ( offset > 2147483647 ) // 2^31 - 1
   {
    
      retVal = fseek( stream, 2147483647, SEEK_SET);
      offset = offset - 2147483647;    // make sure remaining offset is within bounds of signed long int	
      if ( retVal != 0)  return retVal; // first jump failed, stop
      return fseek( stream, (signed long int) offset, SEEK_CUR);

   }

[ddanbe]

fseek32( filepointer , positie , SEEK_SET )
What is the value of positie or is it a typo?

[Georges5]

fseek32( filepointer , positie , SEEK_SET ) What is the value of positie or is it a typo?

Thank you for replying.

positie is actually the position calculated above the fseek32, its just a mistyping.
position is the address of the data in the binary file.

[ArkM]

This fseek32 function is an implementation-dependent nonstandard trick: standard C RTL fseek/ftell expects/returns long int (signed 32-bit for VC++) values only. I think it can't move file pointers over 31-bit boundary (you see that).
Try to rewrite fseek32 with VC++ non-standard _fseeki64 function with long long (64-bit, old typename __int64) offset parameter.

[Georges5]

This fseek32 function is an implementation-dependent nonstandard trick: standard C RTL fseek/ftell expects/returns long int (signed 32-bit for VC++) values only. I think it can't move file pointers over 31-bit boundary (you see that). Try to rewrite fseek32 with VC++ non-standard _fseeki64 function with long long (64-bit, old typename __int64) offset parameter.

Thanks for the valued suggestion.
Never used _fseek64 , can you give a short example, another request, since this _fseek64 is non-standard where can I find it. or which Header file shall I include?

[ddanbe]

It's _fseeki64 I think and it's in

[ArkM]

I never used _fseeki64 too. It's declared in VC++ . It has the same parameters as fseek but offset parameter has long long (64-bit in VC++) type.

You have VC++ 2008 installed. Write _fseeki64, select this id then press F1 ;) or see help on fseek function.

It seems you need a very simple correction:

inline
int fseek32(FILE* stream, unsigned int offset, int origin)
{
    return _fseeki64(stream,offset,SEEK_SET);
}

[Georges5]

I never used _fseeki64 too. It's declared in VC++ . It has the same parameters as fseek but offset parameter has long long (64-bit in VC++) type.

You have VC++ 2008 installed. Write _fseeki64, select this id then press F1 ;) or see help on fseek function.

It seems you need a very simple correction:

inline
int fseek32(FILE* stream, unsigned int offset, int origin)
{
    return _fseeki64(stream,offset,SEEK_SET);
}

Thank you very much ArkM,

_fseeki64() works like charm sweet and smooth.