A few questions about binary i/o..

Please support our C++ advertiser: Intel Parallel Studio Home
Reply

Join Date: Jul 2006
Posts: 1,091
Reputation: MattEvans is a jewel in the rough MattEvans is a jewel in the rough MattEvans is a jewel in the rough 
Solved Threads: 63
Moderator
Featured Poster
MattEvans's Avatar
MattEvans MattEvans is offline Offline
Veteran Poster

A few questions about binary i/o..

 
0
  #1
Aug 17th, 2008
I have a few ( slightly related ) questions about binary i/o in C++, I can't seem to find full answers to these anywhere..

- Is there any way to tell if an i/ostream passed to a function has been opened in binary or text mode?

- If not, are there any nuances to look out for when using put/read/write methods on an i/ostream that was opened in text mode? Would a process that assumes an i/ostream was binary, and worked 'correctly' in that case, be in any way incorrect if the stream it used was instead opened in text mode?

- How does endian-ness work? If two bytes A and B are written ( in that order ) into a file using put, then read on a machine with reversed endian-ness, do the bytes come out in a different order ( i.e. B, A ), or do they come out in the order A, B, but with the actual bits themselves being backwards? Or does the ( binary mode ) i/ostream normalize this so that it's not a problem? What if the bytes are written using ostream.write directly from an array reinterpret_casted to a char * ? ( and then read using istream.read )
Plato forgot the nullahedron..
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 351
Reputation: Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about 
Solved Threads: 62
Radical Edward's Avatar
Radical Edward Radical Edward is offline Offline
Posting Whiz

Re: A few questions about binary i/o..

 
0
  #2
Aug 17th, 2008
Originally Posted by MattEvans
- Is there any way to tell if an i/ostream passed to a function has been opened in binary or text mode?
Not directly through the object by default. The easiest solution if you need that information is to pass it along with the object as a parameter. To save yourself some trouble in keeping up with multiple entities you can save an application specific mode flag in the stream object using xalloc() and iword(), or xalloc() and pword():
  1. #include <fstream>
  2. #include <iostream>
  3.  
  4. namespace EdRules {
  5. using namespace std;
  6.  
  7. int OPEN_MODE = 0;
  8.  
  9. void foo(ostream& os)
  10. {
  11. switch (os.iword(OPEN_MODE)) {
  12. case 0: cout << "Binary\n"; break;
  13. case 1: cout << "Text\n"; break;
  14. }
  15. }
  16. }
  17.  
  18. int main()
  19. {
  20. using namespace std;
  21. using namespace EdRules;
  22.  
  23. OPEN_MODE = cout.xalloc();
  24. cout.iword(OPEN_MODE) = 0;
  25.  
  26. foo(cout);
  27.  
  28. cout.iword(OPEN_MODE) = 1;
  29.  
  30. foo(cout);
  31. }
Originally Posted by MattEvans
- If not, are there any nuances to look out for when using put/read/write methods on an i/ostream that was opened in text mode? Would a process that assumes an i/ostream was binary, and worked 'correctly' in that case, be in any way incorrect if the stream it used was instead opened in text mode?
If the process expects untranslated new-lines and gets translated new-lines, it could fail, but if you don't inspect and work with the characters directly in a way that relies on the transformations that a binary stream does and a text string doesn't do, it shouldn't matter.

Originally Posted by MattEvans
- How does endian-ness work? If two bytes A and B are written ( in that order ) into a file using put, then read on a machine with reversed endian-ness, do the bytes come out in a different order ( i.e. B, A ), or do they come out in the order A, B, but with the actual bits themselves being backwards?
It's all about the interpretation of the bytes when loaded into memory that matters, not how they're stored. If the first machine writes A,B and the second machine reads A,B, the values will be interpreted differently because to the first machine A,B means A,B, but to the second machine A,B means B,A.

Originally Posted by MattEvans
Or does the ( binary mode ) i/ostream normalize this so that it's not a problem? What if the bytes are written using ostream.write directly from an array reinterpret_casted to a char * ? ( and then read using istream.read )
read() and write() don't do any magic with endianness. You get what you wrote, even if what you wrote isn't want you want.
If at first you don't succeed, keep on sucking until you do succeed.
Reply With Quote Quick reply to this message  
Join Date: Jul 2006
Posts: 1,091
Reputation: MattEvans is a jewel in the rough MattEvans is a jewel in the rough MattEvans is a jewel in the rough 
Solved Threads: 63
Moderator
Featured Poster
MattEvans's Avatar
MattEvans MattEvans is offline Offline
Veteran Poster

Re: A few questions about binary i/o..

 
0
  #3
Aug 17th, 2008
Thanks for the info on storing things into the i/ostream directly, that could come in useful elsewhere , but, the reason I wanted to determine the open mode is to stop a method being inadvertently called with a stream in the wrong mode.. so, that won't work.. but I seem to get the same results in light testing when writing to a stream opened in 'the wrong mode' anyway.. hence my second question.

The endian-ness thing, I'm still not sure I get it... a more specific example of what I'm doing, writing 32-bit floats to and from binary files, I don't have access to a machine with a different endian-ness.. But, lets say I work on a PC, and I want my files to open in a Mac ( which is apparently reverse endianess to PC ).. [ I am happy to assume that floats are 32-bit on my targets.. and that they're represented the same... although, is that even a safe assumption on Windows, Mac, Linux? See I really don't want to have to resort to a dodgy handrolled file representation for non-integers, and using a text representation for floating point arrays never really appealled to me. ]

Anyway, assuming that the representation is the same.. if I do this:
  1. int main( void )
  2. {
  3. std::ofstream out( "test.bin", std::ios::binary | std::ios::out | std::ios::trunc );
  4. const float f1 = 123.456f;
  5. out.write( reinterpret_cast< const char * >( &f1 ), sizeof( float ) );
  6. out.close( );
  7.  
  8. std::ifstream in( "test.bin", std::ios::binary | std::ios::in );
  9. float f2 = 0.0f;
  10. in.read( reinterpret_cast< char * >( &f2 ), sizeof( float ) );
  11. in.close( );
  12.  
  13. assert( f1 == f2 );
  14. }
Will it work ( i.e. will the read value be equal the written value ) if the second half of that process is done on a reverse-endian machine? Or is it better ( or no change atall ) to do this?
  1. int main( void )
  2. {
  3. std::ofstream out( "test.bin", std::ios::binary | std::ios::out | std::ios::trunc );
  4. const float f1 = 123.456f;
  5. const char * c1 = reinterpret_cast< const char * >( &f1 );
  6. for( size_t i = 0; i < sizeof( float ); ++i ) {
  7. out.put( c1[ i ] );
  8. }
  9. out.close( );
  10.  
  11. std::ifstream in( "test.bin", std::ios::binary | std::ios::in );
  12. float f2 = 0.0f;
  13. char * c2 = reinterpret_cast< char * >( &f2 );
  14. for( size_t i = 0; i < sizeof( float ); ++i ) {
  15. c2[ i ] = in.get( );
  16. }
  17. in.close( );
  18.  
  19. assert( f1 == f2 );
  20. }
Or, is it necessary to reverse the order of the bits in each byte read if it is determined that the machine reading the file is 'backwards' ?
Plato forgot the nullahedron..
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 351
Reputation: Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about 
Solved Threads: 62
Radical Edward's Avatar
Radical Edward Radical Edward is offline Offline
Posting Whiz

Re: A few questions about binary i/o..

 
0
  #4
Aug 17th, 2008
Edward's rule of thumb for binary I/O is that if you use read() and write(), the file is not portable. If you want portable binary I/O you need to deconstruct the object and write the bytes manually, then read them back in the same order they were written and reconstruct the object manually. That way the result is the same regardless of what system you process the file on.

The manual approach is almost always too involved to be worth it, so for portability, a standardized text format is the way to go.
If at first you don't succeed, keep on sucking until you do succeed.
Reply With Quote Quick reply to this message  
Join Date: Jul 2006
Posts: 1,091
Reputation: MattEvans is a jewel in the rough MattEvans is a jewel in the rough MattEvans is a jewel in the rough 
Solved Threads: 63
Moderator
Featured Poster
MattEvans's Avatar
MattEvans MattEvans is offline Offline
Veteran Poster

Re: A few questions about binary i/o..

 
0
  #5
Aug 17th, 2008
I'm aware of potential issues involved when directly writing struct values to binary, so I already break the objects into primitives, but the smallest primitive I can break down to is a float, I'm happy to do this, it makes a certain amount of sense in a context which is basically, a load of arrays of float and uint32_t, with only a very tiny amount of considered structure ( a 16 byte header for each group of arrays, which may be hundreds/thousands of elements long ). The data is certainly not easily human-readable/writeable, so I wouldn't gain that usual advantage of a text format, only the portability. In any other situation though, I'd certainly go for a text format.

All I really need is a ( somewhat ) platform independant way of storing individual floats in binary format, I say somewhat platform independant, since PC and Mac are the only targets I'm focusing on.

But, based on what you've said, I guess that the second piece of code I posted would work ( reading and writing each byte of each float one at a time using get/put, and always reconstructing in the same order ), as long as floats are 32-bit, and are represented in the same way at bit-level on the target platforms...
Plato forgot the nullahedron..
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 351
Reputation: Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about 
Solved Threads: 62
Radical Edward's Avatar
Radical Edward Radical Edward is offline Offline
Posting Whiz

Re: A few questions about binary i/o..

 
0
  #6
Aug 18th, 2008
Originally Posted by MattEvans
But, based on what you've said, I guess that the second piece of code I posted would work
No, but that's Edward's fault for not being detailed enough. What you're doing with put() and get() is exactly what write() and read() do, and it has the same problems. By manually deconstructing and reconstructing the objects, Ed meant using bitwise operators:
  1. #include <iostream>
  2.  
  3. float ToFloat(const unsigned char *bytes)
  4. {
  5. unsigned long temp = 0;
  6.  
  7. temp |= (bytes[0] & 0xFF) << 24;
  8. temp |= (bytes[1] & 0xFF) << 16;
  9. temp |= (bytes[2] & 0xFF) << 8;
  10. temp |= (bytes[3] & 0xFF) << 0;
  11.  
  12. return *reinterpret_cast<float*>(&temp);
  13. }
  14.  
  15. unsigned char *FromFloat(float value, unsigned char *bytes)
  16. {
  17. unsigned long *temp = reinterpret_cast<unsigned long*>(&value);
  18.  
  19. bytes[0] = static_cast<unsigned char>((*temp >> 24) & 0xFF);
  20. bytes[1] = static_cast<unsigned char>((*temp >> 16) & 0xFF);
  21. bytes[2] = static_cast<unsigned char>((*temp >> 8) & 0xFF);
  22. bytes[3] = static_cast<unsigned char>((*temp >> 0) & 0xFF);
  23.  
  24. return bytes;
  25. }
  26.  
  27. int main()
  28. {
  29. using namespace std;
  30.  
  31. float value = 247.548f;
  32. unsigned char bytes[4];
  33.  
  34. cout << fixed << ToFloat(FromFloat(value, bytes)) << '\n';
  35. }
That works because the shift operator does the right thing to shift the bytes into the correct location regardless of the byte order for the system. It's still not 100% portable because of size and floating point format assumptions, but at least the endianness issue is covered.
If at first you don't succeed, keep on sucking until you do succeed.
Reply With Quote Quick reply to this message  
Join Date: May 2006
Posts: 3,114
Reputation: WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of 
Solved Threads: 281
Moderator
WaltP's Avatar
WaltP WaltP is offline Offline
Posting Sensei

Re: A few questions about binary i/o..

 
0
  #7
Aug 18th, 2008
Originally Posted by Radical Edward View Post
Originally Posted by MattEvans View Post
- How does endian-ness work? If two bytes A and B are written ( in that order ) into a file using put, then read on a machine with reversed endian-ness, do the bytes come out in a different order ( i.e. B, A ), or do they come out in the order A, B, but with the actual bits themselves being backwards?
It's all about the interpretation of the bytes when loaded into memory that matters, not how they're stored. If the first machine writes A,B and the second machine reads A,B, the values will be interpreted differently because to the first machine A,B means A,B, but to the second machine A,B means B,A.
No, it's about how they are stored. The bytes will be backwards in the file when written by a big-endian system but read by a little-endian system.

Matt, see this
Last edited by WaltP; Aug 18th, 2008 at 2:31 pm.
The 3 Laws of the Procrastination Society:
1) Never do today that which can be put off until tomorrow
2) Tomorrow never comes
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 351
Reputation: Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about 
Solved Threads: 62
Radical Edward's Avatar
Radical Edward Radical Edward is offline Offline
Posting Whiz

Re: A few questions about binary i/o..

 
0
  #8
Aug 18th, 2008
Originally Posted by WaltP
The bytes will be backwards in the file when written by a big-endian system but read by a little-endian system.
That's another way of saying the same thing.
If at first you don't succeed, keep on sucking until you do succeed.
Reply With Quote Quick reply to this message  
Join Date: May 2006
Posts: 3,114
Reputation: WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of WaltP has much to be proud of 
Solved Threads: 281
Moderator
WaltP's Avatar
WaltP WaltP is offline Offline
Posting Sensei

Re: A few questions about binary i/o..

 
0
  #9
Aug 18th, 2008
Originally Posted by Radical Edward View Post
That's another way of saying the same thing.
No really. You said "It's all about the interpretation of the bytes when loaded into memory that matters, not how they're stored." I'm saying you must know how they are stored or you can't properly read the data.

I think I know what you were trying to say, but it wasn't clear.
The 3 Laws of the Procrastination Society:
1) Never do today that which can be put off until tomorrow
2) Tomorrow never comes
Reply With Quote Quick reply to this message  
Join Date: May 2008
Posts: 351
Reputation: Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about Radical Edward has a spectacular aura about 
Solved Threads: 62
Radical Edward's Avatar
Radical Edward Radical Edward is offline Offline
Posting Whiz

Re: A few questions about binary i/o..

 
0
  #10
Aug 19th, 2008
Originally Posted by WaltP
I think I know what you were trying to say, but it wasn't clear.
Sorry for not being clear.
If at first you don't succeed, keep on sucking until you do succeed.
Reply With Quote Quick reply to this message  
Reply

This thread is more than three months old.
Perhaps start a new thread instead?
Message:


Thread Tools Search this Thread



About Us | Contact Us | Advertise | DaniWeb | Acceptable Use Policy | RSS Feed

©2003 - 2009 DaniWeb® LLC