padding issue

[pdk123]

Hello All,

I have the following doubt:

Currently both the ipv4/ipv6 headers are such that they donot result in any padding. So it allows us to typecast the input stream to the ipv4/ipv6 header c structures. But don't you feel that this code is error prone ?

int Comp::getfield(uint8* pInPktStream)
{
  int swStreamType = -1;

  // examine the compressible chain of subheaders
  if(getVersion(*pInPktStream) == IP_VERSION_IPv4)
  {
    // IPv4
    st_ipv4* poIp = (st_ipv4*) pInPktStream;

:
:
}

Don;t you feel
st_ipv4* poIp = (st_ipv4*) pInPktStream;
blindly typecasting will result to problems ?

[Ancient Dragon]

typecasting is always error prone.

>>blindly typecasting will result to problems ?
"may" cause problems, not will. That's when you have to know what your doing, and even that is no guarentee that a typecast will be correct.

[pdk123]

yes, my question is, is it ensured that, TCP/IP standards define the headers in such a way that, it can be typecasted ( whatever the machine/compiler is ) !!!..even if they do, it may cause errors later, depending on upgrades in machine/compiler !!!...

any comments /advices are really helpful

[Ancient Dragon]

AFAIK the header files don't change. If a change is needed they create a new structure, leaving the original structure in tact so as not to break existing code. All hell would break out if they changed the structure on everybody without notice.

[pdk123]

i know the headers won't change, but is it ensured that they will never result in padding , lets say either the machine or compiler changes !!! i know it may not happen now, but in long term, there may be possibility of different machine sizes, which i donot know !!!

[Ancient Dragon]

You already stated that there is no padding in that structure
>>Currently both the ipv4/ipv6 headers are such that they donot result in any padding

You can force the padding factor like this using pack pragma

#pragma pack(1) // byte align eveything that follows
struct ipvr4
{
  // blabla
};
#pragma pack() // resume normal padding

[pdk123]

Thanks a lot ...

yes, as of now there is no padding.. and as you adviced that the headers will not change,..

But my concern was when the if at all the machine size/compilers change ( i know it is very unlikely, but is still possible)...

As you suggested if i use the prgma_pack for these structures, i donot need to worry at all...

thanks a lot, it was really helpful

[pdk123]

also currently we define the ipv4/ipv6 headers as class, i guess i can
use the prgma directive around the class ( i am a c programmer, shifting to c++, hope this will not cause any issues ? )

#pragma pack(1)

class st_ipv4 
{
//==================================================================================================
public:
//==================================================================================================

#ifdef _BIG_ENDIAN
  uint8    version:4;    // version 
  uint8    headerlength:4;  // header length 
#else
  uint8    headerlength:4;  // header length 
  uint8    version:4;    // version 
#endif
  uint8    tos;      // type of service 
  uint16  packetlength;  // total length 
  uint16  identification;  // identification 
  uint16  fragmentoffset;  // fragment offset field 
  uint8    ttl;      // time to live 
  uint8    protocol;    // protocol 
  uint16  checksum;    // checksum 
  uint32  sourceaddress;  // source address 
  uint32  destinationaddress;  // destination address 

 //**********************************************************************************************
  //* Description:
  /**  Specify/Return IPv4 Version
  */
  //**********************************************************************************************
  inline void setVersion(uint8 ubyVersion)
  {
     version = ubyVersion;
  }
  inline uint8 getVersion()
  {
    return version;
  }

}

#pragma pack()

Thanks a lot for the help/suggestions...

[Ancient Dragon]

The pragma will solve the problem on your computer but what about the other guy's computer? What is you are running MS-Windows and the os on the other end of the tcp/ip connection is *nix or somethin else?

There are a lot of what-ifs that I know has been discusses and resolved years ago. And that's why I wouldn't bother to use those pack pragmas or worry about paddng for those structures. All you need to worry about is the data you are sending across that tcp/ip connection.

[edit]packing works the same on c++ classes as they do on structures. It affects only the alignment of data objects, not the methods that a class may contain.[/edit]

[pdk123]

i know i.e a problem, but the other guy should take care of interpretting the byte stream properly..i.e his problem right, if he misinterprets the byte stream !!!

[Ancient Dragon]

Of course the receiver has to know what you are sending and he should be aware of the packing factor that you used on the data.

[arkoenig]

As you suggested if i use the prgma_pack for these structures, i donot need to worry at all...

If you use any pragma, you always need to worry.

Pragma has two properties that is guaranteed to cause worry:

1) The very nature of pragma is machine-dependent.

2) An implementation that does not know what to do with a pragma is expected to ignore it.

You do the math.

[pdk123]

Of course the receiver has to know what you are sending and he should be aware of the packing factor that you used on the data.

i guess, there is a slight misunderstanding here, :

i mean: The tcp/ip stream which is sent from the sender, or received by the receiver, will never be packed, it is just the byte stream, just with the data. The problem is how the receiver will see the data stream. The receiver, will need to interpret that as just data stream without any padding. If he wants to typecast the input data stream to a structure ptr, That ptr with which he sees the input data stream , may have the padding issue. i.e the input stream is just data, but the pointer to the structure is machine dependent, and the structure might have padding.. so if we interpret the data stream, by typecasting to structure, there will be problem with the padding...
Basic assumption here is that, whatever is transmitted or received is just the data stream, without any padding...

[Ancient Dragon]

The safest way for the receiver to do it would be to manually insert the streamed data into the structure. For example:

struct st_ipv4 st;
unsigned char buffer[1024]; // assume this contains the streamed data
unsigned char* ptr = buffer;
st.verion = *(uint8*)ptr;
ptr += sizeof(uint8);
st.headerlength = *(unit8*)ptr;
ptr += sizeof(uint8);
// etc. etc for each field in the structure

And an even safer way is for sender to convert everything to text and send the data as text instead of binary. That resolved the big/little endian issues too.

[pdk123]

Thanks a lot Ancient Dragon. Yes, i.e the safest way.

But currently the entire ipv4 compression algo has been written with the typecasting .

So when i was trying to add the code for ipv6, it looked dangerous for me to typecast, ! ( which is obvious)...

As we discussed, to just the hack the existing code, i can put pragma as of now, ..
and when i get enough time, i guess, i need to manually extract each field from the input data stream ...

Thanks a lot, for understanding my problem exactly and suggestions are really helpful.

[mike_2000_17]

I don't mean to interrupt, but usually when you want a really robust binary interface for a byte-stream. You don't cast, or stringify, or pack/pragma, or what give you. You implement a binbag with off-the-hook protection from cross-platform, cross-compiler, cross-compiling-option.. the whole shabang. Implement the binbag's interface on the model of the std::iostreams (with those nice << >> operators) or like Boost Serialization. Then you write a save / load, or send / receive, or pack / unpack, or whatever you like to call it, that will serialize or un-serialize the data by packing it into or extracting from the binbag. This way you put all the nasty conditional compilations that you will need for ensuring off-the-hook binary-compatibility, all in one place (the binbag implementation).

Just my humble input, take it or leave it.

[pdk123]

Thanks a lot Mike, I will go through this option. I guess i need to do some homework for this. ( i come from c prograaming background and new to c++ )

It is really helpful , if i get design options to decide and i do some math.

[pdk123]

I don't mean to interrupt, but usually when you want a really robust binary interface for a byte-stream. You don't cast, or stringify, or pack/pragma, or what give you. You implement a binbag with off-the-hook protection from cross-platform, cross-compiler, cross-compiling-option.. the whole shabang. Implement the binbag's interface on the model of the std::iostreams (with those nice << >> operators) or like Boost Serialization. Then you write a save / load, or send / receive, or pack / unpack, or whatever you like to call it, that will serialize or un-serialize the data by packing it into or extracting from the binbag. This way you put all the nasty conditional compilations that you will need for ensuring off-the-hook binary-compatibility, all in one place (the binbag implementation).

Just my humble input, take it or leave it.

Thankyou Mike, My Kind request is, if possible, can you please elaborate a bit ( as i am basically a c programmer, tend to forget that c++ can offer so many things, which is obvious ). Hope i am not asking too much, without doing my homework

[mike_2000_17]

Ok, well I can put a bit of an example of a binbag: (without OS-specific flags, I don't want to be bothered, that's for you to work on)

#define DEFAULT_SIZE 4096 //or whatever.. like the size of the IP header.
class oBinBag;
class iBinBag;

class serializable {
  public:
    void save(oBinBag& B) = 0;
    void load(iBinBag& B) = 0;
};

class oBinBag {
  private:
    unsigned char* buffer;
    int pos;
    int size;
  public:
    oBinBag() : buffer(new unsigned char[DEFAULT_SIZE]), pos(0), size(DEFAULT_SIZE) { };
    ~oBinBag() { delete[] buffer; };

    oBinBag& operator <<(uint32 value) {
      //say, MSB first:
      if(pos > size - 4) {
        unsigned char* tmp = new unsigned char[size *= 2];
        memmove(tmp,buffer,pos);
        delete[] buffer;
        buffer = tmp;
      };
      buffer[pos++] = value & 0xFF000000;
      buffer[pos++] = value & 0x00FF0000;
      buffer[pos++] = value & 0x0000FF00;
      buffer[pos++] = value & 0x000000FF;
      return *this;
    };
    oBinBag& operator <<(uint16 value) {
      //say, MSB first:
      if(pos > size - 2) {
        unsigned char* tmp = new unsigned char[size *= 2];
        memmove(tmp,buffer,pos);
        delete[] buffer;
        buffer = tmp;
      };
      buffer[pos++] = value & 0x0000FF00;
      buffer[pos++] = value & 0x000000FF;
      return *this;
    };
    
    //etc.. for as many primitive types you need, but watch out for ambiguous overloads.
    
    oBinBag& operator <<(serializable& value) {
      value.save(*this);
      return *this;
    };
};

// the same but in reverse for iBinBag (with >> operators)

// Then, a typical struct like IPHeader, lets say:
class IPHeader : public serializable {
  private:
    uint8 headerlength; // header length
    uint8 version; // version
    uint8 tos; // type of service
    uint16 packetlength; // total length
    uint16 identification; // identification
    uint16 fragmentoffset; // fragment offset field
    uint8 ttl; // time to live
    uint8 protocol; // protocol
    uint16 checksum; // checksum
    uint32 sourceaddress; // source address
    uint32 destinationaddress; // destination address
  public:
    void save(oBinBag& B) {
#ifdef _BIG_ENDIAN
      B << version << headerlength; 
#else
      B << headerlength << version;
#endif
      B << tos << packetlength  << identification << fragmentoffset
        << ttl << protocol << checksum 
        << sourceaddress << destinationaddress;
    };
    void load(iBinBag& B) {
#ifdef _BIG_ENDIAN
      B >> version >> headerlength; 
#else
      B >> headerlength >> version;
#endif
      B >> tos >> packetlength >> identification >> fragmentoffset
        >> ttl >> protocol >> checksum 
        >> sourceaddress >> destinationaddress; 
    };
};

int main() {
  oBinBag bout;
  IPHeader hdr;
  //... init header
  bout << hdr;
  bout.SendThroughIP(); //some function to send the packet.
  //..wait
  iBinBag bin;
  bin.ReceiveFromIP(); //some function to receive packet.
  bin >> hdr;
  //checkout what was put in hdr and do whatever..
};

You get the idea? This way any other types or whatever formats of the data packets, as long as the sequence inside the save and load functions are exactly the same, it is should be fine. And all the bit ordering and padding and whatever, you put in ioBinBag.

Of course, this is just an example, a real implementation could / should be quite a bit different.

EDIT: If this is for a homework, then the above might be overkill. I mean, this is close to what people do in real life.. but again, I don't know how advanced your course is, maybe the prof expects this, or maybe he expect something much simpler, like ignoring completely the issue you raised in this thread and just cast the pointer to a struct.

[pdk123]

Thanks a lot Mike.

I need really to do some home work, before i ask you anymore questions. This is really great.

Btw, it is for real implementation, i know, i need to ask more time now. I appreciate this help very much, i know it takes time for a c programmer to think in c++. Thanks a lot. It is really very very helpful.

[pdk123]

Hello Mike,

i got the output bin bag working fine, except that i had to make the folowing two functions virtual in the class serializable.

virtual void save(oBinBag& B) = 0;
virtual void load(iBinBag& B) = 0;

For iBinBag do you think, i need to explicitly declare the input array ? or create a new one and delete ?
my implementation is giving some compilation errors !

[mike_2000_17]

>>For iBinBag do you think, i need to explicitly declare the input array ? or create a new one and delete ?
You mean, do you need to make the input array as dynamically allocated or not? That depends on the implementation, if you know of a certain maximum size, then a static array would do just fine. If you don't know the size of the array and have no expected maximum (which I doubt, because I remember from some course on internet protocols that there is some maximum packet size), then you need to somehow announce the size of the incoming array to iBinBag and allocate it dynamically.

Compilation errors, well try to solve them as much as you can, if you are stuck, post them here.