How to deserialize binary stream

[thekevin07]

I have written multiple streams of data to a file the format is like this

J|1|2|3|4

that would be 1 record and there are more then one record in the file

my question is how do i read that data in, deserialize it, extract it and assign each of the values into there variables i will eventually want to search for records but i need help with this first. The serialize function i use makes all data pipe delimited.

Thanks

[Lerner]

You could consider reading the entire line in and then parsing it, or you could consider parsing it as you read it in. Either way, you will probably read the data into a series of strings and then convert to numerical data types as desired. You could use getline() using the pipe and newline char as delimeters or you could use strtok() if you're comfortable with that. For that matter, you could write your own function by looking at the information char by char. Once you have the substrings you can use any of a number of techniques to get numerical data depending on your knowledge base and needs.

[ArkM]

The main criteria for the right solution of deserialization is:

you must suspect that all data were crippled, edited by mad operators, data file name was changed, you get another, wrong file etc. Deserialization module must cope with all these situations (at least detect them)...

Probably the best deserialization modules can write paranoics with persecution complex...

[thekevin07]

I c but for some reason i can't read the data in to anything not even a

char buffer[200]

unless I'm doing it wrong here is the function that reads it

char buf[200];
	record rs;
	fs.seekg(0);
	while(!fs.eof())
	{
		fs.read(buf,200);	
	}
cout<<buf<<endl;

but it prints nothing if you want i can post the whole code for the projects

[Lerner]

Can you open a file and see the information in the format posted in your original post or is it in binary format?

Is the format always going to be a char pipe digit pipe digit pipe digit pipe digit newline char?

Post what you coded for the type record.

Don't use the return result of eof() to control the loop. If it doesn't cause a problem this program, it will sooner or later.

read() is only for data written to file using binary format.

I'd be tempted to do this:

char ch
char pipe
int eenie
int meenie
int miney
int mo

ifstream fin(filename)
if(!file)
  cout << err;

while(fin >> ch)
   fin >> pipe >> eenie >> pipe >> meeney >> pipe >> miney >> pipe >> mo;
   //do something with the data;

You might be able to do something like:

string data, eenie, meenie, miney, mo
while(getline(fin, data, '|'))
   getline(fin, eenie, '|')
   getline(fin, meenie, '|')
   getline(fin, miney, '|')
   getline(fin, mo);
  
   //do something with input

Knowing more program details would help.

[ArkM]

I don't know what you expect to print after this obviously incorrect use of unformatted read. Try classic:

std::string line;
...
while (getline(fs,line)) {
    cout << line << endl;
}

[thekevin07]

the file is in binary i can view the .dat file in visual studio 2008 and it will show data and yes it will always be string|int|int|int|double|int here is the entire source code. It is a really simple database

main

#include "stdafx.h"

#include <iostream>
#include <fstream>
#include <string>
#include "record.h"

using namespace std;

#define nameLen 31
#define deathAndTaxes 1

void add(fstream&);
void print(fstream&);
void edit(fstream&);

int _tmain(int argc, _TCHAR* argv[])
{
	char c;
	fstream outfile("parts.dat", ios::binary | ios::out);
	outfile.close();
	outfile.open("parts.dat", ios::binary | ios::in | ios::out);
	if(!outfile.good())
	{
		cerr << "Cannot open parts.dat" << endl;
		exit(1);
	}

	while(1)
	{
		cout<<"p to print, a to add, e to edit"<<endl;
		cin>>c;
		switch (c)
		{	
			case 'p': 
				print(outfile);
				break;
		
			case 'e': 
				edit(outfile);
				break;
	
			case 'a': 
				add(outfile);
				break;

			default:
				cout<<"unknown char"<<endl;
				break;
		}

	}

	system("pause");
	return 0;
}


void edit(fstream &fs)
{}
void print(fstream &fs)
{
	record rs;
	char buf[200];
	fs.seekg(0);
	while(!fs.eof())
	{
		fs.read(buf, sizeof(buf));	
		cout<<buf<<endl;
	}
}

void add(fstream &outfile)
{
	char name[nameLen+1];
	int quantity, partNo;
	double cost;

	while(deathAndTaxes)
	{
		cout << "Please enter record information: name, part number, quantity, cost: " << endl;
		cin.sync();
		cin >> name >> partNo >> quantity >> cost;
		if(cin.eof()) 
		{
			cout << "Exiting." << endl;
			exit(0);
		}
		else if(cin.fail())
		{
			cerr << "Input failed.  Data order is name, part number, quantity, cost" << endl;
			cout << "Retrying: " << endl;
			cin.clear();
			continue;
		}
		else if(cin.bad())
		{
			cerr << "Stream failure.  Giving up." << endl;
			exit(1);
		}
		else
		{
			record *r = new record(name, partNo, quantity, cost);
			if(r) 
			{
				outfile.seekp(ios_base::end,1);
				if(outfile.write( r->serialize().c_str(), sizeof(record)))
				{
					cout << "Record successfully created" << endl;
				}
				else
				{
					cout<<"record create failed"<<endl;
				}
				return;
			}
			else
			{
				cerr << "Record creation failed.  Giving up." << endl;
				exit(1);
			}
		}
	}	
}

record.h

#pragma once
#include <string>
#include <fstream>
#include <iostream>
#include <sstream>

using namespace std;

class record
{
public:
	static int recordIndex;
	static const int nameLen = 31;
	record(void);
	record(char *, int, int, double);
	~record(void);
	void print();
	char * getName() {return name;}
	int getPartNo() { return partNo;}
	int getQuantity() {return quantity;}
	double getCost() {return cost;}
	void setname(char * n) {name = n;}
	void setQuantity(int q) {quantity = q;}
	void setCost(double c) {cost = c;}
	int store(fstream&);
	int search(char *,fstream &);
	const string record::serialize();
	double cost;
	void record::deserialize(char *);

private:
	char * name;
	int idx;
	int partNo;
	int quantity;
	
};

record.cpp

#include "StdAfx.h"
#include <string>
#include <sstream>
#include "record.h"

int record::recordIndex = 0;

record::record(void): idx(recordIndex++)
{
}

record::record(char * s, int pNo, int qty, double cst) :  idx(recordIndex++)
{
	name=s;
	partNo=pNo;
	quantity=qty;
	cost=cst;
}

record::~record(void)
{
}
void record::print()
{
	cout<<quantity<<endl;
	
}
// Convert this to a pipe delimited string
const string record::serialize()
{
  stringstream out;

  out<< name <<'|'<< partNo <<'|'<< quantity <<'|'<< cost <<'|'<< idx;

  return out.str();
}

void record::deserialize(char * s)
{
	stringstream in;
	in.getline(s,strlen(s),'|');

	cout<<s<<endl;


}
//return index of record
int record::search(char * name,fstream &fr)
{
	fr.seekg(0);
	record r;
	fr.read(reinterpret_cast<char *>(&r),sizeof(record));
	while((strcmp(name,r.name)!=0)&&!fr.eof())
	{
		fr.read(reinterpret_cast<char *>(&r),sizeof(record));

	}
	if(fr.eof())
		return -1;
	else
		return idx;
}

[ArkM]

Regrettably, I'm going off-line now.
See you later...

[Lerner]

I wouldn't recommend using read/write with this type of data. I would only try it if each record is guaranteed to be the same length, but it doesn't appear that that's the case given the string in each record.

I would recommend opening the output stream in app mode rather than using seek().

I'd change these:
const string record::serialize();
void record::deserialize(char *);
to these:
const string serialize();
void deserialize(char *);

Why aren't you using STL strings throughout?

You can deserialize() by parsing as reading in as I suggested or by reading in entire line one at a time and then parsing line with a stringstream and converting necessary strings to numerical data as ArkM suggested.

I'd try writing to file in add() like this;

if(r) 
{
    outfile << r->serialize() << endl;
}

I'd try printing file like this:

void print()
{
   ofstream ofs(filename);
   string buf;
   while(getline(fs, buf))
      cout << buf << endl;
}

[thekevin07]

i replaced the print code you gave and i added in the ios::binary flag and it gave me this err

could not deduce template argument for 'std::basic_istream<_Elem,_Traits> &' from 'std::ofstream'

the file is binary and i cant change it to plain ascii

thanks

[ArkM]

Stop. Now I come back to your serialization code. It's incorrect. You evidently got binary/text serialization mixed up:

// Convert this to a pipe delimited string
const string record::serialize()
{
  stringstream out;
  out<< name <<'|'<< partNo <<'|'<< quantity <<'|'<< cost <<'|'<< idx;
  return out.str();
}
...
if (outfile.write( r->serialize().c_str(), sizeof(record))) // <==== !!!???

But sizeof(record) does not bear a relation to a string returned by record::serialize member function! You write fixed number of chars started in the body of the variable length text string formed by

out<< name <<'|'<< partNo <<'|'<< quantity <<'|'<< cost <<'|'<< idx;

There is no sense in any "deserialization" of this BINARY (not text) file with (nearly) garbage contents. At first correct a serialization code in add member function, for example:

if (outfile << r->serialize() << '\n') ...

Please, don't use so common for daniweb visitors feverish copy/paste approach. Think about what happens then correct and try the code. After that come back ;)...

There are other defects in your code. For example, you allocate new record in add but never deallocate this object (have memory leak). So revise your code before send a new help request (it's your homework, not mine ;) )...

Apropos, why you wrote about "multiple streams of data"? You are trying to write an ordinar data structure serialization...

[Lerner]

>>I wouldn't recommend using read/write with this type of data. I would only try it if each record is guaranteed to be the same length

I had thought that if strings were embedded in the user defined type that the type couldn't be written/read by write() or read() because one object may have a string of length x and another of length y, x != y and x and y both less than the capacity of the char array underlying the string. Based on further reading and in a trial program I guess you can read and write user defined types with variable length members, specifically strings; at least if the string member was declared like this: char s[80]. I didn't try it if the string were declared like this: char * s or if an STL string is used (though I will later!).

I also tried reading a file written with write() like this:

struct status
{
  char name[80];
  char address[80];
  double balance;
  unsigned long account_num;
};

char newline = '\n';

status acc;
strcpy(acc.name, "Ralph Trantor");
strcpy(acc.address, "123 Happy Hollow Lane");
acc.balance = 1123.23;
acc.account_num = 34235678;

ofstream fout("balance", ios::out | ios::binary);

fout.write((char *)&acc, sizeof(status));
fout.write((char *)&newline, sizeof(char));

ifstream fin("balance", ios::in | ios::binary);
string temp;
while(getline(fin, temp))
   cout << temp << endl;

That compiled using VC6++, but it had trouble on the output side using cout. Specifically the first char[] was printed appropriately but the rest was garbage. I'll keep working on it to see what I can come up with.

[Lerner]

Using my compiler I was able to use write() and read() successfully with a user defined type that contained and STL string as member variable.

I couldn't get write() and read() to work with user defined type that contained a char * as a variable member, presumably because the size of a char * is the size of a pointer until memory is assigned to it using new[] to get a string after which the lenght of the string may or may not be same size as a char *. However, maybe I haven't figured out the correct syntax to use yet, so maybe someone with more experience can indicate whether a user defined type with a char * as a member variable can be written/read from file using the stream functions read() and write()?