Reading binary file without knowing file format

[gmark]

Dear all,

I have read a lot topics about reading binary files but I didn't find something about my problem. How possible is it to read a binary file that you don't know the file format? I am sure that there are at least chars and float numbers. Thanks a lot!!

[jephthah]

use "fopen()" to open the file, and specify "read, binary" as the mode.

FILE * fp;
fp = fopen("file.dat","rb");

note the following:

In the case of text files, depending on the environment where the application runs, some special character conversion may occur in input/output operations to adapt them to a system-specific text file format. In many environments, such as most UNIX-based systems, it makes no difference to open a file as a text file or a binary file; Both are treated exactly the same way, but differentiation is recommended for a better portability

--http://www.cplusplus.com/reference/clibrary/cstdio/fopen/

in other words, if you think the file has numeric values, read them the same way using fgets() or sscanf() or similar.

.

[nucleon]

It's quite a challenge decoding a binary file with an unkown format! In general, it's not possible. How do you know if the first two bytes are the characters AB or the 16-bit integer with decimal value 1106 (assuming an ascii representation)? You might want to view the file in a hex/ascii viewer.

[gmark]

Thanks for your answer, I have the following code, I will check also your link, I use linux but I can't see an text that should be in the binary.

#include<stdio.h>

int main() {
	FILE *file;
	unsigned char *buffer;
        int i;
	unsigned long fileLen;

	//Open file
	file = fopen("TRACE.0000002958000001000000.mpit", "rb");
	if (!file)
	{
		fprintf(stderr, "Unable to open file %s", "test.mpit");
		return;
	}
	
	//Get file length
	fseek(file, 0, SEEK_END);
	fileLen=ftell(file);
	fseek(file, 0, SEEK_SET);

	//Allocate memory
	buffer=(unsigned char *)malloc((fileLen+1)*sizeof(unsigned char));
	if (!buffer)
	{
		fprintf(stderr, "Memory error!");
                                fclose(file);
		return;
	}
    
	//Read file contents into buffer
 	fread(buffer, fileLen, 1, file);
	fclose(file);

  int ii;
   for(ii = 0;ii < fileLen;++ii)
     printf("%d,  %c n",ii, ((unsigned char *)buffer)[ii]);
	free(buffer);
    printf("%d \n",fileLen);
  }

[gmark]

It's quite a challenge decoding a binary file with an unkown format! In general, it's not possible. How do you know if the first two bytes are the characters AB or the 16-bit integer with decimal value 1106 (assuming an ascii representation)? You might want to view the file in a hex/ascii viewer.

Yes I agree it's difficult. I suppose that the file has data like :

----------------------------------------------------------------
MPI Routine #calls avg. bytes time(sec)
----------------------------------------------------------------
MPI_Comm_size 2 0.0 0.000
MPI_Comm_rank 1 0.0 0.000
MPI_Send 16827 4714.6 0.067
MPI_Recv 16833 4810.6 0.483
MPI_Probe 6 0.0 0.000
MPI_Bcast 201 7.4 0.013
MPI_Barrier 1 0.0 0.000
MPI_Gather 2 4.0 0.000
MPI_Gatherv 3 23240.0 0.013
MPI_Reduce 300 9.3 0.818
MPI_Allreduce 504 13.5 0.028
----------------------------------------------------------------
total communication time = 1.422 seconds.
total elapsed time = 24.526 seconds.
user cpu time = 24.408 seconds.
system time = 0.030 seconds.
maximum memory size = 15428 KBytes.
----------------------------------------------------------------

It is free program and it was producing text files but now they change it to produce binary files in order for another program to reads the files but I don't want the other program.

[nucleon]

So you basically want to write a program that will translate the new format binary file into the old format text file?
If it's a free program, can you get the source?
Or can you find a description of its binary format online?

[gmark]

So you basically want to write a program that will translate the new format binary file into the old format text file?
If it's a free program, can you get the source?
Or can you find a description of its binary format online?

Yes that's exactly. It's free but they give only executable file and there is no documentation about the binary file. I suppose that the binary file format is the same with the text. I sent them an email to ask about it, but as they give another program with license (free of charge) I don't know if they will give me any information. We just don't want to use any license. The other solution is to change the program but this is the easiest and we try first this. So as I see without file format maybe there is no solution.

[nucleon]

You may be able to figure it out with some detective work. Try running the old program and the new program in such a way that they should produce the same data. Then compare the files (viewing the binary file in a hex editor). But they could have added some things or moved some things so it could be difficult to compare them. If you happen to do the above (generate the "equivalent" text and binary files), and they're not too big, zip and attach them.

[gmark]

You may be able to figure it out with some detective work. Try running the old program and the new program in such a way that they should produce the same data. Then compare the files (viewing the binary file in a hex editor). But they could have added some things or moved some things so it could be difficult to compare them. If you happen to do the above (generate the "equivalent" text and binary files), and they're not too big, zip and attach them.

I don't have the old version and I can't find it anywhere! I sent an email to an author's report that I have read about text files version but there isn't such email any more, thanks for the help I will try to see if the binary files are created right because at least I should see the text about some commands so I believe something maybe is wrong about the creation of the files. If I have another question I will write here, thanks!