My own archive format

[BDev338]

Hi there developers!
I'm trying to develop my own online multiplayer game, using the DarkGDK. But I don't want everyone to have access to the media of the game, so I'm working on my own cache archive format.
This is how it works:
.idx0 = textures
.idx1 = sprites
.idx2 = sounds
.idx3 = models
.idx4 = maps
The .idx# files contain 32bit entries. Each entry is 2 integers long. First int of each entry is an offset into the .dat file (the file where all the textures, sprites, sounds, etc, etc, is actually stored). So after you retrieved the offset into the .dat file you need to know how many bytes you have to read to get the requested file. - This is the next integer of the entry. My problem is that I don't know how to unpack the cache again. I use a RandomAccessFile to compress each file, using it's writeInt(int) method. So naturally I used RandomAccessFiles readInt() method. But when I do that the file I try to unpack has an invalid size. How come?
Here's my "compress()" method:

public void compress(int id) throws Exception{
	    path = src.getPath();
		dst = findIdxFile(path);
		byte[] buffer = new byte[(int)src.length()];
		RandomAccessFile outputStr = new RandomAccessFile(dst, "rw");
		RandomAccessFile inputStr = new RandomAccessFile(src, "rw");
		int blockId = id * 2;
		int blockLen = buffer.length;
		String state;
			
		inputStr.readFully(buffer);
		inputStr.close();
			
		outputStr.seek(blockId);
		outputStr.writeInt(blockId);
		outputStr.writeInt(blockLen);
		outputStr.close();

		File dataFile = new File("./cache/main_file_cache.dat");
		
		dataFile.delete();
		dataFile.createNewFile();
		
		RandomAccessFile mainOutStr = new RandomAccessFile(dataFile, "rw");
		
		mainOutStr.seek(blockId);
		mainOutStr.write(buffer);
		mainOutStr.close();
		System.out.println("src: " + src.getName() + " dst: " + dst.getName() + " off: " + blockId + " len: " + blockLen);
	}

Best regards,
Benjamin :).

[~s.o.s~]

Too little information to help you out. Have you got a small standalone code which reproduces your problem which I can run?

[BDev338]

Too little information to help you out. Have you got a small standalone code which reproduces your problem which I can run?

Do you mean like a compressed cache, and a small sample program to extract a file out of the cache, that's ready to run?
If so, yes.
I'll upload a ready to run sample program incl. source and cache, to extract the first texture.
And the cache is compressed using the "compress()" function, so I guess you don't need to the source of the entire compression tool?
Anyway, here's a link:
http://uppit.com/rm528wk4f1hv/sample_program.zip

[~s.o.s~]

I don't really see a problem here; the program when run outputs a file "assault_tropper.bmp" which is around 67 MB due to a bug which causes the length to be shown as 67 x 10^6.

There are couple of things you can do to debug this:
* Write out the idx0 file and immediately read it using the RAF in the same sample code. Verify that the numbers written are indeed the numbers which you intended
* Write out the main cache file (by packing the resources) and in the same sample code try to unpack the now written resources.

If both the above samples work out fine, I don't see why a combination of them won't work.

[BDev338]

I don't really see a problem here; the program when run outputs a file "assault_tropper.bmp" which is around 67 MB due to a bug which causes the length to be shown as 67 x 10^6.

There are couple of things you can do to debug this:
* Write out the idx0 file and immediately read it using the RAF in the same sample code. Verify that the numbers written are indeed the numbers which you intended
* Write out the main cache file (by packing the resources) and in the same sample code try to unpack the now written resources.

If both the above samples work out fine, I don't see why a combination of them won't work.

I have a feeling that it's the way I compress the data into the file. If I System.out.println() the data (blockId and blockLen), I see correct output, so yeah, it HAS to be the way I output data through the RAF. I'm not at my home computer right now, so I'll investigate further when I get home.

[BDev338]

Ahhh... home sweet home.......
Anyway, I modified the "compress()" method to unpack the cache using the blockLen and blockId variables. - Which works fine! So it is the way I write the data to the file..
Here's the modified "compress()" method:

public void compress(int id) throws Exception{
	    path = src.getPath();
		dst = findIdxFile(path);
		byte[] buffer = new byte[(int)src.length()];
		RandomAccessFile outputStr = new RandomAccessFile(dst, "rw");
		RandomAccessFile inputStr = new RandomAccessFile(src, "rw");
		int blockId = id * 2;
		int blockLen = buffer.length;
			
		inputStr.readFully(buffer);
			
		outputStr.seek(blockId);
		outputStr.write(blockId);
		outputStr.writeInt(blockLen);

		File dataFile = new File("./cache/main_file_cache.dat");
		
		dataFile.delete();
		dataFile.createNewFile();
		
		RandomAccessFile mainOutStr = new RandomAccessFile(dataFile, "rw");
		
		mainOutStr.seek(blockId);
		mainOutStr.write(buffer);
		System.out.println("src: " + src.getName() + " dst: " + dst.getName() + " off: " + blockId + " len: " + blockLen);
		/*test*/
		byte[] testBuf = new byte[blockLen];
		File f = new File("./" + src.getName());
		f.createNewFile();
		RandomAccessFile raf = new RandomAccessFile(f, "rw");
		mainOutStr.seek(blockId);
		mainOutStr.read(testBuf, 0, blockLen);
		raf.write(testBuf);
		raf.close();
		/*test end*/
		mainOutStr.close();
		outputStr.close();
		inputStr.close();