Python

Getting size of all files in folder, and finding out some offsets..

Please support our Python advertiser: Programming Forums - DaniWeb Sister Site

Thread Solved

•

Join Date: Aug 2008

Posts: 12

Reputation: Th3_uN1Qu3 is an unknown quantity at this point

Solved Threads: 0

Th3_uN1Qu3 Th3_uN1Qu3 is offline

Offline

Newbie Poster

Getting size of all files in folder, and finding out some offsets..

Aug 18th, 2008

A lot of blah blah, scroll down for the matter i need help with...

Hello everybody.

I'm a moderator of a gaming forum and a big fan of bishoujo games. I sometimes do my own game releases/revamps, but what i've did till now didn't involve much "real" coding.

I want to make a game called Snow Drop compatible with XP and Vista. I've fixed the font animation bug already (just required a registry patch and a nice monospaced font) and remastered the music.

All worked fine on XP SP2 and Vista 32-bit, and replacing the ADPCM codec with an older version fixed voice on updated XP installs. But once i added another 2x 2GB RAM to my main computer, making for a total of 6GB, and installed Vista 64, i hit yet another voice issue.

The ADPCM codec in Vista 64 is screwed up badly, and no file changes would fix it, besides i also had to deal with permissions. So i tried using uncompressed PCM audio for voice, and it worked. The voice file will be 469MB instead of 160MB, but that's a different story, at least it works.

However, the work is far from over. Inside the voice datafile there are 1.564 different wave files. I found a Perl script to extract the WAVs from the main datafile and it worked, however i know exactly zero perl so i'm hoping i could do it in Python. I've written a little playlist converter app for Meizu players using Python a while ago, but i know this is going to be more complicated than copy/pasting a few lines of text.

Stuff of real interest starts here.

So, i have the WAVs extracted and converted. Now i need to get them back into the datafile. Keeping only the index of the voice datafile and using

 Help with Code Tags 
 Python Syntax (Toggle Plain Text) 
copy /b "voice.arc" + "*.wav" "voice.arc"
copy /b "voice.arc" + "*.wav" "voice.arc"

did a nice job at merging the files, but i also have to make the game recognize the new file positions.

The structure of the datafile is as follows:

http://img389.imageshack.us/img389/5168/voicearcgl6.png

The header determins the file type. Then the file names are stored in alphabetical order (yes, it jumps from 0-2, 3-9, etc, no idea why, that's how they were made in the first place). After each file name there is a null character. The following 4 bytes represent the file size in hex, and the 4 bytes following them give that file's offset, in little endian. (It did take a while to figure this out...)

For the 23 music tracks i did all extraction, injection and correction of sizes/offsets by hand. But there's no way in hell i could do the same for the 1.5k+ voice files.

I also have the converted files separately, so i thought of doing this in Python the following way:

1. Give the program the initial offset in decimal (where the datafile's index ends)
2. Get size of every WAV file in the current folder, in alphabetical order, and dump the values into a temp file
3. Calculate the decimal offsets of the WAVs by adding up file sizes to the initial offset, dump those in a temp file as well (yeah i like temp files)
4. Convert sizes and offsets to hex, offsets will also have to be in little endian
5. Dump computed hex data into the index, at the right places.

Right now i'm stuck at step 2, and i only have a bit of crappy code. I think i know how to do steps 3 and 4, but right now i need to do 2.

 Help with Code Tags 
 Python Syntax (Toggle Plain Text) 
import os
filename = raw_input("filename plz: ")
size = os.path.getsize(filename)
filesize = str(size)
output = open("sizes.txt", "w")
output.write(filesize)
output.write("\n")
import os
filename = raw_input("filename plz: ")
size = os.path.getsize(filename)
filesize = str(size)
output = open("sizes.txt", "w")
output.write(filesize)
output.write("\n")

This only gets the size of one file. I have no idea how to make it get the sizes of all files in current directory. I read i'll need to use os.listdir, however i have no idea how to use it for what i need to do.

Any help is welcome.

Last edited by Th3_uN1Qu3; Aug 18th, 2008 at 5:49 pm.

•

Join Date: Jul 2008

Posts: 984

Reputation: Gribouillis is a jewel in the rough

Solved Threads: 222

Gribouillis Gribouillis is online now

Online

Posting Shark

Re: Getting size of all files in folder, and finding out some offsets..

Aug 18th, 2008

I'd suggest something like this

 Help with Code Tags 
 python Syntax (Toggle Plain Text) 
import os
 
def writeSizes(folder, tmpfile):
  temp = open(tmpfile, "w")
  for name in sorted(os.walk(folder).next()[2]):
    p = os.path.join(folder, name)
    temp.write("%s %d\n)" % (name, os.getsize(p))
  temp.close()
 
writeSizes("C:\path\to\my\folder", "C:\path\to\my\temp\file")
import os

def writeSizes(folder, tmpfile):
  temp = open(tmpfile, "w")
  for name in sorted(os.walk(folder).next()[2]):
    p = os.path.join(folder, name)
    temp.write("%s %d\n)" % (name, os.getsize(p))
  temp.close()

writeSizes("C:\path\to\my\folder", "C:\path\to\my\temp\file")

Last edited by Gribouillis; Aug 18th, 2008 at 6:09 pm.

•

Join Date: Aug 2008

Posts: 12

Reputation: Th3_uN1Qu3 is an unknown quantity at this point

Solved Threads: 0

Th3_uN1Qu3 Th3_uN1Qu3 is offline

Offline

Newbie Poster

Re: Getting size of all files in folder, and finding out some offsets..

Aug 18th, 2008

Alright i filled in my folder and temp file names at the bottom, tried both single and double backslashes but it keeps saying

 Help with Code Tags 
 Python Syntax (Toggle Plain Text) 
line 8
temp.close()
 
SyntaxError: invalid syntax
line 8
temp.close()

SyntaxError: invalid syntax

Edit: I think i almost got it, i'll be back later with my results.
Edit 2: There's something wrong with the

 Help with Code Tags 
 Python Syntax (Toggle Plain Text) 
temp.write("%s %d\n)" % (name, os.getsize(p))
temp.write("%s %d\n)" % (name, os.getsize(p))

line, when i type it in the interpreter it looks like it's expecting another block of code.

Last edited by Th3_uN1Qu3; Aug 18th, 2008 at 6:44 pm.

•

Join Date: Jul 2008

Posts: 984

Reputation: Gribouillis is a jewel in the rough

Solved Threads: 222

Gribouillis Gribouillis is online now

Online

Posting Shark

Re: Getting size of all files in folder, and finding out some offsets..

Aug 18th, 2008

ok, the previous line should be

 Help with Code Tags 
 python Syntax (Toggle Plain Text) 
    temp.write("%s %d\n" % (name, os.path.getsize(p)))
    temp.write("%s %d\n" % (name, os.path.getsize(p)))

•

Join Date: Aug 2008

Posts: 12

Reputation: Th3_uN1Qu3 is an unknown quantity at this point

Solved Threads: 0

Th3_uN1Qu3 Th3_uN1Qu3 is offline

Offline

Newbie Poster

Re: Getting size of all files in folder, and finding out some offsets..

Aug 18th, 2008

Should've noticed "path" missing from the function, after all, i had played with that just a few minutes ago...

Thanks, trying it now.

Edit: It kept saying that it can't find the temp file, and didn't work with either single or double backslashes (although the error it gave was a typical "need double backslashes" thing), so i just did this:

 Help with Code Tags 
 python Syntax (Toggle Plain Text) 
import os
 
folder = raw_input("folder path: ")
temp = open("sizes.txt", "w")
for name in sorted(os.walk(folder).next()[2]):
	p = os.path.join(folder, name)
	temp.write("%s %d\n" % (name, os.path.getsize(p)))
temp.close()
import os
 
folder = raw_input("folder path: ")
temp = open("sizes.txt", "w")
for name in sorted(os.walk(folder).next()[2]):
	p = os.path.join(folder, name)
	temp.write("%s %d\n" % (name, os.path.getsize(p)))
temp.close()

Tada. Works perfectly. Now i'll make it only get the size of WAV files. Of course the lazy man's way would be to just move everything else away, and save sizes.txt elsewhere, but hey, you don't learn anything by being lazy.

Last edited by Th3_uN1Qu3; Aug 18th, 2008 at 7:30 pm.

•

Join Date: Jul 2008

Posts: 984

Reputation: Gribouillis is a jewel in the rough

Solved Threads: 222

Gribouillis Gribouillis is online now

Online

Posting Shark

Re: Getting size of all files in folder, and finding out some offsets..

Aug 18th, 2008

I tried it here, and it works very well. But, I'm on linux, so I don't use backslashes

•

Join Date: Aug 2008

Posts: 12

Reputation: Th3_uN1Qu3 is an unknown quantity at this point

Solved Threads: 0

Th3_uN1Qu3 Th3_uN1Qu3 is offline

Offline

Newbie Poster

Re: Getting size of all files in folder, and finding out some offsets..

Aug 18th, 2008

I'd be on Linux too if i weren't a gamer, but i am, and Wine isn't an answer.

Vista just blows. What you gain in speed in 64-bit vs 32-bit Vista you lose in compatibility. I really hope Linux will be ready for the mainstream soon.

•

Join Date: Aug 2008

Posts: 12

Reputation: Th3_uN1Qu3 is an unknown quantity at this point

Solved Threads: 0

Th3_uN1Qu3 Th3_uN1Qu3 is offline

Offline

Newbie Poster

Re: Getting size of all files in folder, and finding out some offsets..

Sep 14th, 2008

Sorry i've been away for so long.

Thanks Gribouillis, your code has been very useful. I eventually finished the task with AutoHotkey automating XVI32, as i am more of a scripting kind of guy. It wasn't the fastest nor the cleanest way, but it worked, and it was fun to watch the computer work by itself.

I'll drop by here when i want to learn more python... Till then, cheers, and once the game release is posted on the DZ board, your name will be in the credits.