Fable: The Lost Chapters Mod Scene
Fast Uncompromising Discussions. FUDforum will get your users talking.

Home » Fable TLC » Development » Advanced Modding » BigTools
BigTools [message #66032] Fri, 09 March 2012 14:19 Go to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

I'm going to have to restart, the original code was a mess, and I've learned more since then. Until then I'm locking the thread.

"All of the work, and none of the play, will surely provide for a speedy decay"

[Updated on: Tue, 18 June 2013 12:21]

Report message to a moderator

Re: .big header [message #66164 is a reply to message #66032] Tue, 20 March 2012 21:48 Go to previous messageGo to next message
xenn is currently offline  xenn
Messages: 17
Registered: February 2012
Location: Canada
morerunes wrote on Fri, 09 March 2012 17:19

Where are there so many nulls at the top of textures.big? There's just tons of them. The wiki says there should be 16 bytes before the number of banks, and that number is among a gigantic blob of nulls, so I don't know what I'm supposed to be looking for.


This hardly belongs under "Advanced Modding".
Try asking within the "Discussion Area" forum next time, please.


To address your confusion, there aren't 16 bytes before the number of banks.
There are 16 bytes that make up the main header from the beginning of the .BIG file.
Most (if not all) .BIG files have zero-data after the header to 800h, where the payload begins.

The 'number of banks' value for each .BIG file is located at the first dword (4-byte integer) pointed to by "Bank address" value (the third dword value in the header).
That is, the bank address specifies the offset of the beginning of the bank index, relative to the beginning of the file.

Perhaps someone can correct me if I'm mistaken.
Re: .big header [message #66166 is a reply to message #66164] Wed, 21 March 2012 05:50 Go to previous messageGo to next message
asmcint is currently offline  asmcint
Messages: 1360
Registered: April 2010
Location: Behind the beef

Moderator
Okay, first off dude, you just earned a lot of respect from me, as this is your first post, it was intelligent, and it wasn't a bump. Second however, is that this probably DOES belong in advanced modding, as advanced modding here is pretty much anything that requires a hex editor and a lot of experience to understand.

;)


Read the site rules, as well as individual thread rules, stickies and announcements, and use search, or you will have smartassy or exasperated ownage rained down upon you by the site's crack team of mods and admins. Also, you can find all you need to get started on modding here.
Re: .big header [message #66174 is a reply to message #66164] Wed, 21 March 2012 13:31 Go to previous messageGo to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

xenn wrote on Tue, 20 March 2012 23:48

This hardly belongs under "Advanced Modding".
Try asking within the "Discussion Area" forum next time, please


I know how this place works, but thanks anyway.


As for your response, I suppose I have been reading the value wrong then. I used that offset and still found nothing. I'll look over my code again.


"All of the work, and none of the play, will surely provide for a speedy decay"
Re: .big header [message #66175 is a reply to message #66032] Wed, 21 March 2012 16:05 Go to previous messageGo to next message
xenn is currently offline  xenn
Messages: 17
Registered: February 2012
Location: Canada
Thank you asmcint.
I've been following your posts particularly for a little while now, shortly after I registered, given your posting habits.

Hope you don't mind.


@morerunes : Do you mean "reading the value wrong" yourself, or having code that handles some value(s) incorrectly ?
What programming language are you working with, if the latter, out of curiosity ?
Re: .big header [message #66176 is a reply to message #66175] Wed, 21 March 2012 17:02 Go to previous messageGo to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

I think I wrote some malfunctioning code, because I haven't done much with binary file io, but I'm starting to get the hang of it now.

I'm writing it in C.

Also, I didn't say it before, but you are indeed much cooler than many of the people who blaze through here.


"All of the work, and none of the play, will surely provide for a speedy decay"

[Updated on: Wed, 21 March 2012 17:02]

Report message to a moderator

Re: .big header [message #66182 is a reply to message #66164] Thu, 22 March 2012 06:58 Go to previous messageGo to next message
Noctus is currently offline  Noctus
Messages: 969
Registered: May 2008
Location: England

Administrator

xenn wrote on Tue, 20 March 2012 21:48

This hardly belongs under "Advanced Modding".
Try asking within the "Discussion Area" forum next time, please.
That probably isn't the way you should be talking to a moderator.

Also this is to do with the game's coding, which doesn't have a specific forum therefore this one is more than adequate.


http://img696.imageshack.us/img696/9540/helpmeaaa.jpg
Re: .big Reading [message #66235 is a reply to message #66032] Wed, 28 March 2012 20:10 Go to previous messageGo to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

Alright, I've written some code that seems to work, but I'm not convinced. The output is giving me a bank address of 401621575, which just seems too big to me. Am I missing something?

- edit -

I've fixed an error, and gotten a more reasonable bank address value: 6570567

My Code


"All of the work, and none of the play, will surely provide for a speedy decay"

[Updated on: Wed, 28 March 2012 20:53]

Report message to a moderator

Re: .big Reading [message #66242 is a reply to message #66235] Thu, 29 March 2012 06:48 Go to previous messageGo to next message
xenn is currently offline  xenn
Messages: 17
Registered: February 2012
Location: Canada
morerunes wrote on Wed, 28 March 2012 23:10

Alright, I've written some code that seems to work, but I'm not convinced. The output is giving me a bank address of 401621575, which just seems too big to me. Am I missing something?

- edit -

I've fixed an error, and gotten a more reasonable bank address value: 6570567

My Code
[/align]


The number that :
Quote:

// Capture the bank address
buffer += 2;
long *bankAddress;
bankAddress = (long *) buffer;
buffer -= 2;

printf("Bank Address: %ld", *bankAddress);



is reading out, is the high word of "Char [4] ‘BIGB’" and the low word of "Char [4] Version", which is, for most of the .BIG files I've checked, the red segment in :

Quote:

BIGBd...

or 00644247 in hex,
or 6,570,567 in decimal.

I think by "buffer += 2;", you mean to add 2 dwords to the buffer address.
Instead, what the program does is only add two bytes to the buffer address.

You can resolve this by using "buffer += 2*4;", or, essentially the same thing, "buffer += 8;".

Fixed,
Quote:

// Capture the bank address
buffer += 2*4;
long *bankAddress;
bankAddress = (long *) buffer;
buffer -= 2*4;

printf("Bank Address: %ld", *bankAddress);



should read out the correct bank address (in decimal).

Of the original files, for example, these .BIG files should read out :

graphics.big -> 243,841,850
shaders.big -> 452,290
frontend.big -> 13,794,314
textures.big -> 533,633,006



http://i.imgur.com/vAT1z.png
Re: .big Reading [message #66243 is a reply to message #66242] Thu, 29 March 2012 13:27 Go to previous messageGo to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

Thanks for the help again, I thought I was advancing the pointer by elements, not bytes. Still new to the whole pointer math thing Razz

"All of the work, and none of the play, will surely provide for a speedy decay"
Re: .big Reading [message #66273 is a reply to message #66243] Mon, 02 April 2012 11:57 Go to previous messageGo to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

I've just updated my first post. Can somebody help me debug this thing? I'm getting closer, but it still prints weird info for the first bank. Block size of 0? Index end of 71?

Ignore the second bank info, I haven't gotten that far yet.


You can run the program by just clicking on it, but you if you want you can give it arguments (it'll tell you how to use the arguments if you mess up)

-- edit --

Just realized that the 'index end' is actually the index size. So that makes sense

-- edit --

Ok, I've done enough for today. Check it out in the first post.


"All of the work, and none of the play, will surely provide for a speedy decay"

[Updated on: Mon, 02 April 2012 19:35]

Report message to a moderator

Re: .big Reading [message #66276 is a reply to message #66273] Wed, 04 April 2012 19:48 Go to previous messageGo to next message
xenn is currently offline  xenn
Messages: 17
Registered: February 2012
Location: Canada
Rather than
Quote:

buffer += 2*4;
bnkAddr = *((long *) buffer);
buffer -= 2*4;

consider
Quote:

bnkAddr = *((long *) buffer+2);

Saves you two steps. Should work the same.

"Bank Address" and "Number of Banks" read okay.

Bank ID doesn't read correctly.
The bank name seems to read okay though.

"Number of Bank Entries", "Index Start", "Index Size", and "Block Size" all seem to not read correctly.

Bank ID was a bit of a bitch to track down.
I don't know C, but I read somewhere that fscanf increments the file pointer as it reads characters, but that it can push the file pointer farther than the string length.
I don't know if that's true or accurate, but in case it is, this seems to be a safer (and working) implementation :
Quote:

/* BEGIN BANK INFO RETRIEVAL */
fseek(input, 4, SEEK_CUR);
fscanf(input, "%s", bankName[i]);
fseek(input,bnkAddr+5+strlen(bankName[i]),SEEK_SET);

The line added (in red) adds the length of the bank name (+1 to account for the null character and +4 to account for the "Number of Banks" dword) to the bank address, and sets the file pointer accordingly.

I hadn't been expecting it to, but fortunately that also seems to fix the next four readouts.
Only tested it with text.big and fonts.big, though.

I didn't look at anything from "NumberFileTypes" ("Number of File Types") onward, and stripped it out of the source for simplicity.
(To avoid crashing).

I'll keep changes I make to the source here : http://pastebin.com/raw.php?i=NNiYQ7HG

Combine the fseek fix with whatever you've got now, do some more of your own testing, and then let us know if you need some help.

It's unrelated, but one last thing that stood out was this :
Quote:

// Begin Reading File
char *buffer = (char *) malloc(5000);

The most memory I've seen your buffer use was something like 36 bytes.
It's probably convenient to choose some arbitrarily large memory amount, but just be careful not to get carried away ;)
It's not so important now, but if (when) you get to loading large chunks of the .BIG files at a time, being careful with your memory management can avoid crashes and hard-to-find bugs.


Good luck with the rest of your work !
Re: .big Reading [message #66277 is a reply to message #66276] Thu, 05 April 2012 05:16 Go to previous messageGo to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

That's incredibly helpful, thank you!

I think what I may do for testing is read the entire file into memory for now, that way I can figure out how to read the file properly. Once I've got my algorithm going I can dumb down my program for less intense computer setups Wink


"All of the work, and none of the play, will surely provide for a speedy decay"
Re: BigTools [message #66354 is a reply to message #66032] Mon, 16 April 2012 13:33 Go to previous messageGo to next message
xenn is currently offline  xenn
Messages: 17
Registered: February 2012
Location: Canada
Hey dude, are you still working on this ?

If not, I'd like to try my hand at making something similar myself, in assembly.

If you're still working on it though, I won't bother - I wouldn't want to encroach on your territory ;)

Re: BigTools [message #66519 is a reply to message #66354] Sun, 29 April 2012 00:05 Go to previous messageGo to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

Just so everyone knows, I have figured out why the file pointer was getting pushed past the string length. "%s" expects to get a series of 'non whitepace characters', not a null terminated string. It seems simple now that I'm thinking about it. Instead of stopping at null, it just kept reading chars from the file until it hit the end of the file and adding them to the string.

Instead I've written my own little method that reads a null terminated string from a file, since I can't find a good one in stdio

Also, I am completely refactoring the program, it should be a bit simpler to follow now, and I don't have everything crammed into one method. I've learned a lot about writing C code in the last few weeks, and I'm applying them now.


"All of the work, and none of the play, will surely provide for a speedy decay"
Re: BigTools [message #66528 is a reply to message #66519] Mon, 30 April 2012 11:39 Go to previous messageGo to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

Alright, I've uploaded a new version. I can't figure out why the second bank header info keeps disappearing after returning from the function.

If anybody has an idea, let me know.


"All of the work, and none of the play, will surely provide for a speedy decay"
Re: BigTools [message #66534 is a reply to message #66032] Mon, 30 April 2012 17:06 Go to previous messageGo to next message
xenn is currently offline  xenn
Messages: 17
Registered: February 2012
Location: Canada
For some reason the "BigTools.exe" in the archive doesn't work, but it works for me if I compile it myself.

You should tell us which .BIG files you're testing with, by the way - or upload them if they're not the original .BIG files.

Anyway, the one that compiles only displays BIGB, Version, Bank Address, and Unknown.

Is that what you meant by "the second bank header info keeps disappearing" ?
Or should it still show, at least, Bank ID to Block Size ?


I'll look into it now, a bit, but you should test your code more thoroughly. ;)
Re: BigTools [message #66537 is a reply to message #66534] Mon, 30 April 2012 19:02 Go to previous messageGo to next message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

I figured it out, and I was right about it being something small. If you'd like to see it working, check out the new version.

"All of the work, and none of the play, will surely provide for a speedy decay"

[Updated on: Tue, 01 May 2012 07:23]

Report message to a moderator

Re: BigTools [message #66556 is a reply to message #66032] Tue, 01 May 2012 09:18 Go to previous messageGo to next message
xenn is currently offline  xenn
Messages: 17
Registered: February 2012
Location: Canada
>Not sure if trolling, or just careless

Your last two BigTools.exe's have crashed on me before displaying anything (on both my netbook & desktop).

Do you test run your release builds before posting them ?
I don't see how it could possibly work for you, so I've gotta believe you test some other, possibly debug, build ?


Take a look at this part of disassembly, where the crash occurs :

00401345 - push eax
00401346 - mov ecx,00000008
0040134B - xor eax,eax
0040134D - mov edi,eax
0040134F - repe movsd


Of course it's going to crash if it tries to move anything to the address 00000000h (which is what EDI is).

Compare it with this code I assembled using Code::Blocks & GCC :

00401478 - sub esp,04
0040147B - mov edx,ebx
0040147D - lea ebx,[ebp-58]
00401480 - mov eax,00000008
00401485 - mov edi,edx
00401487 - mov esi,ebx
00401489 - mov ecx,eax
0040148B - repe movsd


What compiler & IDE do you use ?


Anyway, are you sure you don't mind me whipping my own up in assembly ?

Also, I'm about to try your latest update - hope the above is fixed lol
Re: BigTools [message #66557 is a reply to message #66556] Tue, 01 May 2012 09:26 Go to previous message
morerunes is currently offline  morerunes
Messages: 2154
Registered: June 2007
Location: My desk

Administrator

Most likely ignorance.

I am using Code::Blocks with MinGW

You can try this one, I've been giving you the 'release' build, but here's the debug build.

I don't know why the two would be different.

Also, feel free to write whatever software you want to. I am using this program as an exercise to learn C programming. I have only very recently begun learning to program in C. I don't know the x86 ISA, thus the decompiled assembly is meaningless to me until I get to learning that.
  • Attachment: BigTools.exe
    (Size: 34.51KB, Downloaded 235 times)


"All of the work, and none of the play, will surely provide for a speedy decay"

[Updated on: Tue, 01 May 2012 11:03]

Report message to a moderator

Previous Topic: New lev Crap
Next Topic: Executable Functions
Goto Forum:
  


Current Time: Wed Nov 13 17:14:22 PST 2019

Total time taken to generate the page: 0.01452 seconds