Home » Fable TLC » Development » Advanced Modding » BigTools
BigTools [message #66032] |
Fri, 09 March 2012 14:19 |
|
I'm going to have to restart, the original code was a mess, and I've learned more since then. Until then I'm locking the thread.
"All of the work, and none of the play, will surely provide for a speedy decay"
[Updated on: Tue, 18 June 2013 12:21] Report message to a moderator
|
|
|
Re: .big header [message #66164 is a reply to message #66032] |
Tue, 20 March 2012 21:48 |
xenn
Messages: 17 Registered: February 2012 Location: Canada
|
|
|
|
morerunes wrote on Fri, 09 March 2012 17:19 | Where are there so many nulls at the top of textures.big? There's just tons of them. The wiki says there should be 16 bytes before the number of banks, and that number is among a gigantic blob of nulls, so I don't know what I'm supposed to be looking for.
|
This hardly belongs under "Advanced Modding".
Try asking within the "Discussion Area" forum next time, please.
To address your confusion, there aren't 16 bytes before the number of banks.
There are 16 bytes that make up the main header from the beginning of the .BIG file.
Most (if not all) .BIG files have zero-data after the header to 800h, where the payload begins.
The 'number of banks' value for each .BIG file is located at the first dword (4-byte integer) pointed to by "Bank address" value (the third dword value in the header).
That is, the bank address specifies the offset of the beginning of the bank index, relative to the beginning of the file.
Perhaps someone can correct me if I'm mistaken.
|
|
|
Re: .big header [message #66166 is a reply to message #66164] |
Wed, 21 March 2012 05:50 |
|
asmcint
Messages: 1360 Registered: April 2010 Location: Behind the beef
|
Moderator
|
|
|
Okay, first off dude, you just earned a lot of respect from me, as this is your first post, it was intelligent, and it wasn't a bump. Second however, is that this probably DOES belong in advanced modding, as advanced modding here is pretty much anything that requires a hex editor and a lot of experience to understand.
;)
Read the site rules, as well as individual thread rules, stickies and announcements, and use search, or you will have smartassy or exasperated ownage rained down upon you by the site's crack team of mods and admins. Also, you can find all you need to get started on modding here.
|
|
|
Re: .big header [message #66174 is a reply to message #66164] |
Wed, 21 March 2012 13:31 |
|
xenn wrote on Tue, 20 March 2012 23:48 | This hardly belongs under "Advanced Modding".
Try asking within the "Discussion Area" forum next time, please
|
I know how this place works, but thanks anyway.
As for your response, I suppose I have been reading the value wrong then. I used that offset and still found nothing. I'll look over my code again.
"All of the work, and none of the play, will surely provide for a speedy decay"
|
|
|
Re: .big header [message #66175 is a reply to message #66032] |
Wed, 21 March 2012 16:05 |
xenn
Messages: 17 Registered: February 2012 Location: Canada
|
|
|
|
Thank you asmcint.
I've been following your posts particularly for a little while now, shortly after I registered, given your posting habits.
Hope you don't mind.
@morerunes : Do you mean "reading the value wrong" yourself, or having code that handles some value(s) incorrectly ?
What programming language are you working with, if the latter, out of curiosity ?
|
|
|
Re: .big header [message #66176 is a reply to message #66175] |
Wed, 21 March 2012 17:02 |
|
I think I wrote some malfunctioning code, because I haven't done much with binary file io, but I'm starting to get the hang of it now.
I'm writing it in C.
Also, I didn't say it before, but you are indeed much cooler than many of the people who blaze through here.
"All of the work, and none of the play, will surely provide for a speedy decay"
[Updated on: Wed, 21 March 2012 17:02] Report message to a moderator
|
|
|
|
Re: .big Reading [message #66235 is a reply to message #66032] |
Wed, 28 March 2012 20:10 |
|
Alright, I've written some code that seems to work, but I'm not convinced. The output is giving me a bank address of 401621575, which just seems too big to me. Am I missing something?
- edit -
I've fixed an error, and gotten a more reasonable bank address value: 6570567
My Code
/* Reads a Fable: TLC .big file and exports texture files
* Version: 0.1 - Alpha
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define INFILE "in.big"
#define BUFFER_SIZE 500
int main()
{
// Open a file stream
FILE *inBig;
char *buffer = (char *) malloc(BUFFER_SIZE);
inBig = fopen(INFILE, "r");
size_t numElementsRead;
if (!inBig)
{
printf("Error opening file! Make sure it is named 'in.big' and placed in the correct folder.");
fclose(inBig);
return -1;
}
numElementsRead = fread(buffer, 4, 4, inBig);
if (numElementsRead != 4)
printf("What the hell!? Read: %d", numElementsRead);
// Capture the bank address
buffer += 2;
long *bankAddress;
bankAddress = (long *) buffer;
buffer -= 2;
printf("Bank Address: %ld", *bankAddress);
fclose(inBig);
return 0;
}
"All of the work, and none of the play, will surely provide for a speedy decay"
[Updated on: Wed, 28 March 2012 20:53] Report message to a moderator
|
|
|
Re: .big Reading [message #66242 is a reply to message #66235] |
Thu, 29 March 2012 06:48 |
xenn
Messages: 17 Registered: February 2012 Location: Canada
|
|
|
|
morerunes wrote on Wed, 28 March 2012 23:10 | Alright, I've written some code that seems to work, but I'm not convinced. The output is giving me a bank address of 401621575, which just seems too big to me. Am I missing something?
- edit -
I've fixed an error, and gotten a more reasonable bank address value: 6570567
My Code[code]
/* Reads a Fable: TLC .big file and exports texture files
* Version: 0.1 - Alpha
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define INFILE "in.big"
#define BUFFER_SIZE 500
int main()
{
// Open a file stream
FILE *inBig;
char *buffer = (char *) malloc(BUFFER_SIZE);
inBig = fopen(INFILE, "r");
size_t numElementsRead;
if (!inBig)
{
printf("Error opening file! Make sure it is named 'in.big' and placed in the correct folder.");
fclose(inBig);
return -1;
}
numElementsRead = fread(buffer, 4, 4, inBig);
if (numElementsRead != 4)
printf("What the hell!? Read: %d", numElementsRead);
// Capture the bank address
buffer += 2;
long *bankAddress;
bankAddress = (long *) buffer;
buffer -= 2;
printf("Bank Address: %ld", *bankAddress);
fclose(inBig);
return 0;
}
[/pre] [/align]
|
The number that :
Quote: | // Capture the bank address
buffer += 2;
long *bankAddress;
bankAddress = (long *) buffer;
buffer -= 2;
printf("Bank Address: %ld", *bankAddress);
|
is reading out, is the high word of "Char [4] ‘BIGB’" and the low word of "Char [4] Version", which is, for most of the .BIG files I've checked, the red segment in :
or 00644247 in hex,
or 6,570,567 in decimal.
I think by "buffer += 2;", you mean to add 2 dwords to the buffer address.
Instead, what the program does is only add two bytes to the buffer address.
You can resolve this by using "buffer += 2*4;", or, essentially the same thing, "buffer += 8;".
Fixed,
Quote: | // Capture the bank address
buffer += 2*4;
long *bankAddress;
bankAddress = (long *) buffer;
buffer -= 2*4;
printf("Bank Address: %ld", *bankAddress);
|
should read out the correct bank address (in decimal).
Of the original files, for example, these .BIG files should read out :
graphics.big -> 243,841,850
shaders.big -> 452,290
frontend.big -> 13,794,314
textures.big -> 533,633,006
|
|
|
|
Re: .big Reading [message #66273 is a reply to message #66243] |
Mon, 02 April 2012 11:57 |
|
I've just updated my first post. Can somebody help me debug this thing? I'm getting closer, but it still prints weird info for the first bank. Block size of 0? Index end of 71?
Ignore the second bank info, I haven't gotten that far yet.
You can run the program by just clicking on it, but you if you want you can give it arguments (it'll tell you how to use the arguments if you mess up)
-- edit --
Just realized that the 'index end' is actually the index size. So that makes sense
-- edit --
Ok, I've done enough for today. Check it out in the first post.
"All of the work, and none of the play, will surely provide for a speedy decay"
[Updated on: Mon, 02 April 2012 19:35] Report message to a moderator
|
|
|
Re: .big Reading [message #66276 is a reply to message #66273] |
Wed, 04 April 2012 19:48 |
xenn
Messages: 17 Registered: February 2012 Location: Canada
|
|
|
|
Rather than Quote: | buffer += 2*4;
bnkAddr = *((long *) buffer);
buffer -= 2*4;
|
consider Quote: | bnkAddr = *((long *) buffer+2);
|
Saves you two steps. Should work the same.
"Bank Address" and "Number of Banks" read okay.
Bank ID doesn't read correctly.
The bank name seems to read okay though.
"Number of Bank Entries", "Index Start", "Index Size", and "Block Size" all seem to not read correctly.
Bank ID was a bit of a bitch to track down.
I don't know C, but I read somewhere that fscanf increments the file pointer as it reads characters, but that it can push the file pointer farther than the string length.
I don't know if that's true or accurate, but in case it is, this seems to be a safer (and working) implementation :
Quote: | /* BEGIN BANK INFO RETRIEVAL */
fseek(input, 4, SEEK_CUR);
fscanf(input, "%s", bankName[i]);
fseek(input,bnkAddr+5+strlen(bankName[i]),SEEK_SET);
|
The line added (in red) adds the length of the bank name (+1 to account for the null character and +4 to account for the "Number of Banks" dword) to the bank address, and sets the file pointer accordingly.
I hadn't been expecting it to, but fortunately that also seems to fix the next four readouts.
Only tested it with text.big and fonts.big, though.
I didn't look at anything from "NumberFileTypes" ("Number of File Types") onward, and stripped it out of the source for simplicity.
(To avoid crashing).
I'll keep changes I make to the source here : http://pastebin.com/raw.php?i=NNiYQ7HG
Combine the fseek fix with whatever you've got now, do some more of your own testing, and then let us know if you need some help.
It's unrelated, but one last thing that stood out was this :
Quote: | // Begin Reading File
char *buffer = (char *) malloc(5000);
|
The most memory I've seen your buffer use was something like 36 bytes.
It's probably convenient to choose some arbitrarily large memory amount, but just be careful not to get carried away ;)
It's not so important now, but if (when) you get to loading large chunks of the .BIG files at a time, being careful with your memory management can avoid crashes and hard-to-find bugs.
Good luck with the rest of your work !
|
|
|
|
Re: BigTools [message #66354 is a reply to message #66032] |
Mon, 16 April 2012 13:33 |
xenn
Messages: 17 Registered: February 2012 Location: Canada
|
|
|
|
Hey dude, are you still working on this ?
If not, I'd like to try my hand at making something similar myself, in assembly.
If you're still working on it though, I won't bother - I wouldn't want to encroach on your territory ;)
|
|
|
Re: BigTools [message #66519 is a reply to message #66354] |
Sun, 29 April 2012 00:05 |
|
Just so everyone knows, I have figured out why the file pointer was getting pushed past the string length. "%s" expects to get a series of 'non whitepace characters', not a null terminated string. It seems simple now that I'm thinking about it. Instead of stopping at null, it just kept reading chars from the file until it hit the end of the file and adding them to the string.
Instead I've written my own little method that reads a null terminated string from a file, since I can't find a good one in stdio
Also, I am completely refactoring the program, it should be a bit simpler to follow now, and I don't have everything crammed into one method. I've learned a lot about writing C code in the last few weeks, and I'm applying them now.
"All of the work, and none of the play, will surely provide for a speedy decay"
|
|
|
Re: BigTools [message #66528 is a reply to message #66519] |
Mon, 30 April 2012 11:39 |
|
Alright, I've uploaded a new version. I can't figure out why the second bank header info keeps disappearing after returning from the function.
If anybody has an idea, let me know.
"All of the work, and none of the play, will surely provide for a speedy decay"
|
|
|
Re: BigTools [message #66534 is a reply to message #66032] |
Mon, 30 April 2012 17:06 |
xenn
Messages: 17 Registered: February 2012 Location: Canada
|
|
|
|
For some reason the "BigTools.exe" in the archive doesn't work, but it works for me if I compile it myself.
You should tell us which .BIG files you're testing with, by the way - or upload them if they're not the original .BIG files.
Anyway, the one that compiles only displays BIGB, Version, Bank Address, and Unknown.
Is that what you meant by "the second bank header info keeps disappearing" ?
Or should it still show, at least, Bank ID to Block Size ?
I'll look into it now, a bit, but you should test your code more thoroughly. ;)
|
|
|
Re: BigTools [message #66537 is a reply to message #66534] |
Mon, 30 April 2012 19:02 |
|
I figured it out, and I was right about it being something small. If you'd like to see it working, check out the new version.
"All of the work, and none of the play, will surely provide for a speedy decay"
[Updated on: Tue, 01 May 2012 07:23] Report message to a moderator
|
|
|
Re: BigTools [message #66556 is a reply to message #66032] |
Tue, 01 May 2012 09:18 |
xenn
Messages: 17 Registered: February 2012 Location: Canada
|
|
|
|
>Not sure if trolling, or just careless
Your last two BigTools.exe's have crashed on me before displaying anything (on both my netbook & desktop).
Do you test run your release builds before posting them ?
I don't see how it could possibly work for you, so I've gotta believe you test some other, possibly debug, build ?
Take a look at this part of disassembly, where the crash occurs :
00401345 - push eax
00401346 - mov ecx,00000008
0040134B - xor eax,eax
0040134D - mov edi,eax
0040134F - repe movsd
Of course it's going to crash if it tries to move anything to the address 00000000h (which is what EDI is).
Compare it with this code I assembled using Code::Blocks & GCC :
00401478 - sub esp,04
0040147B - mov edx,ebx
0040147D - lea ebx,[ebp-58]
00401480 - mov eax,00000008
00401485 - mov edi,edx
00401487 - mov esi,ebx
00401489 - mov ecx,eax
0040148B - repe movsd
What compiler & IDE do you use ?
Anyway, are you sure you don't mind me whipping my own up in assembly ?
Also, I'm about to try your latest update - hope the above is fixed lol
|
|
|
Re: BigTools [message #66557 is a reply to message #66556] |
Tue, 01 May 2012 09:26 |
|
Most likely ignorance.
I am using Code::Blocks with MinGW
You can try this one, I've been giving you the 'release' build, but here's the debug build.
I don't know why the two would be different.
Also, feel free to write whatever software you want to. I am using this program as an exercise to learn C programming. I have only very recently begun learning to program in C. I don't know the x86 ISA, thus the decompiled assembly is meaningless to me until I get to learning that.
-
Attachment: BigTools.exe
(Size: 34.51KB, Downloaded 886 times)
"All of the work, and none of the play, will surely provide for a speedy decay"
[Updated on: Tue, 01 May 2012 11:03] Report message to a moderator
|
|
|
|
|
Goto Forum:
Current Time: Thu Nov 21 02:36:29 PST 2024
Total time taken to generate the page: 0.08972 seconds
|