Where do resource files reside (if not in BIF files) ?

[khelban12]

Greetings. I am not very familiar with BG2 internals so i have a stupid question about where do game files reside. I thought that the normal residence is in bif files and those files get overriden by files residing in the override/ folder. Is there any other case ?

Long description of why i am asking:

I use original BG2:Tob on linux using wine / gemrb. On sunday a new wine version was released and i compiled it and upgraded to it. I guess the wine team implemented some new functionality because, alongside the normal window with the game, i now also got a "console window" too with tobex afterlife messages. In this window i got the message that clssplab.2da does not exist.

I searched my tree and clssplab.2da is supposed to be copied by tobex but it is not. Every file is correctly copied except clssplab.2da and item_use.2da.

DEFINE_ACTION_FUNCTION tobex_copy STR_VAR file = "nul" BEGIN
  ACTION_IF NOT FILE_EXISTS_IN_GAME "%file%" BEGIN
    COPY "%TOBEX_MOD_DIRECTORY%/TobEx_redist/override/%file%" "override/%file%"
  END
END

This is the code that does the copying. A simple check for the existence of the file. This code works correctly for every file except for clssplab.2da and item_use.2da. The "FILE_EXISTS_IN_GAME" clause returns that these two files exist. However, neither DLTCEP, nor NearInfinity can find these files. I also opened chitin.key in a hex editor and i couldn't find the resource names there either. I tested a fully patched BG2:TOB tree clean from any mods.

I thought that there must be a bug in the FILE_EXISTS_IN_GAME clause in my weidu (249 version) because i had some other weidu problems lately that appeared suddenly out of nowhere but i ran the following code

ACTION_IF FILE_EXISTS_IN_GAME "clssplab.2da" BEGIN
    PRINT ~HELLO~
    COPY_EXISTING ~clssplab.2da~ ~override/tmp.2da~
    COPY_EXISTING ~item_use.2da~ ~override/tmp2.2da~
END

and lo and behold, the two files get copied correctly to override and have sensible contents (almost same as the ones in tobex but with less columns).

So, my questions are:

a) Where do these two files reside, if not in the override folder or in bif files ?

b) Why does only FILE_EXISTS_IN_GAME find them and not tobex / dltcep / nearinfinity ?

Thank you for your time.

Edited July 12, 2023 by khelban12

[lynx]

These are not vanilla tables, so it's expected that they would be missing on an unmodded install. In your case I'd run that weidu snippet through strace to see where they are pulled from (it also looks in other "override" folders like "movies" and of course "data").

Also GemRB provides them, so if you have gemrb_path.txt, it can pull those. Which would explain why NI and other tools know nothing about them.

Edited July 12, 2023 by lynx

[khelban12]

53 minutes ago, lynx said:

Also GemRB provides them, so if you have gemrb_path.txt, it can pull those. Which would explain why NI and other tools know nothing about them.

Once again, lynx, your help was invaluable. That was the culprit.

I know gemrb creates several files in the tree but they do not cause any problems (my gem-baldur.ini hasn't changed since 2017) so it didn't occur to me that the problem would be due to germb_path.txt. I guess the last time i changed my mod order i used a weidu < 247 and it didn't support gemrb_path so it didn't cause any problems (or most likely i didn't reinstall tobex because i usually change mods after tobex and fixpack).

[lynx]

Well, then we have a problem. The mechanism was added to weidu so mods could easily override our unhardcoded files. In this case one could say that it's not really a problem, since you can't use tobex with gemrb, but that's a bit cheap. If you installed tobex first, perhaps gemrb would have problems then, if the tables are sufficiently different.

Are they from tobex or tobexal? I can look at unifying them, so then nobody will care about the order or mixed installs.

[khelban12]

1 hour ago, lynx said:

Well, then we have a problem. The mechanism was added to weidu so mods could easily override our unhardcoded files. In this case one could say that it's not really a problem, since you can't use tobex with gemrb, but that's a bit cheap. If you installed tobex first, perhaps gemrb would have problems then, if the tables are sufficiently different.

Are they from tobex or tobexal? I can look at unifying them, so then nobody will care about the order or mixed installs.

Won't this put unecessary extra burden on you ? These two files come from tobex but maybe another mod comes with a file which happens to have same name as one in unhardcoded/bg2 and this problem will happen again.

Is it common for users to run both oBG2 (in wine or windows) and gemrb from the same tree ? If it was just an oversight of mine, i can remember to delete gemrb_path.txt before each re-installation.

[lynx]

While tobex was still in development all of us made an effort to not deviate too much:

I think it's unlikely non-engine mods would ship new 2das, so to me this all falls under general compatibility work.

 

[lynx]

The shipped item_use.2da has more columns, but no actual extra data from gemrb's point of view. It would require some work to make tobex fully happy, but since the two problematic lines are not used in bg2, there's no point in trying.

The other one I've made a fix, it's just not submitted yet.

[khelban12]

33 minutes ago, lynx said:

The shipped item_use.2da has more columns, but no actual extra data from gemrb's point of view. It would require some work to make tobex fully happy, but since the two problematic lines are not used in bg2, there's no point in trying.

The other one I've made a fix, it's just not submitted yet.

Thank you again for your swift respone.

Implementing it will not solve the problem with oBG2 so if it doesn't benefit gemrb then of course no point burdening yourself with needless work. I put a note on my mod-instructions-document to remember and delete gemrb_path.txt before installation.

[lynx]

Tobex could just use our tables in such cases, but the weidu code would need adapting. Now that the AL version is maintained, that's within reach.

I don't see a reason for the FILE_EXISTS_IN_GAME check for example, it should just install the files. Maybe with a BUT_ONLY_IF_IT_CHANGES / BUT_ONLY to make purists happy.

[jmerry]

Watch out on the BUT_ONLY clause; what it checks is whether any changes were made since loading the file. If you're using a COPY command to take a file from some repository and put a copy in the override without further changes, you definitely don't want a BUT_ONLY on it. I know BUT_ONLY feels like boilerplate (a convenient way to mark when you're done with something), but there are legitimate cases in which it's important to not use it. I've had errors stem from this mistake in my own modding.

[lynx]

I thought it's there to not do any work, maybe not take coownership, if the specified change would be a noop (in the case of a copy if the exact same file exists already).

[jmerry]

But that's not what a BUT_ONLY checks against. That's what the negated FILE_EXISTS_IN_GAME check is for. A BUT_ONLY on this copy operation would always result in a no-op. Copy the file, do nothing further with it, note that no changes were made since making the copy, don't save the copy. End result: the function does nothing.

Here's an example (from my own mod) of something related, using that BUT_ONLY behavior for a positive purpose:

COPY ~jtweaks/resource/2da/j8hhexft.2da~ ~override~ // Table of data for changes
	READ_2DA_ENTRIES_NOW j8hhexft 42
BUT_ONLY // No need to keep the override copy

Copy a 2da file, use a READ_2DA_ENTRIES_NOW to put its data in an array, then move on without saving the 2da. I don't want to put a random 2da with no gameplay relevance into the override, and the most efficient way to do that is to simply not save it in the first place.