How to generate a list of shields and apply dynamic patching to each?

[CidBahamut]

I'm gonna be real with you guys, the documentation for Weidu is really obtuse and I need some hand holding. I know what I want to do, but not how to do it with Weidu. 
Long and short of it is I'm trying to figure out how to generate a list of every item in the game and then filter it down to just shield type items. From there I'll take that list and for each item I want to add 10% physical damage resistance, done by adding the three effects for slashing, piercing and crushing damage resistance,(not sure if missile damage should be part of this since I didn't see it on Defender of Easthaven in Near Infinity).

 

Some very dirty pseudo code of what I'm trying to go for here:

Spoiler

 

$listOfAllItems = Get-AllGameItems
$listOfShields = @()

#compile a list of every shield in the game
foreach($item in $listOfAllItems){    
    #check if item is a shield
    if($item.Category -eq 12){
        $listOfShields += $item
    }
}

#add damage resistance to each shield
foreach($shield in $listOfShields){    
    #example: SHLD21.ITM
    $fileName = $shield.GetDotItemFileName()
    
    #probably need to calculate next available offset for an effect
    #in here somewhere.    

    COPY_EXISTING "$fileName" "override"
        #add slashing resistance of 10%
        ADD_ITEM_EFFECT
            #parameter to set opcode to 86, slashing resistance
            #parameter to set Target to 1, Self
            #parameter to set Value to 10, 10%
            #parameter to set Timing mode to 2, Instant/While equipped
            #parameter to set Dispel/Resistance to 2, Not dispel/Bypass resistance
            #parameter to set Probability 1 to 100
            #all other effect properties default to zero and don't need to be altered

        #add crushing resistance of 10%
        #add piercing resistance of 10%        
}

Any insights you guys can offer would be much appreciated, because after several hours of staring at documentation my brain is toast.

Edited October 4 by CidBahamut

[Guest Shard of Lost Knowledge]

The basic mindset's wrong and that's not something you should blame WeiDU's doc for.

For starters, building up a list to cache out the shields from all ITMs is unnecessary (since you'll be using this filtered list only once). With your method, you'd iterate all ITMs once and then the shields a second time for no real benefit. You only want to cache results where runtime lookups are slow. (In those cases you store SOURCE_RES within your COPY_EXISTING_REGEXP glob to your predefined variable.)

Just COPY_EXISTING_REGEXP all ITMs as your "foreach" and then use PATCH_IF on a variable read from the shield category offset to filter out the nonshields. BUT_ONLY ensures that you don't duplicate out the files you don't touch at the end.

For ADD_ITEM_(EQ)EFFECT, you don't need to calculate offsets, the macro will handle that to you (ADD_ITEM_EFFECT adds something you can activate on-will, EQEFFECT is more likely what you want here). That's why you want to use the built-in macros/functions, to ensure your math doesn't go off.

Which means that the result is a simple

COPY_EXISTING_REGEXP GLOB ~.*\.itm~ ~override~
	READ_SHORT 0x1C type
	PATCH_IF (type = 12) BEGIN
		LPF ADD_ITEM_EQEFFECT INT_VAR opcode=56 target=1 timing=2 parameter1=10 parameter2=0 resist_dispel=2
		LPF ADD_ITEM_EQEFFECT INT_VAR opcode=57 target=1 timing=2 parameter1=10 parameter2=0 resist_dispel=2
		LPF ADD_ITEM_EQEFFECT INT_VAR opcode=58 target=1 timing=2 parameter1=10 parameter2=0 resist_dispel=2
	END
BUT_ONLY

 

[jmerry]

6 hours ago, CidBahamut said:

Long and short of it is I'm trying to figure out how to generate a list of every item in the game and then filter it down to just shield type items.

Alternative structure: don't build a list. You need to load up all the items to check whether they're shields, so just make the changes immediately once you know it's a shield.

7 hours ago, CidBahamut said:

    #probably need to calculate next available offset for an effect
    #in here somewhere.

That's what functions are for. The ADD_ITEM_EQEFFECT function does what you want here without needing to look up any fiddly bits with offsets.

So anyway, here's my code for this, with plenty of comments.

Spoiler

COPY_EXISTING_REGEXP GLOB ~^.+\.ITM~ ~override~ // Loops through all items, copying them.
    PATCH_IF (SOURCE_SIZE > 0x71) BEGIN // Guards against malformed files that are too small.
        READ_SHORT 0x1c category // Reads the two bytes at offset 1c, which are the item category.
        PATCH_IF (category = 12) BEGIN // Shields. PATCH version of IF because we have a file loaded.
            LPF ADD_ITEM_EQEFFECT // Launch the function
              INT_VAR
                opcode = 86 // Slashing resistance.
                target = 1 // Target self, because that's the wielder.
                timing = 2 // While equipped. The standard for equipment bonuses.
                parameter1 = 10 // Amount to increase resistance by.
                parameter2 = 0 // Increment mode. Default value, so not actually needed.
            END // Done with the function. All other values default.
            LPF ADD_ITEM_EQEFFECT INT_VAR opcode = 87 target = 1 timing = 2 parameter1 = 10 END
            LPF ADD_ITEM_EQEFFECT INT_VAR opcode = 88 target = 1 timing = 2 parameter1 = 10 END
            // Two more uses of the function for crushing and piercing, without the comments and extra space.
        END // Done with main IF block.
    END // Done with outer IF block
BUT_ONLY // Done with this item. Save the copy only if changes were made.

 

Some gameplay consequences may ensue with regard to creatures that have high physical resistance and carry a shield. Though Yaga-Shura puts his shield away in favor of another hammer in the Ascension mod, so the nastiest possible case doesn't apply here.

[CidBahamut]

As long as I've got you here, I'm hoping I can pick your brain a bit more. 
So for specifying to opcode, its decimal value is 86 but in hex it would be 56. In Near Infinity it gets displayed as a decimal value so that's just what I defaulted to but the more I dig into this stuff the more it looks like hex values all the way down inside the files themselves. When is it appropriate to specify stuff via decimal value and when is it appropriate to specify it via hex value? Additionally: You specify the READ_SHORT offset using the format 0x1C rather than 1C. Is there a meaningful distinction between those two formats and if so when should I be making use of one over the other?

I could also use a rundown on INT_VAR from somoene who's more knowledgeable than I am. There's over 500 instances of it in the documentation and I haven't managed to read through to a section where it gets clearly laid out. From what I can piece together it's creating integer variables with specified values and then said variables are immediately getting passed up the chain to the ADD_ITEM_EQEFFECT function as parameters. Now is this a shortcut to bundle up the variable definition or is it syntax specific to the ADD_ITEM_EQEFFECT function? Preliminary efforts to mess around with INT_VAR suggest the latter, but I'm not ruling out user error on my part when trying to use INT_VAR to simply define a variable as standalone fucntionality.

Lastly, you guys define both parameter1 and parameter2. Now in context I can piece together that those correspond to the "Value" and "Modifier type" attributes of the effect as displayed in Near Infinity, but on my own I don't know if I'd have been able to piece that together even whilst staring at the IESDP entry for the .itm file format. What other kinds of discrepancies with Near Infinity should I be on the lookout for? Just common pitfalls or confusion I'll run into if I assume that Near Infinity is feeding me the correct names to be using in my Weidu code. 

Oh one more thing before I forget. When patching an item, if I wanted to display the name of the .itm file, is there a way to do that? It doesn't look like something you can pull by reading an offset within the file itself so I'm wondering if there's another way to do that. Mostly thinking about this for debug purposes.

I really do appreciate you guys taking the time to explain this stuff to me. Ideally I want to get enough of the fundamentals under my belt here that I get to the point where I can do more of the learning on my own. It's more fun once I get to the point where I can do projects on my own instead of coming here and basically asking somoene to do it for me. I don't like being that guy so I want to get beyond that level of understanding sooner rather than later, but I know I can't get there solely on my own.

[kjeron]

INT vs. HEX : doesn't matter which you use as a number, since weidu will convert it to an integer anyway.

4 hours ago, CidBahamut said:

You specify the READ_SHORT offset using the format 0x1C rather than 1C. Is there a meaningful distinction between those two formats and if so when should I be making use of one over the other?

1C / 1c are two different variables to Weidu, with no value unless you've given them one.

0x1c is a hex value to Weidu. (case doesn't matter)

0x... defines a hex value, 0b... defines a binary value, 0o... defines an octal value.

3 hours ago, CidBahamut said:

I could also use a rundown on INT_VAR from somoene who's more knowledgeable than I am.

INT_VAR and STR_VAR simply tell the function which variables are to be passed as integers and which are to be passed as strings, as they are processed differently by the function call. There is no assumed default, so they will always be present when variables are passed within a function call.

3 hours ago, CidBahamut said:

Oh one more thing before I forget. When patching an item, if I wanted to display the name of the .itm file, is there a way to do that? It doesn't look like something you can pull by reading an offset within the file itself so I'm wondering if there's another way to do that. Mostly thinking about this for debug purposes.

READ_STRREF offset variable
PATCH_PRINT ~%variable%~

IDed name of items/spells is at 0xc, UnIDed name is at 0x8.

[jmerry]

On reading the name: yes, the item's name isn't directly in the file. Instead, there's a "string reference" number there, which refers back to the master list of strings in dialog.tlk (a file you should never edit directly, as it's far too easy to break all of the text in the game). READ_STRREF reads that number and looks up the string that corresponds to it, saving that string to the output variable so you can do stuff with it.

5 hours ago, CidBahamut said:

Lastly, you guys define both parameter1 and parameter2. Now in context I can piece together that those correspond to the "Value" and "Modifier type" attributes of the effect as displayed in Near Infinity, but on my own I don't know if I'd have been able to piece that together even whilst staring at the IESDP entry for the .itm file format.

The IESDP entry you should be looking at here is the entry for the opcode. Here's the entry for opcode 86:

Quote

#86 (0x56) Stat: Slashing Resistance Modifier Variants: IWD2
Parameter #1: Statistic Modifier
Parameter #2: Type

Description:

Applies the modifier value specified by the Statistic Modifier field in the style specified by the Type field.
Known values for Type are:

• 0 ⟶ Cumulative Modifier: Resistance = Resistance + 'Statistic Modifier' value
• 1 ⟶ Flat Value Modifier: Resistance = 'Statistic Modifier' value
• 2 ⟶ Percentage Modifier: Resistance = (Resistance * 'Statistic Modifier' value) / 100

Statistic Modifier can be positive or negative.

The effect modifies RESISTSLASHING.

So for this opcode, parameter 1 is the quantity of resistance and parameter 2 is how that quantity is applied.

All of the opcodes have the same parameter fields to work with, but they use those fields differently.

[CidBahamut]

52 minutes ago, kjeron said:

IDed name of items/spells is at 0xc, UnIDed name is at 0x8.

That's more or less the workaround I had puzzled out on my own, but I wanted to know if it was possible to get the actual file name. There's a couple edge cases I can think of where distinct items have identical names but are governed by different files. Might be I'm just trying to over-engineer things prematurely.

 

12 minutes ago, jmerry said:

All of the opcodes have the same parameter fields to work with, but they use those fields differently.

Ah ok. That helps sort things.

I cannot overstate how much I appreciate you guys putting up with my dumb queries. I've been playing this game since launch, so learning how to do all of this is long overdue on my part.

[jmerry]

Ah. File name? When you have a file loaded, that's SOURCE_RES (without the extension) or SOURCE_FILE (with the extension) Or maybe the DEST_ versions if you're changing the name. Relevant passage from the documentation:

Quote

 

COPY commands set the user-defined SOURCE_DIRECTORY, SOURCE_FILESPEC, SOURCE_FILE, SOURCE_RES, SOURCE_EXT, DEST_DIRECTORY, DEST_FILESPEC, DEST_FILE,DEST_RES and DEST_EXT variables based on fromFile and toFile as follows. If fromFile is mymod/cre/bigboss.cre, then SOURCE_DIRECTORY is mymod/cre,SOURCE_FILESPEC is mymod/cre/bigboss.cre, SOURCE_FILE is bigboss.cre, SOURCE_RES is bigboss and SOURCE_EXT is cre. The DEST_ variables are similarly based on toFile. In addition, SOURCE_SIZE is set to the size (in bytes) of the source file.

Very useful stuff when you're opening a bunch of different files with the same command.

All of these are variables that you can alter. But most of the time, you shouldn't. That is a path to great confusion.

So, on that note, here's code that makes a list of shields.

ACTION_CLEAR_ARRAY ShieldList // Clears any previous instance of the array. Not really needed here, but ...
OUTER_SET index = 0 // Just a counter variable.

COPY_EXISTING_REGEXP GLOB ~^.+\.ITM~ ~override~ // Loops through all items, copying them.
    PATCH_IF (SOURCE_SIZE > 0x71) BEGIN // Guards against malformed files that are too small.
        READ_SHORT 0x1c category // Reads the two bytes at offset 1c, which are the item category.
        PATCH_IF (category = 12) BEGIN // Shields. PATCH version of IF because we have a file loaded.        
            SPRINT $ShieldList(~%index%~) ~%SOURCE_RES%~ // Add this filename to the list.
            SET index += 1 // Increment the counter.
        END // Done with inner IF block.
    END // Done with outer IF block.
BUT_ONLY // We didn't make any changes. Don't save the copies.

ACTION_PHP_EACH ShieldList AS index => filename BEGIN // Loop through the list
    // Do stuff with the entries in the list. Not detailed here.
END

I should also note that my code basically always assumes the AUTO_EVAL_STRINGS flag in the preamble; there may be some instances of EVAL needed without it.

Edited October 7 by jmerry