label [Tutorial] How to properly handle mod component dependencies

[AL|EN]

Introduction:

The current way of handling mod dependences is relying on DESIGNATED numbers. Due to how DESIGNATED numbers are inseparably connected to the mod components structure and order, it's inevitable that sooner or later those numbers will be changed and all dependencies will have to be updated. And when they do, everything breaks.

Instead of using REQUIRE_COMPONENT, FORBBIT_COMPONENT with DESIGNATED numbers, we will use mod LABEL with combination of MOD_IS_INSTALLED and ID_OF_LABEL as an improved way that won't rely on DESIGNATED numbers and it won't prevent modders from changing them.

Please remember that it's not only about improving your mods, it's also about giving a good example for new modders. The common practice of new people is reading other mod's code to figure out how to do things instead of reading WeiDU documentation.

Q: What is LABEL and why I should add LABEL's to my mods?
A: LABEL is a better DESIGNATED. It allows assigning a unique textual identifier for a given mod component. And use this identifier to handle internal dependencies between the components of the same mod and external dependencies of the mods from different authors. By using globally unique LABEL's locally, you are saving yourself the trouble of fixing dependences after DESIGNATED changes. By using external mods LABEL's for defining dependencies, you avoid 'locking' DESIGNATED numbers changes of those mods so the author can freely change them when he needs to.

Q: I would like to add LABEL's for all my mods so other modders can use it. How to create proper LABEL?
A: Read this guide: What is LABEL, why you should create it and how to do it properly

 

How it was handled before:

1. mod provide DESIGNATED numbers assigned to it's components
2. other mods and tools create dependences based on those numbers
3. after a mod update, the DESIGNATED numbers had to be changed and with it, all mod dependencies need to be updated

How it is handled now:

1. mod provide globally unique LABEL's assigned to it's components
2. other mods and tools create dependences based on those LABEL's
3. even if DESIGNATED numbers will be changed due to the evolution of the mod, their LABEL's won't so all mod dependencies will still be correct

Advantages:

• using the new way will allow for the dependences to no longer rely on DESIGNATED numbers and survive when they change
• avoid 'locking' of external mods DESIGNATED numbers changes, author will be able to freely change them when they need

  
Prerequisites:

• the mod that we want to add dependencies must have LABEL's added to it's components

Example:
Let's consider a NPC mod that has 3 components:

• the component that add NPC with all it's content into the game
• the component that allows to change NPC portrait, this component requires main component to be installed
• the component that allow assigning a special kit from an external mod, this component requires an external mod main component to be installed

NPC.tp2:

 BACKUP "weidu_external\backup\NPC" // move backup folder outside of the main mod data folder
SUPPORT "forum.com/thread-1"
VERSION "0.1.0"

AUTO_EVAL_STRINGS

BEGIN "NPC Main Component" LABEL "NPC-Main" DESIGNATED 1000

    OUTER_SET ID-NPC-Main = ID_OF_LABEL "NPC.tp2" "NPC-Main"
    PRINT "ID of the NPC-Main LABEL: %ID-NPC-Main%"

// Example of local dependency
BEGIN "NPC Portrait Component" LABEL "NPC-Portrait" DESIGNATED 2000
    REQUIRE_PREDICATE MOD_IS_INSTALLED "NPC.tp2" ( ID_OF_LABEL "NPC.tp2" "NPC-Main" ) "This component requires main component."

    OUTER_SET ID-NPC-Main = ID_OF_LABEL "NPC.tp2" "NPC-Main"
    PRINT "ID of the internal NPC-Main LABEL: %ID-NPC-Main%"

    OUTER_SET ID-NPC-Portrait = ID_OF_LABEL "NPC.tp2" "NPC-Portrait"
    PRINT "ID of the internal NPC-Main Portrait: %ID-NPC-Portrait%"

// Example of external dependency
BEGIN "NPC Kit Component" LABEL "NPC-Kit" DESIGNATED 3000
    REQUIRE_PREDICATE MOD_IS_INSTALLED "Kit.tp2" ( ID_OF_LABEL "Kit.tp2" "Kit-Main" ) "This component requires the main component from the external Kit mod."

    OUTER_SET ID-Kit-Main = ID_OF_LABEL "Kit.tp2" "Kit-Main"
    PRINT "ID of the external Kit-Main LABEL from Kit mod: %ID-Kit-Main%"

Kit.tp2 (for reference, must be installed before NPC.tp2):

 BACKUP "weidu_external\backup\Kit" // move backup folder outside of the main mod data folder, change only your mod data folder name
SUPPORT "forum.com/thread-1"
VERSION "0.1.0"

AUTO_EVAL_STRINGS

BEGIN "Kit Main Component" LABEL "Kit-Main" DESIGNATED 9999
    OUTER_SET ID-Kit-Main = ID_OF_LABEL "Kit.tp2" "Kit-Main"
    PRINT "ID of the internal Kit-Main LABEL: %ID-Kit-Main%"

Possible WeiDU improvements:

• remove the requirement of having '.tp2' for the ID_OF_LABEL tp2 name parameter (for now, you can use a local variable with full tp2 name)
• add a possibility to use ID_OF_LABEL for REQUIRE_COMPONENT and FORBBIT_COMPONENT so the syntax can be simplified

Edited March 9, 2021 by AL|EN

[Incrementis]

Hi,

I have some general and perhaps trivial questions as I don't have enough experience with components and/or external components of external mods.

Before I start the questions, I would like to thank you for the code examples as they helped me understand the concept a little better. The many links and sublinks to content, while helpful, are confusing in scope and take too much time to understand the simple beginner questions I have in my head.

1. Should every mod use LABELS? (Example: I have an itemset mod with no subcomponents.)
2. `

   BEGIN "NPC Main Component" LABEL "NPC-Main" DESIGNATED 1000

   ` Is DESIGNATED <number> needed or can I leave this part out?
3. If “DESIGNATED” is not needed, what is the reason to keep it?

If you can, please answer these questions in simple words, otherwise I may not understand them, and I cannot use things that I do not understand.

[AL|EN]

1. Yes. It serves both WeiDU and PI.

2. DESIGNATED should be added regardless of LABEL. My suggestion is to use 1000 for the first component, 2000 for the second etc so in case when you would like to add a new component between two already existing ones, you will have room for it aka 1100, 1200 etc

3. DESIGNATED is needed to keep WeiDU backup/reinstall/uninstall system happy. 

[Incrementis]

23 hours ago, AL|EN said:

3. DESIGNATED is needed to keep WeiDU backup/reinstall/uninstall system happy. 

Thank you for your short explanation. It makes things much clearer and understandable.

So far it's convinced me to just use LABEL, but there's still something I don't fully understand.

As far as I know, LABEL is the preferred method, because You wrote:

“LABEL is a better DESIGNATED…”

This leads me to believe that DESIGNATED is not necessary for completely new mods.

Could you therefore please go into more detail about the quoted point 3.(Maybe with a short and easy to understand example)?

[CamDawg]

DESIGNATED is important and should still be used in conjunction--WeiDU still uses component numbers for command-line installs as do some other tools. I'd recommend new mods use DESIGNATED from the start.

The important thing with DESIGNATED is that its entire purpose is to lock a component to a number, so changing the DESIGNATED number of something undermines the entire purpose. IOW once you set a component a DESIGNATED value, don't change it. Like, ever.

[Incrementis]

Thank you, as thinks get clearer with every new comment.

1 hour ago, CamDawg said:

IOW once you set a component a DESIGNATED value, don't change it. Like, ever.

But your quoted sentence is no longer a problem if LABEL is used correctly in combination with DESIGNATED? At least that's what I understand from what AL|EN wrote:

"By using globally unique LABEL's locally, you are saving yourself the trouble of fixing dependences after DESIGNATED changes. By using external mods LABEL's for defining dependencies, you avoid 'locking' DESIGNATED numbers changes of those mods so the author can freely change them when he needs to."

I apologize if you answer will be a simple "yes", but I need such clarification as it will take away all my uncertainties on this matter(I also believe this will increase the value of this tutorial).

Edited March 2 by Incrementis
typo

[CamDawg]

The claim is that it frees us from the tyranny of fixing a DESIGNATED value, which is nonsensical, as that's the whole point of DESIGNATED. It's like arguing that by using DESIGNATED it frees us to change LABELs, an equally nonsensical proposal.

DESIGNATED was created so that mods with multiple components (e.g. Tweaks) could present the components in a sensible order in the installer without causing massive compatibility headaches down the line. Prior to DESIGNATED, WeiDU simply numbered components itself starting with 0, so if you wanted to add components they had to be at the end of existing ones. Tweaks is barely manageable as it is; I want you to imagine the horrific mess it would be if it was organized in a purely chronological order of the components by creation date. With fixed DESIGNATED values I can freely add components where I think they fit in the order, move them around, or do whatever without making David and other modders having to constantly shift around their compatibility checks.

AL|EN's doing a little not-so-subtle lobbying here: no one is using LABELs to check against other components--at least not yet, and this paradigm shift was proposed three years ago. Keeping your component numbers DESIGNATED is more important, and I don't foresee that changing any time soon.

This isn't to say that LABELs aren't useful: they very much are, and you should use them, and they'll become even more useful is we ever get a new version of WeiDU with the proposed changes to LABEL handling. I'm simply clarifying that DESIGNATED is still relevant and not to be discarded, and that changing a DESIGNATED value is exceptionally bad practice.

[suy]

I would go even further. I wish the labels could replace the component numbers entirely (e.g. by writing in the weidu.log file #{label-name} instead of #XYZ, or some other extension to the file format that would not break too much), but in the current situation, I'm just not interested in adding labels. If I could install components by name instead of number (by calling weidu --force install label-name instead of a number), that's something that I can easily test, and which is easy to me. As is right now, I can't test labels, and serve no use to me, so I'm sticking to DESIGNATED without labels for now.

[AL|EN]

On 3/3/2024 at 1:14 PM, suy said:

I would go even further. I wish the labels could replace the component numbers entirely (e.g. by writing in the weidu.log file #{label-name} instead of #XYZ, or some other extension to the file format that would not break too much), but in the current situation, I'm just not interested in adding labels. If I could install components by name instead of number (by calling weidu --force install label-name instead of a number), that's something that I can easily test, and which is easy to me. As is right now, I can't test labels, and serve no use to me, so I'm sticking to DESIGNATED without labels for now.

This is what PI Install Sequence already does. Some missing pieces are still left for me to add, but it's a good start. I do not hope this will end up in weidu (tip: you need to cover one more feature of DESIGNATED to be a successful replacement) but I would be happy to be proven wrong.

As for testing labels: sure you can, use --list-components-json, parse it, and make a general shell script that can launch weidu installation like this "Install-WeiDUMod -Label Ascension-RewrittenFinalChapterOfToB" etc. Also adding labels for your mods are mostly for other modders to be able to use them in the first place but ofc it can be used to cover local component dependences.

[AL|EN]

On 3/2/2024 at 7:32 PM, Incrementis said:

Thank you for your short explanation. It makes things much clearer and understandable.

So far it's convinced me to just use LABEL, but there's still something I don't fully understand.

As far as I know, LABEL is the preferred method, because You wrote:

“LABEL is a better DESIGNATED…”

This leads me to believe that DESIGNATED is not necessary for completely new mods.

Could you therefore please go into more detail about the quoted point 3.(Maybe with a short and easy to understand example)?

Yes, DESIGNATED is not necessary for completely new mods if you are providing LABEL's all components. Having to not provide DESIGNATED sends a clear message to the modders: "When you want to check if one of my mods/components is installed, use LABELS.". It's safer that way.

 

[DavidW]

2 minutes ago, AL|EN said:

Yes, DESIGNATED is not necessary for completely new mods if you are providing LABEL's all components. Having to not provide DESIGNATED sends a clear message to the modders: "When you want to check if one of my mods/components is installed, use LABELS.". It's safer that way.

I recommend against this. I agree with Cam that you should use DESIGNATED whether or not you are using labels.

[DavidW]

On 3/2/2024 at 3:38 PM, CamDawg said:

AL|EN's doing a little not-so-subtle lobbying here: no one is using LABELs to check against other components--at least not yet, and this paradigm shift was proposed three years ago.

Speaking personally, I'm not using it because adding LABELs to everything still leads to painful slowdowns in mods with very many components.

[suy]

24 minutes ago, AL|EN said:

Yes, DESIGNATED is not necessary for completely new mods if you are providing LABEL's all components. Having to not provide DESIGNATED sends a clear message to the modders: "When you want to check if one of my mods/components is installed, use LABELS.". It's safer that way.

 

You are also sending a clear message to users: you care less about them.

Don't get me wrong. Your previous message made me consider making a small program that wraps weidu and can accept installing something by label name (I'd make it cross platform, open source, and hopefully easy to use, etc., so it can spread usage of LABEL for more people, not just me). But given that weidu.log only stores component numbers, not using designated component numbers at all, and given David's findings on the bad performance of LABEL, this all points to a promising future for the labels, maybe, but a pretty limited present. This needs work on weidu's side. Note also that the JSON output is marked as experimental, so it's another problem to spread its usage.

[AL|EN]

6 hours ago, DavidW said:

Speaking personally, I'm not using it because adding LABELs to everything still leads to painful slowdowns in mods with very many components.

I'm not saying that you are wrong on this but my tests aren't confirming this. Here are my results when all components are installed (except 100):

| 'ToF Beta 7 tp2: | 'ToF Beta 7 tp2 with LABELS: |
|------------------|------------------------------|
| 18,06 minutes    | 18,32 minutes                |

| 'SCS 35.10 default tp2: | 'SCS 35.10 tp2 with LABELS: |
|-------------------------|-----------------------------|
| 22,44 minutes           | 25,15 minutes               |

Testing procedure, repeat for default tp2 and tp2 with added LABELS:

1. Clean BG2EE, clean extraction of SCS/ToF, don't forget to uninstall SCS if you are testing ToF!
2. Open PowerShell, change the directory to the game dir and paste the code below:

• SCS:

  • Measure-Command -Expression {
      & '.\setup-stratagems.exe' --no-exit-pause --noautoupdate --language 0 --force-install-list 1500 1510 1520 1600 1610 2000 2010 2020 2030 2040 2050 2060 2070 2080 2500 2510 2520 2900 3010 3015 3017 3020 3021 3022 3040 3041 3500 3501 3505 3510 3540 3541 3550 3551 3552 3580 4000 4020 4030 4050 4051 4052 4093 4099 4100 4115 4130 4135 4140 4145 4146 4150 4160 4161 4162 4163 4164 4170 4171 4172 4173 4174 4190 4210 4215 4216 4217 4218 4230 4240 4250 5000 5070 5080 5900 6000 6010 6020 6030 6040 6100 6200 6300 6310 6320 6500 6510 6520 6540 6550 6560 6570 6580 6590 6700 6800 6810 6820 6830 6840 6850 7000 7010 7020 7030 7040 7050 7060 7070 7080 7090 7100 7110 7130 7140 7150 7200 7210 7220 7230 7250 7900 8000 8010 8020 8040 8050 8060 8070 8080 8085 8090 8100 8110 8120 8130 8140 8150 8160 8170 8180 8190
    }

• ToF:

  • Measure-Command -Expression {
        & ".\setup-dw_talents.exe" --no-exit-pause --noautoupdate --language 0 --force-install-list 1500 1510 1520 1600 1610 2000 2010 2020 2030 2040 2050 2060 2070 2080 2500 2510 2520 20000 40000 40100 40200 40300 40310 40400 40450 40460 40470 40500 40600 40610 40650 40660 40700 40750 40751 40752 40753 40800 40850 40900 40925 40950 41000 50000 50100 50200 50300 50400 50500 55000 55100 55200 55300 55400 55500 55600 55700 55800 55900 60100 60200 60300 80000 80001 80100 80101 80102 80103 80104 80150 80500 81000 81010 81011 81012 81020 81030 81100 90000 90100
    }

I'm more than happy to run your tests and If the situation is that bad then I will make some noise about it.

Edited March 6 by AL|EN
Formatting

[DavidW]

Those numbers look broadly compatible with my own observations: a slowdown of about 30-60 seconds at the very start of running the mod, and subsequent slowdowns at the start of every component. The 2.5 minute slowdown for SCS is if anything worse than I recall (but then the architecture of SCS has changed a bit since then).

That might be tolerable for end users (though a 10% slowdown for SCS is not nothing in a mod which already takes far too long to install) but they are intolerable for development, in the quite literal sense that I put LABELs in some years ago, couldn't tolerate the slowdowns, and took them out again.