projectinfinity Dynamic Install Order - providing enforced install order rules for you mods

[AL|EN]

Let's try an alternative look at how modders/maintainers can take care of the install order:

Dynamic Install Order

It's a rule-based order of the mods that use BEFORE/AFTER 'rules' in order to determine the relationship between source mods and destination mods. Generally, it is a recreation of the install order recommendations from mods 'ReadMe' files, but in 'machine-readable' format, which the mod manager can use, apply, and react to when rules are not fulfilled.

Goals:

• you care only about install order/dependencies of your own mods or mods that you maintain
• sufficient for low to medium installation, for huge mega-installation of 100+  mods, a big install order list will be a better choice

Non-goals:

• to handle every possible type of future mods
• a perfect install order for an unlimited amount of mods
• replacement for using an install order list for a huge amount of mods

Features:

• rules cannot be disabled by players, you can be sure that your mods will be installed using the provided install order
• using mod ID as tp2 filename without extension and 'setup-' prefix
• everything is evaluated before installation even starts so the player won't ever face 'you must install x before y' after hours-long installation
• allows only to set rules for your own mods relating to other mods, it doesn't allow you to alter install order rules of other mods
• all rules are loaded dynamically, according to whenever the mod is selected for installation and the mod to which the rule applies is also selected
• all rules are evaluated dynamically, according to whenever the mod is selected for installation and the mod to which the rule applies is also selected
• the installation will be possible only when all rules for all mods will be fulfilled
• works globally, which means it will check mods already installed, and currently being installed and it also 'will be fired' when someone installs mods later

Covered use cases:

The idea for this topic is regarding install order/installation dependencies only, not mod conflicts. So in order to clarify the scope of this discussion let's limit it only to these cases:

Assuming 'ModA' is the source of the rules aka 'if I'm editing metadata of ModA.ini, the rules are for other mods related to it:

Implemented:

1. BEFORE = ModB
   Mod A should be installed before Mod B, this is only a single 'rule/requirement', which doesn't mean that ModA requires ModB to be installed.
    
2. AFTER = ModC
   Project Infinity feature status: Implemented

   Mod A should be installed after ModC, this is only a single 'rule/requirement', but it doesn't mean that ModA requires ModC to be installed.

   Example:
   AFTER = BobShop, Caravan, TeamBG-Armors
   The mod will add new items into every game shop regardless of whether BobShop, Caravan, or TeamBG-Armors mods are installed or not.

Not yet implemented: please don't use those until there is a working implementation as things might be changed

Spoiler

1. REQUIRE-EARLIER = Y

   Mod A require Mod Y to be installed earlier in order for Mod A to work. Those are actually two rules: "REQUIRE Y" and "BEFORE Y". Due to how IE/WeiDU mods works, both rules are implied. Players however can interpret this only as 'two mods must be installed, that's it, I fulfill the requirements which is not sufficient.

   Example:
   "SCS: Smarter Abazigal" require "Ascension: Tougher Abazigal"

2. REQUIRE-LATER = Y

   Mod A require Mod Y to be installed later in order for Mod A to work. Again two rules: "REQUIRE Y" and "AFTER Y", both are implied. 

   Example:
   The Drizzt Saga Main component > Worldmap variant
   Installing this component means that player is required to install Worldmap later. The Worldmap MUST be installed after. 

Example of rules definition for Deities Of Faerun mod:

Project Infinity feature status: Implemented

• mod ID as tp2 filename without extension and 'setup-' prefix
• ID's are separated by commas

DeitiesOfFaerun.ini

[Metadata]
Name = Deities Of Faerun

BEFORE = IHateUndead, MyMod

AFTER = AllThingsMazzy, OtherMod

 

Example of defining install order rules directly inside mod components by using WeiDU 'METADATA' keyword in order to provide data for PI's Dynamic Install Order feature:

Project Infinity feature status: Implemented

• mod ID as tp2 filename without extension and 'setup-' prefix
• ID's are separated by commas
• the format stays the same as for global rules with the addition of the usual WeiDU keyword syntax
• you can't put BEFORE/AFTER into a single METADATA occurrence
• you can have multiple METADATA keyword statement
• you can put global rules inside ini and a different one directly inside components - component local rules will override global rules from global ini

LocalRules.tp2

BEGIN "Local Rules Main" LABEL "LocalRules-Main"
METADATA "AFTER = Ascension, DSotSC"
METADATA "BEFORE = X, Y"

BEGIN "Local Rules Extra" LABEL "LocalRules-Extra"
METADATA "AFTER = C, D"
METADATA "BEFORE = FaithsAndPowers, TomeAndBlood"

Visual representation:

 

If you have any questions, or feedback regarding details of the above system, feel free to ask.

Edited October 4, 2023 by AL|EN
added clarification regarding scope, cosmetics

[Raduziel]

Well, you know I'm all for it. I think this idea is awesome as I always seeing new players struggling with this subject and not rarely us modders keep scratching our heads to solve a bug that is purely related to a bad install order.

I would add another command, though: ForbidInstall (or something like that), as some mods are just impossible to be used at the same time; this one should be able to relate to another mod's component only if necessary. For instance,  DoF can't be used with some components from Tweaks Anthology but many others (most of them) are fine.

Good initiative, Alien.

Edited March 7, 2020 by Raduziel

[grodrigues]

I would suggest a slightly different implementation: move the data to a .2da file with columns mod_name, component_number, install_order. The first two are self-explanatory, the last takes values 0 or 1, with 0 meaning must be installed (when the mod is installed) and 1 must not be installed (when the mod is installed).

pros:

- weidu can read it and it is relatively easy to weidu-code to force these requirements.

cons:

- mod names cannot have spaces. This could be solved with an extra layer of indirection (e.g. put a tra reference instead).

- do not know how PI could handle, but I am assuming it should not be too hard.

I have already implemented something like this to use on my mods; the idea is to shift as much as possible from code to data.

note(s): a further con:

- as far as Raduziel's idea, the problem is that the requirement is a global one, not a local one, meaning it is not enough to make the check at mod install time, but rather the check must be done every time a mod is installed after the mod with such a requirement. This implies some kind of mod coordination, which while a tool like PI can probably handle, weidu has more difficulty as it would need global mod coordination.

[subtledoctor]

AFAIK Weidu cannot read a .2da file as part of a REQUIRE_PREDICATE operation.  It would be amazing if it could, because then this idea could work and every mod could simply append a line (or a number of lines with relative install order prescriptions) to a common .2da file.

I agree that there should be the reverse of the current FORBID_COMPONENT, something that actually forbids a future component (the current one should really be called "FORBIDDEN_BY_COMPONENT").  I've asked Wisp for this but I don't think it's forthcoming.  Realistically, the way to do it might be to have FORBID_FUTURE_COMPONENT just add a special marker file, and have Weidu skip components when it detects their special marker files.

But, I'm not holding my breath. 

[Leeux]

 

4 hours ago, subtledoctor said:

Realistically, the way to do it might be to have FORBID_FUTURE_COMPONENT just add a special marker file, and have Weidu skip components when it detects their special marker files.

But, isn't that a bit heavy handed? As long as mods use it for what it's meant (i.e. absolute conflicts like, editing some NPC dialogue in ways that can't be reconciled, f.e.) there should be no problem, but I can envision situations where a mod can forbid to install, I don't know, let's say, some Tweaks Anthology components because they just break the "balance" they intend to be.  I.e. it'd be a bit open to over-use, IMO. 

On topic for the thread:  @ AL|EN, that's looking awesome! Could it be extended so that the rules have messages attached, that can be used as to why that rules is in place? I.e. a reason why a mod has to install after or before such a mod... It's just something cosmetic ofc. but IMO gives more context to the player that could be using the tool to understand the thinking behind the enforcement of the rules.

Of course, they should be optional and not required... but in cases where the reason is known, it'd be great to have information associated with the rules. It'd also help to keep the rules from getting outdated... i.e. if someone says, mod X needs to install after mod Y because "tweaks stores and mod Y adds a new store to the game", for example. Maybe at some point mod Y changes and it has no store anymore, and thus a player getting that message could then prompt the author of mods X or Y, or whomever is in charge of maintaining the rules repository, to tweak the rule. 

 

Edited March 7, 2020 by Leeux
clarification + separated paragraph

[subtledoctor]

1 hour ago, Leeux said:

But, isn't that a bit heavy handed? As long as mods use it for what it's meant (i.e. absolute conflicts like, editing some NPC dialogue in ways that can't be reconciled, f.e.) there should be no problem, but I can envision situations where a mod can forbid to install, I don't know, let's say, some Tweaks Anthology components because they just break the "balance" they intend to be.  I.e. it'd be a bit open to over-use, IMO.

I mean, if a mod abuses it then the mod won't have a great reputation.  People don't have to use it.  That's on that modder.  (And of course if it prevents a combination that people really want to use, people can simply edit the mod to allow things to proceed.

It's really born of a practical matter: as an example, look at the Tome & Blood "bonus spell slots for high INT/CHA" component.  That has to alter the wizard/bard/sorcerer spell tables, in order for the slot calculation to work right.  If someone comes in later and installs the CDTweaks component that overwrites spell tables, everything gets screwy.  FORBID_COMPONENT can only work on mods already installed, so the later mod must be the one to control the conflict.  But in this case, the later mod (CDTweaks) is not actively maintained and has not and maybe will not ever add that predicate.  So the only way to prevent the conflict is for players to be totally on top of all the fine print.  That's not the best solution.  Better to give modders a tool to address the issue.  Tools can always be misused, but on balance we are better off with them.

[grodrigues]

On 3/7/2020 at 4:36 PM, subtledoctor said:

AFAIK Weidu cannot read a .2da file as part of a REQUIRE_PREDICATE operation.

I know -- don't use REQUIRE_PREDICATE. Instead move everything to a (action) function that checks requirements and FAIL's if they are not met. So basically, it would mean moving from such commands as REQUIRE_PREDICATE or REQUIRE_COMPONENT to using MOD_IS_INSTALLED checks inside the function that checks requirements.

 

And the system can of course be improved, e.g. with per component restrictions, or with special messages (e.g.an extra column with tra references).

[AL|EN]

On 3/7/2020 at 2:53 AM, Raduziel said:

I would add another command, though: ForbidInstall (or something like that), as some mods are just impossible to be used at the same time; this one should be able to relate to another mod's component only if necessary. For instance,  DoF can't be used with some components from Tweaks Anthology but many others (most of them) are fine.

@Raduziel
What you are describing is actually a global mod conflicts. This functionality is next in line. Implementation hover is like opening Pandora's Box.

@grodrigues and @subtledoctor 

My idea for this topic was regarding install order/installation dependencies only, not mod conflicts. So in order to clarify scope of this discussion let's limit it only to this cases:

Spoiler

 

1. BEFORE = B
   Mod A should be installed before Mod B, this is only single 'rule/requirement', it doesn't mean that Mod A require Mod B to be present.
    
2. AFTER = B

    Mod A should be installed after Mod B, this is only single 'rule/requirement', it doesn't mean that Mod A require Mod B to be present.

   Example: 

   
   Mod description: This mod adds new items into every game shop.
   AFTER = BobShop, Caravan, TeamBG-Armors
   The mod will add new items into the every game shop regardless whenever BobShop, Caravan, TeamBG-Armors mods are installed or not.

    

3. REQUIRE FORMER/Pre-Install = B

    Mod A require Mod B to be installed before in order for Mod A to work. This is actually two rules: DependOn B and BEFORE B. Due to how IE/WeiDU mods works, both rules are implied. Gamers however can interpret this only as 'two mods must be installed, that's it, I fulfill the requirements' which is not sufficient.
   
   Example: 

   
   "SCS: Smarter Abazigal" require "Ascension: Tougher Abazigal"

    

4. REQUIRE LATER/Post-Install = B
   Mod A require Mod B to be installed after in order for Mod A to work. Again two rules: DependOn B and AFTER  B, both are implied. 
   
   Example:

   
   The Drizzt Saga Main component > Worldmap variant
   Installing this component means that player is required to install Worldmap later. The Worldmap MUST be installed after. 

 

I've added this into first post, the discussion regarding mod conflicts will have to wait until I will have something ready.

Edited March 9, 2020 by AL|EN

[subtledoctor]

I've resisted responding directly to the first post because I don't want to be a Debbie Downer about it.  But my first instinct is, while lovely in theory, the idea of each mod specifying its own relative install order vs. other mods will either 1) not give the installer enough information to go on, or 2) result in logical impossibilities that cause the world's computer systems to asplode.  A good way to fight Skynet* but may not work for BGEE installs.

Okay, well, it probably won't destroy any computers, but it might easily lead to the situation from the bad old days when the BW Guide said

• Mod A should go before Mod B
• Mod B should go before Mod C
• Mod C should go before Mod A

One overall big prescribed install order seems better to me.  Make it open-source and editable by anyone, so anytime a conflict or imperfection arises the solution will be crowd-sourced.  That seems good enough.

Of course I may be worrying for nothing - it may be fine.  Hard to say from this early vantage point.

 

* or "Legion?" wut?

[AL|EN]

@subtledoctor This needs further clarification, apologies to not including this before:

The system is not mean to replace 'big prescribed install order list'. Both solutions have strength and weakness. It has limited scope: you care only about relationship of you own mods or mods which you maintain. You don't care about all current mods and possible new ones because it's impossible. This allows for defining for eg: NPC + Kit relationship, cross-mod banter-packs and other requirements which are already established inside mods readme.

Regarding not give the installer enough information to go on:
Again, with limited scope, the information which you provide should be pretty much sufficient. An example for faiths_and_powers:

BEFORE = might_and_guile, cdtweaks, refinements, scales_of_balance

Ask yourself how many times people asking about 'how to install faiths_and_powers might_and_guile and scales_of_balance?". Even such simple rule is very valuable for defining install order.

Regarding logical impossibilities:
They exist. They are literal truth. If you present them as rules, gamer atleast can decide which mod he should skip. Or notify both modders to find solution. When you using install order list, this information is lost. I think that it's better for the player to skip one mod instead of ignoring install rules and see what happen after 2 weeks of playing. 

2 hours ago, subtledoctor said:

One overall big prescribed install order seems better to me.  Make it open-source and editable by anyone, so anytime a conflict or imperfection arises the solution will be crowd-sourced.  That seems good enough.

I did. No one show the will to contribute. That's why I'm looking for alternative because currently, there is nothing except mods readme and you own list.

[subtledoctor]

4 minutes ago, AL|EN said:

Again, with limited scope, the information which you provide should be pretty much sufficient. An example for faiths_and_powers:

BEFORE = might_and_guile, cdtweaks, refinements, scales_of_balance

Ask yourself how many times people asking about 'how to install faiths_and_powers might_and_guile and scales_of_balance?". Even such simple rule is very valuable for defining install order.

My concern is, what if a player installs Ascension, Faiths & Powers, Song & Silence, and Dual to Kit?  None of them might mention each other, so there is no guidance... but there is definitely a proper install order for them. 

If the installer uses a static list as baseline - or, perhaps, defaults to a category-based order - and then checks individualized relative instructions, that would be fine.  It would just be more work for you (or whoever else is creating/administering this hypothetical system).

[AL|EN]

@subtledoctor The ability for mods to add themselves into install order group-based (categories can confuse people) can be done without problem as part of this system. But I'm not gonna implement anything which require 24/7 maintenance. So the data needs to be filled by modders/maintainers, regardless if it is a rule definition + placing the mod into install order group or putting the mod into online install order list.

Regarding you example: sure, that might be true for now. The readme of Song & Silence doesn't mention about those mods. Nor there is any other source of the correct install order for those mods, except you list. So let's change that:

- please post all reasons why each of those mods needs to be installed BEFORE/AFTER

- I can then add it to Ascension, you can add it to Faiths & Powers

- CamDawg can add it to Song & Silence

- kjeron can add it to Dual to Kit

Profit?

 

[subtledoctor]

11 minutes ago, AL|EN said:

@subtledoctor The ability for mods to add themselves into install order group-based (categories can confuse people) can be done without problem as part of this system. But I'm not gonna implement anything which require 24/7 maintenance. So the data needs to be filled by modders/maintainers, regardless if it is a rule definition + placing the mod into install order group or putting the mod into online install order list.

Regarding you example: sure, that might be true for now. The readme of Song & Silence doesn't mention about those mods. Nor there is any other source of the correct install order for those mods, except you list. So let's change that:

- please post all reasons why each of those mods needs to be installed BEFORE/AFTER

- I can then add it to Ascension, you can add it to Faiths & Powers

- CamDawg can add it to Song & Silence

- kjeron can add it to Dual to Kit

Profit?

Seems like too much work.  I would have to add hundreds of such rules to every component in my mods (and there are probably over 100 such components, so... > 10,000 rules to add?  Not gonna happen anytime soon.

[AL|EN]

@subtledoctor You don't add hundreds of rules into each component. You add it globally for mod, same as when you add a mod to install order list. And you limit relationship to the crucial ones.

[Magus]

I don't see what's dynamic about it. It's just the same static rules, only in machine readable format.

But assuming it's not too taxing to maintain, it of course could be useful.