Errors When Using Weidu to Automate Installation

[Bartimaeus]

Basically, what I want to know is if weidu spitting out approximately ten billion "possible missing evaluate_buffer" errors is actually a cause for concern when doing a mass component install.

Background: Okay, so say I have a weidu.log from an old install. Say, for simplicity's sake, it's simply two mods. weidu.log:

Spoiler

~TOBEX/TOBEX.TP2~ #0 #100 // TobEx - Core: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #101 // Apply Concentration Check On Damage [WIP]: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #102 // Awaken On Damage: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #103 // Blindness As Spell Description: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #105 // No Spell Interruption On Zero Damage: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #106 // Use Caster Level On Mirror Image [C]: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #107 // Allow All Races to Dual Class: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #108 // Allow Equipping Armor in Combat: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #109 // Disable Experience Boost: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #110 // Disable Force Inventory Pause: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #111 // Disable Silence On Charm: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #112 // Level One Proficiency Restrictions: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #113 // Remain Hidden On Pickpocket Success [C]: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #116 // Enable Animation Attack Sounds: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #119 // Remove all race-class restrictions: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #120 // Drop Inventory on Disintegrate: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #121 // Drop Inventory on Frozen Death: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #122 // Drop Inventory on Stone Death: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #123 // Enable Auto-Pause On All Screens: Beta 0025
~TOBEX/TOBEX.TP2~ #0 #124 // Make All Attack Animations Genuine Attacks: Beta 0025
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #0 // BG2 Fixpack - Core Fixes: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #1000 // BG2 Fixpack - Game Text Update -> GTU Light (by Wisp): v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #3 // BETA Core Fixes (please check the readme!): v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #100 // Party Gets XP for Sending Keldorn to Reconcile With Maria: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #101 // Improved Spell Animations: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #104 // Ghreyfain's Holy Symbol Fixes: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #106 // Giants Receive Penalties When Attacking Halflings, Dwarves, and Gnomes: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #107 // Remove Dual-Classing Restriction from Archers and Stalkers: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #108 // Remove Second Attribute Bonus for Evil Path in Wrath Hell Trial: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #109 // Corrected Summoned Demon Behavior: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #110 // Additional Script Fixes: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #111 // Bard Song Fixes: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #112 // Wizard Slayers Cause Miscast Magic on Ranged Attacks: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #113 // Additional Alignment Fixes: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #115 // Paws from Shapeshifting Can Not Be Dispelled: v13
~BG2FIXPACK/SETUP-BG2FIXPACK.TP2~ #0 #116 // Remove Thieving Start Bonuses from Bard and Ranger Skills: v13

I have these two mods in my game directory, but they aren't actually installed. I place this weidu.log into my game directory, open up the installer for ToBEx, and tell it to reinstall the very first component, and then exit. It'll give errors like "Error Uninstalling [TOBEX/TOBEX.TP2] component 100:; Sys_error("TobEx/backup/100/UNINSTALL.100: No such file or directory")" and will skip it, and then once those are complete, will start the installation of all these components from the beginning properly, and everything seems to work properly. In other words, I've basically automated the installation of individual components through the use of a weidu.log. The problem I'm running into is it seems whenever I do this for more than like a mod or two at a time, things seem to start going crazy and every mod gets marked with "installed with warnings" and as I'm watching the installation go, I'm seeing just an endless amount of the aforementioned "possible missing evaluate_buffer" errors while installing mods and components that would not normally be spitting out these errors, but for some reason do so when doing a reinstall. Are these actually installing properly, or is stuff breaking?

[Jarno Mikkola]

48 minutes ago, Bartimaeus said:

Are these actually installing properly, or is stuff breaking?

Well, depends on your point of view... as you shouldn't fuck up the system that's keeping your afloat by puching holes at it.

What you might want to do is to ask the modmanager to make you an an autoinstallation instructions based on your weidu.log, and not just force the weidu.exe to make it do it by itself, as in that case there is, a likely a case where some settings are not made at all... say like ACTION_READLN's, as those are not set in the weidu.log, so they are not set.

[Bartimaeus]

1 hour ago, Jarno Mikkola said:

Well, depends on your point of view... as you shouldn't fuck up the system that's keeping your afloat by puching holes at it.

What you might want to do is to ask the modmanager to make you an an autoinstallation instructions based on your weidu.log, and not just force the weidu.exe to make it do it by itself, as in that case there is, a likely a case where some settings are not made at all... say like ACTION_READLN's, as those are not set in the weidu.log, so they are not set.

Yeah, I was thinking of cases like BGT and 1pp where something is input that can't be read from the weidu.log. And I see that there are these very helpful "readln.0/text" files stored in the backup folders of components that use it that store what commands you used. Testing it out now...and yep, that was exactly what I need to fully automate installs through weidu. Thanks, .

Beautiful stuff.

Edited July 1, 2020 by Bartimaeus

[Guest guest]

Are you making for yourself an "alternative" to PI ?

[Bartimaeus]

1 hour ago, Guest guest said:

Are you making for yourself an "alternative" to PI ?

Never used it, but I used the BWS a few times in eons past and I remember that while it was nice in some ways, it was difficult to manage and error-prone in others (errors during installation would sometimes mysteriously happen, components that never got put into the BWS and therefore weren't selectable/installable, inflexible installation order and the modification thereof, imaginary/taken care of conflicts that it would keep yelling at me for, some mods that I used missing entirely, etc). It occurred to me that weidu itself should theoretically be capable of doing it all by itself because it can sequence the reinstallation of components via the weidu.log, so that was what I was trying to figure out how to do, especially because I already keep for myself a master weidu.log of all the mods that I like that I modify depending on what I want to install and use to reference for order and re-arranging.

Now I can just sequence virtually an entire installation through the weidu.log - and if I want to insert a new mod somewhere, I can just insert it into the weidu.log exactly where I want it. READLNs which choke weidu doing this are thankfully pretty rare, and I just did a quick scan of all my .tp2 and .tpa files to identity which of my mods use them and planted the correct READLN files to handle it. Probably not the most realistic option for most other people who don't keep as strict record of their files as I do, but it's perfect for me.

I should probably give Project Infinity a try sometime to see how it compares, though.

Edited July 1, 2020 by Bartimaeus

[subtledoctor]

Wait how do you do more than one at atime??

[DavidW]

Those 'possible missing evaluate buffer' WARNINGs are (I think) happening because some mod in your install stack has the MODDER flag left on in its tp2. Those aren't warnings that are supposed to show up in live use of a released mod, but I think if a MODDER flag is set in one mod's tp2 it can propagate to the installation of other mods too.

As to whether they're something to worry about: Probably not. My experience of that particular MODDER setting is that it generates lots of false positives, and most importantly, doesn't really know about the AUTO_EVAL_STRINGS setting. If you've got mods that use AUTO_EVAL_STRINGS, particularly if they also use lots of functions (e.g., SCS) then there'll be huge numbers of false positives, and you shouldn't worry about them. 

[Bartimaeus]

52 minutes ago, subtledoctor said:

Wait how do you do more than one at atime??

Sorry, I'm afraid I don't understand: more than one mod at a time, or..? In my above example, I have TobEx installed first, then the BG2 Fixpack - only, I don't actually have them installed, it's just what the weidu.log says and what the weidu exe is reading, so when I open up the ToBEx setup and reinstall the first component, it "uninstalls" all the components that came after and then "reinstalls" them (installs them for the first time since they were never installed to begin with) in the same exact order I specified in the weidu.log afterwards. You can take this to an extreme with having a hundred components "installed" in your weidu.log and then just reinstalling the very first component and then weidu will attempt to install everything that's supposed to come after it (however, components that ask for manual user input via READLNs will die if the READLN files are not supplied in advance - it'd be nice if there was like an "interactive reinstall" mode for weidu that would pause and let you input for any READLNs that came up, that'd make automating everything through weidu much easier and safer, especially seeing as READLNs are pretty rare - in my install, I only came across 5 total: BGT, BGGraphics, 1pp, Widescreen, and the Level 1 NPC Project).

@DavidW: Thanks for the info. I found the culprit via a mass Notepad++ scan: BGGraphics sets it. I'll fix that to prevent it happening in the future - it seemed to massively slow installation when it was set (I had like a 40MB debug file on my first attempt, which is ludicrous), so I ended up doing it in chunks and got past it by happenstance instead.

Edited July 1, 2020 by Bartimaeus

[subtledoctor]

1 hour ago, Bartimaeus said:

Sorry, I'm afraid I don't understand: more than one mod at a time, or..?

Yeah, how do more than one mod. But I think I understand now. You “reinstall” the very first mod (or even just the first component) in the list, and then it will “reinstall” all mods that are after it?

Gotta say, that’s sort of genius. If you don’t have any mods that use READLN (pretty sure I don’t) then it should be pretty seamless.

[Bartimaeus]

17 minutes ago, subtledoctor said:

Yeah, how do more than one mod. But I think I understand now. You “reinstall” the very first mod (or even just the first component) in the list, and then it will “reinstall” all mods that are after it?

Gotta say, that’s sort of genius. If you don’t have any mods that use READLN (pretty sure I don’t) then it should be pretty seamless.

Yep, exactly that - only need the first component of the first mod of the chunk of mods/components you want, then you can just type "q" for quit and let it do everything else (because it already thinks everything after should be installed, a 'quit' makes it think you're telling it to do "no other changes" and redo the installation of everything else in the weidu.log).

For years and years, I've always thought it *should* work, but always had problems actually making it do so - figured I would actually finally ask and see if it was workable, which it certainly seems like it is thanks to Jarno and DavidW's input here.

Edited July 1, 2020 by Bartimaeus

[Caedwyr]

This trick is very helpful for me and kind of obvious in retrospect.   Thanks for sharing. 

[Bartimaeus]

20 minutes ago, Caedwyr said:

This trick is very helpful for me and kind of obvious in retrospect.   Thanks for sharing. 

It's a little bit of a shame that I didn't ask for help with it like ten years ago when I initially thought up of the idea - tried a few times to make it work, never quite got it to, and never quite got around to asking either. A lot of my own wasted time attending installations when I could've just had weidu do 99% of it all by itself. I'm sure other people could've benefited too...whoops.

If your installation is truly massive, it may make sense to do it in chunks instead of all at once - I think the installation starts getting slower as more is done. Also, if something goes wrong, it's easier to go back and fix it when only doing, say, a couple hundred components at a time instead of a thousand, .

Edited July 1, 2020 by Bartimaeus

[Jarno Mikkola]

Care to share the weidu.log ?

[Caedwyr]

I tried this earlier and ran into some issues.

Quote

Not enough backed up entries in your repliesStopping installation because of error.
Stopping installation because of error.

This error shows up fairly often and entire components just fail to install.

SETUP-DLCMERGER.DEBUG

[Bartimaeus]

2 hours ago, Caedwyr said:

I tried this earlier and ran into some issues.

This error shows up fairly often and entire components just fail to install.

SETUP-DLCMERGER.DEBUG 574.73 kB · 0 downloads

It's the thing Jarno warned of, READLNs that do not have a value set because you did not already install the mod. For example:
 

Spoiler

 

Select the interval between Garrick's dialogues:

Please choose one of the following:
[1] 1 hour real time (standard) minimum between dialogues
[2] 45 minutes real time minimum between dialogues
[3] 30 minutes real time minimum between dialogues
[4] 15 minutes real time minimum between dialogues
[5] 1 hour 30 minutes (extended) real time minimum between dialogues

 

Unlike when you install real subcomponents, READLNs are basically when weidu asks for your input to set a variable in a totally free way - e.g. if you're using the Widescreen mod and try to install it, it asks you to enter your X and Y resolution sizes for it to set; if you try to install BGT, it asks you to enter the path to your Baldur's Gate 1 folder, etc. With your selected components, you ran into it 3 times, including the one above - the listed options aren't their own subcomponents, it's just a user prompt that's setting a variable that never shows up in your weidu.log. If these components had already been installed, whatever input you used would have been saved into the backup folder of each individual component that needed it (for example, for BGT it's \bgt\backup\0\READLN.0 and READLN.0.TEXT). If the component is ever reinstalled, it will just re-use the input saved in these files - but because you didn't actually install it yet, these files don't yet exist and the installer doesn't know what to enter when it comes across it.

I wrote above: "however, components that ask for manual user input via READLNs will die if the READLN files are not supplied in advance - it'd be nice if there was like an "interactive reinstall" mode for weidu that would pause and let you input for any READLNs that came up, that'd make automating everything through weidu much easier and safer, especially seeing as READLNs are pretty rare - in my install, I only came across 5 total: BGT, BGGraphics, 1pp, Widescreen, and the Level 1 NPC Project"

Some mod installs may admittedly have more - possibly many more if there are particular mods that like to use READLNs instead of components (there's really not much reason the above example, Garrick's Infatuation, uses a READLN to set an option instead of simply using a subcomponent...but the fact remains that it does). I started saving the READLN files of all the components that I use that need it (which isn't very many), so that if I start a new install, I can just drop and drag them into my BG2 folder in advance and have it all taken care of.

https://dl.dropboxusercontent.com/s/q2p1eaohcexfw1t/PiyHiIGIXZ.mp4

...This may be too annoying for most other people to setup, though, . It'd be awesome if whoever's maintaining weidu would make it so that when reinstalling, if a READLN is needed but not already supplied, it would simply ask you to enter it again as they come up, rather than failing the component entirely and trying to continue without and thereby breaking your install. That would fix that problem without changing functionality for normal installation.

Edited July 2, 2020 by Bartimaeus