*Possible* unintentional behavior with the "Wider selection of random scrolls" component for BG2EE

[AFootBison]

In Irenicus' Dungeon a goblin dropped what I thought was a scroll of fireball, but when I had Imoen use it, she cast Selune's Curse (which completely obliterated the Duergar I was fighting). I know the component adds any spell scroll to the list of available random scrolls, but this doesn't seem intentional.

[jmerry]

[Edit again] Short explanation: the macros SCS uses to populate those random scroll lists pick the wrong "Fireball" scroll, because there's a weird unused item in the game files. See below for details.

Was the scroll named "Fireball"? A scroll with one spell's name that casts a different spell is definitely a bug, if that's what happened. But not necessarily a bug with SCS.

The general procedure here is that scroll items are created when the spells are added. Then, later, the random scroll component picks those up when it's building the new tables for random scrolls.

So, what mod is "Selune's Curse" from?

[Edit] Wait, no, Selûne's Curse exists in the base game for scripted events. SPWI953, symbolic name DRADEEL_BLUE_FIREBALL, appears in BG2 Dradeel's script, not learnable because it's an innate spell, and it has a scroll SCRL5Z for no good reason. Which is named "Fireball". So, then, that scroll somehow got onto the random scroll lists. I need to take another look at how the SCS component populates those lists.

Which is ... take the sfo_arcane_scrolls array. Exclude anything in hidespl.2da, plus Find Familiar. Cap the spell levels, appropriate to the campaign. What's left is what goes in the lists. So, naturally, the next step is looking at sfo_arcane_scrolls. After some searching, the macro that creates it is data_internal_scroll_resrefs in SFO's lib_data.tph. Which requires data_spell_resrefs to be run first. Read spell.ids. Parse the entries to generate resource references and, for SPWI/SPPR entries, assumed spell levels. Do some more stuff to handle the extended namespace DW mods use. And you have the list. Nowhere in this process was the actual spell ever looked at.

Now, for the scrolls, we go through all the items and check - is this a spell scroll, and if so, for what spell? Anything with at least one "Cast Spell" or "Learn Spell" effect counts, unless there are multiple "Learn Spell" effects. Whatever the last of these effects references is now considered this scroll's spell. (A couple of items that would be problematic get skipped - the tutorial scrolls TTSCRL01 and TTSCRL02. Put a pin in this...) If this was a success, add to the array - that spell's scroll is now this item. If multiple scrolls are found for the same item, I'm pretty sure the last one wins.

All right. What went wrong? Well, the problem is that scroll SCRL5Z. It has two abilities. First, a "Cast Spell" effect for Selûne's Curse. Second, a "Learn Spell" effect for Fireball. So that array building macro goes through the items alphabetically, comes to SCRL1G, and says "That's the Fireball scroll". Then, later, it comes to SCRL5Z and says "That's the Fireball scroll. Forget the other one". The random scrolls component populates its lists based on this, so when it picks a Fireball scroll it picks SCRL5Z instead of SCRL1G. They're both equally functional for learning the spell, but radically different for casting from the scroll; Selûne's Curse is 20d6 magic damage (save for half) with a white fireball projectile. Yeah, that's the wrong scroll.

How do we fix it? Well, go back to those exclusions. This problematic scroll exists in both BGEE and BG2EE, and adding it to the skip list is the simplest fix. Replace this

        PATCH_MATCH "%filename%" WITH
        TTSCRL01 TTSCRL02 BEGIN END // tutorial scrolls - skip them

with

        PATCH_MATCH "%filename%" WITH
        TTSCRL01 TTSCRL02 SCRL5Z BEGIN END // problematic scrolls - skip them

(sfo2e/lib_data, lines 291-292 in current version) and it should work out. At least in BGEE/BG2EE ... might need to do something more complicated if IWD has a legit scroll at that resref.

Edited December 1 by jmerry

[Graion Dilach]

Heh, never noticed this scroll. Note that the RGT scroll-generator isn't affected because I keep the first instances so in this case I end up keeping SCRL1G and noting SCRL5Z as a duplicate (the duplicate detection is logged for anyone to check their setup).

There are a few additional bogus scrolls I'd blacklist along, SCRLADJ has a bogus op145 lacking the charge, which was cloned into PSSCRL02 in Planar Sphere, and maybe SCRLZY/SCRLZZ with their Summon Cow spells.

Edited December 1 by Graion Dilach

[jmerry]

I'd consider SCRLZZ as the "legit" Summon Cow scroll. Which doesn't normally go on the random scroll list because that also excludes everything in HIDESPL.2DA. No need for an explicit blacklist, because some loony might want Summon Cow in their game. SCRLZY has a "Kill Target" effect instead of "Cast Spell", so it's bogus. SCRLADJ ... yeah, that's bogus. Learn Summon Cow only there's no charge, and no second ability. PSSCRL02 ... what mod did that cloning? Not in base BG2EE. (But out of all these "Summon Cow" scrolls, the SCS macro randomly picks the legit one anyway)

[CrevsDaak]

This is an arcane bug if I've seen one. Very cool find.

[jmerry]

A bit more about this macro. First safeties in the system. The macro checks for the "scroll" item type. Anything else that casts a spell, nope. The Book of Infinite Spells is not ten different spell scrolls. Also, it checks against the IDS map for the final list; if the spell is anything outside the wizard/priest namespaces. it won't get added to the wizard/priest list. No MDK2 spell for you. But that's about it. If a mod creates a cast-only scroll for an arcane spell and happens to have a prefix that's later than SC in the alphabet, its scroll can displace the real spell scroll in random drops with this component. Oops.

So, thinking about more complicated solutions ... what if instead of just requiring a cast or learn ability for arcane scrolls, we required both? And matching resources, naturally. That would clear out the various bogus scrolls (SCRL5Z, SCRLZY, SCRLADJ) mentioned above. It would also remove Identify, so you'd need to whitelist that one. And the two tutorial scrolls would still need to be blacklisted, since they have both abilities. But I think it's worth it - it would also be much more robust against mods accidentally poisoning the scroll list.

[ScrambledKhegs]

Another quick fix is just to overwrite the offending scroll:

COPY_EXISTING ~SCRL1G.ITM~ ~override/SCRL5Z.ITM~

 

[Anterwaare]

There's also a scroll with ghost armor image, greater malison description and it casts Selune's Blessing on use. So same story pretty much

[jmerry]

Okay. That's SCRL5W, SCRL5X, and SCRL5Y. All three of them learn Ghost Armor, as their second abilities. They all cast different spells; SPWI932 (Die and be replaced with Aec'Letec, or just crash the game if it's BG2), SPWI952 (Selûne's Blade; equip a 1d12+4 weapon with +4 to hit, 50% fire resist, and +1 AC), and SPWI954 (Selûne's Blessing; Globe of Invulnerability and +5 AC)

And because SCRL5Y is the last of the three, it's the one that displaces the real Ghost Armor scroll (SCRL1T) in random drops. But if you just excluded that one, one of the others would just take its place.

I'm definitely favoring the more complex solution over enumerating bogus spell scrolls to exclude.

[Graion Dilach]

8 hours ago, jmerry said:

I'd consider SCRLZZ as the "legit" Summon Cow scroll. Which doesn't normally go on the random scroll list because that also excludes everything in HIDESPL.2DA. No need for an explicit blacklist, because some loony might want Summon Cow in their game.

If the restoration is made explicit with a custom scroll, that prevents any confusion on the intention. Shard of Ice's Summon Cow component does this and in hindsight, I'd say that's the better option. Just in case HIDESPL is handled...

8 hours ago, jmerry said:

PSSCRL02 ... what mod did that cloning? Not in base BG2EE.

When I said Planar Sphere, I meant the Planar Sphere mod. Note that in The-Gate-Project, I persuaded the others to delete this opcode from the item, because it's bogus and it doesn't belong there anyway (both SCRLADJ and PSSCRL02 are "info" letters afterall).

RE: Ghost Armor; another case where my setup keeping the first valid instance avoids the issue. I'm not convinced that additional validation is necessary, I don't think I'll add that to RGT. (Keep in mind that if you stick to the first instance, you don't need to explicitly blacklist the tutorial scrolls either.)

6 hours ago, jmerry said:

If a mod creates a cast-only scroll for an arcane spell and happens to have a prefix that's later than SC in the alphabet, its scroll can displace the real spell scroll in random drops with this component.

Honestly, that's not really an issue, even if we opt for keeping the first take instead of the last one. Mods don't often add duplicates of vanilla spell scrolls (usually only the SoD Faerie Fire/Darkness 15' Radius and Dancing Lights trio gets duplicates), otherwise you tend to end up with IWD spell scroll duplicates for the most part (where it doesn't really matter which mod you end up sourcing the scrolls from IMO). Sure, it can affect stacking (and maybe some art style differences, like Check the Bodies' scrolls using IWD2 scroll art if that's the one getting rolled in) but as long as the mod sets up the scrolls right, it should be fine.

But if we involve mods, I'd say sticking to the first valid scroll remains a better option. For the most common case of the CamDawg-DavidW IWD library vs argent77's ToTLM port, the former uses CD prefix for the scrolls, while the latter uses ID#, so the library wins. Other cases which come to my mind:

• DSotSC-Trilogy uses SCRLDS for the SoD duplicates, so BD wins.
• BG1NPCSoA can install spells and scrolls using a BG prefix, but all those spells are complete individual duplicates not even using SPELL.IDS, so they don't pass the validation (and I'm not sure if this one's worth the trouble to fix).
• Weigo's recent CtB patch resolving all it's shortcomings wrt spell introduction leads to CtB's CB prefix beating IWDSpell's CD.
• So far SoD2BG2_IU reused the Beamdog spell scrolls.

Edited December 1 by Graion Dilach

[AFootBison]

@jmerry I appreciate the deep dive on this. I suppose I should have prefaced that I'm on a relatively light modded install (mostly tweaks/SCS) and I'd looked around NI to see what I could figure out before posting here.

As for the first solution, would adding the selune spells to the blacklist be better or would @Graion Dilach's component would be better for now

[Andrea C.]

The thing I’m the most curious about—was the fireball, in fact, blue?

[AFootBison]

2 hours ago, Andrea C. said:

The thing I’m the most curious about—was the fireball, in fact, blue?

It was indeed. Ilyich and his Duergar had no idea what hit them, and if it wasn't for the explicit cast warnings component from the Tweaks Anthology master I wouldn't have known either

[Graion Dilach]

4 hours ago, AFootBison said:

As for the first solution, would adding the selune spells to the blacklist be better or would @Graion Dilach's component would be better for now

My mod uses a parallel distribution system where I add new items to mages/clerics instead of fiddling with the game's own random treasure (which is also why I offer multiple options for this). You can do both.