request for v4: preserve overwritten spells

[subtledoctor]

One of the unfortunate things I've found using SR is that the way it overwrites some vanilla spells acts to remove several good assets from the game. There have been a number of conversations about the issues involved in replacing Cacofiend with Summon Death Knight, but here I'll try to focus the issue and use Blindness/Obscruring Mist as an example.

 

Let's say I have a kit mod and I want it to have Blindness as an innate ability. And I don't want to supply my own spell, I want it to be consistent with the rest of the game. (The player may have used other mods that alter the spell... *ahem* like, the way this mod tweaks various spells.)

 

So my mod will use a command like this:

COPY_EXISTING ~spwi106.spl~ ~override/kitblind.spl~
   LPM spell_to_innate

But if SR is installed, now my kit will get Obscuring Mist instead of Blindness. And I don't know if the game includes any variant of the Blindness spell for me to use. So SR has actually destroyed something. This is why people have complained about Cacofiend - the original spell is not even there for a later mod to grab and re-introduce into the game.

 

Some spell changes made by SR are minor and not worth worrying about - the save penalty from Malison drops from -4 to -2, but it's functionally the same effect. With other spells, though, like Blindness/Obscuring Mist, there is a huge difference. SR is really deleting one spell and inserting a totally new spell. The better situation would be to have *both* spells in the game, by adding Obscuring Mist as a new spell. SR adds tons of new spells, and I haven't seen a full explanation of logic behind which ones get added as new, and which ones replace already-existing spells.

 

Setting aside the original logic behind the decision, we seem to be bound by it thanks to the collaboration between Demi and DavidW, which means SCS expects certain things to be in certain places and this cannot be changed. But, there are two solutions to this problem that SR could employ, with differing levels of difficulty.

 

1) The ideal solution is for SR to use the modern and flexible ADD_SPELL convention to add all new spells. This would be best for overall compatibility with other mods, but it would be a bit of work to convert the SR code. Essentially, the code would have to use something like this:

ADD_SPELL ~spell_rev/spells/spwi106.spl~ 2 1 WIZARD_BLINDNESS

LAF RES_NUM_OF_SPELL_NAME
   STR_VAR spell_name = ~WIZARD_BLINDNESS~
   RET spell_res
END

COPY_EXISTING_REGEXP GLOB ~.*\.spl~ ~override~
   LPF ALTER_EFFECT 
      match_resource = ~spwi106~ 
      resource = EVAL ~%spell_res%~
   END
BUT_ONLY

That's not perfect or complete code, but it's roughly what would be necessary.

 

2) The less ideal solution, but one which I would urge at the least, is to simply preserve the overwritten spells before overwriting them. In essence, change this:

COPY ~spell_rev/spells/spwi106.spl~ ~override~

... to this:

COPY_EXISTING ~spwi106.spl~ ~override/dvwi106.spl~
COPY ~spell_rev/spells/spwi106.spl~ ~override~

Then at least the original asset would be findable, and SR compatibility for my hypothetical kit mod would be simple:

ACTION_IF NOT FILE_EXISTS_IN_GAME ~dvsrv4here.mrk~ BEGIN
   COPY_EXISTING ~spwi106.spl~ ~override/kitblind.spl~
      LPM spell_to_innate
END
ACTION_IF FILE_EXISTS_IN_GAME ~dvsrv4here.mrk~ BEGIN
   COPY_EXISTING ~dvwi106.spl~ ~override/kitblind.spl~
      LPM spell_to_innate
END

Pretty please?

[Jarno Mikkola]

Pretty please?

When the SR moves to the use of the ADD_SPELL function, it will be not overwriting the existing spells anymore... issue solved.

And they said they will. The bonus being also that they get 60+ HLA spell slots ... from the code.

[subtledoctor]

When the SR moves to the use of the ADD_SPELL function,

Well, I'm not sure they've actually committed to do that for v4, so I want to advocate at least for this simpler alternative.

[Mike1072]

You can think of what SR is doing here as accomplishing 2 things: adding a new spell and removing an old spell. Depending on which replacement you are talking about, the new spell may be completely distinct from the old (True Strike vs. Infravision) or it may serve the same purpose (Battering Ram vs. Knock) or a similar one (Summon Shambling Mound vs. Summon Earth Elemental). For a lot of these, there isn't a way to make the original spell unique enough from the replacement to justify keeping them both accessible to the player. Right now, we are only planning on using ADD_SPELL for spells that are plopped into new spell slots, not these replacements.

 

You may want to consider whether your kit is gaining access to a specific Blindness ability (that should function in a specific way) or whether it's gaining access to a 1st level arcane spell that blinds people. This can guide whether you should be including the spell in your mod or using one which may have been changed by other mods.

 

From a technical perspective, rather than us implementing option #2, you could just include the vanilla spells in your mod and in any place you would copy dvwi106.spl, just copy your version of spwi106.spl.

[subtledoctor]

You can think of what SR is doing here as accomplishing 2 things: adding a new spell and removing an old spell.

Right, I understand that's what SR does here, but I don't understand what is accomplished by removing old spells. To clarify: I completely understand why SR would want to remove those spells from the player; I just don't see why SR thinks it is important to remove them from the game entirely.

 

Except of course that it's easier and less effort - which is a real factor, I understand that. But I guess what I'm trying to communicate, is that for lots of friendly modders and players, that effort would be worthwhile. It shouldn't be *too* much effort - we're only talking about a dozen or so spells here, so maybe a dozen or so extra COPY commands. As I said, it's really only needed where the replacement spell is totally different from the original - like Obscuring Mist, and True Strike, and Death Knight, but not Battering Ram or Greater Malison. Heck, I'd be happy to take the time to add in the required commands myself.

 

From a technical perspective, rather than us implementing option #2, you could just include the vanilla spells in your mod and in any place you would copy dvwi106.spl, just copy your version of spwi106.spl.

I could re-make the spell myself and include it in a mod... but I would have to make the effort to make sure different variations for different games are all correct, and have the correct name and description strings, and I would be completely unable to prevent possible inconsistencies if other mods go in before mine and make changes. It could certainly be done, and the problems are generally edge case problems... but honestly, it's extremely silly to go to all that trouble just to duplicate an asset that is already in the vanilla game - or would be, if it hadn't been affirmatively deleted by SR.

 

And to clarify a bit more, I'm not actually making a kit, so this isn't a question about helping me figure out how to solve some particular problem I have. It's just an example. The Cacofiend thing is another example, there is a whole thread with people talking about ways to make the situation more flexible but unable to find a good solution. I'm saying, it would be seriously complicated and time-consuming for one of use to solve that problem, whereas all it would really take is a single line of code in SR.

[Fiann of the Silver Hand]

I've always thought of SR as replacing all vanilla spells, so for a later mod to assume vanilla is untouched is a bit of a stretch of goodwill. It's not ideal, but no mod that expects to be so early in install order ever is.

[Demivrgvs]

Mike and Fiann already said most of what I could have said.

 

Right, I understand that's what SR does here, but I don't understand what is accomplished by removing old spells. To clarify: I completely understand why SR would want to remove those spells from the player; I just don't see why SR thinks it is important to remove them from the game entirely.

I don't think there is a single case where I want a spell to be removed from players but left for the AI, and most importantly adding a new spl doesn't make it used by SCS, replacing a spl does. I admit that V4 is doing it a bit more often, and that some changes may not be welcomed by everyone (e.g. Chaos was a fan favorite spell because of its ridiculous power) but the current approach make me both remove an "unwanted spell" and let the new spell be used by SCS (e.g. you'll face both Confusion and Waves of Fatigue instead of Confusion and its copy/paste Chaos).

 

P.S I'll try to re-think the whole Death Knight thing, but I'd like to remind that its related issues arise only when too many mods are messing with the same thing. SCS was actually coded to use SR's DK and we had no issuesback then, but then aTweak fiends were released and everything started to become much more complicated. Don't get me wrong, aVENGER's work is fantastic, and players have all the rights to wish the three mods to work together, I'm just saying the current issues with DK/Cacofiend isn't just SR's fault.

[subtledoctor]

most importantly adding a new spl doesn't make it used by SCS, replacing a spl does.

Actually that's not true. If you use

ADD_SPELL ~spell_rev/waves.spl~ 5 2 WIZARD_CHAOS

...then you will be adding a new spell and not destroying any existing game assets, *and* making sure that SCS and AI and scripts etc. use the new spell in place of the original spell... in other words, achieving your goal completely. Deciding to only use ADD_SPELL for new spells, and not for spell replacements, kind of misses the point of that command. It's designed specifically for spell replacement.

 

P.S I'll try to re-think the whole Death Knight thing, but I'd like to remind that its related issues arise only when too many mods are messing with the same thing. ... I'm just saying the current issues with DK/Cacofiend isn't just SR's fault.

I'm not saying anyone's at fault, and I really hope this isn't coming across as critical. I'm just pointing out that there's a solution to some of those issues that SR could implement extremely easily. (Whereas, if you read the DK/Cacofiend threads and look at the hoops redo was jumping through, it's clear that it's very difficult for other people to solve them.)

[Jarno Mikkola]

@subtledoctor, because the ADD_SPELL function is a bit odd to use, could you show a live example of the replacement of an actual spell in code ? Obviously a one that the SR does already, and comparison of the codes ?

Aka, use a SR beta as the from, .tp2 file... for the whole portion of the spell copying, editing etc. So the code is complete for that one spell. Demi etc can handle the rest.

Hopefully you have an access to a beta of a recent while. I think are at them... I am probably the only one that isn't... cause I don't have a computer to install and test it to, so to speak.

[subtledoctor]

I'd be happy to. Does anyone know of a particular AI Mage that uses a particular spell, so it would be easy to test? I don't have any savegames so maybe a low-level would be best... is there an AI Mage near Candlekeep/FAI/Beregost who is scripted to cast Blindness at Charname? Or, does anyone have a savegame near a Mage who is scripted to cast Chaos?

[subtledoctor]

Btw part of what makes it difficult to determine the scope of what I'm talking about is that I can't find anywhere a proper list of what's added, what's deleted, and what's changed by v4. (There being no readme and all, which is perfectly natural since the mod's not done yet.)

 

Anyway, I went through NI myself and made such a list while working on another mod, so maybe it would be useful, for this discussion or just for other modders to get a handle on SR's changes.

 

"changed" SR spells:

sppr209 = know opponent

(was know alignment)

sppr210 = resist elements

(was resist fire/cold)

sppr218 = gust of wind *

sppr307 = break enchantment

(was remove curse)

sppr506 = stoneskin

(was ironskin)

sppr513 = righteous fury

(was righteous magic)

sppr515 = repulsion *

(was repulse undead)

sppr702 = shambling mound *

(was earth elemental)

sppr703 = death knight *

(was gate)

(use wizard gate instead)

sppr716 = symbol weakness *

(was symbol fear)

spwi106 = obscuring mist *

(was blindness)

spwi107 = monster summ 1 *

(was friends)

spwi108 = expeditious retreat *

(was pro petrification)

spwi111 = true strike

(was infravision)

spwi207 = battering ram *

(was knock)

spwi208 = know opponent

(was know alignment)

spwi223 = sound burst

(was deafness)

spwi402 = dimension jump

(was dimension door)

spwi403 = acid sheath

(was fire shield blue)

spwi410 = break enchantment

(was remove curse)

spwi423 = monster summ 4

(was spider spawn)

spwi501 = summon shadow

(was animate dead)

spwi508 = waves of fatigue

(was chaos)

spwi510 = dispelling screen *

(was spell imm)

spwi605 = banishment

(was death spell)

spwi614 = acid fog

(was death fog?)

spwi619 = monster summ 6

(was wyvern call)

spwi623 = skeleton warrior *

(was carrion crawler)

spwi708 = prismatic mantle *

(was mantle)

spwi7__ = death knight *

(was cacofiend)

spwi808 = moment of prescience *

(was imp. mantle)

spwi818 = icy grasp

(was clenched fist)

 

New SR spells:

sppr114 = faerie fire

sppr115 = strength of stone

sppr116 = sunscorch

sppr117 = regenerate light

sppr118 = goodberry

sppr120 = animal summ 1

sppr121 = obscuring mist

sppr122 = curse (?)

sppr216 = fire trap

sppr217 = regenerate moderate

sppr220 = animal summ 2

sppr320 = animal summ 3

sppr322 = storm shield

sppr323 = regenerate serious

sppr324 = magic fang

sppr325 = spike growth

sppr418 = ice storm

sppr419 = regenerate critical

sppr519 = polymorph other

sppr520 = pro acid

sppr521 = pro cold

sppr522 = pro electricity

sppr523 = pro fire

sppr524 = mass regenerate

sppr525 = animal growth

sppr619 = regeneration

sppr620 = banishment

sppr621 = air elemental

sppr622 = earth elemental

sppr623 = skeleton warrior

sppr716 = animal summ 7

spwi225 = resist elements

spwi226 = monster summ 2

spwi526 = acid sheath

spwi724 = monster summ 7

spwi801 = ghost form

spwi802 = mind blank

 

The instances with stars are the ones where, instead of changing a spell, SR really overwrites the old spell with a completely different effect. Reasonable minds can disagree about some of them - should Monster Summoning 4 be included? Should Gust of Wind or Battering Ram be excluded? - but it's those 12-15 spells that I'm talking about here. So 12-15 extra COPYs.

[Roxanne]

Btw part of what makes it difficult to determine the scope of what I'm talking about is that I can't find anywhere a proper list of what's added, what's deleted, and what's changed by v4. (There being no readme and all, which is perfectly natural since the mod's not done yet.)

 

Anyway, I went through NI myself and made such a list while working on another mod, so maybe it would be useful, for this discussion or just for other modders to get a handle on SR's changes.

 

"changed" SR spells:

sppr209 = know opponent

(was know alignment)

sppr210 = resist elements

(was resist fire/cold)

sppr218 = gust of wind *

sppr307 = break enchantment

(was remove curse)

sppr506 = stoneskin

(was ironskin)

sppr513 = righteous fury

(was righteous magic)

sppr515 = repulsion *

(was repulse undead)

sppr702 = shambling mound *

(was earth elemental)

sppr703 = death knight *

(was gate)

(use wizard gate instead)

sppr716 = symbol weakness *

(was symbol fear)

spwi106 = obscuring mist *

(was blindness)

spwi107 = monster summ 1 *

(was friends)

spwi108 = expeditious retreat *

(was pro petrification)

spwi111 = true strike

(was infravision)

spwi207 = battering ram *

(was knock)

spwi208 = know opponent

(was know alignment)

spwi223 = sound burst

(was deafness)

spwi402 = dimension jump

(was dimension door)

spwi403 = acid sheath

(was fire shield blue)

spwi410 = break enchantment

(was remove curse)

spwi423 = monster summ 4

(was spider spawn)

spwi501 = summon shadow

(was animate dead)

spwi508 = waves of fatigue

(was chaos)

spwi510 = dispelling screen *

(was spell imm)

spwi605 = banishment

(was death spell)

spwi614 = acid fog

(was death fog?)

spwi619 = monster summ 6

(was wyvern call)

spwi623 = skeleton warrior *

(was carrion crawler)

spwi708 = prismatic mantle *

(was mantle)

spwi7__ = death knight *

(was cacofiend)

spwi808 = moment of prescience *

(was imp. mantle)

spwi818 = icy grasp

(was clenched fist)

 

New SR spells:

sppr114 = faerie fire

sppr115 = strength of stone

sppr116 = sunscorch

sppr117 = regenerate light

sppr118 = goodberry

sppr120 = animal summ 1

sppr121 = obscuring mist

sppr122 = curse (?)

sppr216 = fire trap

sppr217 = regenerate moderate

sppr220 = animal summ 2

sppr320 = animal summ 3

sppr322 = storm shield

sppr323 = regenerate serious

sppr324 = magic fang

sppr325 = spike growth

sppr418 = ice storm

sppr419 = regenerate critical

sppr519 = polymorph other

sppr520 = pro acid

sppr521 = pro cold

sppr522 = pro electricity

sppr523 = pro fire

sppr524 = mass regenerate

sppr525 = animal growth

sppr619 = regeneration

sppr620 = banishment

sppr621 = air elemental

sppr622 = earth elemental

sppr623 = skeleton warrior

sppr716 = animal summ 7

spwi225 = resist elements

spwi226 = monster summ 2

spwi526 = acid sheath

spwi724 = monster summ 7

spwi801 = ghost form

spwi802 = mind blank

 

The instances with stars are the ones where, instead of changing a spell, SR really overwrites the old spell with a completely different effect. Reasonable minds can disagree about some of them - should Monster Summoning 4 be included? Should Gust of Wind or Battering Ram be excluded? - but it's those 12-15 spells that I'm talking about here. So 12-15 extra COPYs.

Just for a simple modder to understand correctly - it means if I use some of the spells from that list above within my mod (regardless whether in creature creation, progression or contents wise) and SR would be installed, the result will be far from the intended but what SR makes out of it?? E.g. Imoen taking some lessons with a thief companion and "learning" Knock would end up with "battering ram" (whatever that is)? And my priest who was discovering a traitor by casting "know alignment" will now fail the quest because SR has changed this spell into something else??

[subtledoctor]

Precisely.

 

To really spell this out at a nuts & bolts level: some of SR's "revisions" really substitute completely new and different spells rather than just revising the vanilla spell. In some instances, the new spells are added as new files; but in some instances the SR devs want AI scripts to use the new spell every time it would have used a certain vanilla spell. Since the method to do this is somewhat complicated and arduous (I described a simplified representation of the method above), SR instead just overwrites the vanilla spell file itself.

 

To be clear: I understand why that is done, and I actually think it is a good and efficient decision. It would just be nice if SR preserved the vanilla spell in some way, so that way other mods using that spell could simply use the backup version, and instruct users to install the mod after SR to make sure it gets taken into account. Problem solved.

[Roxanne]

Precisely.

 

To really spell this out at a nuts & bolts level: some of SR's "revisions" really substitute completely new and different spells rather than just revising the vanilla spell. In some instances, the new spells are added as new files; but in some instances the SR devs want AI scripts to use the new spell every time it would have used a certain vanilla spell. Since the method to do this is somewhat complicated and arduous (I described a simplified representation of the method above), SR instead just overwrites the vanilla spell file itself.

 

To be clear: I understand why that is done, and I actually think it is a good and efficient decision. It would just be nice if SR preserved the vanilla spell in some way, so that way other mods using that spell could simply use the backup version, and instruct users to install the mod after SR to make sure it gets taken into account. Problem solved.

I simply call that lack of respect for other people's work...So they do their changes and then other mods need to take care to revert it back and another mod comes along and tweaks it back in again...

I would throw a fireball at those nerds but I am sure they will tweak it into frog's spittle before it hits them.

[subtledoctor]

I simply call that lack of respect for other people's work...

Well, I wouldn't go that far. They can't know how every mod will affect or use every spell... hard to 'respect' what you're not even aware of. And in dealings on these forums and others, the SR guys have been nothing but gentlemen.

 

I'm just posting this here because I've identified a way that SR is *inadvertantly* causing bad effects for other mods, and trying to help them finds ways to make SR as friendly to other mods as possible, with as little effort as possible (15 lines of code!)

 

I make mods specifically designed to be installed late in the order, so I accept that it's up to me to make sure they are compatible with stuff like SR. But I try to open a dialogue to communicate ways that SR could make that easier. Like, y'know, adding 15 lines of code.