Preprocessing and templating

[Magus]

I'm entertaining an idea to add some preprocessing and/or templating capabilities to MLS/IElib. Looking to gather some feedback before endeavouring.

Gcc was my first thought, but it's not great with loops, so maybe jinja2 instead (or in addition).

Some example snippets to get the ball rolling:

baf

IF
  See(NearestEnemyOf(Myself))
  Global("abi_dalzim","LOCALS",0)
  LevelGT(LastSeenBy(Myself),15)    // Level 16+
  !GlobalTimerNotExpired("delay","LOCALS")
THEN
  RESPONSE #100
    SetGlobalTimer("delay","LOCALS",6)
    IncrementGlobal("doomed","LOCALS",1)
    IncrementGlobal("abi_dalzim","LOCALS",1)
    ReallyForceSpell(NearestEnemyOf(Myself),WIZARD_ABI_DALZIMS_HORRID_WILTING)
END

IF
  See(NearestEnemyOf(Myself))
  Global("efreeti","LOCALS",0)
  LevelGT(LastSeenBy(Myself),15)    // Level 16+
  !GlobalTimerNotExpired("delay","LOCALS")
THEN
  RESPONSE #100
    SetGlobalTimer("delay","LOCALS",6)
    IncrementGlobal("doomed","LOCALS",1)
    IncrementGlobal("efreeti","LOCALS",1)
    ReallyForceSpell(Myself,WIZARD_SUMMON_EFREET)
END

IF
  See(NearestEnemyOf(Myself)
  Global("sunfire","LOCALS",0)
  LevelGT(LastSeenBy(Myself),15)    // Level 16+
  !GlobalTimerNotExpired("delay","LOCALS")
THEN
  RESPONSE #100
    SetGlobalTimer("delay","LOCALS",6)
    IncrementGlobal("doomed","LOCALS",1)
    IncrementGlobal("sunfire","LOCALS",1)
    ReallyForceSpell(NearestEnemyOf(Myself),WIZARD_SUN_FIRE)
END

gcc equivalent

#define HASH1 # // special character shenanigans
#define HASH2(x) x
#define cast_spell_level(SYMBOL, TARGET, SPELL_VAR, MIN_LEVEL) \
  IF \
    See(NearestEnemyOf(Myself)) \
    Global(#SPELL_VAR,"LOCALS",0) \
    LevelGT(LastSeenBy(Myself),MIN_LEVEL) \
    !GlobalTimerNotExpired("delay","LOCALS") \
  THEN \
    RESPONSE HASH2(HASH1)100 \
      SetGlobalTimer("delay","LOCALS",6) \
      IncrementGlobal("doomed","LOCALS",1) \
      IncrementGlobal(#SPELL_VAR,"LOCALS",1) \
      ReallyForceSpell(SYMBOL, TARGET) \
  END

cast_spell_level(WIZARD_ABI_DALZIMS_HORRID_WILTING, NearestEnemyOf(Myself), abi_dalzim, 15)
cast_spell_level(WIZARD_SUMMON_EFREET, Myself, efreeti, 15)
cast_spell_level(WIZARD_SUN_FIRE, Myself, sunfire, 15)

jinja2

{% for item in data.battle_spells %} // list defined elsewhere
IF
  See(NearestEnemyOf(Myself))
  Global("{{ item.var }}","LOCALS",0)
  LevelGT(LastSeenBy(Myself),{{ item.min_level }})
  !GlobalTimerNotExpired("delay","LOCALS")
THEN
  RESPONSE #100
    SetGlobalTimer("delay","LOCALS",6)
    IncrementGlobal("doomed","LOCALS",1)
    IncrementGlobal("{{ item.var }}","LOCALS",1)
    ReallyForceSpell(NearestEnemyOf(Myself), {{ item.symbol }})
END
{% endfor %}

Of course, complex baf is already dominated by SSL, so this is mostly useful in less complex scripts.

But on the other hand, the applications are not limited to baf.

// never again confuse "GLOBAL" and "LOCALS" types
#define LocalGT(x,y) GlobalGT(x, "LOCALS", y)
IF
  LocalGT("my_var", value)
  ...


// just because I'm tired of typing COPY_EXISING_REGEXP all the time
#define copy_er COPY_EXISING_REGEXP
copy_er GLOB ~*.itm~ ~override~


// or take it further
#include header.h
// header.h:
#define POUND1 $ // more shenanigans
#define POUND2(x,y) x ## y
#define POUND3(x,y) POUND2(x,y)
#define POUND_APPEND(x) POUND3(x,POUND1)
#define copy_erg(FILE_EXT) COPY_EXISTING_REGEXP GLOB ~^.+\.POUND_APPEND(FILE_EXT)~ ~override~
// actual code
copy_erg(dlg)
  DECOMPILE_AND_PATCH BEGIN x = 0 END
BUT_ONLY


// this is too long...
IF ~~ study1
  SAY @14 
  IF ~OR(2) Global("wm_book_spell","LOCALS",0) Global("wm_book_spell","LOCALS",1)~ REPLY @20 GOTO 00   // -> +6 = 6/7
  IF ~OR(2) Global("wm_book_spell","LOCALS",0) Global("wm_book_spell","LOCALS",6)~ REPLY @22 GOTO 02   // -> +1 = 1/7
  IF ~OR(2) Global("wm_book_spell","LOCALS",7) Global("wm_book_spell","LOCALS",8)~ REPLY @23 GOTO 03   // -> +2 = 9/10
  IF ~OR(2) Global("wm_book_spell","LOCALS",7) Global("wm_book_spell","LOCALS",9)~ REPLY @24 GOTO 04   // -> +1 = 8/10
// how about this
#define if_node(value1, value2, tra, goto) \
  IF ~OR(2) Global("wm_book_spell","LOCALS",value1) Global("wm_book_spell","LOCALS",value2)~ REPLY tra GOTO goto
IF ~~ study1
  SAY @14
  if_node(0,1,@20,00)
  if_node(0,6,@22,02)
  if_node(7,8,@23,03)
  if_node(7,9,@20,04)

Technically, all this already can be done manually. But with some glue code in various places it could be automated similarly to how SSL works.

 

Edit: no one, really? OK, I guess I'll just do my own thing, as always.

Edited July 10, 2021 by Magus

[suy]

Hi. Thank you for bringing this topic up. I wanted to make public my own party AI script soon (which is mostly inspired by the existing ones, and it's not too long, as I prefer to control almost everything, it just helps with the tedious part). Very early in the development process I noticed the problem of needing replacing (for the DisplayStringHead message), and also I did notice that there are too many repetitive tasks. I knew of SSL, but it's a bit like relying on a moving target without much documentation when I am a noob in scripting anyway.

I ended up using cpp (the C/C++ preprocessor), as it's pretty ubiquitous and I'm familiar with it's syntax quite well. I also looked at other macro languages like M4, but it felt quite crazy. I went for cpp and accepted the repetition because I hate multiple line expansions and the ugly backslash at the end that it would be required for that.

Since now you mentioned text templating systems, I went for Mustache in my case. It's a "logic less" template system with tons of implementations in different languages and a formal specification. It's well stablished. The default/initial Mustache implementation (Ruby based) is a bit dated, but has a command line tool, so you can just define the data in JSON/YAML and the template. You call the tool, and it produces the output, no code involved. However, I needed a more up to date implementation to use the inheritance feature, so I went with a C++ library called bustache. I can easily make a command line tool that just reads the files from input, like SSL does, and produces output. Let me know if anyone is interested. I could even make that into a GUI application, though I don't know if that's any benefit. For my use case, I wrote a program for myself, but again, can easily made abstract so you just need files and don't need to code a thing.

Now, this is how the "code" looks.

Templates that I need to reuse verbatim, just look like the output, so like BAF:

// This is in one file that I've called "ClassesWithStealth"
OR(3)
    Class(Myself,THIEF_ALL)
    Class(Myself,RANGER_ALL)
    Class(Myself,MONK)

// This is another, named "ModalStatesDisabled"
!ModalState(DETECTTRAPS)
!ModalState(STEALTH)
!ModalState(TURNUNDEAD)
!ModalState(BATTLESONG)

The following is a helper that I need to use twice, with a small differences, and which use the previous templates by including them. This is a bit more advanced because uses a feature called "inheritance". The next template I think it's called a "parent", because you can use it like this, or "inherit" it by overriding parts. It's simpler to see it than to explain it, so let's look:

// This line is to change the delimiters. Just because in my keyboard is a pain to write
// {{foo}} (the default in Mustache and which gives it its name), so I'd rather prefer
// to write |foo|. Can be changed to almost anything you like. The next examples don't do it.
{{=| |=}}
IF
    // The notation with the dollar marks a chunk of text with a default value
    // but that can be overridden (in one case I use 0 and no extra BAF triggers, but
    // in another I add a check for enemies, and with a different variable value).
    // See the following file (in the next code block) to see how this is used.
    Global("suy#fight","LOCALS",|$fight|0|/fight|)
    Global("suy#mode","LOCALS",1)
    ActionListEmpty()
    !GlobalTimerNotExpired("suy#hide","LOCALS")
    // Likewise. The default definition on this template has no extra content,
    // but a user can override this, like if it were a function call. See below.
    |$check|
    |/check|
    // This reuses the snippets mentioned above. Just copy them verbatim.
    |>ClassesWithStealth|
    |>ModalStatesDisabled|
THEN
    RESPONSE #100
        SetGlobalTimer("suy#hide","LOCALS",ONE_ROUND)
        Hide()
END

Now with this last template defined in AttemptToHide it can be used in the main template like this:

// Just use the helper template, without overriding anything.
{{<AttemptToHide}}
{{/AttemptToHide}}

// Same helper template, but this time override the defaults to check
// for a different value of the variable, and add an extra line for an
// additional trigger.
{{<AttemptToHide}}
{{$fight}}1{{/fight}}
{{$check}}
    !See([EVILCUTOFF])
{{/check}}
{{/AttemptToHide}}

The main template follows like the next snippet. It looks 100% like regular BAF in the first block, but with the second uses one template (that we've seen above) as "helper", to avoid repetition.

IF
    Global("suy#fight","LOCALS",1)
    Global("suy#mode","LOCALS",1)
    ActionListEmpty()
    !See([EVILCUTOFF])
    Class(Myself,BARD_ALL)
    !ModalState(BATTLESONG)
THEN
    RESPONSE #100
        BattleSong()
END

IF
    Global("suy#fight","LOCALS",1)
    Global("suy#mode","LOCALS",2)
    ActionListEmpty()
    !See([EVILCUTOFF])
    OR(2)
        Class(Myself,THIEF_ALL)
        Class(Myself,MONK)
    {{>ModalStatesDisabled}}
THEN
    RESPONSE #100
        FindTraps()
END

So that's the idea. You write BAF inside a template, but when you see repetitive parts, you move them to files and include them N times. If those need to be reused but with changes, put the default values wrapped in {{$some_name}}default value{{/some_name}}. If you find the double braces too verbose, do as in the example above, and change the delimiters to whatever you like (I did change them to simple vertical bars, as you saw).

I did not need to use loops here, but that's possible as well. Normally the data is defined elsewhere though. I could simulate SSL's target/trigger blocks by having those strings in an array. The template can loop over them, with different values in each iteration.

Conclusions?

I don't have them yet. I just ported my script which was 95% regular BAF, to this templated solution, so I'm still a noob on this approach. It looks a lot clearer in a sense (things have names, and there is no repetition, so I won't go back to writing all BAF by hand, as it's too error prone), but the syntax is too verbose. Having to open and close each use of a different file is too noisy. It should look like a function call, much terser. Given this limited scope (just a simple party AI script), I probably could do writing a program in any language and save snippets in variables and use the formatting capabilities to have default values or their replacement. This seems easy to do in a bunch of languages I have experience with.

I think I can do everything in SSL with Mustache templates, but the number of pros (documented, well implemented) comes with the big, clear con of not being as concise, unless I figure a way how to do it. For example, copying this from SSL lesson six:

IF TRIGGER
TargetBlock(PCMages|PCsInOrderShort)
TriggerBlock(Paralyse|MR)
THEN DO
Action(Spell,WIZARD_HOLD_PERSON)
END

This notation (if I'm not mistaken) would entail having to merge the list of entries of PCMages and PCsInOrderShort somehow (I don't think it can be done at the template level, so it would have to be in some other logic), and likewise for Paralyse and MR. Then, make a nested loop so each combination of target+trigger is looped over, and expand the action. SSL wins hands down in being somewhat simpler once you've learnt it.

Note that I've not written anything with SSL yet, but I'll try to write my party AI script with it to compare side by side, though it will take time, and I'm not even sure it will be easier for this purpose.

Cheers, and a cookie for you if you read the wall of text.

[Magus]

Well, it's probably a no-brainer for a c++ programmer, but as for me, I don't find the syntax too intuitive. My jinja2 example above seems much clearer for casual reader.

Also not sure if I like the logic-less approach, could be too limiting for this case.

Of course, there's no reason why diffferent engines can't be used. Or even combined. But highlighting and intellisense become a bitch in that case.

In any case, I wouldn't go for hardcoding snippets. Rather, I'd publish a library to let anyone who want to use it include it.

[suy]

10 minutes ago, Magus said:

Well, it's probably a no-brainer for a c++ programmer, but as for me, I don't find the syntax too intuitive.

Well, I don't find it intuitive either. The Mustache initial implementation was Ruby, and a lot of support and proposals for improvements came from Twitter folks doing JavaScript, I think. C++ is not even a language in which is common to use template systems because it's not popular for making web pages. All the other template systems that I've seen have the problem that are their own language. So they might be intuitive initially, but you have to learn them anyway to write code. The logic-less approach may not make complete sense for generating code, I give you that. But in that case, I'd rather go with a template system that is the very opposite of this, like the one in underscore/lodash, based on John Resig's approach. That allows you to use the real language.

27 minutes ago, Magus said:

But highlighting and intellisense become a bitch in that case.

But do you have code completion, validation and so on on the language server for BAF or BAF+jinja2? I definitely have to check it out by myself, but I just don't use VS Code and I can't switch so easily.

My examples, are just the very proof of concept, and the first use I've done with this, ever. I've written very little BAF code so far. But it would not be hardcoded, as all would be files that you supplement the main template with, and you can provide as many or as few as you want.

[Magus]

22 minutes ago, suy said:

But do you have code completion, validation and so on on the language server for BAF or BAF+jinja2? I definitely have to check it out by myself, but I just don't use VS Code and I can't switch so easily.

Switch from what, notepad++? Or do you write BAF in some IDE that has no support for it at all?

Yes, MLS has some support for BAF. It's very basic, but better than nothing.

If and when there's some consensus about templating for weidu (which preferably shouldn't be limited to BAF), then extending MLS for supporting it might be considered.

[suy]

I do most of my editing in Vim (now NeoVim). I wrote the syntax highlighting for BAF myself because I could not find one online for it. 5 years ago I had to also start using an IDE for C++ development because the tooling wasn't there, and there was no LSP (and in the IDE I still use vi-mode). Now I'm considering moving that part of development to NeoVim, but I need time to invest on it. MLS is another reason to start learning about using LSP on this editor. Though, to be honest, I probably will never write stuff in WeiDU. But I still want to read code from existing WeiDU mods.

[Magus]

Oh, I see you're a man of culture as well.