Does ChatGPT know WeiDu?

[RoyalProtector]

Out of curiosity, I asked it if it could produce a script to make Magic Missile do 1 extra point of damage per projectile. This was the result:

// Define the spell resource name
DEFINE_ACTION_FUNCTION modify_magic_missile
BEGIN
    // Extract the spell resource
    COPY_EXISTING ~spwi405.spl~ ~override/spwi405.spl~

    // Read the spell's properties
    READ_SHORT 0x1e num_projectiles
    READ_BYTE 0x20 projectile_dice
    READ_BYTE 0x21 projectile_sides
    READ_SHORT 0x22 projectile_bonus

    // Modify the damage
    WRITE_SHORT 0x22 (%projectile_bonus% + 1)

    // Write the modified properties back
    WRITE_SHORT 0x1e %num_projectiles%
    WRITE_BYTE 0x20 %projectile_dice%
    WRITE_BYTE 0x21 %projectile_sides%
    WRITE_SHORT 0x22 %projectile_bonus%

    // Display a message indicating the modification
    SAY NAME1 @100
    SAY ~Magic Missile damage increased by 1 point.~

    // Apply changes
    COMPILE EVALUATE_BUFFER
    COPY ~override/spwi405.spl~ ~override/spwi405.spl~

    // Clean up
    DELETE_FILE ~override/spwi405.spl~
END

ACTION_IF FILE_EXISTS_IN_GAME ~spwi405.spl~ THEN BEGIN
    // Execute the modification function
    modify_magic_missile
END ELSE BEGIN
    // Display error message if the spell file is not found
    SAY ~Error: spwi405.spl not found!~
END

Is this even remotely accurate? Anyone has had some success getting coding help from AI for WeiDu? Just curious. Cheers.

Edited March 16 by RoyalProtector

[DavidW]

It’s superficially-plausible gibberish.

[argent77]

That's a good example how deceiving ChatGPT's answers can be. At first glance this script looks plausible (except for two glaring syntax errors), but the code itself is nonsense and looks more like a mishmash from several different (and unrelated) mod scripts.

[RoyalProtector]

The examples given for mainstream, general-purpose programming languages tend to be on point more often, although they sometimes miss crucial context experienced developers would take into account.

I guess WeiDu being a fringe programming language, the examples would degenerate significantly, since it was fed much less information.

I was kinda hoping I could use it as a useful learning tool, but it doesn't look like that will work out. Oh well

[jmerry]

It's not the first time I've seen ChatGPT fail at producing code in this modding scene's obscure languages. Here's a juicy example (I took apart one chunk of it in detail): https://forums.beamdog.com/discussion/88152/need-help-in-making-vampire-kit-spells-scripts-creatures-items

Anyway, for this one...

- SPWI405 isn't Magic Missile. It's Improved Invisibility (in the BG series). So right off the bat, we're editing the wrong spell.

- Then we read some binary data. Two bytes at 0x1E (some of the kit exclusion flags), one byte at 0x20 (more kit exclusion flags), one byte at 0x21 (the last of the kit exclusion flags), and two bytes at 0x22 (the casting animation). Values 4096, 0, 0, and 13, by the way.

- Then we write something. Add one to the number at 0x22. Now the spell has a Conjuration animation instead of Necromancy.

- Then we write more things. In the first three cases, write what's already there. In the fourth, undo the change we just made in the last step.

- Then, in the name of printing an unearned victory message, we bring in a string @100 (OK, I'll assume that there's a tra file somewhere for that to point to, as that's the more entertaining option). Write it at the NAME1 offset. Which is actually the name of the spell. So we've renamed "Improved Invisibility" to ... something.

- The next line is a glaring syntax error; SAY needs two arguments, and only one is provided. The first argument has to be an integer, and that string can't be converted. So the code crashes here. All right, let's proceed as if the "SAY ~Magic Missile damage increased by 1 point.~" doesn't exist.

- The line after that is another glaring syntax error. COMPILE ... what? EVALUATE_BUFFER is a modifier, not the argument that action needs. All right, ignore that line too.

- Now, we close the spell (by performing an ACTION). That action is to immediately open up the same thing we were just working with. I really don't see the point.

- Finally, the DELETE_FILE command tells us to get rid of everything we just worked on. Except - no, that command doesn't actually exist in WeiDU. So that's a third syntax error, a bit subtler than the last two.

- OK. We've just defined a function. Which, contrary to what it's called, renames Improved Invisibility. At least, if we ignore the syntax errors. Now, we have a code snippet that invokes the function. I'll be generous and suppose this is part of a mod component, and we just don't have the boilerplate around it.

- If Improved Invisibility exists, "modify_magic_missile". Wait. You need to LAUNCH functions in WeiDU. That's another syntax error. And what if the spell doesn't exist? Well, then we SAY something. Which is a syntax error for two reasons: that there's only one argument, and that SAY is a PATCH and we don't have a file loaded.

So, as a summary, there are five syntax errors, and the parts that are technically functional have absolutely nothing to do with the stated purpose.

[Isewein]

As a "modern-day luddite", I somehow find this quite comforting.  

[Yovaneth]

I've also seen it lift WHOLESALE code from other language projects. In real life I also program flight simulators - specifically Lockheed-Martin's Prepar3D platform. I hit a bit of a narly problem with a C++ function and having found the information I needed (on Stack Overflow!) to get me past my codeblock, just for the hell of it I threw the question at ChatGPT. Well, f*** me if it didn't throw some my own publically-available code back at me!!!! Of course it was wrong.

Oh yes; I wish people would stop calling it AI. It's not Artificial Intelligence because that name/statement implies self-awareness. It's just an extremely well written Turing machine.

Edited March 16 by Yovaneth

[RoyalProtector]

26 minutes ago, Yovaneth said:

Oh yes; I wish people would stop calling it AI

Hah, funny, I was having that same conversation today with a friend. I agree. It's a fancy, complicated and useful machine that sort-of mimics human intelligence, but it doesn't have self awareness, and/or reasoning capabilities (really). People got a bit too insane with the claims about AI. I also think the claims that it will take their jobs are overblown, but I digress.

1 hour ago, jmerry said:

So, as a summary, there are five syntax errors, and the parts that are technically functional have absolutely nothing to do with the stated purpose

My goodness. Total disaster!

Edited March 16 by RoyalProtector

[jmerry]

Of all the errors in the snippet provided, I find the DELETE_FILE one particularly vicious. Because, even as someone who knows WeiDU pretty well, I had to search the documentation to spot that one. The real command is just DELETE, by the way. With a big warning in the documentation to not use it without a good reason.

[paladin84]

Yes, I tried at some point ChatGPT for weidu coding. The thing is that it is able to help you with the mainstream languages, because there lots of code online ChatGPT was studied on. There is not enough weidu code online, so it can try to create somewhat realistically looking, and probably "understand" basic ideas of programming, but weidu syntax is unknown to it. You can actually make it provide working weidu code, but you need to explain all its mistakes and ask to fix it. Like, for the first error, you should say: "SPWI405" is not a Magic Missile, but "proper_resource_name" is, fix the code to use the proper resource name.

And then after 20-40 iterations you probably going to have a code, that does what you want.

Edited March 17 by paladin84

[RoyalProtector]

Sounds barely worth it, but I get it haha.

[polytope]

In my impression, generally speaking, the more niche the topic, the more technical and specific the question the less likely the language learning model (or machine learning in general) can produce anything that applies the body of knowledge appropriately, let alone specifically delivers the answer you were looking for. A fan's interface for modding an old computer game is definitely not something it's likely to be able to work with.

This applies more generally, remember last month's story about the AI illustrated article in Frontiers in Cell and Developmental Biology? People were laughing at how it attempted to depict the rat's testicles, but in fact the depiction of the JAK-STAT signalling pathway is also a total mess, because it's a little known subject except to immunologists and hasn't been described and illustrated enough times for an artistic AI to train on it, so the AI produces a lot of nonsensical filler instead of a useful diagram.

[jmerry]

Yeah. The AI-generated pictures can figure out how to replicate the style of a scientific illustration, but don't have a clue that it's meant to convey accurate information. Plus picture-generating AIs in their current form are very bad with text. So you get labels that superficially look like words but aren't real words, and often contain glyphs that aren't in any standard alphabet.

Oh, and all of those pictures were total messes. The rat testicle illustration was just the easiest one to laugh at. (Hey, that picture managed to have one accurate label: "Rat")

[RoyalProtector]

I've seen my share of screwups with image generators, particularly generators of portraits or character concepts. The most common issues in my experience are incorrect number of fingers, awkward hand holding of objects, messed up eyes, unnatural positions, lack of continuity in objects that are partially covered by something else, and sometimes also inconsistent skin tones or eye colors when the prompt is very specific about it. My Pinterest has some examples of this though I mostly upload things after fixing them.

It has issues but it also can produce a lot of good stuff. Thankfully I'm good enough to fix the majority of issues with Photoshop.

The part about text is very true, I've tried generating things like "evil priest of Shar" to find the generator very poor attempt at writing that very sentence at the bottom of the image...

Which of course I had to remove. With AI I might add. Photo editing gets more consistently better results in my experience, particularly when removing stuff from a picture. Generating, not as good.

But I digress.

Edited March 17 by RoyalProtector

[Bartimaeus]

"AI" might be useful for coding one day, but it'll be through models trained specifically for and ONLY on whatever language you're trying to get it to do something for - not ChatGPT, which is just a language model that has no fundamental understanding of any of the code that it reads or writes, so of course it just spits out vaguely similar-looking pseudo-code. Garbage input leads to garbage output, and generalized models like ChatGPT are 100% pure unadulterated garbage input.

Edited March 18 by Bartimaeus