Things that prevent WeiDU successor or alternative?

[mickabouille]

When confronted with regexes, I always lament the weidu support, which is as far as I understand, limited by the ocaml regex support.

I thought it would be probably better to just fork any external interpreter (probably one with pcre engine) but suspect nobody would actually go that way after all

[marco_costa]

My hunch is also that it would benefit more from incremental improvement than a rewrite, but Ocaml isn't a very popular language and the current build tree isn't very legible to modern tooling and IDEs - I remember playing with it a year back to try and convert the build from make to Ocaml's "official" build system Dune, which would make pinning compiler for the build, IDE integration, autoformatting, etc all easier, made some progress but couldn't quite get the executable out

SFO's example for the problem presented indicates WeiDU could use a 'standard library' to provide a good way to handle the stuff that's doable in WeiDU but involve a lot of boilerplate or footguns. Seems like a lot of mods come with 'helpers' of their own anyway.

[AL|EN]

On 3/12/2024 at 5:07 PM, Incrementis said:

This is something I need to test the mods. If the successor doesn't have that, I don't know if I want to use it.

Correct me if I'm wrong, you specifically have 'an installation routine as of:

1. create tp2, install it via setup-mymod
2. launch the game, verify results and quit
3. make edits to tp2, reinstall mod via setup-mymod
4. launch the game, verify results and quit
5. repeat

right? Can I assume that this is the 'spoiled' part?

On 3/12/2024 at 2:49 PM, mickabouille said:

For me that’s the only real blocker.

In my (really limited more proof-of-concepty) experiments with this, I just generate a weidu mod that I then install to keep weidu happy.

Never went really far, really, just generating fixup minimums without needing to remember the baroque syntax of tpX files.

I'm assuming that you creating this "fake tp2" is for mods-in-the-middle? What if the 'backup' system were decoupled and externalized from WeiDU? Would this solve the problem of installing mods/uninstalling only the last one/uninstalling only a few?

[Incrementis]

1 hour ago, AL|EN said:

Correct me if I'm wrong, you specifically have 'an installation routine as of:

Yes, that is exactly the basic routine that I go through. I'm trying to reduce these steps by using Near Infinity. Basically I use WeiDu for implementation purposes and NI for debugging/analysis/testing purposes.

[mickabouille]

Quote

I'm assuming that you creating this "fake tp2" is for mods-in-the-middle? What if the 'backup' system were decoupled and externalized from WeiDU? Would this solve the problem of installing mods/uninstalling only the last one/uninstalling only a few?

1. Yes

2. Probably... Maybe?

 Yes I guess that would help not having to generate a tp2 file just to copy over the generated resources.

Though the .tp2 generation is not the hardest part as it has nearly no logic inside by itself, just a COPY instruction.

[AL|EN]

5 hours ago, Incrementis said:

Yes, that is exactly the basic routine that I go through. I'm trying to reduce these steps by using Near Infinity. Basically I use WeiDu for implementation purposes and NI for debugging/analysis/testing purposes.

So you don't even start the game, you just refresh resources inside NI and check/verify the desired outcome, correct?

Edited March 16 by AL|EN
typo

[subtledoctor]

AL|EN you are being very coy here… the thread is already on the third page. 

[Incrementis]

48 minutes ago, AL|EN said:

So you don't even start the game, you just refresh resources inside NI and check/verify the desired outcome, correct?

It depends on what is needed or what I want to do.

Let me do a simple example to make things a little more vivid.

When I create an item mod, I first create a detailed concept. I implement things step by step in WeiDU, using the steps you already described to validate changes in the game:

1. create tp2, install it via setup-mymod
2. launch the game, verify results and quit
3. make edits to tp2, reinstall mod via setup-mymod
4. launch the game, verify results and quit
5. repeat

If the item doesn't behave as expected or the installation fails and I have a strong suspicion as to why, I make the changes in WeiDU without using NI and follow steps 1 through 5.

When I need to troubleshoot problems that require a lot of changes, to validate the item's behavior, I use NI to isolate the problem. In this case, it is not necessary to reinstall the mod, but to restart the game with every change. The problem in this case is outside the concept or my understanding of why I use NI.

I hope this makes things a little clearer and more understandable.

 

 

[AL|EN]

3 minutes ago, subtledoctor said:

AL|EN you are being very coy here… the thread is already on the third page. 

I needed some time to gather my thoughts and reflect on them, especially when it comes to the different proposal and their scale. But hey, as you wish  

The number of active WeiDU-only developers is decreasing. Modding with WeiDU involves programming, and there are many programmers out there. We just need to attract them. One of the ways is by allowing modders to use other tools alongside WeiDU. I strongly believe that this could be very beneficial. Creating a WeiDU-like library in Python, C#, or Java could attract hobbyists and professional programmers. They could develop mods using their favorite language or technology, bringing expertise, knowledge, and better tools that could result in better mods. I don't think sticking solely to WeiDU without even considering alternatives is wise. Imagine if the authors of ToBEx/EEEx had thought, "It's not WeiDU, so I'm not going to do it," etc.

In my opinion, it's not about the amount of time needed to learn WeiDU. It's about the fact that you can't use it elsewhere for any other purposes than modding IE games. Even in the hobbyist/professional developer world, language syntax and feature sets are the main reasons why some programmers won't touch another language at all. Of course, some use them all, but that's the minority.

Possible approaches:

- Developing a WeiDU replacement that functions and behaves like WeiDU and uses the same tp2 syntax.
  Apparently, this is in progress by Wisp. https://forums.pocketplane.net/index.php?topic=30011.msg340720#msg340720

- Creating a WeiDU replacement that works like WeiDU but uses tp2 syntax with some additional improvements, also known as tp3.
  Instead of revolution, it's evolution. Depending on the implementation, this could also be what Wisp is currently working on.

- Introducing small, side-by-side tools that are alternatives for specific parts of WeiDU, called by WeiDU itself or outside of it.
  This doesn't pressure the author to be the "next big thing." You can quickly check what works and what doesn't. In the future, you can replace that small part of the file processing with yet another thing. Unlike one big monolith, it has a low cost of making improvements that are breaking changes. I like this approach.

- Developing a WeiDU side-by-side alternative that can do what WeiDU can but doesn't use the same syntax and is not compatible with WeiDU.
  It's a lot of work, yet according to DavidW, "90% of mods use only 10% of WeiDU," so maybe it's not as daunting as it seems? It would be nice for a new tool to have lots of good stuff and/or a so-called "killer mod/feature" to have chances to attract modders.

- Combining the current WeiDU, WeiDU side-by-side alternative, and side-by-side tools with the freedom to call all of them outside of WeiDU at any given time.
  This allows for total freedom in using old and new tools to modify game resources. An easy uninstall/reinstall procedure is pretty much required for a modder's workflow.

I would love to play with the IE engine. Unfortunately, WeiDU puts me off. I don't want to spend another year learning WeiDU only to find that working with it is cumbersome. I would rather use anything else. But going against everything may not be the right solution. Since no alternative to WeiDU has emerged then there must be some deeper reasons/problems for it. Now at least I know that besides binary file modifications, dialogs and installation/reinstallation system are equally important. etc.

 

[DavidW]

I repeat my previous observation that mere unfamiliarity is not by itself a good reason to change WEIDU. If you are a competent programmer then it should not be difficult to learn a new language, nor time-consuming compared to the time to execute a complex mod. (I learned LUA in an afternoon.) If you are not a competent programmer then you can’t make a mod that involves complex programming anyway (and it is trivial to learn enough WEIDU to install premade files.)

[paladin84]

Sure, you can study everything, providing you have enough time and/or motivation. And then you write 20 lines of code using this "unique, unlike other" language while it would be enough to write 10 lines of code for the same functionality using mainstream languages. Looking at weidu code discourage me enough, so I don't have enough motivation to study it.

[DavidW]

I don’t think WEIDU is a “unique, unlike other” language. It’s an imperative weakly typed language with basic support for functions and a mildly awkward syntax. Its regexp and control flow is pretty standard. Its array support is slightly idiosyncratic but not radically so. Its low-level I/O is unusually good and I’ve actually used it occasionally for work for that reason. It’s not the language I’d choose to do a general-purpose programming task but it’s perfectly functional.

[RoyalProtector]

From my outlook as a developer, the impression that WeiDu gave me is that it's annoyingly low-level, without much in terms of high-level utilities. And very limited in terms of design.

Whenever I look at the code, all I can think of is something along the lines of "I wish I could encapsulate every annoying thing here, and every convoluted stuff in high level sets of classes and functions, and never have to touch it otherwise."

Heavens and hells, I'll take C and C++, at least there is more high level stuff there.

But I don't understand what the point of the discussion is regardless, since it doesn't seem like it's possible.

Edited March 16 by RoyalProtector

[DavidW]

46 minutes ago, AL|EN said:

I don't want to spend another year learning WeiDU only to find that working with it is cumbersome.

It won’t take you a year to learn WEIDU. At least to understand it as a programming language, it will take you an afternoon. What takes somewhat longer is understanding the various built-in WEIDU commands that manipulate IE objects at a low level. But any language will have to manipulate those objects at a low level unless it is actively enriched with functions that manipulate them at a high level. And no such functions exist in other languages.

If the issue is a lack of high-level tools to interact with IE objects then I sympathize. But that’s not really about WEIDU per se, it’s about the best way to build those tools. 

[DavidW]

14 minutes ago, RoyalProtector said:

From my outlook as a developer, the impression that WeiDu gave me is that it's annoyingly low-level, without much in terms of high-level utilities. And very limited in terms of design.

Whenever I look at the code, all I can think of is something along the lines of "I wish I could encapsulate every annoying thing here, and every convoluted stuff in high level sets of classes and functions, and never have to touch it otherwise."

That’s more a feature of how people use WEIDU. You can code in a much more high-level way - look at ToF sometime - though to be sure it’s not as fully-featured as a proper modern language.

Mostly when I code low-level it’s for speed.