tool Infinity Auto Packager - automatically generate mod packages when you publish a release

[lefreut]

On some of my repos, I use a customized version of the workflow where I run a command between the checkout and the CreateIEModZipPackage so I don't think I can use this new version.

[Austin]

On 10/6/2023 at 10:59 PM, jastey said:

@Austin I updated this for BG1NPCs in SoA&ToB.

Thank you very much!)

[Austin]

(DELETED)

Edited October 10, 2023 by Austin

[suy]

On 10/6/2023 at 11:52 PM, lefreut said:

On some of my repos, I use a customized version of the workflow where I run a command between the checkout and the CreateIEModZipPackage so I don't think I can use this new version.

I have been finishing the packaging of a mod myself, and I think I suffer the same. I have the documentation written in AsciiDoc in the README.adoc, so browsing it on Github/Gitlab/etc has a pretty frontpage, but I "compile" it to HTML so users can see that in the browser normally, instead of having to see the AsciiDoc markup.

I will probably need other steps as well in the future.

BTW, I would advice against referencing files or actions at a branch instead of a tag or commit. This breaks any chance of reproducing a build, even though might seem very convenient (to not speak of the security risks).

[suy]

On 2/25/2020 at 11:42 AM, AL|EN said:

It contains the latest WeiDU executable for Windows and macOS (for Linux is impossible) and .command file for macOS.

Additionally, I'd like to point out that this is not correct. Not only WeiDU has (as of 249) 3 different Windows versions available to download in its releases, with only 1 for Linux and Mac, but it is perfectly possible to have unique installers for Linux. AppImage has existed since long, and it's only necessary for applications which use shared libraries. WeiDU is statically linked.

[Graion Dilach]

Fair, but I'd presume you can only AppImage a weidu mod if the mod itself implemented encapsulation and doesn't attempt rewriting itself (which some mods do, for example, SotSC on being installed into EET).

That requirement doesn't apply to this library atm.

Edited October 12, 2023 by Graion Dilach

[suy]

No, I meant making just the WeiDU executable an AppImage, not the whole packaging. So, same as the Windows version.

[suy]

Additionally, I want to mention something that I am not sure if it's a bug or not, but just in case...

I wanted to see exactly the "status quo" of packaging solutions, and what they are providing, so I downloaded the latest stable release of Forgotten Armament, and compared the IEMOD package and the "zip for linux" one. Then I extracted both and, compared them, ignoring file name case, and trying to ignore the changes in line ending (which I don't get... as I don't think any change of this is necessary?). The result was a bit puzzling, as both are almost the same, but:

• The Linux ZIP version has item-set-bonuses.html, which the IEMOD lacks.
• The IEMOD version has a g3.ico and a g3banner.bmp that the ZIP lacks.

I don't understand this disparity, and I don't suppose is intended nor relevant, but just in case.

[AL|EN]

I have investigated the 'custom step' case and my suspicions were confirmed. If the state of the mod inside the repository doesn't represent a full working mod, and there is a custom action that modifies the package content, it breaks the PI "mod update" feature because it resets the local mod directory to the repository's contents from the last release tag. While there is some guidance about this, it should also be reflected here.

To summarize, if you want your mods to have rock-solid updates, don't alter the package creation in any way.

 

 

[AL|EN]

@suy

regarding the 'compile' step:

The solution to this problem is moving the custom step into a new workflow and running it on every commit along with committing changes, this might be helpful. I'm sure that you can handle this on your own.

regarding using tags:

It's against the design of this tool, especially when you need to recreate the old packages because of bugs/changes. If you really want to 'bound' IAP to a specific version, you can use commit IDs to do this.

regarding the inclusion weidu Linux executable inside the mod package:

I'm not quite sure what AppImage has to do with this so let me reword the 'impossible' statement:

"Currently, It's not possible to include both Mac and Linux weidu executables, inside a single zip mod package, because both those files are provided without file extension"

There ofc ways to handle this problem but it requires either modification of how mod launches weidu or modifying weidu code itself.

regarding the "Forgotten Armament" package difference:

The zip package is not created via IAP, it's done by G3 ModPackaging and added to release after creation. The iemod package doesn't have "*.html" files because those are added via G3 ModPackaging md2html custom action. Thanks for bringing up this problem, I will report it to the mod author and hopefully, the solution will be found.

[suy]

15 minutes ago, AL|EN said:

new workflow and running it on every commit along with committing changes

I hate that. Generated files don't belong to a repository alongside the source. This is not a good practice at all.

16 minutes ago, AL|EN said:

It's against the design of this tool, especially when you need to recreate the old packages because of bugs/changes.

If you make a new package, you change the source, hence you bump the version. I don't see a problem. Otherwise you end up in a communication nightmare: "which version are you using? x.y.z? But the old x.y.z or the new x.y.z?". This is again not a good practice.

17 minutes ago, AL|EN said:

Currently, It's not possible to include both Mac and Linux weidu executables, inside a single zip mod package, because both those files are provided without file extension

That's even more shocking (because it wasn't phrased like that in the original post, and it was biased clearly against one of the platforms) and even worse reason to say it so (because the technical solution is so easy that it's trivial). You don't need a file name convention on either Mac or Linux to run a progam, so you can just use different names. And you can use a script that delegates to one or the other, invoking it with the right arguments, so it can be easier.

And something which is new, and did not exist (I think) when you wrote the post, is that nowadays you can use the same exact file to run on all those 3 platforms. The OCaml version is called esperanto. Those are usually named 'something.com'.

[AL|EN]

@suy Those files are not simply generated, they are as you said, an internal part of the mod in order to provide easy-to-read HTML files. You update them along with AsciiDoc, the only difference is that for HTML, it's done via automation. Not having those files in the repository means that you can't immediately verify the conversion via diff, in order to make sure that the files added into the package are what you expected to be. Is this a best practice?

The 'best practice' can be evaluated only with the goals. The goal of IAP with the combination of placing mods into the GitHub repository is to offer modders rock-solid, bulletproof delta updates for their mods. Having those goals means that some compromises had to be made. You might have different goals so the 'best practices' may differ.

When it comes to IAP versioning, the 'master' branch model has proven to be working. And it's the best one when you take into consideration extra GitHub Actions stuff. The alternative you are proposing will require constant nagging regarding IAP updates, which I have no desire to do.

Regarding Linux stuff, I feel that I'm hitting a wall here. Even if I don't use Linux myself, during discussions regarding iemod format, I invest some amount of time to handle this problem. I will be very happy if you provide a package where Windows, Mac, Linux executables are present and users can launch mod installation via double-click. So far you have posted a link to the experimental project which I had no clue about, that requires changing the whole build system of Weidu, and probably changing the codebase to be able to compile the weidu using the modern OCaml version and testing everything. If you think is "easy" feel free to do it, just don't tell me that I'm biased.

 

 

[suy]

32 minutes ago, AL|EN said:

You update them along with AsciiDoc, the only difference is that for HTML, it's done via automation.

No. AsciiDoc is the input, HTML the output, and the tool is the generator. I don't update the HTML along the AsciiDoc, because I preview the AsciiDoc either directly in my browser via an extension, or on Github/Gitlab/etc. Same as any other generated file. I don't diff those, as different parameters to the generation create different outputs. I just ship it to users because they have no use of the source, but of the output. Same that I don't diff the generated assembly of the code that I just compiled.

37 minutes ago, AL|EN said:

Regarding Linux stuff, I feel that I'm hitting a wall here. Even if I don't use Linux myself, during discussions regarding iemod format, I invest some amount of time to handle this problem. I will be very happy if you provide a package where Windows, Mac, Linux executables are present and users can launch mod installation via double-click. So far you have posted a link to the experimental project which I had no clue about, that requires changing the whole build system of Weidu, and probably changing the codebase to be able to compile the weidu using the modern OCaml version and testing everything. If you think is "easy" feel free to do it, just don't tell me that I'm biased.

I feel like you are mixing stuff here.

• The IEMOD format has no binary embedded inside, like IMHO, it should be. It's just a ZIP file.
• The claim that the ZIP file doesn't have a Linux binary because "for Linux is impossible" is wrong, and I just pointed out that.
• You said that you meant something else, which is fine, but then I point out that then it makes even less sense any claim that you can't put a Linux binary (and I insist, Linux binary... Linux-only binary), because it can just have a different name than the other two. The MacOS version already has a wrapper to run it, so the Linux one could have it as well, and pass the arguments to WeiDU so it can be "double clicked to run", if anyone is interested in that. I'm not asking for you to do the work (I'm not asking anyone to do the work, because I don't think any Linux users are gonna care at all).
• Then I mention an example of an entirely different approach. Apologies for bringing that up if that was confusing, but that's an example of something entirely different that sidesteps the previous discussion because it's a single binary for many platforms. And yes, it requires working on the OCaml side of things (a lot less than what you implied, though, it's a different toolchain, which is not uncommon in compiled projects), which few are going to be motivated to even look at. So I would like to focus on the previous points.

To summarize: IEMOD seems fine to me, and I don't see why the need of a ZIP package with binaries inside, but if it has to have them, better do it right. I see more Linux users on Discord than Mac ones (though might be survivor bias), and with the Steamdeck this can go even higher.

[Graion Dilach]

Actually, this sounds like a valid enough reason to me to repack the Linux WeiDU to AppImage, because then the Linux and the Mac binary will differ on extension and can be supplied side-by-side.

[suy]

But as I was saying, you only need to make the AppImage of a binary if it uses shared libraries (or other assets), and WeiDU doesn't. It has everything statically linked. If you want to provide side by side the mac and linux binaries, just give them different names. If the issue is "but then both need to be 'setup-foo' to run automatically the 'foo' mod", then run the linux version through the shell script, the same way the mac one is run through the .command file (which just seems a weird shell script to me).