Jump to content
AL|EN

Mod metadata - brainstorming

Recommended Posts

Hi,

let me start from make clear statement that NOTHING IS REQUIRED. Did I say that nothing is required? Yes, that's true, nothing is required. You don't have to do anything. My proposition/design also aims for be "less selfish as possible" aka "don't require anything and if, require minimal amount". I hope that's clear now.

The main idea behind this is to give back the control of mod metadata in the hands of modders. Who else know the current and most up to date mod name, description or the working link to the place where players could receive support? That's right, the modder itself.

The first idea was to have a website where modders could provide mod metadata. Creation of such website is very simple and can be done in one day. But then there is a problem when some other people will provide 'malformed' info about other people mods. Trolling is an art. It's impossible to counter without some sophistic login/verification method. And ofc "yet another website" will be instantly boycotted, even if it would be offer a Genie in a bottle.

So without website, similar to "mod configuration file", a mod could provide few important metadata like:

- Full name of the mod itself, without version number
- Mod description
- a direct download link to the latest version of the mod (which is intend to be used by players so no ALPHA/BETA/DEV)
VERSION is dynamically read from tp2. All other missing info I can read from tp2: AUTHOR/README, but if they are inside INI, INI will have preference over tp2.

This is my take on this matter:

0. File encoding: It can be ANSI if it won't offer to write Unicode strings. UTF8 no BOM, if it does.

1. File format of such file must be 'industry standard', one for all mods. And please forget about xml :p I can't write a custom parser for each mod only because it's author like to put spaces before and after '=' character. Or use TAB to separate key and values. Or include weidu commands which the tool can't understand. It would be too much work and errors. So let's choose one, modern 'industry standard' file format. I vote would be for INI as a first choice and HJSON/JSON5/JSON (JSON doesn't' allow comments) for more advanced modders who understand benefits of such modern format.
 
2. File extension of 'mod metadata file': Almost irrelevant, follow the file format definition associated extension. Those files shouldn't be modified by players anyway.
 
3. Filename of such 'mod metadata file'. There is one, very simple solution: <modTp2name>.ini/HJSON/JSON5/JSON. This is what is currently implemented.
- it doesn't require file header / ini sections added [Mod] [Metadata] to avoid false detection
- it prevents people from making multiple, outdated version of this file
 
Other solutions require file headers/scanning mod directory and parsing files etc.
 
4. File content: nothing is required, you can even put one, single line Key=value pair with only mod full name. One single line.

INI:

Name = NPCs Enhanced for Everyone
Author = Argent77
Description = This is intended to be a successor to the fantastic "Level 1 NPCs" mod. L1NPCs was never updated for the EE games, and its author is no longer active, and it is an incredible, complicated piece of work that is difficult for someone else to update. So I made this.
Download = http://lynxlynx.info/ie/modhub.php?UnearthedArcana/NPC_EE

Remarks:

For some modders, tool could use industry standard JSON file or JSON5/HJSON, it has very strict rules, only for modders who wan't to benefit from the advantages of the JSON file format itself. Order of the preference: JSON > JSON5/HJSON > INI . It means that if you provide only INI, everything works. Later, if someone wan't to switch to more advanced metadata format, mod metadata will be read from them. I think that INI would be enough but I don't want to limit modders.

5. Presentation to user
- mod metadata will be displayed inside "Mod info textbox"
- in addition, mod name will be displayed at treeview/mod list
- mod downoad link would be used to download new mod version (if it is supported)
 
68747470733a2f2f692e696d6775722e636f6d2f
 
 
outdated proposal
tp2:
AUTHOR ~SubtleD~

ini:

Name=Scales of Balance
Download=http://lynxlynx.info/ie/modhub.php?UnearthedArcana/Scales_of_Balance
Information=http://forum.baldursgate.com/discussion/33657/scales-of-balance-post-hac-kits-and-tweaks
Collected data:
Name=Scales of Balance
Author=SubtleD (extracted from tp2)
Information=http://forum.baldursgate.com/discussion/33657/scales-of-balance-post-hac-kits-and-tweaks
Download=http://lynxlynx.info/ie/modhub.php?UnearthedArcana/Scales_of_Balance
 
Example of INI with all metadata provided:
tp2:
AUTHOR ~email@example.com~

ini:

Author=subtledoctor (tp2 AUTHOR is ignored)
Name=Scales of Balance
Information=http://forum.baldursgate.com/discussion/33657/scales-of-balance-post-hac-kits-and-tweaks
Download=http://lynxlynx.info/ie/modhub.php?UnearthedArcana/Scales_of_Balance

Collected data:

Name=Scales of Balance
Author=subtledoctor (tp2 AUTHOR is ignored)
Information=http://forum.baldursgate.com/discussion/33657/scales-of-balance-post-hac-kits-and-tweaks
Download=http://lynxlynx.info/ie/modhub.php?UnearthedArcana/Scales_of_Balance

That's everything which comes to my mind. I'm sure that everyone will have their own vision and ideas of how it's should be done etc. If I miss some keywords/mod metadata, please let me know.

EDIT: Fixed post duplicated new lines after forum engine update.

Edited by AL|EN

Share this post


Link to post

Where does the key "Information" point to for a mod that has a readme, a forum, and a website all hosted online, and not necessarily at the same base URL?

 

Also, if you mandate everyone conform exactly to "an industry standard", you must point to strict documentation of that standard. Preferably, the standard and documentation would be rigorously maintained by a recognized authority. Ideally, the format chosen would be established and widespread enough to be *natively* supported by many programming languages, and IMO be human-readable.

Share this post


Link to post

Where does the key "Information" point to for a mod that has a readme, a forum, and a website all hosted online, and not necessarily at the same base URL?

 

Also, if you mandate everyone conform exactly to "an industry standard", you must point to strict documentation of that standard. Preferably, the standard and documentation would be rigorously maintained by a recognized authority. Ideally, the format chosen would be established and widespread enough to be *natively* supported by many programming languages, and IMO be human-readable.

 

Ad 1.

It display the the value. So if you enter homepage, it will display homepage. You want three separate keys for online readme, online forum/post and homepage? No problem :) You have to define extra ini keys, I would need to add them to parser and "Mod Info box" it will display all of them.

 

Ad 2. Good point:

INI: https://en.wikipedia.org/wiki/INI_file

JSON: Definition https://www.json.org, Validator https://jsonlint.com/

JSON5: Definition https://json5.org/, it's just the same as JSON with possible comments and less rules, no online validator yet :/

HJSON: Definition: it's just the same as JSON with possible comments and less rules, online validator http://hjson.org/try.html

 

All of those formats are "human-readable".

Edited by ALIENQuake

Share this post


Link to post

It would be a very bad idea to write this on the assumption that [mymod].ini is metadata. Doing so would be to reserve a chunk of namespace inside people's mod folders that has no particular connection to the purpose (you can't tell from the fact that it's called [mymod].ini that it's metadata) and that conflicts with already-existing usage in at least some mods (notably SCS).

 

This is bad practice in an abstract space, but more importantly it runs a serious risk of false positives, i.e. of a file being parsed by your autoinstaller as if it's metadata (or, worse, configuration options) when it was not intended to be parsed that way by the writer. (If you had implemented this proposal without me being aware of it, you might well have ended up offering SCS's and Wheels of Prophecy's debug options to the end user as installation choices.)

 

If you want to do this (and it sounds sensible) then you need conventions whereby files only get treated as processable metadata if they are explicitly and intentionally marked as such by the mod maintainer (or, I guess, by a BW Fixpack change if we're talking about not-currently-maintained mods). There are lots of ways to do that:

 

1) My preferred option, as I said on the other thread, is to embed a keyword in the comment field of the file, as in

;;;IE_METADATA_FILE;;;
2) Alternatively, you could mark a file through a particular key-value pair, as in

treat_this_as_ie_metadata=1

3) If you really want to insist on doing this through namespace, pick a bit of namespace that you can be really confident isn't going to be used except for the intended purpose. metadata.ini is the obvious choice; using a .metadata suffix would also work. Theoretically this is less safe than either of the first two proposals; in practice I doubt you're going to get anyone who's already (e.g.) using "metadata.ini" to indicate something that's not metadata.

Share this post


Link to post
On 9/16/2018 at 6:06 PM, DavidW said:

It would be a very bad idea to write this on the assumption that [mymod].ini is metadata. Doing so would be to reserve a chunk of namespace inside people's mod folders that has no particular connection to the purpose (you can't tell from the fact that it's called [mymod].ini that it's metadata) and that conflicts with already-existing usage in at least some mods (notably SCS).

 

This is bad practice in an abstract space, but more importantly it runs a serious risk of false positives, i.e. of a file being parsed by your autoinstaller as if it's metadata (or, worse, configuration options) when it was not intended to be parsed that way by the writer. (If you had implemented this proposal without me being aware of it, you might well have ended up offering SCS's and Wheels of Prophecy's debug options to the end user as installation choices.)

 

You concerns about 'false-positive detection" are valid. I might have been gone to far in terms of "require nothing". That is why I wanted to receive feedback from modders. You "metadata.ini" is a good suggestion. The name reflects it's purpose. With one minor defect: It would be impossible to store multiple metadata files inside one directory. So how about <*-metadata>.ini ? So it could be "modtp2name-metadata.ini" but it doesn't have to be. The important pat is thee 'metadata.ini" part. I think it's very safe and no modder will create such file by accident.
 
As an alternative to you comment/key-value, how about using a INI "Section", let's call it [Mod] or [Metadata] inside the ini file of any given name, which will be inside main mod folder, where tp2 file is:
[Metadata]
Name=NPCs Enhanced for Everyone
Description=This is intended to be a successor to the fantastic "Level 1 NPCs" mod. L1NPCs was never updated for the EE games, and its author is no longer active, and it is an incredible, complicated piece of work that is difficult for someone else to update. So I made this.
Download=http://lynxlynx.info/ie/modhub.php?UnearthedArcana/NPC_EE

 

Such method can also be applicable to JSON/HJSON/JSON5.
 
I believe that combining those two requirements could be very, very safe and no false-positive detection could occur. Unless there will be a major opposition to use INI Section, and a clear message from other modders that "filename schema will be enough" I would like to go for it.
Edited by AL|EN

Share this post


Link to post

Ad 2. Good point:

INI: https://en.wikipedia.org/wiki/INI_file

JSON: Definition https://www.json.org, Validator https://jsonlint.com/

JSON5: Definition https://json5.org/, it's just the same as JSON with possible comments and less rules, no online validator yet :/

HJSON: Definition: it's just the same as JSON with possible comments and less rules, online validator http://hjson.org/try.html

 

All of those formats are "human-readable".

The wikipedia page for INI doesn't mention it, but I'm pretty sure any language relying on Windows API to read INI files will limit the length of a key's value to 65,535 characters. This presumably includes AutoIt (and so BWS) and AutoHotkey. Just a tidbit to keep in mind in the event of rather verbose descriptions...

Share this post


Link to post

Such method can also be applicable to JSON/HJSON/JSON5.

You forgot that the file needs to work in weidu to be read from ... now, of course if you can get Wisp to allow such bastardizations... Edited by Jarno Mikkola

Share this post


Link to post

Forget about BWS and it's limitations, I'm not loading the content of the INI using 80's win32 API. But you really would need to put whole book into "Description" value :p

Edited by ALIENQuake

Share this post


Link to post

In what case would a mod want to include more than one metadata file? Sounds like an unnecessary complication.

 

Since you're already parsing weidu — why not store all of it there, just like author?

Share this post


Link to post

Two metadata files can coexist to provide different kind of metadata: for a mod information and for mod dependencies ;)

 

I requested new keywords for tp2 long time ago. The wisp conclusion was that the tp2 code is not a good place for those information. Metadata files are much better suit for such data. Code vs Data separation: modification of the metadata it can be added/modified without touching tp2 file. They are industry standard, parser and tools are very solid etc.

Edited by ALIENQuake

Share this post


Link to post

In what case would a mod want to include more than one metadata file? Sounds like an unnecessary complication.

 

Since you're already parsing weidu — why not store all of it there, just like author?

Good question and my answer, of course, is that you wouldn't... well, perhaps in the case of Level 1 NPCs NPC-components, you could see it, as that's per NPC, but other wise, nope to me.

I intend for the file to contain all the variables I use as the inserts ... so it needs to complete as a weidu's .tpa file, which is just a part of a .tp2 injected in, so it needs to follow the regular .tp2 coding. Dah... which is what I refered in the above post.

We don't care if another program can get something out from it.

Share this post


Link to post

Still sounds like needless proliferation.

With a fixed name it's also always obvious what's inside (and less chances for misspelling/miscasing).

 

What problem does it actually solve? If the metadata is stored with the source, one always has to download it to check if it changed.

Share this post


Link to post

Still sounds like needless proliferation.

With a fixed name it's also always obvious what's inside (and less chances for misspelling/miscasing).

 

Well, that's valid too, but unless you convince DavidW, I need to keep "<modname>-metadata.ini" scheme.

 

 

 

What problem does it actually solve? If the metadata is stored with the source, one always has to download it to check if it changed.

 

The source of the mod metadata is the mod itself, not some external database.

Edited by ALIENQuake

Share this post


Link to post

 

Still sounds like needless proliferation.

With a fixed name it's also always obvious what's inside (and less chances for misspelling/miscasing).

 

Well, that's valid too, but unless you convince DavidW, I need to keep "<modname>-metadata.ini" scheme.

 

What do you need to convince me of? I'm neutral as to whether you need multiple metadata files or just one - going from "metadata.ini" to "xx-metadata.ini" was your idea. I'm also fine if you want to read metadata out of mymod.ini provided there is a completely unambiguous way to designate it as metadata that won't lead to false positives - your suggestion of using .ini section headers is less hacky than my ideas.

 

I don't think I'd seen "store the metadata in the tp2" mentioned before, but I'm strongly against it - code/data separation, please.

Share this post


Link to post
Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...