Issues with item icons

[Sam.]

I have done some initial research into the current state of the item (inventory) icons in the Baldur's Gate series of games:  oBG1 (AKA TotSC v1.3.5521), oBG2 (AKA ToB v2.5.26498), SoD v2.6.6.0, and BG2EE v2.6.6.0.  I have attempted to capture all item icons, regardless of whether or not they are explicitly used by any item files.  If you find any I have missed, please bring it to my attention.  It is also likely a few BAMs which are not item icons have slipped through my filtering into this list.  Included are a list of warnings given by PS BAM for each item icon, along with a separate estimation of the primary palette index used for the shadow (which is assumed to exist) on the large item icon.  Here is the comparison.

Exactly how a BAM is rendered depends on the game engine and engine version, game configuration settings, context of usage within the game, computer hardware, and also sometimes graphics settings at the OS level.  After more than a decade of studying BAMs and how they are rendered, the following are some general rules to follow to ensure the optimal display of a BAM under all possible scenarios.

1. The transparent color should be pure green RGBA(0,255,0,0) at palette index 0.  No other palette entry should be this color; therefore, no pixel that should not be transparent should use this color.
2. The shadow color should be pure black RGBA(0,0,0,0) at palette index 1.  This color at this palette entry is generally rendered as ~50% transparent.  No other palette entry should be this color; therefore, no pixel that should not be rendered as a semi-transparent shadow should be this color.
3. Outside of the IE, an alpha value of 0 in 32-bit images indicates that color is completely transparent and an alpha value of 255 indicates that color is completely opaque.  However, for backwards compatibility purposes in a BAM's palette, an alpha value of 0 is treated as completely opaque whereas an alpha value of 1 is almost completely transparent.  As a result, any program intending to render a BAM must scan the alpha values in the palette.  If any alpha value is non-zero, the BAM is rendered as 32-bit (and any alpha values of 0 are made 255).  However, if all alpha values in the BAM's palette are 0, then it can be directly rendered as a 24-bit image.  The takeaway is that there is no functional difference between a BAM palette using alpha values of 0 and 255, but using 0 can result in more efficient rendering at no cost and with no functionality sacrificed.  This is relevant only for EE games.

Additionally, the following best practices specifically apply to item icon BAMs.  For reference, traditionally an item icon consists of two frames.  Sequence 0 contains the large item icon (with a shadow) that is displayed when the item is picked up in the inventory.  Sequence 1 contains the small item icon (without a shadow) that is displayed when the item is placed in an inventory (including backpack) slot.

1. The large item icon (Sequence 0) must be no larger than 64x64 px.
2. The X and Y offsets of this frame are calculated according to the following rules:
   •  CenterX=Width//2
   • CenterY=Height//2
3. In the classic engines, the small item icon (Sequence 1) must be no larger than 32x32 px.
4. The X and Y offsets of this frame are calculated according to the following rules:
   • CenterX=(Width-32)//2
   • CenterY=(Height-32)//2
5. In the EEs, the "small" item icon (Frame 1 in Sequence 0) must be no larger than 64x64 px.
6.  I suspect (can't verify as offsets are ignored) the X and Y offsets of this frame should be calculated according to the following rules:
   • CenterX=(Width-64)//2
   • CenterY=(Height-64)//2

In the above, // is a floor divide (drop the remainder)

The EEs do not use the small item icon and BG2EE has, in general, dropped this frame from the BAMs entirely.  Instead, the EEs use the large item icon (with the shadow) for both when the item is picked up and when it is placed in an inventory slot.  However, the classic item icon behavior can be restored by adding the small item icon as a second frame to Sequence 0.  Doing so is backwards compatible with the classic engines.  In oBG1 and oBG2, the small icons filled the inventory slot and would become bigger than the slot when picked up.  In the Enhanced Editions, the inventory slots are larger (doubled in size).  As a result, the smaller frame from the original icons is too small, even when used.  The EEs use the larger frame in order to fill the larger slots, which are now sized to fit an image up to 64x64 px (used to be 32x32 px).

Issues to consider

At the engine level:
IIRC, the EEs used to (pre v2?) calculate the inventory icon offsets as follows.  If the X and Y offsets were 0, then the frame would be centered automatically based on the width and height of the frames.  If the X and Y offsets were non-zero, those values would be respected and used to center the frame.  See above for details on how they should be calculated.  The current EE behavior (from v2 onwards?) does not seem to behave this way.  Instead, any frame offsets are completely ignored, and are instead centered based solely on the frame width and height.  The EEs (particularly SoD) seem to have made an effort to center the large item icon based on the primary image (excluding the shadow).  This places the shadow down and/or to the right of the center of the inventory slot.  For obvious reasons, this can't be done when the large frame is approaching or at the maximum dimensions (64x64 px).  If the frame dimensions were respected by the engine, this modified centering could be done simply by adjusting the frame X and Y offsets by the number of columns and rows of pixels made up exclusively of shadow.  With a well-formatted shadow, this would be easy to do programmatically.  Since the frame offsets are not currently respected, however, this modified centering has been performed by adding rows and columns of transparent colored pixels to the frame to artificially adjust the width and height (which are then used to center the frame).  Many of the item icon BAMs (both original and new) have been done this way in SoD.  To me, this seems like a major regression of engine behavior.  It has no conceivable benefit (other than masking the laziness of having not made appropriately sized "small" item icon frames), and comes at the cost of requiring extravagant and byte-consuming manipulations of the frame images themselves in order to reproduce the old behavior.

At the BAM level:
(in no particular order)

1. At some point, a few of the item icon frames used in the EEs got hosed compared to the originals.  Examples include imisc1j and IMISC5N in SoD and BG2EE, IMISC61 and IMISC61 in SoD, ISW2H22 in BG2EE, etc.
2. No shadow at all on the large icon.  A few examples include IBOLTS01, IMISC13 in SoD, and IMISC17, IMISC18, and IMISC19 in BG2EE, and IAX1H44, IMISC72 and IMISC73 in all games.
3. Clearly bad shadow color (noticeably not black).  See IBRAC26, IGLBGRE1, IMISC40, IMISC43, etc.
4. Beamdog has generally inserted a mid-level grey color, RGBA(128,128,128,255), into the palette entry 1 (the shadow color) of item icons.  Not only is this (IMHO) a poor choice of shadow color, but this palette entry is rarely (if ever) even used for the actual shadow in the frame.
5. Shadows using pure black, RGBA(0,0,0,0), but not located at palette entry 1.
6. Shadows using an off-black color (e.g. RGBA(1,1,1,255) in imisc3n), sometimes located at palette entry 1 but more often not.
7. Use of an alpha value of 255 in an otherwise non-semi-transparent BAM.  While not strictly wrong, I feel it is sub-optimal for reasons explained above.  This is trivial to correct with PS BAM.
8. Noticeable differences in icon sizes between items of the same type.  See IRING01 vs IRING02.  While you wouldn't want to upscale ALL icons to the max 64x64 px (i.e. a sling bullet shouldn't be the same size as a two-handed sword), there is some room for improvement.  See also https://forums.beamdog.com/discussion/81852/bg-ee-sod-bgii-ee-inconsistent-icon-size/p1
9. Transparent color that is not pure green (usually BAMWorkshop's colors).  See ISW1H58 etc.  This is trivial to correct with PS BAM.
10. Inconsistencies between games for the same item icon.  See IXBOW06 and ICHAN01.  Personally, I generally prefer the oBG1 style over the oBG2 style where applicable.  Where BD has done the best job of updating/improving the icons (IMO) are the scrolls and spell icons.  See SPPR417A as an example.  They have also improved the shadows of some item icons.  See ICHAN01.
11. Inconsistency in the function of scrolls between SoD and BG2EE, and within the same game.  Specifically, whether the scroll gets rolled up when picked up.  See SPWI101A vs SPPR732A etc.
12. Inconsistency in shadow offset direction (sometimes down, sometimes right, sometimes both) and offset amount (see IBLUN06 vs IBLUN07).
13. Inconsistency in magnitude of shadow offset.  See IBZPLO01.
14. Inconsistency in shadow size (a handful of shadows are considerably smaller than the original item).  For example, see IBLUN09.
15. Some of Beamdog's new item icons are larger than 64x64 px.  Depending on game configuration settings (like Scale User Interface) and possibly OS level display settings (like DPI settings), these icons can overflow their bounding box in the user interface (inventory screen, store screen, etc.).  They should be trimmed, and if still too large, scaled down.
16. Anything else I haven't thought of or noticed?

The way I see it, the most thorough (and effort intensive) method of resolving these issues would be this:

1. For each item icon, pick the best one available between all games, preferably one with a "small" item icon (with no shadow).
2. Correct the transparent color if needed.
3. Set all alpha values in the palette from 255 to 0.
4. Remove any existing shadow from the large frame
5. Resize frames up or down as necessary, leaving enough space (within the 64x64 px limit) to re-add the shadow back later.
6. Replace any remaining pure black pixels with a slightly off-black color.
7. Trim all frames.
8. Create the small item icon if necessary
9. Create a pure black shadow at consistent offsets on the large icon, and save this as a third frame.
10. Optional alternative:  Either recreate or upscale the original small item icon to be twice the original size.  This will become the "large" item icon with no shadow.
11. Rebuild the item icon BAM with the proper transparent and shadow colors at the proper palette entries, and with 3 frames.
    • Sequence 0 Frame 0 is the large item icon frame with the shadow
    • Sequence 0 Frame 1 is the large item icon frame without the shadow
    • Sequence 1 Frame 0 is the small item icon
12. Calculate the proper frame offsets as described above.  Optionally, adjust the offsets to account for the rows/columns taken up by only the shadow.
13. Compress the BAM.
14. Use this new BAM in all the EEs, and backport them to the classic games via a mod, to be installed very early in the install order.
15. Modify the Enhanced Edition game engines to respect the item icon frame offsets if they are non-zero, but calculate them automatically if they are 0.

Between PS BAM and Imagemagick, I think I can mostly automate 2, 3, part of 4, part of 5, 6, 7, 8, 9 with some guidance, possibly some of 10 with some guidance, 11, 12, 13, and the mod part of 14.  What I'd need significant help on would be 1, 4 simply for the amount of time and effort involved, 5 if we want to scale any of them to be a consistent size within family, deciding the offset of the shadow for 9 (doesn't necessarily have to be the same amount for all item types), probably some instances of 10 since I'm no artist, the native EE usage of 14, and of course 15 is out of my hands.  Number 4 is significantly more complicated than you might expect, and I know of no straightforward way to do 5 that gives universally optimal results, so some trial and error is in order.  Considering the scale of what I'm proposing, this quickly becomes a significant time sink.  

Does this all seem like a reasonable assessment of the current situation?  Worth the effort, or should the scope be scaled back to only those item icons that are obviously and visibly broken in-game?  I fully recognize that not everything in this list constitutes a "bug" in the context of EEFP, but a good number of them may.  I'm looking for feedback on what, if anything, we may want to address here.  For example, some shadows are clearly the wrong color while others are a bit off but hard to distinguish in-game.  Where to draw the line?

 

[CamDawg]

One of the neat things about collaborative projects is that folks will make posts like this, full of all sorts of close, technical analysis that is just awesome in its scope and detail. I f'ing LOVE stuff like this.

[Graion Dilach]

I just dumped all the classic icons (GoG BG ToTSC) on the other day - it seems to me that IAMUL17 in oBG1 ended up with anti-aliasing issues causing the totsc shadow note. It is especially noticable in this comparison that the second frame got squashed horizontally for some reason.

[CamDawg]

To attempt an answer to the questions at hand: I suspect we may be able to crowdsource #1 to some degree. All it needs is NI and eyeballs, so hopefully we can get some volunteers to make suggestions. As I've been spending the better part of two weeks removing shadows and making small icons, I know how time-consuming #4 will be. Given that it's a purely graphical task, we may be able to get some help there as well. #5 doesn't always require a re-sizing, as in some cases, like weapons, you can just use a crop of the hilt (swords) or head (e.g. maces, halberds). In terms of quality I don't think upscaling will work very well--I tried it for the light crossbow fix. Generally, outside of a tiny upscale, the results are suboptimal--though perhaps someone knows some kickass upscale algorithm, though I'm skeptical. I'm not in favor of mass compression, though I'll admit that's because I want to incorporate the result of all of this for IWDFP without extra work. 

More broadly speaking, in terms of this being Fixpack material: I think there's quite a bit here that qualifies. The scroll icons have bothered me forever and, once you notice them, you can never unsee the inconsistent shadows, and there are any number of one-offs where something has been FUBAR'd. I'd be thrilled converting anything we don't deem a strict 'fix' into a general icon/art OBC regardless, so the work absolutely would not be wasted. And like I mentioned above, this work would also find a home in IWDFP and BG2FP as well.

One thing to keep in the back of your mind: the BD build process builds BAMs from PNGs at build time. (This allowed the art team to simply update the art as needed with Photoshop without having to get someone to make a BAM.) So while we'd ship EEFP with finished BAMs, having a pile of source PNGs would be easier for BD to use in future patches

[Sam.]

1 hour ago, CamDawg said:

I'm not in favor of mass compression, though I'll admit that's because I want to incorporate the result of all of this for IWDFP without extra work. 

PS BAM is extraordinarily good at compressing BAMs, far beyond any other tool available.  Smaller mods, smaller games, faster downloads, faster rendering in-game.  There are significant benefits.  AFAIK, only oBG1 and oPST do not support the BAMC format, in which case use a version of Miloch's "Decompress a BAM" patch when installing on those games.  Am I missing something else?

 

1 hour ago, CamDawg said:

One thing to keep in the back of your mind: the BD build process builds BAMs from PNGs at build time. (This allowed the art team to simply update the art as needed with Photoshop without having to get someone to make a BAM.) So while we'd ship EEFP with finished BAMs, having a pile of source PNGs would be easier for BD to use in future patches

This seems to be a significant source of errors and inconsistencies, and I STRONGLY recommend against doing that at this stage of development of the EEs.  It may have made sense early on, but that is no longer the case.  They will simply end up reintroducing many of the issues we are attempting to address.  Bammer was a nice idea, but it has flaws.  Edit:  and if we're aiming for backwards compatibility, transparency handling must be kept in mind.

Edited April 3, 2022 by Sam.

[CamDawg]

7 minutes ago, Sam. said:

This seems to be a significant source of errors and inconsistencies, and I STRONGLY recommend against doing that at this stage of development of the EEs.  It may have made sense early on, but that is no longer the case.  They will simply end up reintroducing many of the issues we are attempting to address.  Bammer was a nice idea, but it has flaws.  Edit:  and if we're aiming for backwards compatibility, transparency handling must be kept in mind.

I totally agree. When I submit tickets for this I'm going to push for just including the finished BAMs, but I'm also preparing for the contingency where they don't want to mess with the build process.

[Graion Dilach]

I am still processing all this so sorry if my questions being sporadic:

Are you sure neither of these files have any transparency in the EEs? Additionally, the source PNG method makes sense to me because that allows everything to be BAMv2 and I am not sure if BAMC is actually better than BAMv2 in EE perf.

EDIT: Actually, what is the problem with Beamdog's runtime-PNG-import process? What is the exact data which that process cuts out? Because I don't see what would be lost from a PNG frame which has correct 8bit palette and storing it's frame offset metadata as a tEXt value (or however BD expects the metadata be supplied, but an embedded tEXt block would make the most sense to me)...

Edited April 4, 2022 by Graion Dilach

[Sam.]

13 hours ago, Graion Dilach said:

Are you sure neither of these files have any transparency in the EEs? Additionally, the source PNG method makes sense to me because that allows everything to be BAMv2 and I am not sure if BAMC is actually better than BAMv2 in EE perf.

The images in my table essentially have the alpha channel turned off.  Looking through the item icon BAMs from BGEE, SoD, and BG2EE, none of them are BAM V2, and there are only a handful of BAMs that have alpha values in the palette other than 0 or 255 (both values render as completely opaque).  They are IAX1H19, IBLUN40, ISTAF25, ISW2H22, OHRICLK3, and SPWI920A all in BG2EE.  I'll see about re-exporting them with alpha enabled for comparison.  BAM V2 isn't supported in all applications, and I'm unsure if they can be used for item icons.  Even if they could, no modding tool I'm aware of can currently perform the bin packing across multiple files, so you'd end up with 1 PVRz page for each BAM, which would be a non-starter.

 

13 hours ago, Graion Dilach said:

EDIT: Actually, what is the problem with Beamdog's runtime-PNG-import process? What is the exact data which that process cuts out? Because I don't see what would be lost from a PNG frame which has correct 8bit palette and storing it's frame offset metadata as a tEXt value (or however BD expects the metadata be supplied, but an embedded tEXt block would make the most sense to me)...

Presumably, they would be storing the offsets in the BAMD file, not in an extra chunk in the PNG itself.  However, they don't even seem to be doing that.  Instead, they are leaving the offsets in the compiled BAMs at 0 and letting the engine calculate the offsets on-the-fly based solely on the dimensions of the frame.

Beyond that, how deep down the rabbit hole do you want to go?  I played around with bammer (Beamdog's BAM building tool) a few years back, and while I'm not familiar with the Go programming language, I have poked around the source code just a bit.  It appears to reserve pure green in palette index 0 for the transparent color, and then quantize the image down to 255 colors, which of course also includes the green of the background which may or may not get mixed with other colors in the process.  Then the original image is mapped back to the combined palette.  The result is either a duplicate of the transparent color in the palette, or an off-green palette entry that probably isn't used.  Furthermore, bammer often hardcodes a non-shadow color (not pure black) in the shadow index of the palette in addition to two additional hard-coded palette entries.  This false shadow color palette index is rarely even used, as the actual shadows in the image don't use this color.  This is the source of many of the warnings issued by PS BAM in the comparison table.  The end result is that each BAM may use up to 4 fewer colors in the EEs than they did in the classic games.

Furthermore, looking at the quantizer, it does not appear that the alpha channel is even being fed into the quantizer to generate the palette.  If this is really the case, then trying to map the original RGBA colors to the palette generated is going to result in huge errors if the original image had semi-transparent pixels.  I haven't studied the code extensively, but I get the feeling this is a straight median cut quantizer.  A modified median cut algorithm would produce much better results, tho with very small images like these, it might not be that noticeable.

[Acifer]

On 4/3/2022 at 11:02 PM, Sam. said:

PS BAM is extraordinarily good at compressing BAMs, far beyond any other tool available.

PS BAM is a killer tool for bam postproduction. I recently tried to enhance my XDR Dragon animations I created some time ago, and PS BAM reduced the size of the 567 bam files from 543 MB to 11.4 MB!

Not only did it compress the bams, it optimized each frame in every possible way, reducing the size of many bams that only contained transparent frames from 743KB to zilch (1KB). Awesome!

Edited April 5, 2022 by Acifer

[Graion Dilach]

@Sam. Sorry, you're right and wrong at the same time. Bammer can pull the source image palette with the usepalette parameter, but seemingly Beamdog doesn't use that param in their shipping config. Technically, even that parameter hurts the image somewhat, because it overwrites slot 0 to be the green and if there was a transparent slot then it switches that slot with whatever color slot 0 had before 0 gets overwritten to be green, but I guess that is acceptable. It might be possible that PS-BAM is better but Beamdog can fix this without it.

[Sam.]

46 minutes ago, Graion Dilach said:

Sorry, you're right and wrong at the same time.

Like I said, I'm unfamiliar with the language and didn't spend all that long trying to decipher it.  All I can say is that from personal experience, I am not happy with how it handles palettes.

48 minutes ago, Graion Dilach said:

Bammer can pull the source image palette with the usepalette parameter

You assume you have an existent and appropriate palette to pull from, and that all frames use the same palette.  Unless you have taken extreme and meticulous care to ensure this is the case, it won't be.

52 minutes ago, Graion Dilach said:

Beamdog can fix this without it

Could bammer be improved?  Absolutely and that would be great.  Better tools available to modders and developers is never a bad thing.

Will bammer be improved?  Maybe... but I won't hold my breathe.

If bammer is improved, will it be a magic fix for all the issues there are with item icons?  I think not.

 

Regardless of the above, you are of course welcome to take on fixing/improving anything you like in whatever manner you deem best.  If the community agrees it's a solid improvement or correctly fixes a genuine bug, it'll end up in a fixpack or tweakpack.

[Graion Dilach]

I'm trying to understand the situation. Yes, I do assume Beamdog uses paletted PNGs in the build process and it only needs to add the -usepalette switch for bammer to not break them.

I haven't had issues with creating unison paletted images - when I was converting my own animations I was creating in particleIllusion to Red Alert 2, I just grabbed GIMP, threw everything into one image, then generated the palette from that project (GIMP doesn't flatten the image so it is perfect for this). If I had to do realignment because hardcoded gradient overlap, I could do that in GIMP. Then I exported the resulting palette and used that for reimporting the image to RA2 sprite format. Yes, Photoshop flattens and destroys the image for this, but GIMP doesn't and gives you WYSIWYG results. And I formed this workflow in 2013 or so (my biggest issue with it was that RA2 lacks blending modes and I had to always manually void out the close-to-black edges because pI rendering always looked best on black backgrounds and brrr hard edges).

I'm just trying to get a full picture because I see "creature animations are finite, consider this" and also "CRE variables X Y and Z has remap colors" and both Infinity Animations and Beamdog introducing literal recolors of stuff where the CRE-based recoloring seemingly would have worked without wasting a slot... and these just confuse me a lot.

[Sam.]

On 4/4/2022 at 1:29 PM, Sam. said:

I'll see about re-exporting them with alpha enabled for comparison.

As promised, here it is.  You can select whatever background color you want, and hover the mouse over each image to view it with transparency enabled.  The shadows are only made 50% transparent if they are RGBA(0,0,0,0) at palette entry 1.  If there is a real desire to also do this for RGBA(0,0,0,255) at palette entry 1, I can accommodate.

[Sam.]

I've been updating PS BAM, creating scripts, and building toolchains to help automate this as much as possible while still allowing human inspection for quality control.  Some BAMs are one and done, while others require pixel by pixel level edits in the vein of 1pp (which have been integrated into the EEs).  I'm over 100 item icon BAMs in of oBG1 (TotSC), but the going is slow, as predicted.  I'm willing to continue on this trek (which will likely take months), but I could use some help. So far in particular I could use some help removing the shadows from the mage robes.  Specifically iclck10, iclck11, iclck12, iclck14, iclck15, and iclck17 (but all of them really).  Theo goal is to keep the original graphics as original as possible while resolving the stated inconsistencies/issues.

@Lava are you interested?

Edited April 20, 2022 by Sam.

[Lava]

Just to make it clear: you need me to drop shadows manually for these items: iclck10, iclck11, iclck12, iclck14, iclck15, and iclck17? Right?