Colors/palette issue in BAM file

[Roberciiik]

Hi, 

I'm trying to fix an issue with colors of map icons introduced in the BP-BGT-Wordmap modification after v10.6. Currently (v11, or pre-v12 master branch), the icons seem to be much darker than those in the v10.6 version. The problematic file is bam/bpbgtmap.bam.

I've tried so far:
- replace the v11 palette using the one from the v10 BAM file - resulted with even worse colors,
- append all frames to the BAM files from v11 to v10 and from v10 to v11 - no change, imported icons are the same as from the original file,
- export EE version BAM file as Legacy v1 (EE has correct coloring scheme) - incompatibility because of transparency I think.

Do you have any idea how to properly handle such conversion? I prefer not to override all the frame sequences from the scratch as it is more than 200 frames now... It seems that exporting EE BAM file to legacy format is the easy way, however I don't know how to migrate the palette.

Thanks for any advice and help!

Example icon from v10: 

Example icon from v11: 

[Graion Dilach]

Are you sure this isn't intended? Have you tried bisecting which commit introduced this? I mean, the v10 BAM example looks worse to me, because there's a transparency issue on the tree trunks.

EDIT: Actually, wasn't it https://github.com/SpellholdStudios/BP-BGT-Worldmap/pull/20?

Edited June 23, 2023 by Graion Dilach

[Roberciiik]

I've only checked the v10.6 is the working version. The EE version hasn't been changed, and it is still brighter, so I think it is not intentional. Maybe @Weigo could give us a hint if this is expected. I agree with you about BG1 icons' transparency issue, unfortunately, even BG2 icons are also darker, so it looks quite bad on the brighter Worldmap background. More examples:

v10: 

v11: 

Edited June 23, 2023 by Roberciiik

[Weigo]

Of course, it could be that the palette will eventually run out of colors. There are soon 7000 frames and there are only 256 colors in the palette. That means Near Infinity will have to average colors at some point. But this is only a guess.

What is the transparency incompatibility from v1 to PVRZ?

In case of need you would have to recreate the bam again, maybe it will split the palette a bit better then and we get more beautiful colors

[Roberciiik]

So you are saying that appending a new BAM file causing palette recalculation? Do you remember what changes were made to the bpbgtmap.bam file during your UI improvements?

About EE BAM file: I open bpbgtmap.bam from EE directory, then click Edit BAM, export in Legacy v1 format, and got such result after loading the file:

I think about recreation of BAM file as a finality... It could require some input files from other mods that I would need to search for.

Edited June 23, 2023 by Roberciiik

[Roberciiik]

Hey @argent77, could you confirm that this is the way how NI handles appending new BAM files according to what Weigo said? Maybe you know the proper way to export a v2 BAM as a v1 BAM without loosing color information?

Thanks

[Graion Dilach]

So I've grabbed the current tip of the V2 BAM and exported it out with NI v2.3-20230303 (the throwaway test folder I've installed the mod to wasn't updated with a newer NI yet). I don't see the issue within my result of the BAM file.

Conversion attached for individual confirmation.

bpbgtmap.7z

[Roberciiik]

In my example, I used "Edit BAM" and then "Save output file" button to export file. "Export" button works to generate compressed/uncompressed v1 BAM file, unfortunately the colors are still wrong/dark as in your package: 

 

[argent77]

It looks like color conversion of NI's BAM Converter is currently not very accurate when alpha-blended graphics is involved.

With a bit of experimenting I came up with the following procedure that should produce acceptable results (using EE version of the BAM as source):

1. Use "Mass Exporter" to batch export and convert all BAM-related PVRZ files to PNG (use "Resource Name Filter" to filter list of files to export).
2. Use "Image to PVRZ" converter to convert PNG files back to PVRZ: Use compression type "DXT1" to get rid of alpha!!!
3. Use "BAM Converter" to export the EE version of the BAM with the now alpha-less frames to BAM V1 format (don't forget to enable "Compress frame" checkbox for all frames).

 

Edited July 7, 2023 by argent77

[argent77]

I have fixed some issues in NI's BAM Converter that adversely affected the conversion quality for graphics with alpha blending. You can download this version from the Nightly Releases section.

[Roberciiik]

Hi again, it looks like the issue is now showing up in the EE-format too. I've attached the current BAM file in EE format from worldmap master branch. When you append the additional icons from the second BAM file (file attached), the colors are now darker in similar manner as in legacy v1 BAMs files. I thought v2 BAMs should use extended palette in such case?

Steps to repro:
1. Load BAM file from worldmap-master.zip in NI
2. Click "Edit BAM" button
3. Then "Add" -> "Import BAM files"
4. In the preview tab, you can see the incorrect colors of icons.

@argent77 Could you please verify/debug if this is the same running-out-of-colors issue of the palette, or maybe some NI issue?

Do you know if there is any other tool that supports PVRZ-based BAM files, so I could compare results?

 

EDIT: False alarm. It seems that I was mistaken: you need to choose V2 PVRZ output file type to see proper preview. So appending new files is working for me for EE version. However, abalabokhin was able to generate a broken palette for EE-PVRZ BAM file somehow (here), so that's why I reopened the issue here and I thought that this is a global issue.

worldmap-master.zip bams-to-append.zip

Edited August 10, 2023 by Roberciiik

[Roberciiik]

Alright, I post my findings just for the record:

I've tried creating the BAM v1 file starting from version 10.6 (the "working" one), and after appending some icons the color palette started degrading eventually. So this is a dead end. 

However, above solution with exporting PVRZ and removing the alpha channel brings awesome results and the final BAM file is nearly identical as EE version! So big thanks Argent77 for this! 

Changes are implemented here: https://github.com/SpellholdStudios/BP-BGT-Worldmap/pull/49