Area door/animation graphic (black line/seam) issues

[Sam.]

The PVRz format for area animations and doors has resulted in graphical glitches (usually described as seams or black lines between the tiles).  Some have been reported and fixed, but I doubt all of them have.  Here is an example of one such report.  Is there a list of any such issues that remain in the current builds?  I can describe how I have gone about fixing it for mods, but the process is currently a bit tedious.  More to the point, is this something EEFP is interested in addressing?

[CamDawg]

20 minutes ago, Sam. said:

More to the point, is this something EEFP is interested in addressing?

The answer to this is nearly always going to be "yes", including this.

[Sam.]

On 5/10/2023 at 7:47 PM, Guest The_sextein said:

Looks like the spinning crystal in the high hedge still has a crease through it.

Here's the fix for the horizontal line through the crystal in High Hedge AR3202.  I have included screenshots of before and after in the archive.  This should be compatible with both BGEE and SoD.  The background of the animation still looks a bit dark compared to its surroundings, so if an artist in interested in cleaning them up I have included the images in the resource folder.  Send them back to me and I'll rebuild the TIS.

Also, I really need more upload space here.

[Sam.]

And here's my attempt at cleaning up the dark background.  I'd say it's a noticeable improvement.  There is still some pixel flicker if anyone would like to clean that up.

[Guest the_sextein]

Do you think you could give me some pointers on fixing door frames?  I've been fighting this thing all night and can't even fix a single one.  I understand how to convert the files and overide but building the tile sets...  Where are you finding the animated bits at?  How are you rebuilding them?  BTW, great job and thank you for your efforts.

[Guest The_sextein]

So I see all of the doors included at the bottom of the .TIS tileset.  I export the .TIS to PNG and then copy and past the doors until they line up with the rest of the .TIS image.  Then I save it and convert it pack to .TIS file and then drop it into the overide folder. Is this correct?  I can do this with photoshop or even microsoft paint right?  Sorry for all the questions but it's been years since I did any modifications to this engine and it took me forever just  to remember how to install near infinity cause I forgot about JAVA.  uhhh!  Anyway, If there is anything I'm missing or doing wrong please let me know.

[Guest the_sextein]

I can't seem to get this to work.  Didn't use to be like this but now with PVRZ I can't figure it out.  If I export the pallet based TIS file then how do I open it and manipulate the graphics?  I've converted the TIS files to PNG and then manipulated the graphics and then converted them back to PVRZ based TIS files but I continue to get the same lines around doorways regardless of what I do.  Very frustrating, it looks like the new PVRZ format just doesn't work with the TIS files and converting the TIS files to PVRZ based doesn't help with the problem.  If you use legacy then it doesn't work with the PVRZ backdrops and if you use PVRZ TIS then the door ways are incompatible.  Manipulating the textures doesn't seem to matter at all.  I've overlapped them, under sized them, stretched them, you name it, it won't work with PVRZ based TIS files.  I'm sure I'm missing something but I can find any information on the internet at all regarding these changes to the infinity engine or how to deal with them.  If we are expected to open the Pallet Based Tis files then I am unaware of what software I would need to do that or how it would work.  I'm starting to regret wasting my time on this.  I don't mind spending hours working on improving things but this just looks like incompatible file structures and tools that can't work with them properly to me. 

[Sam.]

Maybe it would be best if I go into a bit of detail about the root cause of these issues, and the general methodology of what eliminating them entails.  The Infinity Engine breaks area graphics down into individual 64x64 pixel blocks known as “tiles”.  In the EEs, these tiles are stored in PVR(Z) files.  PVRZ files are Z-Lib compressed versions of the PowerVR container file format (PVR).  This file format supports a large number of texture compression algorithms, but the EEs use the Block Compression (BC) aka DXT family of LOSSY texture compression algorithms.  A side effect of the nature of these compression algorithms is that the color of a pixel will “bleed” into the pixels that surround it.  This is the root cause of the graphical artifacts that are the topic at hand.  Beamdog’s particular variant of the PVR file format seems to be limited to 1024x1024 pixels (16x16 tiles).  The most space efficient way to pack tiles into a PVR(Z) file is to add them sequentially until the space is filled, and then start a new PVRZ file.  Several of the modding tools available for building PVRZ-based tilesets do exactly this, but because of the color-bleeding nature of the underlying compression algorithms, this will trash your area graphic.  Instead, you should only build the PVRZ files from contiguous blocks of adjacent tiles.  To put it another way, you must only add tiles to the PVRZ file that are surrounded by the tiles that will surround them in the actual area graphic.

This sounds easy enough, but the devil is in the details.  For the base area graphic, this is straightforward if you use the proper tool.  Use Near Infinity’s Tools->Convert->Image to TIS and select the PVRZ-Based variant for the tileset.  This is the only modding tool for building PVRZ-based tilesets that I have found that stores tiles as contiguous blocks in the PVRZ files.  But that is just for the base graphic.  The problem tiles come in three flavors:  true overlays (like water or lava), doors, and tileset animations (like many fireplaces or the crystal in High Hedge).  These require special handling.  Specifically, you can’t just leave them added sequentially to the end of your area graphic, you have to surround each tile with the tiles that will surround it when placed inline in your area in order to ensure no stray colors bleed into your overlay/door/animation tiles.  See my example for High Hedge in the archive above for a demonstration.

It is EXTREMELY important to note that once a tileset has been packed improperly and stray colors have bled into your overlay tiles, the damage has already been done.  You can’t simply rearrange them after the fact and expect the graphical artifacts to go away!!!  You must either go back and start with the original palette-based tileset or you must spend a vast amount of time touching up the graphics by hand.  Edit2:  Or better yet, when the true original is available (like for custom mod areas), go back to that as palette-based tilesets are also lossy, tho in a different way.

Now for the hard part.  True overlays (especially water) are formatted/constructed very differently in oBG1 than they are in oBG2 than they are in BGEE/SoD than they are in BG2EE.  There are tools out there that can help with the conversion, but if you’re this far into area graphic creation/manipulation you probably already know about them.  More to the point at hand, rearranging the overlay/door/animation tiles at the end of the tileset require you to update the area’s WED file accordingly.  The structure of this file is complex, and the nature of the required update is different for true overlays vs doors vs tileset animations.  In typical textbook fashion, I will leave the details of this as an exercise for the reader.

Edit:  As far as what tools I use, I use a few AHK scripts I've written myself, ImageMagick, Near Infinity, and Microsoft Paint.  And when the occasion requires it, the tools used to help convert the water overlays from one game variant to another.

Edited May 14, 2023 by Sam.

[Guest The_sextein]

Thanks for the explanation.  So these backdrops have all of the doors at the bottom of the .TIS file.  Are you saying that I need to select each one and paste them into a black canvas so they are surrounded by black and save each one individually before assembling them on the .TIS file?  I don't think their are any other reasouces for these backdrops.  I have not seen the doors to these backdrops anywhere else other than the .TIS file.  If they have been packed like this and will have light bleed then I would assume you mean that I should surround them by dark colors and save them one by one before importing them back onto the .TIS file.  Is this correct?

To clarify, I should convert the .TIS file to PNG using near infinity and then pull the door resources onto a black canvas and save each one before importing them back onto the PNG on the proper building and then convert the backdrop back to .TIS?  After that I need to update the .wed file.  How do I do that?  Do I just export the .wed file for the area after I have updated the .TIS file and drop it into the override folder like everything else?

[Guest The_sextein]

Oh wait, I think I get what you mean.  You are saying I should cut parts of the buildings out of the .TIS file that surround the doors and surround the doors with those graphics and save them.  Then cut the door out of the file and import it back into the .TIS file and place them on the building they need to be on.  Is this correct?

[Guest the_sextein]

Alright, so I cut the door frame out of the building and placed it in a microsoft paint canvas and then I cut the doors out of the bottom of the .Tis file being careful to undercut 1 pixel on all sides.  I rebuilt the doors inside the door frame inside microsoft paint and saved the file.  Then I copied the whole door frame and doors as a single texture out of microsoft paint and imported it onto the .Tis backdrop and aligned it well enough that I couldn't see anything wrong with it.  I then converted it from PNG back to .TIS and it created a bunch of PVRZ files of the tiles that make up the backdrop.  When I imported them into my override folder and launched the game.... The door frame had doors on it with white creases as usual but when I opened the door a perfect set of doors with no creases appeared instead of a door frame that should have had no doors. 

So, I successfully added doors to a backdrop without artifacts showing up but I'm confused as to why I'm seeing doors with white lines on them when I start out and why my backdrop tile has doors on it when I pasted them onto the .TIS file only.  The game engine seems to transform your .TIS on the fly based on something but I'm confused and don't really understand how the engine works.  I'm using Tipun's IWD1EET mod so maybe it's falling back onto something he has installed when the .TIS file has no doors on it.  It's strange that I had doors pasted onto the .TIS file and saved, and then when I converted it from PNG  to .TIS the doors disappeared from the .TIS file and showed up on my first backdrop PVRZ tile. I'm pretty confused but if you have any ideas I would appreciate it if you could share them.  I feel like I almost have a grip on this but the way the engine works is still throwing me a few curve balls.

[Sam.]

It sounds like you failed to update the area's WED file to actually point the doors structures to the new tile numbers where you moved your doors to at the end of the tileset.

 

[Guest the_sextein]

So I need to alter my .TIS file by "selecting all" and pasting it into a larger canvas and then adding the image of the doors and doorframe that I made somewhere at the bottom after all the other doors that are already stored there?  I opened the .WED file and it looks like you can add a new door easy enough but I don't understand how this engine works haha.  I wouldn't know what to put for tilemap indices or how tilemap index and offsets work.  Think this may put an end to my little adventure.  I've been training a new AI upscaler and I wanted to enhance the backdrops of IWD1EET but all of the doors are wrong and simply upscaling the image and resizing it back to it's original pixel size of 1024x1024 still causes the door frame creases to show up where there were not before and it makes existing ones worse sometimes. 

AI scaling is quick enough but replacing these doors would probably take years if I could even figure out these tools.  I appreciate the help but I think I'm in over my head with the .WED file thing.  I'm used to modding 3D shooter games that are easier to work with and I figure I would probably have to "trial and error" this for months before I got anywhere with it.  Probably just gonna have to accept that the game is broken at this point.  I can see that Beamdog's .TIS file for Easthaven in the EE version of the game has much larger cutouts for door frames.  Like someone went in and specified larger sections around each door in the game engine and then did what I did with the door frame and doors.  Too bad I can't find how they set it up or where their door assets are.  I figured since IWDEE already worked that it would be a simple matter of porting the updated data over to IWD1EET via "override" folder but that's clearly not the case and I can see why nobody has fixed this problem including Tipun or interested people like myself.

[Guest The_sextein]

This isn't related to seams on doors but being that you have knowledge on how to manipulate the .WED files you may want to look into the town of Bergost in Baldur's Gate EE.  Just below the smithy's building there is a tree just sitting in the road where it clearly shouldn't be.  I don't know if this has been reported or fixed before but I thought I would mention it.

[zenblack]

On 5/12/2023 at 8:48 PM, Sam. said:

Also, I really need more upload space here.

I hope you get what you need. These introduced graphical glitches in the EE are frustrating and anyone willing to take the time and have the knowledge you have to actually fix them is a huge contribution to BG2:EE staying alive. If someone is already critical about dated graphics, this can be annoying. I think with how big BG3 is becoming that there will be fans who will go back and check out the epic backstory.

Edited July 13, 2023 by zenblack