BG2 graphics upscale brainstorm

[Golden Pantaloons]

I investigate possibility to increase BG2 resolution (example) and interpolate game animations (example) using machine learning.

In this post I'll try to summarize my thoughts in a checklist of what should be done (and if it can be automated).

I'm pretty new to IE and would be grateful for any corrections/advises/thoughts you can provide.

 

Increase resolution

• Upscale backdrops rebuilding its TIS and reassigning its WED tile indices
• WED/ARE - adjust all absolute X/Y
• BCS - adjust alter all absolute X/Y/range/distance arguments in scripts
• BMP - upscale all search/light/height maps
• BAM - upscale all frames (should be ok for new sizes >256 in EE?)
• INI - adjust "move_scale" to fix movement speed ?
• PRO - adjust "speed"/"area of effect" ?
• SPL - adjust "Range (feet)" ?
• ITM/SPL - adjust "range" ?

Anything else?

Any chance there's can possibly be an alternative solution without rebuilding TIS and messing original coordinates?

 

Interpolate animations

• Find BAM area/creature/effects/projectiles animations and interpolate them
• baldur.lua - adjust "fps"
• VVC - adjust "duration"
• ARE - adjust "frame number" ?
• INI - adjust all "*_frame" ?
• PRO - adjust "speed" ?
• EFF/ITM/SPL - "duration" here is not in frames, so don't touch ?

Anything else?

[Jarno Mikkola]

Increase resolution... is done by the Widescreen mod in non-EE games. So what's this ... nad the other is used by EE to allow zooming ? So, tahw siht ?

Yes, the widescreen mod allows any resolution, not just some, but all that are above 642x482. So there.

[Sam.]

Have you tried upscaling the original MVE movies to the EE resolution?  I did my best with BG: EE Classic Movies, but your NN algorithms might produce even better results.  I can provide an original movie as an uncompressed AVI if you want to try.  They suffer heavily from compression artifacts.

[aqrit]

Quote

Any chance there's can possibly be an alternative solution

Maybe there is some trick for "EE" but...

I don't think the upscaling method as proposed would work. There is always a hard coded distance check somewhere...

The only solution is engine modding. The game itself would be blissfully unaware of any changes.

One would do the typical "DxWnd" style window mode mod. Then hook into the entire graphics sub-system.

 

I hooked into the original engine's ddraw system to improve the fog of war effect,  it might be of interest:

https://github.com/aqrit/ddwrapper/blob/95c64084f640d51552f0e77c140d9af71af7cc76/torment.cpp#L398

 

 

[Kilivitz]

I would be very much interested in a way of automating upscaling for item, spell icon and paperdoll BAMs.

The EEs have increased the size of most GUI elements to 125% of their original counterparts. This made all item slots (both in the inventory and action bar) 25% larger as well. This is not a bad thing - playing the game in modern resolutions (1366x768 and up) makes the slots uncomfortably small.

However, a side effect of that is that the EEs also eschew the default behavior of item BAMs - in the originals, they had two different frames: a "close-up" one used in the slots, and a "zoomed out" one used when you pick an item up in the inventory. Now, items only use the zoomed out BAM since it fits the larger icon slot a bit better.

For years, I've been working on-and-off in restoring the original UIs to the EEs. Part of that effort involves restoring the original behavior of item BAMs. However, this means the close-up icons are too small, so upscaling them would be in order. However, it's just not feasible to do it unless the process could be automated somehow.

Edited January 8, 2020 by Kilivitz

[Adul]

I too am interested in the prospect of AI-upscaling all of the in-game graphics in both Baldur's Gate games, though lately I'm considering the appropriate engine to start out from to be GemRB instead of the IE:EE—if for no other reason then at least because of its open-source codebase, which at this point I consider to be an essential feature to the long-term health of these games.

Do note that the above opinion is largely unresearched, as I barely know anything about the general state of GemRB.

[lynx]

There was another ML thread a while ago, but I can't find it. Didn't go into much detail about actually using this, since the algo was too intensive and unreliable to do at runtime.

With GemRB one can emulate a good ol' CRT by just resizing the window — SDL does the simple scaling and of course the best results are at integer factors. It has some options (eg. anisotropic filtering on top), but this is far from just having greater detail.

[Joneleth Irenicus]

I looked into this for the last few days and I am going to start the project.

I think the first bit that makes the most sense to start with and would honestly be pretty simple is only upscale the interface.  Then potentially add zooming.  This would enable people to use the larger interface and then also not be worried about the areas being so small.

Then we could slowly work out the methods to do all the other bits.

 

Here is an example of what the start screen looks like at 4k.  Excuse the ugly fonts.  I just grabbed the first I could find for testing.  It is at 48px.

Edited August 8, 2020 by Joneleth Irenicus

[Joneleth Irenicus]

Here is an example of the visual quality from AR0111.  Disregard the visual elements still on the screen.  I am messing with some stuff.  Obviously the characters are the wrong size and other things, but this is at 4k.

 

Here is the original just so you can get an idea of the improved fidelity.

Edited August 9, 2020 by Joneleth Irenicus

[Magus]

I'm interested in the topic, but I'm not sure what to look at in the last screenshots.

[Jarno Mikkola]

39 minutes ago, qwerty1234567 said:

I'm interested in the topic, but I'm not sure what to look at in the last screenshots.

Well, if you look at the view in it, the icons are very small, but they are in their correct places(centered), if they were just a lot larger, to cover the entire left panel of the screen. Compared it to the gray stone panel in it that's the original, where the icons would have been, originally. The screen size would be a 4k... technically meaning 3840x2160. Originally from 640x480.

So replacing the icons with centered larger ones would be in the doable cathegory of options, if one were to just supply the large enough icons. 

Edited August 9, 2020 by Jarno Mikkola

[Joneleth Irenicus]

1 hour ago, qwerty1234567 said:

I'm interested in the topic, but I'm not sure what to look at in the last screenshots.

Really the main take away from the last two images is the area.  Look at the different sizes.  I haven't perfected the GUI yet as I have only just started the project. 

The resolution in both the images is 4k. 

They are both the same areas, but the one on the top is much larger and more clear.  This is without any zoom functionality.  Now for the bottom it is just the original.

I am thinking this is simply modification to GemRB adding an extra setting called "UpScaleFactor".  Then we can create a "mod" that would have all the upscaled assets we need. 

I am working on tools to increase the speed at which I can create the assets.  This will be a new addition to NearInfinity that will come out soon.

I have already converted all areas and many portions of the GUI, but still working on a few other things.

I will post a fork of GemRB with a new branch that we can merge if we are successful and I will also post the mod github soon so we can all work on that piece.

Edited August 9, 2020 by Joneleth Irenicus

[Guest Guest]

The mod will be available only for gemRb?

[Joneleth Irenicus]

Going to focus on GemRB first and then maybe EE, mostly because EE requires more work since I don't have access to the engine code.

Edited August 9, 2020 by Joneleth Irenicus

[Joneleth Irenicus]

Okay as promised I have finally got SOME of my work public.  Still working on getting more done.  I will get the assets setup for download soon so we can work on some sort of installer hopefully.  I should add the screens below are 2k not 4k.  Going to do that soon.

Near Infinity video

Please join in if you can.  Right now I am trying to focus on work in Near Infinity, but I have some GemRB changes I will upload soon. 

https://github.com/Goddard/Project-IE-4k

Edited August 17, 2020 by Joneleth Irenicus