Displaying Faerûn correctly in ingame BG2 Text

[Bill Bisco]

Dear all,

 

I'm trying to get the word Faerûn to display correctly. See an example of my problem below.

 

 

Is there something I can do to get the engine to display Faerûn correctly? Or do I need to do some rewrites?

 

Thanks,

 

Bill

[Jarno Mikkola]

My recommendation is that you don't even try... it can be done. However, is this in BG2 OR in an EE game ?

The English -tra file needs to be encoding that includes the character:

Character Sets

 

To get a lot of fancy characters in for the translators, the EE versions use a different character encoding. This causes problems for non-English tra files in particular, but can also cause problems for English tra files. Non-converted tra files can introduce garbled text or even crash the game for your players. Again, Wisp to the rescue with HANDLE_CHARSETs.

 

As of v236, this is a standalone library; starting with v237 it will be included in WeiDU itself. The basic idea is that, when a mod install detects an EE game, HANDLE_CHARSETs will get invoked and convert your translation files to the proper UTF-8 encoding. For Windows packages you'll need to include a copy of the iconv utility; OS X and Linux users will either already have this utility or will need to install it themselves. Rather than re-hash what Wisp has already documented, I'll simply refer you to the linked thread above for help and questions.

[Bill Bisco]

This is for regular BG2 that I'm having a problem. I'll add support for BG2EE down the road later.

[Jarno Mikkola]

Then what is your Windows Region Language Codec ? You can access it via the Control Panel...

[argent77]

You should encode all of your TRA files in ANSI and use WeiDU's HANDLE_CHARSET for the necessary charset conversions. The text error in your screenshot is typical for an UTF-8 encoded text displayed in the original BG2. Advanced text editors such as Notepad++ provide options for converting text into different character encodings.

[Bill Bisco]

The screenshot of my windows 7 laptop looks the same as Jarno's. English United States.

 

Argent, thanks for the tip. Could you link me an example? So to display this correctly, I need to do all the TRA Stuff now as opposed to the end?

[argent77]

I suppose putting all your text into TRA files is required if you want to rely on WeiDU's character conversion functions. You can probably use any recent mod that supports both the original games as well as the Enhanced Edition games as example.

In the case of my Improved Archer mod I've placed the HANDLE_CHARSETS function into an ALWAYS block, so I don't have to call it manually for each component. The whole code should be pretty self-explanatory. I've excluded the "setup.tra" since its content is not written into the dialog.tlk. See the english translation file (or the italian/russian translations if you want to see more special characters). I have noted the character encodings on top of each TRA file.

And that's basically it. You have to include the iconv tool in your mod which handles all the conversion stuff. The WeiDU Readme provides a link to the Windows version of the tool.

[Mike1072]

That's a good example, but if you want something that explains everything step-by-step, you can also check out "Real Mod". It's a tutorial mod that explains how to use HANDLE_CHARSETS and other tasks, like installing audio or tilesets, with thoroughly-commented code courtesy of CamDawg. It includes the binary tools needed for these types of conversion (iconv for charsets, tisunpack for tilesets, oggdec/sox for audio).

 

You can view the whole thing on GitHub here or take a look at just the .tp2 code here.

[Bill Bisco]

Mike thank you for your reply and thank you Argent.

 

Do I need to run this every time I want to TRA files in order to display the correct characters?

 

Can I just arbitrarily set my own @1000 , @1005, etc. Numbers and then define them in the tph file for that translation?

[Mike1072]

A .tra file can have any number of TRA references (@0, @1, etc.). Those references can be used throughout your mod so long as you have loaded the appropriate .tra file.

 

There are different ways to load .tra files:

• In the LANGUAGE section, you can specify .tra files that should be loaded when the mod installation starts (so they are available in every component).
• You can use LOAD_TRA anywhere in a component to load .tra files.
• When compiling specific files, you can specify .tra files to be loaded along with them: COMPILE ~MyMod/quest/sword.d~ USING ~MyMod/tra/%s/sword.tra~. (The %s is replaced with the name of the language directory specified in the LANGUAGE section.)
• You can use AUTO_TRA ~MyMod/tra/%s~, which is a way to achieve the behaviour of COMPILE USING without the work - when you use COMPILE ~MyMod/quest/sword.d~, it will automatically load the .tra file in ~MyMod/tra/%s~ that has the same name (sword.tra) if it exists. This method is usually preferred in favour of COMPILE USING.

 

HANDLE_CHARSETS simply deals with fixing the encoding of the .tra files for EE games. You still need one of the above methods to actually use the .tra files in your mod.

 

The information you need to provide to HANDLE_CHARSETS is where your .tra files are located (tra_path), which .tra files should not be converted because they contain text used in the installer rather than text that will be viewed in game (noconvert_array), and which .tra files need to be reloaded after conversion because they were loaded in the LANGUAGE section before the conversion was applied (reload_array).

 

You only need to run HANDLE_CHARSETS once, in the ALWAYS section of your mod's .tp2 file to convert all of your .tra files.

[Bill Bisco]

The instructions above don't cover what Kulyok said here on the third post

 

Traifying the mod from the beginning: all you need is a Weidu.exe and A COPY(you NEED a backup) of your dialogue file in the same directory.
Just run
weidu --traify MyFile.d --nogame --out MyFile.tra

That's all!

For each file, yes, but it's as simple as that. Finally, for your setup file,
weidu --traify Setup-MyMod.tp2 --nogame --out MyMod.tra

Later, you need to create English folder for your new tra files, and place your new d files into Dialogue folder.
And place this in your .tp2:
AUTO_TRA ~MyMod/%s~
LANGUAGE ~English~ ~English~ ~MyMod/English/Setup-MyMod.tra~

Some people use the whole Translations/English/ subfolder, but it's really a pain afterwards. You can create Translations/English, if you want, though. Just use this in your tp2:
LANGUAGE ~English~ ~English~ ~MyMod/Translations/English/Setup-MyMod.tra~

There're more complex cases, like compiling a AAAA.d file which uses lines from BBBB.tra. In this case, write this in your tp2:
COMPILE ~MyMod/Dialogue/AAAA.d~ USING ~MyMod/%s/BBBB.tra~

 

 

I don't fully understand Argent's code. Why is a separate TPA for EEs necessary? I assume HANDLE_CHARSETS uses iconv somehow even though it's not directly called out?

 

I have tried to combine a mishmash of advice here

 

I'm assuming that there's no support for TRAIFING during the install process so that I don't have to Re-Traify every time I add new content to the mod? This might allow me to display Faerûn correctly without having to deal with all of the @30, @31, etc. numbers and referencing multiple files at the beginning.. Perhaps it's not recommended?

[argent77]

I don't fully understand Argent's code. Why is a separate TPA for EEs necessary? I assume HANDLE_CHARSETS uses iconv somehow even though it's not directly called out?

In many cases it's not necessary. My EE-specific TRA file only deals with new and different text content, such as item descriptions with and without usability information or innate ability descriptions which are not needed for the original games.

 

[lynx]

going back to the original questoin, û isn't part of ASCII, so unless you're using the German (latin1) or maybe Polish version, I doubt you can do much. I don't know if just recoding dialog.tlk would work or if the exes are also different in that regard.

[Avenger]

Your real problem is that the original engine used static fonts. The fonts simply don't contain the needed character without modifying their bams.

The second problem is, that normal BG2 doesn't support unicode (or any double byte character schemes). I think the chinese version does.

All this is moot in EE, because EE uses truetype fonts and already has this character correctly employed.

[Bill Bisco]

I guess that's why the original game never said Faerûn. Ironic since that's the name of the larger landmass that everyone is on.