Edvin69

2 minutes ago, jastey said:

Hm. How about you use the correctly formatted trat files for the EE games - without HANDLE_CHARSET?  - Might be a bit difficult if it's an existing mod with several languages, but not impossible I would think.

Do I get your problem right that if you provide the tra files in ANSI then there is symbols/letters missing and that this is the problem why HANDLE_CHARSET doesn't convert it into utf-8(without BOM) properly ?

Actually, that's one of the things I've been thinking about lately.

Some old translations of old mods work without problems (old versions of weidu?), but translations of new EE mods always have a problem with fonts.
I usually use UTF-8 without BOM, then mod instalation converts my texts to ANSI and problems arise. I tried to convert my texts to ANSI and it was much better, but a few letters were still missing. The old translators obviously did something right because their translations work and mine don't (unless it's just because old version of weidu work better with czech).

57 minutes ago, AL|EN said:

@Edvin69 The authors of Tactics or even recent BG2EE-compatible version eetact2 are inactive for long time. I doubt that you will get any answers from them.

I know about that, but right now I have about 7 translated mods (I'm planning much more) and they all have the same problem.
Once I find a solution, it should be possible to apply it to everything.

4 minutes ago, argent77 said:

I can't say much about Tactics. But Ascension has encapsulated the HANDLE_CHARSETS call in another function (look for "handle_charsets_immutably" in ascension/lib/sfo_lite.tph). You won't have much freedom for that mod either.

Hmm, what if I put Czech texts in a separate folder outside the folder lang?
Wouldn't it help to avoid changes that are applied to the lang folder?
Or apply my changes only to my texts outside the lang folder.

7 minutes ago, argent77 said:

Not quite.
1. The "/czech" part from the tra_path parameter is not needed, since it's derived from the LANGUAGE directive.
2. The "noconvert_array" parameter expects the array name, which is "noconvert" (just as outlined in my code snippet above).

But from what I see you're trying to add or update a language for Ascension. In that case there is already a HANDLE_CHARSETS call defined somewhere in the script.

Since HANDLE_CHARSETS treats tra files for every language equally you might have no choice but to follow the text encoding conventions defined for that mod. I'd suggest to contact the mod author or maintainer directly if you have trouble adding your translations to the mod.

Actually, I'm trying to translate Tactic

This was just an example. I wanted to know if I understood correctly.
But Ascension is also one of the mods I'm working on (same problem with bad letters) and doesn't have HANDLE_CHARSETS in scripts (I just checked it).

My plan is to find a working solution first and then contact the authors of the mods.

7 hours ago, argent77 said:

WeiDU's HANDLE_CHARSETS function allows you to exclude tra files from the conversion process. It's commonly used to exclude setup.tra, which contains only strings for the WeiDU installer. The code can be found in the mod's .tp2 script file.

Let's take this code snippet as an example:

ACTION_DEFINE_ARRAY noconvert BEGIN ~setup~ ~lines-ee~ END
ACTION_DEFINE_ARRAY reload BEGIN ~lines~ END
LAF HANDLE_CHARSETS
  INT_VAR
    infer_charset = 1
  STR_VAR
    tra_path        = ~MyModFolder/languages~
    noconvert_array = ~noconvert~
    reload_array    = ~reload~
END

The "noconvert" array contains all tra filenames (without .tra extension) which should be excluded from the conversion process. In this example the files setup.tra and lines-ee.tra will be excluded. You can add your own tra files to this array to have them excluded as well.

Yes, that's exactly what I need!
I just I make sure that I understood correctly.

Let's say this is the path:             ascension/lang/czech

Tra files are:         ascension.tra, setup.tra, epilogues.tra

 ACTION_DEFINE_ARRAY noconvert BEGIN ~ascension~ ~setup~ ~epilogues~ END

ACTION_DEFINE_ARRAY reload BEGIN ~lines~ END

  LAF HANDLE_CHARSETS
    INT_VAR
      infer_charsets = 1
    STR_VAR
      tra_path                =   ~ascension/lang/czech~
      noconvert_array  =  ~ascension~ ~setup~ ~epilogues~
      reload_array         =  ~reload~
  END

Is this correct?

On 8/5/2019 at 7:48 PM, argent77 said:

Converting the files with Notepad++ may be complicated, since it converts from UTF-8 into the default ANSI codepage (whatever that is on your system). I'd suggest to use the command line tool 'iconv' instead (the same tool WeiDU calls internally).

iconv binary can be found here: http://gnuwin32.sourceforge.net/packages/libiconv.htm

The syntax is:

iconv -c -f utf-8 -t cp1250 sourcefile-in-utf8.tra >newfile-in-cp1250.tra

 

Let's say I want a mod that will be compatible only with EE and not original BG2.
(If I understand well, this is the main reason why the texts must be converted.)
Would be possible to use some script that tells weidu "DO NOT TOUCH MY TRA FILES!!!" ?

18 minutes ago, Jarno Mikkola said:

See the thing is, we kinda need the whole mod ... as we can't just use a .tra file, as we don't have a .tp2 file to add in the HANDLE_CHARSET command, and it needs another folder and a file too.

It would be much easier if I didn't have to work with such an ancient mod...

It have two .tp2 file and so confusing setting that I was unable to install Czech until I rewrote the English language.

I've been trying to solve this for several days.

If you can figure out why it does not install Czech (I'm almost sure that I set the installation path correctly) and how to edit text to display them properly, I will call you Sensei.

https://gofile.io/?c=eJ5m6h

13 minutes ago, Jarno Mikkola said:

The way you say this is horrific...

When making the .tp2 file, it needs to be encoded in the " standard UTF-8 withnout BOM " ... the .tra files can be either ANSI if you know what that ends up being, or in CP1250, if you don't, and then reset that to be the ANSI encoding with Notepad++ for example... which switches the special characters to be "normal" characters until they get reverted back during install.

And the weidu install will then switch the ANSI characters to be the CP1250 ... by the HANDLE_CHARSET, when the install is done via Czech language.

 

So, the .tp2 and .tpa are encoded in UFT-8 without BOM.

And the .tra files are encoded in ANSI.

I liked the simple way he said it.
From your complex explanation I'm just more confused. 

Ok, I'll try it another way.
This is one of my translated dialog files.

Could someone please modify it to make it work correctly?
Then I use reverse engineering to apply it to other files.

ilysetup.tra

11 minutes ago, argent77 said:

Text in mods that are not exclusively aimed at Enhanced Edition games has to be encoded in ANSI. By (WeiDU) default this is CP1250 for Czech. The mod itself will convert it automatically to UTF-8 if needed.

You can check if text conversion is performed by looking for a call to HANDLE_CHARSETS in the mod's .tp2 script file. More details about how text conversion works can be found in the WeiDU Documentation.

That's exactly the information I needed.

Thank you very much.

So if I convert my translations into CP1250, everything should work properly, right?

Recently I have translated a large number of modes into the Czech language.
I use standard UTF-8 withnout BOM.

But for some reason, during the installation of the mods, all the texts I translated are converted to ANSI (coding in the files I translated is physically changed).
The problem is that the Czech language has a large number of special letters that are not supported by ANSI (š,č,ř,ž,ý,é,á  etc.)
So the translated files are now full nonsenses and same nonsenses can be seen in the game.

I tried to convert texts ANSI (cut the texts, convert the file and reinsert them into the ANSI tra file) and to my surprise it almost worked.
I still lost a few letters, but it was much better than before.

Can anyone advise me what I am doing wrong, where is the problem, or how to fix it?
(Preferably all three.)