Synchronism between BG:EE & BG2:EE (& IWD:EE)

[Jazira]

Hi,

While working on synchronism of French texts between BG:EE & BG2:EE, Lefreut and I noticed a lot of inconsistencies about the English texts between BG:EE and BG2:EE (and IWD:EE). I'm talking about the "shared" texts, the one supposed to be exactly the same, for example: spells, items, user interface, etc.

Sometimes it's just a few comma and stuff, but sometimes it is more "crucial". And there is hundreds (if not thousands?) of occurrences.

Here are just a few examples:

  

On 8/4/2022 at 11:51 AM, lefreut said:

Some inconsistencies in spell descriptions.

Cone of Cold Area of Effect: BGEE/SoD 17-ft. cone with 90-deg. arc, BG2EE 18-ft. cone with 90-deg. arc, IWDEE 20-ft. cone with 60-deg. arc.

Glyph of Warding Area of Effect: BGEE/SoD 25-ft. radius, BG2EE/IWDEE 12-ft. radius.

 

Quote

BG:EE: @26287 = ~Shield of the Archons
(Abjuration)

Level: 7
Sphere: Protection
Range: Touch
Duration: 3 rounds/level
Casting Time: 9
Area of Effect: The caster
Saving Throw: None

This powerful abjuration spell causes the spells cast against the priest to be absorbed and consumed. This affects a total of spell levels equal to half the level of the caster. This includes spells cast from scrolls and innate spell-like abilities, but excludes the following: area effects that are not centered directly upon the priest, as well as area effects that are stationary such as Cloudkill and Stinking Cloud. As long as the spell is cast directly at the priest, it will be absorbed provided that there are spell levels remaining. For example, if there is only 1 level left and a 3rd-level spell is cast at the priest, the spell will be absorbed while canceling the shield. This also works against Dispel Magic.~

BG2:EE @11015  = ~Shield of the Archons
(Abjuration)

Level: 7
Sphere: Protection
Range: Touch
Duration: 3 rounds/level
Casting Time: 9
Area of Effect: The caster
Saving Throw: None

This powerful abjuration spell causes the spells cast against the priest to be absorbed and consumed. This affects a total of spell levels equal to half the level of the caster. This includes spells cast from scrolls and innate spell-like abilities, but excludes the following: area effects that are not centered directly upon the priest, as well as area effects that are stationary, such as Cloudkill and Stinking Cloud. As long as the spell is cast directly at the priest, it will be absorbed provided that there are spell levels remaining. For example, if there is only 1 level left and a 3rd-level spell is cast at the priest, the spell will be absorbed while canceling the shield.~

These spells, items, etc. are supposed to act the same with the shared engine, right?

I know it is not game breaking, but beside the obvious inconvenience, it makes any attempt to automate synchronization of translations way harder.

So, do you think anything could be done to harmonize it, other than manually proofread it all? If not, which game is the more correct, the most proofread? The one to blindly follow? To me, from my experience, BG2:EE English texts seems to be the most proofreaded one, but I might be wrong.

Edited September 22, 2022 by Jazira

[jmerry]

With these descriptions, we should check if the effects are actually the same or different.

I don't have IWDEE, but I can check on BGEE/SoD/BG2EE.

Cone of Cold uses the CONECOLD projectile. Trap size 250, explosion size 300, flagged as cone-shaped. Identical between BGEE and BG2EE.

Glyph of Warding uses the TRAPGLYP projectile. Trap size 150, explosion size 200. Identical between BGEE and BG2EE. Though ... where are you getting that "25 ft" description? The descriptions are identical between BGEE without SoD, BGEE with SoD, and BG2EE for me.

[CamDawg]

Spells, generally, should be the same between BG, SoD, and BG2.

IWD runs its own spell systems so, despite the spell names being the same, it's OK if they vary--I don't think we can necessarily generalize one way or the other. (PsT is its own thing altogether.) Spells imported into IWD from the BG games generally should be the same as their BG counterparts, but even then there are exceptions. Mass Cure Lights Wounds was imported into IWD from BG, but it was modified to not work on undead and constructs like other IWD healing spells, as an example. The imported BG spell cosmetics and audio underwent a big revamp in 2.6 to better fit the IWD aesthetic as well.

Like jmerry, Glyph lists a 12-ft radius on all games for me (which is correct). Cone of Cold probably does need to be synced as 17 or 18 feet for BG/SoD/BG2 (the actual projectile value to actual in-game feet is an estimate at best, and using the IESDP conversion sets it as 17.6 feet), but the IWD cone is indeed different (and correct) at 60 degrees, 20 feet. For Shield of the Archons, we need to verify it one way or the other.

[Jazira]

13 hours ago, jmerry said:

Glyph of Warding uses the TRAPGLYP projectile. Trap size 150, explosion size 200. Identical between BGEE and BG2EE. Though ... where are you getting that "25 ft" description? The descriptions are identical between BGEE without SoD, BGEE with SoD, and BG2EE for me.

 

12 hours ago, CamDawg said:

Like jmerry, Glyph lists a 12-ft radius on all games for me (which is correct). Cone of Cold probably does need to be synced as 17 or 18 feet for BG/SoD/BG2 (the actual projectile value to actual in-game feet is an estimate at best, and using the IESDP conversion sets it as 17.6 feet), but the IWD cone is indeed different (and correct) at 60 degrees, 20 feet. For Shield of the Archons, we need to verify it one way or the other.

Sorry, I didn't have double-checked this one.

The 25 ft radius come from the old BG1, not in function anymore, spell description in BG:EE.

Quote

@12200 = ~Glyph of Warding
(Abjuration, Evocation)

Level: 3
Sphere: Guardian
Range: 60 ft.
Duration: Permanent until discharged
Casting Time: 9
Area of Effect: 25-ft. radius
Saving Throw: Neg. 

A Glyph of Warding is a powerful inscription magically drawn to prevent unauthorized or hostile creatures from passing, entering, or opening. It can be used to guard a small bridge, to ward an entry, or as a trap on a chest or box. Any creature violating the warded area is subject to the magic it stores. A successful Saving Throw vs. Spell enables the creature to escape the effects of the glyph. Multiple glyphs cannot be cast on the same area; although if a cabinet had three drawers, each could be separately warded. When the spell is cast, the priest weaves a tracery of faintly glowing lines around the warding sigil. When the glyph is activated, it deals 1d4 points of electrical damage per level of the caster to the victim.~

Still, those are just a very few examples, there are many more.

Edited September 22, 2022 by Jazira

[jmerry]

On Shield of the Archons - the effect of SotA is essentially identical to that of Spell Deflection, except for the numbers. It does not block AoE spells or spells with the MAGICATTACK type, and it's not dispellable, so it simply has no interaction with Dispel Magic.

Actually, we should probably just take the last phrase from the Spell Deflection description for this one: "It is not affected by a Dispel Magic".

[Jazira]

While I'm glad you responded to me about these specifics examples, what about the hundreds, if not thousands of other occurrences? And I do not exaggerate here, the following took me 3 minutes to find.

What about the "shared" texts, the one supposed to be exactly the same, for example: spells, items, user interface, etc.?

@6641  = ~Fallorain's Plate +1~ / @6641   = ~Plate Mail +1~

@6644  = ~Evermemory~ / @6644   = ~Reaching Ring~

@6652  = ~Leather armor, despite the popular misconception, is not soft and supple like the leather used to make a Ranger's boots or a Druid's robe. That kind of leather offers no better protection than common clothing.

STATISTICS:

Armor Class: 8 (10 vs. piercing and missile)
Requires:
 4 Strength

Weight: 15~

@6652   = ~Basic armor made of thick, wax or water-hardened leather. It is sturdy protection, not supple like boots or a cloak. Soft garment leathers like that would offer no more protection than common clothing. 

STATISTICS:

Armor Class: 8 (10 vs. piercing and missile)
Requires:
 4 Strength

Weight: 15~

I do believe there is a huge subject to discuss here. What should/could be done?

Edited October 3, 2022 by Jazira

[Sam.]

@Jazira

Personally I like the idea of BGEE/SoD vs BG2EE overlapping texts being unified, but I also don't consider myself an authority on what the "correct" wording is.  I could probably provide technical details like a list of which StrRefs are binarily different, but I'm not sure where that gets us.  If there are as many differences as you suggest there are (and I don't doubt there are), I struggle to envision a method of presenting the differences between A and B in such a way as to be easily digested by the masses so as to form a consensus.  Moreover, in many cases I doubt the "best" wording is either A or B, but some combination of the two.  Sure if "technical" descriptions differ but the underlying ITM files are the same there may be one correct answer, but this surely won't be the case for many of the discrepancies.  I'm willing to help, but will need suggestions on how to do so.

[jmerry]

#6641: Genuinely different items between BGEE and BG2EE. PLAT02 is Fallorain's Plate +1 in both games, with name #6641 in BGEE and #103095 in BG2EE. PLAT10 is Plate Mail +1 in both games, with name #61978 in BGEE with SOD and #6641 in BG2EE. The large numbers, in both cases, are Beamdog additions.

Actually, what was PLAT02 called in pre-EE BG2? I suspect that it was just called "Plate Mail +1", meaning that you could import the unique "Fallorain's Plate +1" and have it become generic "Plate Mail +1" in BG2. Then Beamdog renamed that item in BG2EE, creating a new string. And added generic +1 plate mail to SoD, with a new string for that name. So basically, this one's already fixed.

#6644: Genuinely different items between BGEE and BG2EE. Sure, both versions of RING08 add spell slots, but that's the only thing they have in common. BG1/BG2 did something silly, and we kind of just have to live with it.

#6652: Descriptions for LEAT01. OK, we probably don't need two different descriptions for basic leather armor. Not that I have any idea which one would be standard. This one might merit a change.

Sure, you can automate a tool to find possible discrepancies between text entries. But then every single one of them requires individual attention to even see if it's a real issue. This is starting to feel like that creature stat harmonization idea that went mostly nowhere, with stuff like hundreds of creatures that have two of their base saves inexplicably swapped. It's just too big, with too many things to do.

Edited October 3, 2022 by jmerry

[Jazira]

2 hours ago, jmerry said:

#6641: Genuinely different items between BGEE and BG2EE. PLAT02 is Fallorain's Plate +1 in both games, with name #6641 in BGEE and #103095 in BG2EE. PLAT10 is Plate Mail +1 in both games, with name #61978 in BGEE with SOD and #6641 in BG2EE. The large numbers, in both cases, are Beamdog additions.

Actually, what was PLAT02 called in pre-EE BG2? I suspect that it was just called "Plate Mail +1", meaning that you could import the unique "Fallorain's Plate +1" and have it become generic "Plate Mail +1" in BG2. Then Beamdog renamed that item in BG2EE, creating a new string. And added generic +1 plate mail to SoD, with a new string for that name. So basically, this one's already fixed.

#6644: Genuinely different items between BGEE and BG2EE. Sure, both versions of RING08 add spell slots, but that's the only thing they have in common. BG1/BG2 did something silly, and we kind of just have to live with it.

#6652: Descriptions for LEAT01. OK, we probably don't need two different descriptions for basic leather armor. Not that I have any idea which one would be standard. This one might merit a change.

Sure, you can automate a tool to find possible discrepancies between text entries. But then every single one of them requires individual attention to even see if it's a real issue. This is starting to feel like that creature stat harmonization idea that went mostly nowhere, with stuff like hundreds of creatures that have two of their base saves inexplicably swapped. It's just too big, with too many things to do.

I appreciate your desire to provide answers to each of the examples cited, but I didn't even bother to look beyond that in Near Infinity when I posted it. These are just examples to make my point. This subject is vast and is not limited to a few entries.

That's what scares me, it might require too many human resources that you (English-speaker) might not have, or don't have enough experience about, there are very few English translators/proofreaders as far as I know, translation often goes one way. I wish I could help, but I'm not English native, and honestly we're already exhausted with the work we got on the French side of the topic to provide anything meaningful.

But, the good news is that it just require a few insane, very motivated people. If a few French fools could do it, why not some English one?

edit: Also, another good news, it is not required to have advanced technical knowledge. It's really accessible to anyone who can use a text editor and Near Infinity. Beside the testing part.

Technically, this is just about the spells, the items, the user interface lines, and the engine lines. I'm pretty sure it's below the ten of thousand entries to compare and proofread. With something like what... maybe 25% of binary variations within. The hard part could be the different location (stringrefs) in the files.

2 hours ago, Sam. said:

@Jazira

Personally I like the idea of BGEE/SoD vs BG2EE overlapping texts being unified, but I also don't consider myself an authority on what the "correct" wording is.  I could probably provide technical details like a list of which StrRefs are binarily different, but I'm not sure where that gets us.  If there are as many differences as you suggest there are (and I don't doubt there are), I struggle to envision a method of presenting the differences between A and B in such a way as to be easily digested by the masses so as to form a consensus.  Moreover, in many cases I doubt the "best" wording is either A or B, but some combination of the two.  Sure if "technical" descriptions differ but the underlying ITM files are the same there may be one correct answer, but this surely won't be the case for many of the discrepancies.  I'm willing to help, but will need suggestions on how to do so.

I know this is a sensitive subject, which will raise many questions and even conflict.

As you said, I agreed the best wording isn't A or B, but a combination of the two. This is what we're trying to do with the French mods correcting the texts & sounds, most of the latest commits are good examples of this approach.

BG1:EE (alpha - do not install it) / BG2:EE (beta)

But we are at a point where the non-synchronism of the English texts between BG1:EE and BG2:EE start to handicap us. And we mostly left those entries untouched yet. This is the reason of this topic.

Edited October 4, 2022 by Jazira

[subtledoctor]

I don't fully understand what the point is? BGEE and BG2EE have different descriptions for leather armor... but they are two different games? I'm sure Dragon Age and NWN have different descriptions for leather armor as well. Why do they need to be the same? What problem does this solve?

That probably sounds dismissive but it is not my intention. My point is simply to understand better what the problem is.

[Jazira]

Again, this is not about the few descriptions and examples I cited, but hundreds, if not thousands of them. I didn't even proof check most of them because there are too many. This post is not about correcting those examples, but to discuss the subject as a whole.

@11988  = ~ALIGNMENT~ / @11988 = ~Alignment~
@11989  = ~Mage Book~ / @11989 = ~Mage Spells~
@11990  = ~GENDER~ / @11990 = ~Gender~

Sometimes it is just a little visual or logical inconvenience, a few commas, capital letters, different wording, etc. But sometimes the difference is more critical, different range, different information about spell effects, etc.

The same identical objects, spells, interface panels names, etc. are slightly, if not totally different between two games that are supposed to share the same engine, lore, spells, items, etc.

One could argue BG1:EE and BG2:EE are two different games. Story wise, yes, but the ruleset and the lore is the same for both. And for the sake of continuity and convenience, I do believe this should be synchronized.

Edited October 5, 2022 by Jazira

[jmerry]

On a systematic level, we're not going to overwrite any of those old strings that don't match. Mods don't do that. The EEs didn't do that either. If one of those needs to be replaced by a new string, we'll be making a new string and replacing references to it. Which means that we need more than just "these strings are different". We need to know where they're being used, too. After all, they might have been fixed already by replacing one or both of those references.

Taking your latest post, I'm not sure those strings are even used in the game right now. "Mage Book" and "Mage Spells"? Those don't show up as string references to 11989. They show up as literal strings in the L_en_US.lua file, and their translations would be in the corresponding files for other languages. (Relevant entries: rows 424 and 425, MAGE_BOOK_LABEL="Mage Book" and MAGE_SPELLS_LABEL="Mage Spells")

And yes, it's all about the examples. However many there are, they're just a list of examples in the end. And we can't just automate this, because lots of different things are going on and every one of them needs an individual look. Noting a discrepancy is only the first step, and it doesn't even guarantee that there's a problem at all.

[Jazira]

2 hours ago, jmerry said:

On a systematic level, we're not going to overwrite any of those old strings that don't match.

Honest question, why not?

(By "strings that don't match", you do mean "strings that are not identical"? Not the old, inactive ones, right? For the latest, of course, why would we want to lose our time about it.)

2 hours ago, jmerry said:

Mods don't do that.

Our does, French localization had the same problem but far worst, but it is understandable, since it's been through the same process of translation several times by several teams. It's still WIP, and I guess we have to wait for an "official" stance, or at least a consensus about the remaining English entries that are not identical.

I don't know if any other localization have already done the same. Anyone?

And most of the time, those corrections should have been applied directly to the Enhanced Edition through localization, not mods. Like everything present in the EEfixpack. But since 2.7 is so far off...

2 hours ago, jmerry said:

The EEs didn't do that either.

It should have been done in the Enhanced Edition, in my opinion. I am both very grateful to Beamdog over some subject, and very annoyed by their laziness over some other.

2 hours ago, jmerry said:

If one of those needs to be replaced by a new string, we'll be making a new string and replacing references to it. Which means that we need more than just "these strings are different". We need to know where they're being used, too. After all, they might have been fixed already by replacing one or both of those references.

So, we do agree. Some people should investigate the subject more deeply. That is the reason of me reporting a lot of inconsistencies. Nothing else.

2 hours ago, jmerry said:

Taking your latest post, I'm not sure those strings are even used in the game right now. "Mage Book" and "Mage Spells"? Those don't show up as string references to 11989. They show up as literal strings in the L_en_US.lua file, and their translations would be in the corresponding files for other languages. (Relevant entries: rows 424 and 425, MAGE_BOOK_LABEL="Mage Book" and MAGE_SPELLS_LABEL="Mage Spells")

And yes, it's all about the examples. However many there are, they're just a list of examples in the end.

And what about all the other valid examples? As I said, it's cool that you debunked some of them, but it is not the point of this topic.

2 hours ago, jmerry said:

And we can't just automate this, because lots of different things are going on and every one of them needs an individual look.

So, what do you all suggest?

Personally, I would try to export all the lines for both games, by categories (spells, items, ui, engine and so on), automatize a process to eliminate all the identical and unused ones. Then do a first manual pass to eliminate the ones that are not present in both game (only present in BG1:EE, for example). And then a second pass to harmonize whatever is left with the approach @Sam. (quoted below) suggested, or something like that.

On 10/4/2022 at 1:20 AM, Sam. said:

Moreover, in many cases I doubt the "best" wording is either A or B, but some combination of the two.

2 hours ago, jmerry said:

Noting a discrepancy is only the first step, and it doesn't even guarantee that there's a problem at all.

You just saw 0.1% of it by looking at my non-proof checked examples. Since some of them are valid, don't that prove there might be a problem that should be investigated more deeply?

Edited October 6, 2022 by Jazira

[Sam.]

For whatever reason I was under the impression that the first X dialog.tlk entries between BGEE/SoD and BG2EE were "theoretically" supposed to be the same (in other words the BG2EE dialog.txt started with the BGEE dialog.txt, and then added its new entries from there).  A quick test of using WeiDU to traify the DLG files and diffing the two shows this is absolutely not the case.  Is my original assumption completely wrong, or is my method flawed?

 

[Jazira]

1 hour ago, Sam. said:

For whatever reason I was under the impression that the first X dialog.tlk entries between BGEE/SoD and BG2EE were "theoretically" supposed to be the same (in other words the BG2EE dialog.txt started with the BGEE dialog.txt, and then added its new entries from there).  A quick test of using WeiDU to traify the DLG files and diffing the two shows this is absolutely not the case.  Is my original assumption completely wrong, or is my method flawed?

Can't tell about the .tlk or .txt files, I only work with the .tra one.

In BG2:EE dialog.tra, the lines are "arbitrarily" mixed together. The old ones (from BG1:EE in BG2:EE) are mostly concentrated at specific spots (with the same string ref in both game), but also contain for sure some inactive ones replaced with others (mostly in BG1:EE) later in the file, or in .lua file. That is the reason of all the false flags in my examples.

So, to complicate things even more... the active entries of the same object, spell, etc. present in both games could have different strings ref between BG1:EE and BG2:EE.

 I'm almost sure the 1046 first entries from BG1:EE and BG2:EE are story wise entries, and are consequently different. Or at least not shared together. Can't tell about IWD:EE.

You can easily export the latest dialog.tra by downloading the 2.6 patch of both BG1:EE and BG2:EE (& IWD:EE), then export the dialog.tra with Near Infinity:

Edit > String table > Export > as TRA file.

To change localization:

Options > TLK Language (EE only) > the language you want

Edited October 6, 2022 by Jazira