Porting WeiDU mods to Linux - initial thoughts

[Avenger]

I didn't know there are so many people who run the IE on linux.

[Fyorl]

This seems to be the place to discuss it so I thought I'd just say that I switched from using Cedega to run BG2 to Wine after I compared performance with thebigg.

 

Wine offers far superior performance for me. With Cedega I had to disable party AI until a battle and run in 640x480 to get it playable. With Wine I can happily run around in 1024x768 (btw, I never realised it was so huge o_o).

 

Software Mirror BLT needs to be on for Cedega or Wine though. Also, Wine does have audio/mouse pointer issues as thebigg mentioned. If you press alt+f4 and select 'No' (when Boo says he'll miss you) then that will apparently clear up the mouse pointer and sound echo issues although for me it just crashes.

 

Also, I have to set my screen's resolution to match that of BG2 when using Wine or otherwise it crashes.

 

So there are a few more issues with Wine than Cedega but Wine does give much better performance for me. It also has the benefit of detecting my CDs correctly so I don't have to resort to dirty no-cd patches with Cedega.

 

I had no idea you could do WeInstall foo... I just installed a huge load of mods with just WeInstall setup-foo.tp2. Worked fine for me which is a credit to the creator ^_^ of WeInstall

 

Also, I always download the windows versions and just run ark foo.exe &

Then I tell it to open it as a rar archive and it works fine. For the executables that aren't self-extracting rars then running Wine on them works fine.

 

So far I've just been ignoring bat files completely and I haven't noticed any problems but then again I only half an idea of what I've installed so I could be missing stuff and just not notice it. Sound unpacking seems to be the biggest problem. I had no idea my NPC mods had sound until I read this thread and realised all those ogg files in my override were supposed to be decoded.

[ronin69hof]

Something weird with bg2fixpack v.4 core fixes.

 

Cedega 6.0.4 and full install of SoA and TOB with official patch and bg2fixpack v.4 fails on spcl212.spl (paladin detect evil) searched in the BG2 dir and didn't find it there.

 

Wine 0.9.52 with also full install of SoA and TOB with official patch and bg2fixpack v.4 fails on the same spell.

 

Did complete reinstalls in different directories and double checked to make sure that they were full installs and same thing happens

 

If I comment it out then it fails on the very next one (spin120.spl detect evil (bhaalspawn innate)) and so on and so forth.

 

Same thing on Winblows XP and it does not fail. Compared the windows installation with SoA/TOB and official patch with the linux one and they are identical.

 

 

What gives?

 

OS is Ubuntu 7.10

[Guest erik]

Hmm. Installing the mods with wine setup-blargmod.exe or with linux weidu (tolower; weinstall blarg)?

 

(Not that I've had any problems with either approach, but... last time I used the fixpack v4 OSX package and weinstall)

[ronin69hof]

I used WeInstall, I havent tried the OSX package so I will give that a go. Before someone asks I did run tolower in the BG2 directory.

[cmorgan]

OK, I surrender - and throw myself on all you folks mercy.

 

the bigg was very cool and tried to help me figure this out, but I must be missing something, and that means probably anyone else who is not a serious linux person will to...

 

Converting from the OSX package to the LIN package.

 

The contents of myModunpack.sh:

 

#!/bin/bash

mkdir music/mxm\#am //make a directory called music/mxm <<unknown>>

 

cp sarahsoa/audio/*.ogg music/mxm\#am // copy the .oggs to music/mxm <<unknown>>

 

cp sarahsoa/audio/snd2acm.exe music/mxm\#am // copy the .exe to music/mxm <<unknown>>

 

cd music/mxm\#am // change to the new directory

 

oggdec mxm\#*.ogg // run oggdec from usr/bin on all the .ogg files

 

rm mxm\#*.ogg // delete all the .ogg files

 

wine snd2acm.exe -wav mxm\#K#SaraM1.ogg mxm\#K#SaraM1.acm // run the .exe to convert the .wav files to .acm files, one at a time (bloody long list)

 

wine snd2acm.exe -wav mxm\#K#SaraM2.ogg mxm\#K#SaraM2.acm (etc.)

 

rm mxm\#*.wav // delete all the old .wav files

 

rm snd2acm.exe // delete the copy of the .exe

 

cd .. // move up the directory tree

 

cd .. // move up the directory tree

 

cd override // move to override

 

OK, now I think I have most of this... except how to move them to the override. I think the bigg was using ADD_MUSIC, but I am not sure -

 

will simply

 

cp sarahsoa/audio/*.acm override/#am

 

work, without the directory changes?

 

 

on OSX I do

 

for file in $ogg_files; do
 echo Converting $file
 SarahSoA/BatchFiles/sox SarahSoA/audio/$file override/${file%.ogg}.wav
done

 

and we already have

 

ACTION_IF ("%WEIDU_OS%" STRING_COMPARE_CASE ~WIN32~ = 0) THEN BEGIN // if Windows
 AT_NOW ~SarahSoA/BatchFiles/sarahsoaaudio.bat~
 AT_INTERACTIVE_UNINSTALL ~SarahSoA/BatchFiles/sarahsoaaudiouninstall.bat~
END ELSE BEGIN // if OS X or Linux
 AT_NOW ~sh SarahSoA/BatchFiles/sarahsoaaudio.sh~
 AT_INTERACTIVE_UNINSTALL ~sh SarahSoA/BatchFiles/sarahsoaaudiouninstall.sh~
END

 

 

This is a little like DOS; just enough to get me confused...

 

second problem - do i need to escape the # in a community prefix?

 

like for instance k# or c- or b!

[Steve]

> will simply

> cp sarahsoa/audio/*.acm override/#am

> work, without the directory changes?

 

Yes to not needing the directory changes. But if you mean to prepend #am to each .acm file, then no. The corresponding lines for bash would be:

for file in sarahsoa/audio/*.acm; do

cp -f $file override/#am${file##*/}

done

 

> second problem - do i need to escape the # in a community prefix?

 

No, unless the # is the first character of the word, which the shell would then interpret as a comment like the C++ // token.

 

> OK, now I think I have most of this... except how to move them to the override. I think the bigg was using ADD_MUSIC, but I am not sure -

 

Well, I'm not sure since the ADD_MUSIC command is undocumented, but in the Chloe mod, the .tp2 says:

ADD_MUSIC ~ChGTKUCR~ ~Chloe\Music\CRblank1.mus~

and later on says:

COPY ~Chloe\Music\ChGTKUCR.crm~ ~override\ChGTKUCR.ogg~

and converts it to .wav in an AT_EXIT batch file. It seems that ADD_MUSIC just adds an entry to the SONGLIST.2DA file, probably checking it doesn't go over 100 entries.

 

> AT_NOW ~sh SarahSoA/BatchFiles/sarahsoaaudio.sh~

 

I don't know if all versions of Linux have sh being a version of bash, so it's probably safer to say:

AT_NOW ~bash SarahSoA/BatchFiles/sarahsoaaudio.sh~

 

The #!/bin/bash in your file is ignored unless it's run as a program rather than an input file to a shell.

 

P.S.- I have the source to snd2acm/acm2snd if you want it. I found it on the TeamX Russian site a while ago. I modified it to run on Macs, but if you delete the -DMACOSX from the makefile it should compile on Linux.

 

P.P.S.- It sounds like you're making an installer for Linux. I wrote a converter/installer for Macs that automatically converts Windows mods. It also uses bash so if you want to look at it, it's here: http://home.comcast.net/~shorter18/Install-tools.zip

[cmorgan]

> will simply

> cp sarahsoa/audio/*.acm override/#am

> work, without the directory changes?

 

Yes to not needing the directory changes. But if you mean to prepend #am to each .acm file, then no. The corresponding lines for bash would be:

for file in sarahsoa/audio/*.acm; do

cp -f $file override/#am${file##*/}

done

 

I guess what I am missing is what the /#am is actually doing - is it regexp filtering for just files that begin with "am"?

 

So that the shortcut for all of this would be

 

for file in sarahsoa/audio/*.acm; do

cp -f $file override/${file##*/}

done

 

 

> AT_NOW ~sh SarahSoA/BatchFiles/sarahsoaaudio.sh~

 

I don't know if all versions of Linux have sh being a version of bash, so it's probably safer to say:

AT_NOW ~bash SarahSoA/BatchFiles/sarahsoaaudio.sh~

 

The #!/bin/bash in your file is ignored unless it's run as a program rather than an input file to a shell.

 

Hmmm... that means I had better look up the easy way to express "If windows do x, if mac do y, if linux do z" using %WEIDU_OS%.

 

P.S.- I have the source to snd2acm/acm2snd if you want it. I found it on the TeamX Russian site a while ago. I modified it to run on Macs, but if you delete the -DMACOSX from the makefile it should compile on Linux.

 

 

Thanks, but I had better stick with the standard snd2acm.exe - I am the equivalent of a guy who rides bicycles trying out a 747 cockpit for the first time - with the Microsoft Flight Simulator docs as a guide

 

P.P.S.- It sounds like you're making an installer for Linux. I wrote a converter/installer for Macs that automatically converts Windows mods. It also uses bash so if you want to look at it, it's here: Install-tools.zip

 

 

Thank you!

 

I will take a look at it immediately. The goal is to get the G3 mods distributed in all three flavors (win/osx/lin) and while I can do the windows and follow the template of the osx materials, the linux stuff is definitely kicking me hard.

[Guest erik]

for file in sarahsoa/audio/*.acm; do

cp -f $file override/#am${file##*/}

done

 

I guess what I am missing is what the /#am is actually doing - is it regexp filtering for just files that begin with "am"?

 

Heh. You're thinking far too complex right now. The only tricky part is the bit inside the curly brackets - the rest is no more complex than dos. Think "copy fromname toname".

 

cp $file override/nicenameprefix${file##*/} ;

 

The bits inside the curly brackets strips away the directory part of the filename.

 

cp $file override/nicenameprefix$(basename $file) ;

 

While this is a slower, more compatible and more readable variant you might want to use instead.

 

So that the shortcut for all of this would be

 

for file in sarahsoa/audio/*.acm; do

cp -f $file override/${file##*/}

done

 

Yep, mostly. Add semicolons like this and it should work:

 

for file in sarahsoa/audio/*.acm; do

cp -f $file override/${file##*/};

done;

 

> AT_NOW ~sh SarahSoA/BatchFiles/sarahsoaaudio.sh~

 

I don't know if all versions of Linux have sh being a version of bash, so it's probably safer to say:

AT_NOW ~bash SarahSoA/BatchFiles/sarahsoaaudio.sh~

 

They don't. Ubuntu is a case in point. A bunch of things break because of that.

 

Doing it the $(basename $file) way doesn't need bash, however.

 

I will take a look at it immediately. The goal is to get the G3 mods distributed in all three flavors (win/osx/lin) and while I can do the windows and follow the template of the osx materials, the linux stuff is definitely kicking me hard.

 

... and the bat-to-bash autotranslating script linked above will probably kick you harder. I wouldn't want to understand it, that's Nythrun-grade magic. Stay away from the missile launch controls, please

[Steve]

> I guess what I am missing is what the /#am is actually doing - is it regexp filtering for just files that begin with "am"?

 

I see the confusion. You wanted to know what this line meant:

mkdir music/mxm\#am

All it's doing is making a directory called music/mxm#am . The backslash is used to escape the next character (make the shell ignore any special meaning it may have). If all you're doing is copying files, then just:

cp -f sarahsoa/audio/*.acm override

 

> Hmmm... that means I had better look up the easy way to express "If windows do x, if mac do y, if linux do z" using %WEIDU_OS%.

 

You could, but I imagine Mac and Linux would be the same. Macs use bash as its sh.

[cmorgan]

New syntax from the bigg, for ease of future win/osx/linux comfort:

 

// scripts to compress/decompress sounds
ACTION_IF (~%WEIDU_OS%~ STRING_EQUAL_CASE ~win32~) THEN BEGIN
 AT_NOW				   ~bg1npcmusic/allaudio_win.bat~
 AT_INTERACTIVE_UNINSTALL ~bg1npcmusic/allaudio_win.bat~
END ELSE BEGIN
 ACTION_IF (~%WEIDU_OS%~ STRING_EQUAL_CASE ~osx~) THEN BEGIN
AT_NOW				   ~sh bg1npcmusic/allaudio_osx.sh~
AT_INTERACTIVE_UNINSTALL ~sh bg1npcmusic/allaudio_osx.sh~
 END ELSE BEGIN
AT_NOW				   ~bash bg1npcmusic/allaudio_lin.sh~
AT_INTERACTIVE_UNINSTALL ~bash bg1npcmusic/allaudio_lin.sh~
 END
END

 

And a reconfirmation of a sample audioscript for Linux distributions, to make sure I am doing this right: if I have all of my materials in ../music/

 

(and instead of compressing all the temp files, just dump 'em like we do on Windows when we just move them to override)

 

audio_install_linux.sh

#!/bin/bash
mkdir music/temp
cp myMod/music/*.ogg music/temp
oggdec music/temp/*.ogg
rm music/temp/*.ogg
for file in myMod/music/*.wav; do
cp -f $file override/${file##*/};
done;
rm music/temp/*.wav

 

audio_uninstall_linux.sh

rm override/myAudioFile1.wav
rm override/myAudioFile2.wav
rm override/myAudioFile3.wav
rm override/myAudioFile4.wav

 

Can someone give me a hand and rebuild my work into something pretty failsafe?

(speak slowly, as if to a very stuipd person ewith minimal language skills, please!)

 

 

edit:

or is it even simpler:

 

#!/bin/sh

for file in gavin/audio/*.ogg; do
oggdec $file gavin/audio/${file##*/};
done;

for file in gavin/audio/*wav; do
mv $file override/${file##*/};
done;

 

or simpler still:

#!/bin/sh
oggdec gavin/audio/*.ogg
mv gavin/audio/*.wav override

[Steve]

I don't know if 'oggdec' is standard on all Linuxes but I'm pretty sure 'sox' is.

 

Simpler is better:

 

#!/bin/sh

oggdec gavin/audio/*.ogg

mv -f gavin/audio/*.wav override

 

The biggest problem you'll have with Linux is that it runs on a case-sensitive file system, unlike Windows and Macs. You will want to lowercase all filenames and shell scripts in the distribution. For audio_uninstall_linux.sh:

 

rm -f override/myaudiofile1.wav

rm -f override/myaudiofile2.wav

rm -f override/myaudiofile3.wav

rm -f override/myaudiofile4.wav

 

Or, if you want to be clever you can do this: (yes, now I'm just showing off)

 

rm -f `ls gavin/audio | sed -n 's,\(.*\)\.ogg,override/\1.wav,p'`

 

Edit: let's really be clever and make it actually work.

[cmorgan]

So use sox or use oggdec?

 

I have already lowercased all files across all materials in each package, and removed the .exe files, as per the bigg's instruction. That meant rechecking that folks had used patching routines that were case insensitive, but 99% were, so the last scare is the audio, doing it right.

 

The instructions with each mod have the user download the latest WeiDU files for Linux to usr/bin/ ,

That would just leave tisunpack and snd2acm to be called and used in directory structures.

 

(That last line looks way cool, but at this point I am operating on

 

10 BEGIN

20 GOTO 30

30 EXIT

 

 

wine snd2acm.exe -wav directory\myFile1.ogg directory\myFile1.acm

 

I assume this is compressing the sound.

[cmorgan]

OK, trial conversion, working through the purmutations, to double check my thinking - then I am going to beg/plead/bribe someone to test this on their Lnux install so I have confrimation. Starting with a relatively simple one.

 

Song and Silence. All files, directory names, etc. lowercased. tp2 checked for STRING_EQUALS or special stuff like DavidW's PEARL materials which need a rebuild of his .exe. [in this case, no problems found]. Updated audio scripts:

 

Windows: ships with oggdec, .bat files saved with Windows CR LF

audio-install-win.bat

@echo off
song_and_silence\oggdec song_and_silence\items\a!ra0*.ogg
move song_and_silence\items\a!ra0*.wav override

audio-uninstall-win.bat

@echo off
del override\a!ra0*.wav

 

OS X: ships with sox, .sh files saved with UNIX LF

audio-install-osx.sh

#!/bin/sh

ogg_files="a!ra01.ogg a!ra02.ogg a!ra03.ogg a!ra04.ogg"

for file in $ogg_files; do
 echo Converting $file
 song_and_silence/sox song_and_silence/Items/$file override/${file%.ogg}.wav
done

audio-uninstall-osx.sh

#!/bin/sh

wav_files="a!ra01.wav a!ra02.wav a!ra03.wav a!ra04.wav"

for file in $wav_files; do
 rm override/$file
done

 

Linux: ships with oggdec saved as UNIX LF

audio-install-lin.sh

#!/bin/sh

oggdec song_and_silence/Items/*.ogg

mv -f song_and_silence/Items/*.wav override

audio-uninstall-lin.sh

#!/bin/sh

rm -f override/a!ra01.wav 
rm -f override/a!ra02.wav 
rm -f override/a!ra03.wav 
rm -f override/a!ra04.wav

 

It seems like the special character escape was applying in mid word in that above scripts - even though it is not in the first position, should that last be

 

rm -f override a\!ra01.wav

 

or could be

 

rm -f override "a!ra01.wav"

 

Since it is within a shell, no special escaping required.

 

For mods that require audio compression, (it looks like Amber does this because she adds tons of .wav files to the override, and the .acm files are smaller...)

 

oggdec directory/myFile1.ogg
wine snd2acm.exe -wav directory/myFile1.wav directory/myFile1.acm

[Steve]

> So use sox or use oggdec?

 

I see that the WeiDU distribution for Linux includes both so you get to choose. These two fragments are functionally equivalent:

 

oggdec gavin/audio/*.ogg

mv -f gavin/audio/*.wav override

 

for x in gavin/audio/*.ogg; do

sox $x ${x%.ogg}.wav

done

mv -f gavin/audio/*.wav override

 

 

> wine snd2acm.exe -wav directory\myFile1.ogg directory\myFile1.acm

 

I don't think this will work. You need to convert from .ogg to .wav first:

 

oggdec directory/myFile1.ogg

wine snd2acm.exe -wav directory/myFile1.wav directory/myFile1.acm

 

The snd2acm source that I have can accept only Raw or WAV at 22050Hz as input. The output is Interplay's own compressed WAV format.