GemRB for Android compilation attempt..

rmtew · July 7, 2012

Alright, I've done some preliminary attempts to compile GemRB with commandergenius. Here's the steps I cobbled together that work for me, with Windows and MingGW installed.

git clone https://github.com/p...ommandergenius
cd commandergenius
pushd
cd project/jni/application
ln -s gemrb src
cp src/AndroidAppSettings.cfg ../../
popd
./build.sh
# This gets to the actual build step and chokes on "export $NDKBUILDPATH ..." due to MingW not liking spaces in the path.
... edit build.sh and replace "env PATH=$NDKBUILDPATH nice -n19 ndk-build V=1 -j4" with "env PATH=$NDKBUILDPATH nice -n19 ndk-build V=1 -j4"
export PATH=$PATH:/d/Code/android-ndk-r8
./build.sh

At this point, the following compilation errors occur:

In file included from jni/../jni/sdl-1.3/src/video/android/SDL_androidinput.c:43:jni/../jni/sdl-1.3/include/SDL_screenkeyboard.h:1: error: expected identifier or '(' before '.' token

This file was mismerged on github, and contains garbage.

If I clear out this file so this error and fallout from it do not occur, then further errors occur:


jni/../jni/sdl-1.3/src/video/android/SDL_touchscreenkeyboard.c:980: error: 'SDL_ANDROID_SCREENKEYBOARD_BUTTON_5' undeclared (first use in this function)

It looks like the commandergenius project is currently buggy. There's also some shenanigans where files are supposed to be symlinks, but actually (at least on my windows clone) appear to be just files containing the given filename.

rmtew · July 7, 2012

Hmm, the symlinked files being turned into files with the path of the symlinked file, is discussed here on stackoverflow.

rmtew · July 7, 2012

Okay, manually symlinking the sdl-1.2/include/SDL_screenkeyboard.h into 1.3 has the same problem in MingGW. So I copied the file in, and then compilation proceeded satisfactorily. But it does not appear to be compiling gemrb itself, only sdl I think.

Here are my new steps:

git clone https://github.com/p...ommandergenius
cd commandergenius
rm AndroidAppSettings.cfg
ln -s project/jni/application/gemrb project/jni/application/src
cp project/jni/application/src/AndroidAppSettings.cfg . # WINDOWS /
cp project/jni/sdl-1.2/include/SDL_screenkeyboard.h project/jni/sdl-1.3/include/
sed -i 's/env PATH\=\$NDKBUILDPATH nice \-n19 //g' build.sh

This is what it exits with:

./obj/local/armeabi/objs-debug/sdl_main/sdl_main.o: In function `Java_net_sourceforge_gemrb_DemoRenderer_nativeInit':
d:\VCS\git\commandergenius\project/jni/../jni/sdl_main/sdl_main.c:13: undefined reference to `SDL_main'
collect2: ld returned 1 exit status
make.exe: *** [obj/local/armeabi/libsdl_main.so] Error 1

My current assumption is that commandergenius compiles SDL for a given application in a custom way, and then compiles the application. What I need to do next is compile the application, and presumably that requires the Java SDK to be installed and "ant release" to be run in the project directory. Working on seeing if this is correct.

SyntaxError · July 7, 2012

your github links are 404s (they have a "." at the end)

also have you tried building official SDL? Its unclear if this fork supports the multitouch API. it is still versiond as 1.3 so im guessing it hasnt remerged with upstream changes in quite some time.

Edit: in fact if you read the readme on that page it says to use official SDL 2 if possible (and i dont see why its not).

rmtew · July 7, 2012

Okay, finally managed to get java using the jdk, rather than the jre.

$ ant release
Buildfile: d:\vcs\git\commandergenius\project\build.xml
BUILD FAILED
d:\vcs\git\commandergenius\project\build.xml:7: Source resource does not exist:
d:\vcs\git\commandergenius\project\local.properties
Total time: 0 seconds

Apparently "android project update" needs to be executed to generate this. Still working out how to do this from mingw.

rmtew · July 7, 2012

your github links are 404s (they have a "." at the end)

also have you tried building official SDL? Its unclear if this fork supports the multitouch API. it is still versiond as 1.3 so im guessing it hasnt remerged with upstream changes in quite some time.

Edit: in fact if you read the readme on that page it says to use official SDL 2 if possible (and i dont see why its not).

It's all undefined. Where are the instructions to build GemRB with SDL 2? There's barely any instructions that work to do it with commandergenius, and look at how complicated that is turning out to be.

SyntaxError · July 7, 2012

to compile gemrb with sdl 2 all you need to do is have libSDL2 and then compile SDL20Video.cpp/h instead of SDL12Video.cpp/h

rmtew · July 7, 2012

to compile gemrb with sdl 2 all you need to do is have libSDL2 and then compile SDL20Video.cpp/h instead of SDL12Video.cpp/h

I can compile applications for Windows or Linux no problem. But how to compile for Android is the unknown :-)

rmtew · July 7, 2012

Alright, after stalling for a while, I've made some progress. I'm going to keep writing up my notes, so there's less work to do later on.

Get commandergenius and gemrb linked. Can also use a symbolic link, but the mingw ln command is.. unique in behaving.. differently.

git clone https://github.com/p...ommandergenius
git clone https://github.com/gemrb/gemrb.git
pushd .
cd gemrb
git archive --format=tar -o ../commandergenius/project/jni/application/gemrb/gemrb/gemrb.tar HEAD
cd ../commandergenius/project/jni/application/gemrb/gemrb
tar xf gemrb.tar
rm gemrb.tar
popd

Now to fix the problems with commandergenius. These are caused on two fronts, GIT and MinGW. Both GIT and MinGW do the wrong thing with regard to symbolic links. GIT will on cloning, instead of making a link, will make a file with the path to the file it should be linked to inside. MinGW will do the same, if asked to make links to files. MinGW will also choke on "env PATH=$XXX command" if XXX has spaces in it.

cd commandergenius
rm AndroidAppSettings.cfg # Git corruption.
cp project/jni/application/src/AndroidAppSettings.cfg . # Git corruption
cp project/jni/sdl-1.2/include/SDL_screenkeyboard.h project/jni/sdl-1.3/include/ # Git corruption / MinGW file links
sed -i 's/env PATH\=\$NDKBUILDPATH nice \-n19 //g' build.sh # MingW path spaces.

Now to set up commandergenius for gemrb.

ln -s project/jni/application/gemrb project/jni/application/src
./build.sh

Okay. Now this compiles SDL, and gemrb. But I get errors:

cc1plus.exe: error: jni/../jni/application/../python/include: not a directory
cc1plus.exe: error: jni/../jni/application/../python/include: not a directory
cc1plus.exe: error: jni/../jni/application/../python/include: not a directory
make.exe: *** [obj/local/armeabi/objs-debug/application//gemrb/gemrb/gemrb/core/ActorMgr.o] Error 1

This actually looks easy to fix. And it is, it is the GIT symlink problem. If I manually make this directory symlink, then compilation proceeds towards more errors.

In file included from jni/../jni/application//gemrb/gemrb/gemrb/core/System/String.h:24,
			 from jni/../jni/application//gemrb/gemrb/gemrb/includes/win32def.h:33,
			 from jni/../jni/application//gemrb/gemrb/gemrb/core/AmbientMgr.h:25,
			 from jni/../jni/application//gemrb/gemrb/gemrb/core/AmbientMgr.
cpp:21:d:/Code/android-ndk-r8/sources/cxx-stl/gnu-libstdc++/include/cstring:76: error:'::memchr' has not been declared
d:/Code/android-ndk-r8/sources/cxx-stl/gnu-libstdc++/include/cstring:77: error:'::memcmp' has not been declared
d:/Code/android-ndk-r8/sources/cxx-stl/gnu-libstdc++/include/cstring:78: error:'::memcpy' has not been declared

Not so sure about this one. There are references when googling, to it being because of an include in a namespace. Not the case.

My best guess is that this is related to order of imports.

rmtew · July 7, 2012

This is a wasted effort. Advice to other Windows git users: do the following command:

git config --global core.symlinks true

Now you will get a warning when your git is making a corrupt fake symlink, rather than having your code silently choke on it. Git for Windows/msysgit will not support symlinks. cygwin does not support symlinks. git under both creates corrupt fake symlinks.

Pelya's advice to use Portable Ubuntu, for Windows users, is probably the best idea.

rmtew · July 8, 2012

The include ordering problem is now sorted, and the symlink problem seems to be sorted for now as well.

The include ordering problem is caused because the Android developers intentionally decided to move the NDK-related include paths, which of course include the platform and STL include files, last. This means that on a case-insensitive OS like Windows, GemRB's String.h is going to be found BEFORE the platform string.h. Suddenly the std namespace won't be providing string-related members. The linked post handwaves impractical solutions like "change your project so you don't have this problem" but in the real world, you can't reasonably be expected to do this for other people's projects you are using for whatever reason.

The symlink problem was sorted by writing a bash script. Symbolic linking within cygwin, mingw/msys or msysgit is broken, given that it doesn't work properly -- if at all. When it doesn't work properly, chances are it errors with no clear explanation why. This may be because it won't create symlinks in other directories from your current one, but it will create the exact same symlink if you make the directory the link is to be created in your current one. When it does work properly, it either creates a corrupted symlink, which will be a text file containing the path where it should be symlinking. Or it may proceed to copy the directory you are trying to make a link to. And don't even think of making a symlink to the current directory, from within the current directory. That just does not work, because symlinking in these environments does not really work. But you're not going to know that from the error messages..

As they, whomever they may be, say.. experience is what you get when you don't get what you want.

Here's the script. Run it within Mingw shell, but make sure you run Mingw shell as administrator. This is needed for the Windows symlinking to work.

#!/bin/sh
# This code is licensed under the MIT license.
git ls-files -s | awk '/120000/{print $4}' | while read FILEPATH
do
if [ ! -e $FILEPATH ]; then
	echo "Please check that GIT core.symlink for this project is 'false'" 1>&2
	exit 1
fi
if [ ! -L $FILEPATH ] && [ -f $FILEPATH ]; then
	wc -l $FILEPATH | while read LINES DISCARD
	do
		if [ "$LINES" = "0" ]; then
			FILEDIR=`dirname $FILEPATH`
			FILENAME=`basename $FILEPATH`
			LINKPATH=`cat $FILEPATH`
			pushd . > /dev/null
			cd $FILEDIR
			if [ -e $FILENAME ]; then
				if [ -e $LINKPATH ]; then
					echo "Repairing symlink: $FILEPATH"
					if [ -f $FILEPATH ]; then
						rm $FILENAME
						cmd /c "mklink ${FILENAME//\//\\} ${LINKPATH//\//\\}"
					else
						rm $FILENAME
						cmd /c "mklink /d ${FILENAME//\//\\} ${LINKPATH//\//\\}"
					fi
				else
					echo "$FILEPATH: symlink path '$LINKPATH' invalid" 1>&2
				fi
			else
				echo "$FILEPATH: problem finding this file"
			fi
			popd > /dev/null
		else
			# ERROR: Expected a file with one line for the link path
			echo "$FILEPATH: bad link file: expected 0 lines, got $LINES" > /dev/null # 1>&2
		fi
	done
else
	echo "$FILEPATH: already a symlink" > /dev/null # 1>&2
fi  
done

rmtew · July 8, 2012

A lesson in further problems with symbolic links on Windows...

Let's say you make a Windows symbolic link with mklink, you've even run as administrator to do it. Now you can see the link in an explorer folder view with a shortcut icon.

MinGW shell almost believes it works when you use the file command on it:

$ file project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
project/jni/sdl-1.2/src/video/android/SDL_androidinput.h: ASCII C program text, with CRLF line terminators

If you try and copy the file that it links to over top of it, that seems to think they're the same:

$ cp -f project/jni/sdl-1.3/src/video/android/SDL_androidinput.h project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
cp: `project/jni/sdl-1.3/src/video/android/SDL_androidinput.h' and `project/jni/sdl-1.2/src/video/android/SDL_androidinput.h' are the same file

Heaven forbid you try and delete the symbolic link, even though you're running as adminstrator:

$ rm -f project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
rm: cannot remove `project/jni/sdl-1.2/src/video/android/SDL_androidinput.h': Not owner

A bit of googling suggests trying the rmdir command:

$ rmdir project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
$ file project/jni/sdl-1.2/src/video/android/SDL_androidinput.h
project/jni/sdl-1.2/src/video/android/SDL_androidinput.h: ERROR: cannot open `project/jni/sdl-1.2/src/video/android/SDL_androidinput.h' (No such file or directory)

Why am I deleting this file?

Compile thumb  : sdl-1.2 <= SDL_androidinput.c
/d/Code/android-ndk-r8/toolchains/arm-linux-androideabi-4.4.3/prebuilt/windows/bin/arm-linux-androideabi-gcc -MMD -MP -MF ./obj/local/armeabi/objs-debug/sdl-1.2/src/video/android/SDL_androidinput.o.d.org -fpic -ffunction-sections -funwind-tables -fstack-protector -D__ARM_ARCH_5__ -D__ARM_ARCH_5T__ -D__ARM_ARCH_5E__ -D__ARM_ARCH_5TE__  -Wno-psabi -march=armv5te -mtune=xscale -msoft-float -mthumb -Os -fomit-frame-pointer -fno-strict-aliasing -finline-limit=64 -Ijni/../jni/sdl-1.2/include -Ijni/../jni/sdl-1.2 -DANDROID -O3 -DSDL_JAVA_PACKAGE_PATH=net_sourceforge_gemrb -DSDL_CURDIR_PATH=\"net.sourceforge.gemrb\" -DSDL_TRACKBALL_KEYUP_DELAY=1 -DSDL_VIDEO_RENDER_RESIZE_KEEP_ASPECT=1 -DSDL_VIDEO_RENDER_RESIZE=1 -DSDL_ANDROID_KEYCODE_0=LCTRL -DSDL_ANDROID_KEYCODE_1=c -DSDL_ANDROID_KEYCODE_2=NO_REMAP -DSDL_ANDROID_KEYCODE_3=NO_REMAP -DSDL_ANDROID_KEYCODE_4=e -DSDL_ANDROID_SCREENKB_KEYCODE_0=LCTRL -DSDL_ANDROID_SCREENKB_KEYCODE_1=c -DSDL_ANDROID_SCREENKB_KEYCODE_2=NO_REMAP -DSDL_ANDROID_SCREENKB_KEYCODE_3=NO_REMAP -DSDL_ANDROID_SCREENKB_KEYCODE_4=e -Wa,--noexecstack -O0 -g -O2 -DNDEBUG -g -I/d/Code/android-ndk-r8/platforms/android-8/arch-arm/usr/include -c  jni/../jni/sdl-1.2/src/video/android/SDL_androidinput.c -o ./obj/local/armeabi/objs-debug/sdl-1.2/src/video/android/SDL_androidinput.o && ./obj/convert-dependencies.sh ./obj/local/armeabi/objs-debug/sdl-1.2/src/video/android/SDL_androidinput.o.d
jni/../jni/sdl-1.2/src/video/android/SDL_androidinput.c:41:30: error: SDL_androidvideo.h: No such file or directory
...

The links just aren't globally usable. Most things work with them, but somewhere deep down in the compilation process, something chokes on them.

rmtew · July 8, 2012

Alright, I have the apk building.

The code quoted below will enable someone using MingGW/msys on Windows to do the same.

Requirements:

MingW/msys installed, so you can run MinGW Shell with "run as administrator".
A git command which is accessible within MinGW Shell.
A cloned copy of the commandergenius project.
A cloned copy of the gemrb project, with this patch in place. The script below expects it to be located in the same directory as the commandergenius project.
The Android SDK installed, and the tools directory within it added to your path.
The Android NDK downloaded and extracted, and it's directory added to your path.
java ant downloaded and present in your path.
My "fix-msysgit-symlinks.sh" script, and the path to it below corrected.
The section with "Edit commandergenius\project\jni\application\Android.mk" applied to that file manually, as it is not valid shell commands.

That's it! It should work with that in mind.

#!/bin/sh
export PATH=$PATH:/d/Code/android-ndk-r8
cp /d/Code/scripts/fix-msysgit-symlinks.sh .
pushd .
cd project/jni/application
cmd /c "mklink /d src gemrb"
popd
cp -R ../gemrb/* project/jni/application/src/gemrb/
./fix-msysgit-symlinks.sh
sed -i 's/env PATH\=\$NDKBUILDPATH nice \-n19 //g' build.sh
# Enable for debugging ease thereof.
#sed -i 's/-j4/-j1/g' build.sh
rm -rf project/jni/vcmi
# These two lines might be irrelevant now.
cp -f project/jni/sdl-1.2/include/SDL_config_android.h project/jni/sdl-1.2/include/SDL_config.h
cp -f project/jni/sdl-1.2/include/SDL/SDL_config_android.h project/jni/sdl-1.2/include/SDL/SDL_config.h

# Edit commandergenius\project\jni\application\Android.mk
# Fix include paths.
"""
ifneq ($(NDK_R7_TOOLCHAIN)$(CRYSTAX_R7_TOOLCHAIN),) # NDK r7 broke it even more
LOCAL_C_INCLUDES := $(NDK_PATH)/sources/cxx-stl/gnu-libstdc++/include $(NDK_PATH)/sources/cxx-stl/gnu-libstdc++/libs/armeabi/include $(LOCAL_C_INCLUDES)
LOCAL_C_INCLUDES := $(NDK_PATH)/platforms/android-8/arch-arm/usr/include $(LOCAL_C_INCLUDES)
LOCAL_LDLIBS += -L$(NDK_PATH)/sources/cxx-stl/gnu-libstdc++/libs/$(TARGET_ARCH_ABI) -lgnustl_static
else
ifneq ($(NDK_R6_TOOLCHAIN),) # NDK r6 broke it
LOCAL_C_INCLUDES := $(NDK_PATH)/sources/cxx-stl/gnu-libstdc++/include $(NDK_PATH)/sources/cxx-stl/gnu-libstdc++/libs/armeabi/include $(LOCAL_C_INCLUDES)
LOCAL_C_INCLUDES := $(NDK_PATH)/platforms/android-8/arch-arm/usr/include $(LOCAL_C_INCLUDES)
LOCAL_LDLIBS += -L$(NDK_PATH)/sources/cxx-stl/gnu-libstdc++/libs/$(TARGET_ARCH_ABI) -lstdc++
endif
endif
"""
# Android SDK should be in path, so we can run android.bat
cmd /c "android update project -p project -t android-15"
cls; ./build.sh

My latest "fix-msysgit-symlinks.sh" script:

#!/bin/sh
git ls-files -s | awk '/120000/{print $4}' | while read FILEPATH
do
if [ ! -e $FILEPATH ]; then
	echo "Please check that GIT core.symlink for this project is 'false'" 1>&2
	exit 1
fi
if [ ! -L $FILEPATH ] && [ -f $FILEPATH ]; then
	wc -l $FILEPATH | while read LINES DISCARD
	do
		if [ "$LINES" = "0" ]; then
			FILEDIR=`dirname $FILEPATH`
			FILENAME=`basename $FILEPATH`
			LINKPATH=`cat $FILEPATH`
			pushd . > /dev/null
			cd $FILEDIR
			if [ -e $FILENAME ]; then
				if [ -e $LINKPATH ]; then
					if [ -d $LINKPATH ]; then
						rm $FILENAME
						cmd /c "mklink /d ${FILENAME//\//\\} ${LINKPATH//\//\\}"
					else
						echo "Faking symlink with copy: $FILEPATH"
						rm $FILENAME
						cp -f $LINKPATH $FILENAME
					fi
				else
					echo "$FILEPATH: symlink path '$LINKPATH' invalid" 1>&2
				fi
			else
				echo "$FILEPATH: problem finding this file"
			fi
			popd > /dev/null
		else
			# ERROR: Expected a file with one line for the link path
			echo "$FILEPATH: bad link file: expected 0 lines, got $LINES" > /dev/null # 1>&2
		fi
	done
else
	echo "$FILEPATH: already a symlink" > /dev/null # 1>&2
fi  
done

Good luck! If you're doing it on a non-Windows OS, you probably skip all the complications and just do steps 2-7 as appropriate for your platform, then follow the instructions in the commandergenius readme.txt. Which adapted would be something like:

rm project/jni/application/src
ln -s project/jni/application/gemrb project/jni/application/src
# make sure that project/jni/application/gemrb/gemrb contains the gemrb source
./ChangeAppSettings.sh -a
android update project -p project -t android-15
Then edit file build.sh if needed to add NDK dir to your PATH, then launch it.

lynx · July 8, 2012

nice battle, you earn 50XP.

Avenger · July 8, 2012

We should probably rename our String.h/String.cpp to something more unique.

Sign In

GemRB for Android compilation attempt..

Recommended Posts

rmtew

rmtew

rmtew

SyntaxError

rmtew

rmtew

SyntaxError

rmtew

rmtew

rmtew

rmtew

rmtew

rmtew

lynx

Avenger

Archived

Website

Forums

My Activity Streams

Downloads

Gallery