Jump to content
Sign in to follow this  

HOWTO: See binary file differences for mods using github (or other git hosting)

Recommended Posts

I realised this could be handy to more people developing mods, so here it goes.

If you used git log, git diff or just viewed some changes in github, you probably realised that it only displays text files and their differences (weidu code, documentation and similar). Any update of an IE format file is just considered done, misunderstood — you don't get to see the actual change.

This can be fixed though! Not on github, but for everyone developing or viewing through their own clones of the repositories, the process is relatively simple.

1. Get either iesh or ielister (and compile it).
2. Copy the files somewhere stable. For iesh this means everything, for ieparse the executable will do.
3. Tell git to use it when comparing IE formats.
3.1. Open .git/config of each mod repository you downloaded and add the following, replacing PARSERPATH with the path to either iesh's ieparse.py or ielister:

[diff "ie"]
textconv = PARSERPATH
cachetextconv = true

(if you use a global git config, you can do it there)
4. Tell git how to reckognise files in IE formats.
4.1. Again for each repo, add or edit .gitattributes with:

*.bcs diff=ie
*.chu diff=ie
*.pro diff=ie
*.spl diff=ie
*.vvc diff=ie
*.cre diff=ie
*.wmp diff=ie
*.gam diff=ie
*.sav diff=ie

You can add other formats as needed.


5. That's it, now viewing git log -p, git diff and similar will first convert the files to plaintext and then diff them normall. Example:



commit 571100d62479082b9f7a4b52abae94384928e540
Author: Jaka Kranjc
Date: Fri Sep 18 08:26:58 2015 +0200

iwd2: fixed a few recent clab names

diff --git a/gemrb/unhardcoded/iwd2/39280.spl b/gemrb/unhardcoded/iwd2/39280.spl
index f26a0a5..74ae130 100644
--- a/gemrb/unhardcoded/iwd2/39280.spl
+++ b/gemrb/unhardcoded/iwd2/39280.spl
@@ -1,5 +1,5 @@
00h signature spl v2.0
-08h unid name 0000998f
+08h unid name 00009970
0ch id name 0098967f
10h completion
18h attributes 00000000

commit bc395d1c839396fba28588223101be2c02326198
Author: Jaka Kranjc
Date: Mon Jul 20 18:59:03 2015 +0200

iwd2: wizard variant of confusion targets everyone

diff --git a/gemrb/unhardcoded/iwd2/confusw.pro b/gemrb/unhardcoded/iwd2/confusw.pro
index ea6e585..b1912a9 100644
--- a/gemrb/unhardcoded/iwd2/confusw.pro
+++ b/gemrb/unhardcoded/iwd2/confusw.pro
@@ -137,7 +137,7 @@
01f8h unknown 00000000
01fch unknown 00000000
0200h aoe section
- 0200h aoe flags 000004c0
+ 0200h aoe flags 00000400
0204h trigger radi 012c
0206h effect radiu 012c
0208h explosion so are_m21


You can test it by deleting a file and seeing how it is displayed in git diff.


Note: both parsers are mostly complete, but neither has been updated with any of the EE structural changes yet.

Edited by lynx

Share this post

Link to post

This looks very useful. I tried compiling iesh with Python 3, but it didn't work. I guess it needs Python 2?


ielister link above goes to a 404 page.

Share this post

Link to post

Sourceforge has problems today, check tomorrow (re ielister).


iesh: correct, it was only partly ported to 3.

Share this post

Link to post

I'm encountering an error when trying to install iesh.  I'm using Python 2.7.17 and running python ./setup.py install in the repo folder.  I receive a message about a syntax error on this line.

The full output is below.


running install
running build
running build_py
running build_scripts
running install_lib
byte-compiling D:\Programs\Python2\Lib\site-packages\infinity\cache.py to cache.pyc
running install_scripts
running install_data
running install_egg_info
Removing D:\Programs\Python2\Lib\site-packages\ie_shell-0.0.3-py2.7.egg-info
Writing D:\Programs\Python2\Lib\site-packages\ie_shell-0.0.3-py2.7.egg-info
  File "D:\Programs\Python2\Lib\site-packages\infinity\cache.py", line 33
SyntaxError: invalid syntax

This is from the 2nd time I ran the command.  The first time I ran it, I encountered the same error but there were additional output messages in the earlier steps.


Share this post

Link to post

Yeah, those two lines are clearly broken. Never ran into that though, since I don't actually install it — you can run it from the checkout normally. You can try changing them to "return rec".

Share this post

Link to post
On 7/10/2016 at 9:37 AM, qwerty1234567 said:

Cool. I guess with hooks it can be embedded into commit messages themselves.

Following up:

$ cat .git/hooks/prepare-commit-msg


Binary changes listed below with iesh:

for f in $(git diff --name-only --cached); do
  fname=$(basename -- "$f")
  case $ext in
      file_diff="$(git diff --cached "$f")"
    *) true ;;

if [ "$iesh_diff" != "$base_msg" ]; then
  echo $(cat "$1") "$iesh_diff" > "$1"

Example result.

Share this post

Link to post
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

  • Create New...