Further reverse engineering

[WizzardMaker]

Figure out how

• *.pol
• *.dat
• *.mcf
• *.csv
• *.raw
• *.smk

are structured

[WizzardMaker]

There is already C++ (Or rather old C) code that can decode the *.csv files. They are encoded with an xtRLE method. http://karlchenofhell.org/kram/xtRLE_dec.c is working 100%
Also here https://www.gog.com/forum/airline_tycoon_deluxe/editing_game_data_csv_files_to_customizes_cities_routes_etc

We have to add a smk decoder as well, for all the character animations in the rooms, a C# version is already in the open source StarCraft remake (https://github.com/kangaroo/scsharp/tree/master/SCSharp/SCSharp.Mpq.Smk), we should ask them if we can use and modify their code, would save us some work.

I don't know what the *.dat files contain, but the misc folder has *.res files which should contain all the dialog if my memory serves me right.

the *.mcf files contain the font data.

[WizzardMaker]

The *.dat files hold the airport gfx position data.

I managed to extract the information of those files. I created a script which reads the file and then creates a scene from it.

The file has the following structure:
Header:
4 bytes (int) - Entries in file
4 bytes - ? often null

For each entry:
4 bytes (long) - file id (from brick.csv)
4 bytes (int) - x position
4 bytes (int) - y position
1 byte - other info, maybe room number
1 byte - null

There are still some artifacts, like misplaced tiles or multiple entries on the same position and its still missing a method to load animated entries.

Here is a little preview of the assembled airport from those files:

[Serphentas]

Amazing work ! The preview looks great, despite the few details here and there. I think that's a good basis to build upon.

I'll try to write a CSV handler/editor so that we can customize routes in-game.

[Serphentas]

Found info on xtRLE (but it's in German): http://www.karlchenofhell.org/kram.php?part=rhodan#xtRLE

[AWKIF2005]

I stumbled upon this project while trying desperately to find a way to decrypt the .csv files in ATD (specifically routen, planetyp, and city). First of all, I am beyond excited about the work you guys are doing to open ATD and bring it into the future. This is a classic game! I am not a coder or programmer, so most of what I see here goes completely over my head and I doubt I can be much help, but I will be following this project closely! Now to see if I can figure out how to use this script to open up and edit the .csv files...

Keep up the good work!

[Serphentas]

Thanks for the comment ! Much appreciated.

As for the script, it can only be used on relation.csv because other ".csv" files are either already CSV or something else (not xtRLE).

[WizzardMaker]

As for the script, it can only be used on relation.csv because other ".csv" files are either already CSV or something else (not xtRLE).

That's not 100% true. I successfully decoded all csv files with the aforementioned c file, doesn't mean your files are the same as mine, but they should be decryptable

I am porting the c file to C# atm. That should show us whether your files can be decoded or not.

xtRLE is also used in the Dialog files, but the current implementation of the script has some problems with the file, but that should be fixable.

[Serphentas]

That's what I get for en/data/*.csv:

brick.csv:    ISO-8859 text, with CRLF line terminators
builds.csv:   ASCII text, with CRLF line terminators
city.csv:     ISO-8859 text, with CRLF line terminators
clan.csv:     ISO-8859 text, with very long lines, with CRLF line terminators
names.csv:    ISO-8859 text, with CRLF line terminators
piloten.csv:  Non-ISO extended-ASCII text, with CRLF, NEL line terminators
planetyp.csv: ISO-8859 text, with very long lines, with CRLF line terminators
pnames.csv:   ISO-8859 text, with CRLF line terminators
relation.csv: data
routen.csv:   ASCII text, with CR line terminators
stdpaper.csv: ISO-8859 text, with CRLF line terminators

Do you have a similar layout ?

[WizzardMaker]

Well, seems like our versions differ a bit. My relation.csv and builds.csv are not encrypted and all other .csv files are encrypted with xtRLE

I always just look at the HEX content of the files to see whether they are encrypted or not

[AWKIF2005]

Perhaps the .csv files are in different states of coding depending on different versions or releases of the game. My Strategy First release of ATEvolution had the routen.csv file in regular text, so I was able to use Excel to easily mod the in-game route structure in ATE to my tastes, within some limits before the the game would start crashing on start up. I tried using the regular text routen.csv from ATE, manually added the new routes from ATDeluxe, following the same format, and then put that new routen.csv in the ATDeluxe data folder. It didn't work, the game crashes. I tried multiple variations of city spelling, etc, but no joy. Hence my desire to decode the actual routen.csv from ATD :) My version of ATDeluxe is a download from GOG games.

Interestingly, the builds.csv file is normal text in both versions, allowing easy modding of the "custom airplane" parts. I use this to create custom planes that mimic the specs of real aircraft that are not included in the original game (A330, A380, etc)

Hopefully part of the open ATD project will be to allow these types of mods more easily. Since I am a coding neanderthal, I would be happy to beta play the game for you guys, when it's ready! :)

[AWKIF2005]

WizzardMaker, sounds like you and I have the same version of the game as far as which .csv's are encrypted. Serphentas, your ATDeluxe routen.csv is already in normal text? Interesting!

[WizzardMaker]

Serphentas is using the Linux version (as far as I can see it) which was ported over, so they probably changed some parts of how they handle the files.

[AWKIF2005]

That probably explains it then.

[WizzardMaker]

Aaaand now we should be able to decrypt all the xtRLE files.

It seems like the c source from http://karlchenofhell.org/kram/xtRLE_dec.c had some problems with bigger files. I tested the following files with my C# decoder and all got decrypted successfully:

• misc/ein_ger.res
• misc/dlg_ger.res
• misc/std_ger.res
• data/brick.csv
• data/city.csv
• data/clan.csv
• data/Names.csv
• data/piloten.csv
• data/planetyp.csv
• data/pnames.CSV
• data/routen.CSV
• data/stdpapter.csv

I need to clean my code, but it should be committed in a couple of hours