[4.0 Beta] Save game trade analyser

[harkov]

I was wondering how much money my traders and stations were making and the new in-game charts did not provide me with an overview of all my assets.
Since the 4.0 Beta, Egosoft has added transaction logs to the save files. I've used these transactions to calculate the per ship trade value and display these stats in a few graphs. Only trades performed after the update to 4.00 are visible. Below are some examples. These stats are also available per ship in a table.

The program only interacts with the savegame file which you can copy into the program folder so your game will remain unmodded.

I have some ideas to expand the statistics page but before I do I would like to see if there are more people interested in an empire dashboard like this.

If you want to give this a try, head on over to github for the source code:
https://github.com/harkovs/x4stats

Fair warning, while loading the save this program eats RAM. To give you an idea, my compressed save is 51MB and takes about 4-5 GB of RAM to process. After analysing the save RAM usage lowers significantly. This is also why you want the 64-bit version of python.

I'd love to hear your feedback

Planned features:
-List of inactive traders and miners

update 2020-12-15
-A savegame directory can now be specified. The save file with the latest modification date will be loaded
-Added a list of all transactions per ship or station
-Profit and margin scatter chart is now per commander instead of per ship
-Reload button added on the dashboard
-Player wharfs should now also show ship transactions. (Still needs testing)

Profit per ship
https://i.imgur.com/PCve76j.png

Profit and margin
https://i.imgur.com/ghT2KL8.png

Sales and purchases per ware
https://i.imgur.com/N4ZqFwX.png

[XTC0R]

Thats pretty cool! Will check it out.
Could you also do live updates during the game?
Maybe using SirNukes API? There was also another guy who pulled live data from X4. But I couldn't find the mod right now.

[harkov]

For now the refresh is manual by copying the save and restarting the program. What I want to do eventually is monitor the save game folder, load the latest save file and then have the dashboard refresh.

Another possibility is just monitoring a single file so you can refresh the dashboard from the game by saving to that file.

It is unlikely that I will make it work on live data since that would require installing mods. I'm trying to avoid that because I'm one of those people who likes chasing in-game achievements. It gives me something to aim for.

[kuertee]

Any chance you can leverage this X4 Rest Server mod? viewtopic.php?f=181&t=426061&p=4986734#p4986734

[XTC0R]

Any chance you can leverage this X4 Rest Server mod? viewtopic.php?f=181&t=426061&p=4986734#p4986734

That's the one I also had in mind but couldn't remenber.

[harkov]

Given enough time I could probably get it to work. That is if the rest API has all the information I need. It would mean starting over though because the current setup only supports a refresh initiated by the client and not from the server.

To be honest I don't see it happening any time soon as I simply do not have the time for it right now. I do plan to make the back-end pick up save game changes so when you reload the page, the information will be up-to-date.

[harkov]

Yesterday it was brought to my attention that ships sold by player wharfs may not be included in the sales amounts. I don't have a savegame with a player wharf in it so I am unable to check if this is the case. It would be greatly appreciated if anyone can share a 4.00 savegame which includes a player wharf.

[XTC0R]

I, as many others here in the modding forum, have a save including a shipyard. But I have installed a lot of mods which won't you load the game.
Maybe easier for you to use a cheat mod.

[harkov]

Thank you for the reply. I've managed to get my hands on vanilla save with a wharf. I was able to update the analyser to reflect the sales of wharfs and shipyards as well. The latest version is on GitHub.

Note: ship sales and repairs are not specified in the save game file but believe I've managed to find a way around that. Still need to fully test it

[drJulien]

I wish I had found this before and save me some time to write my own code based on this other tool: viewtopic.php?f=181&t=411091
This other one is much more complicated buggy but I am working on an even more complicated one (see latest comments) with cool features but very hard to install and only pros can dev on it. dotnetcore, entity framework, mysql web kendo ui.

I have a question why is this considered a mod?

My comment: It seems high ram usage means you're parsing the whole xml of the save file and/or you're loading the uncompressed file into RAM. It's better to use the disk for that - uncompress the file and save it as a file then delete the file when done. About the xml serialization, other project only skips to the interesting part which is 50MB for a 50MB save file (500MB uncompressed). further optimized by changing ship names and ware names to an integer ID but I digress.

[drJulien]

How do you calculate profit?
I use "money" which seems to be x100 in the <log><entry>
Then substract the average market price from a static table of wares
But maybe I shouldn't substract for mined materials since we get them free.
PS: To use in my c# project

[harkov]

When a sell transaction has a partner id. That partner is the buying ship or station. Giving you the exact amount that was paid for the wares.

So basically I sum all transactions where the ship is either the selling or buying party.

[Perry-Rhodan49]

I am now coming out as a complete newbie, but how do I get your promising program to run -> python stats / x4stats.py won't do anythingk? If it's a stupid question, I'll apologize in advance.
Hope you are still working on this.....

[harkov]

Does the command print any output in the console?

Also note I haven't played X4 in a while and haven't tested it with the final version of 4.0.

[Perry-Rhodan49]

thx harkov for your swift reply, no unfortunately there is no ouput. After installing python I made the following steps

Start Console
cd x4stats-main
python -m venv venv
venv\Scripts\activate.bat
pip install -e .
______________________________
Everthing works fine

renaming stats/config.example.py to stats/config.py
copied a compressed savegame into the stats/saves folder
edit the stats/config.py so that SAVEGAME_LOCATION reflects the savegame path

Start Console
python stats / x4stats.py
-> no output, nothing happens

[harkov]

I'm not sure what the problem could be. The only thing that comes to mind is that the command prompt pauses when you right click it. Then it's in select mode. Does the title of the command prompt mention 'select'?

[Perry-Rhodan49]

no, its in the same mode. e.g. "pip install -e ." is working just fine. I don't want to take up too much of your time, especially since you don't play X4 anymore, don't worry.

[harkov]

No worries, I've been playing X games of and on for ten years now. I'll check whether I can get it to run on 4.0 tomorrow night.

[Perry-Rhodan49]

Very kind of you, I'm curious if it works for you

[harkov]

I just tested it and I think I know what's up.

You're trying to launch the file in the x4stat\stats dir. What you should be doing is simply typing "x4stats" in the top directory
So let's say D:\x4stats is your main directory, do this:

Code: Select all

D:\x4stats> .\venv\Scripts\activate.bat
D:\x4stats> x4stats

Then wait for it to start and point your browser to http://localhost:2992/stats