[TOOL] X4LogAnalyzer - Help analyse the trade log

[Roeleveld]

I hope people in this forum can comment if there's another solution to this problem:

Manage your fleet of traders and miners to identify the best and worst performers. This is something basic for strategy games!
But I'm not angry at X4 - except that it took me so long to discover this incredible game, that I've been waiting for 24 years in my life. No joke.

@MagicTrip are you still active? I tried to ping you on github with an Issue. I'm a developer. I can contribute! But I have missing references.

So if there's a better tool already, please point me to that.
If this is the best thing to start from - this is still great! I'd like to add features and convenience. Maybe I can branch out without your permission or get permission to contribute?

If anyone is interested in helping with the project - just by helping me understand the log file, so I can code around it - that's also great

I would be willing to assist, but only if we can get this tool ported and supported on different platforms.

[drJulien]

I would be willing to assist, but only if we can get this tool ported and supported on different platforms.

You're on Linux?
This application is currently .NET Framework 4.6.1
If I convert it to .Net Core, it can run as a standalone web application on any platform. One click and it opens a browser. you could even be using your mobile or to access the application while you're playing the game!
Visual Studio Code is also available on Linux so some limited help with the coding can be made by developers on Linux.

Well that's a lot of work. And already I can't compile now because the project uses something called Livecharts and 3 other components that are not on Nuget and probably licensed. They're not particularly good, the graphs and menus. But basically, I have to rewrite and rethink everything except the classes and log import code in C# (which are priceless btw).

I'm an experienced Developer with .Net and I think I can start a brand new project based on my personal work on a proper web application with security, charts and grids using Kendo. Kendo is not free but a trial version has some "you're using a trial version" once per session and u just refresh the page to use the application. The grids and charts are out of this world, though. One "problem" is my application uses MySQL database so it's another installation step for a poor user... But I'd like a database to store the log file data so it's easier to work with. Maybe it can be contained in the application but I never did this.

On the other hand if a good samaritan hosts the website for everyone - they register and upload their save file, update their save file and see their results. we could even share our empire's performance with other players! Each save file is 50MB though...

edit: another decision considering a web application is how to access the log file. ultimately the web application can be hosted on the internet. the user then has to upload his file. it can be stored on the server or locally but it has to be processed on the server. the application could also run on the X4 machine and have direct access to the save files on the disk.
So, what do you think? Why different platforms? The community here is multiplatform? I'm new!

[Roeleveld]

I would be willing to assist, but only if we can get this tool ported and supported on different platforms.

You're on Linux?

Yes, and there are several others.
A tool like this should be available on the same platforms that is supported by X4. (In my opinion)

This application is currently .NET Framework 4.6.1
If I convert it to .Net Core, it can run as a standalone web application on any platform. One click and it opens a browser. you could even be using your mobile or to access the application while you're playing the game!
Visual Studio Code is also available on Linux so some limited help with the coding can be made by developers on Linux.

My preference would be to split the code into 2 sections:
1) Log/savegame parser
2) Display

The parser should be written in standard C/C++ using non-platform specific libraries or using libraries that are available on all platforms.
The display-part should use a framework per platform to make it look nice.
As the display only shows tables and graphs, it shouldn't change too often, the parser will need updating along with the savegame format.

Well that's a lot of work. And already I can't compile now because the project uses something called Livecharts and 3 other components that are not on Nuget and probably licensed. They're not particularly good, the graphs and menus. But basically, I have to rewrite and rethink everything except the classes and log import code in C# (which are priceless btw).

I'm an experienced Developer with .Net and I think I can start a brand new project based on my personal work on a proper web application with security, charts and grids using Kendo. Kendo is not free but a trial version has some "you're using a trial version" once per session and u just refresh the page to use the application. The grids and charts are out of this world, though. One "problem" is my application uses MySQL database so it's another installation step for a poor user... But I'd like a database to store the log file data so it's easier to work with. Maybe it can be contained in the application but I never did this.

On the other hand if a good samaritan hosts the website for everyone - they register and upload their save file, update their save file and see their results. we could even share our empire's performance with other players! Each save file is 50MB though...

edit: another decision considering a web application is how to access the log file. ultimately the web application can be hosted on the internet. the user then has to upload his file. it can be stored on the server or locally but it has to be processed on the server. the application could also run on the X4 machine and have direct access to the save files on the disk.
So, what do you think? Why different platforms? The community here is multiplatform? I'm new!

My preference, and I think quite a few others as well, would be something that doesn't require an internet connection. And if it can be pointed at a newly created savegame without having to upload it first would be my preference.

[drJulien]

I would be willing to assist, but only if we can get this tool ported and supported on different platforms.

You're on Linux?

Yes, and there are several others.
A tool like this should be available on the same platforms that is supported by X4. (In my opinion)

This application is currently .NET Framework 4.6.1
If I convert it to .Net Core, it can run as a standalone web application on any platform. One click and it opens a browser. you could even be using your mobile or to access the application while you're playing the game!
Visual Studio Code is also available on Linux so some limited help with the coding can be made by developers on Linux.

My preference would be to split the code into 2 sections:
1) Log/savegame parser
2) Display

The parser should be written in standard C/C++ using non-platform specific libraries or using libraries that are available on all platforms.
The display-part should use a framework per platform to make it look nice.
As the display only shows tables and graphs, it shouldn't change too often, the parser will need updating along with the savegame format.

Well that's a lot of work. And already I can't compile now because the project uses something called Livecharts and 3 other components that are not on Nuget and probably licensed. They're not particularly good, the graphs and menus. But basically, I have to rewrite and rethink everything except the classes and log import code in C# (which are priceless btw).

I'm an experienced Developer with .Net and I think I can start a brand new project based on my personal work on a proper web application with security, charts and grids using Kendo. Kendo is not free but a trial version has some "you're using a trial version" once per session and u just refresh the page to use the application. The grids and charts are out of this world, though. One "problem" is my application uses MySQL database so it's another installation step for a poor user... But I'd like a database to store the log file data so it's easier to work with. Maybe it can be contained in the application but I never did this.

On the other hand if a good samaritan hosts the website for everyone - they register and upload their save file, update their save file and see their results. we could even share our empire's performance with other players! Each save file is 50MB though...

edit: another decision considering a web application is how to access the log file. ultimately the web application can be hosted on the internet. the user then has to upload his file. it can be stored on the server or locally but it has to be processed on the server. the application could also run on the X4 machine and have direct access to the save files on the disk.
So, what do you think? Why different platforms? The community here is multiplatform? I'm new!

My preference, and I think quite a few others as well, would be something that doesn't require an internet connection. And if it can be pointed at a newly created savegame without having to upload it first would be my preference.

I don't know any way to downgrade to C++ and be multiplatform. There are tools like Xamarin (linux gtk) to create ports in various platforms but it's probably licensed?
.net core is not platform specific except it can only run web applications in a browser. using a browser is the modern way to be accessible to anyone. And to be clear it doesnt need to be on the internet.
It's always a good idea to separate the project like you say into the business logic that processes the log file and the UI. Multiple UI projects for each platform can connect to that one. In visual studio terms, that's a solution with multiple projects.

[Roeleveld]

I don't know any way to downgrade to C++ and be multiplatform. There are tools like Xamarin (linux gtk) to create ports in various platforms but it's probably licensed?
.net core is not platform specific except it can only run web applications in a browser. using a browser is the modern way to be accessible to anyone. And to be clear it doesnt need to be on the internet.
It's always a good idea to separate the project like you say into the business logic that processes the log file and the UI. Multiple UI projects for each platform can connect to that one. In visual studio terms, that's a solution with multiple projects.

Isn't C# the same as C++ with a whole bunch of additional libraries?
I was thinking of going through the savegame myself to see what I can find.

GTK is a graphical toolkit, sticking to something opensource like "Boost" should help with keeping the parser portable.
For the graphical display part, QT (is available on different platforms), GTK (Not sure if it exists for MS Windows) or something similar should easily be applicable.
But the graphical part can be seperated per platform as long as all the data can be made available.

[drJulien]

C# also has useful shortcuts
.net is used in the code we have now with Linq for example
You'd have to go line by line and rebuild it.

I can't help you with all the stuff you listed to get to the point of drawing graphs and information grids with c++ open source libraries it would take me weeks just to get started. I'm too busy to learn new (and especially old ) stuff

the magic guy doesn't reply to my messages im sure it's legal/appropriate to "be inspired" by his project, the classes and the writeLogEntry function. If you progress on your side let us know here. My timeline is 2 days if I get started. I dont want to start yet if I step one someone's toes or double-work.
https://github.com/magictripgames/X4Mag ... ogAnalyzer

keep in touch

[Roeleveld]

C# also has useful shortcuts
.net is used in the code we have now with Linq for example
You'd have to go line by line and rebuild it.

I can't help you with all the stuff you listed to get to the point of drawing graphs and information grids with c++ open source libraries it would take me weeks just to get started. I'm too busy to learn new (and especially old ) stuff

the magic guy doesn't reply to my messages im sure it's legal/appropriate to "be inspired" by his project, the classes and the writeLogEntry function. If you progress on your side let us know here. My timeline is 2 days if I get started. I dont want to start yet if I step one someone's toes or double-work.
https://github.com/magictripgames/X4Mag ... ogAnalyzer

keep in touch

I don't see any license mentioned, but as long as you credit the source (eg. "magictrip" and a link to the github repo) it should be fine.
It's also not been touched for nearly 2 years, which points to abandonware in my book.

My timeline will be longer than a few days.

[drJulien]

Ok so I was stressed today so I started working on it.
I wanted to have fun so I used my familiar tools. Sorry it's in evil .net core
A lot of things to fix but once it's somewhat cleaner I'll put this on github if anyone is interested.






PS: Integrating the other project's code was not pleasant. Some bits are brilliant but it's a bit messy overall.

EDIT: Wait what the frick is happening to my profit the past 2 hours???

[Roeleveld]

Please share the code (and a compiled version) for others to use, if you don't mind.

.net-code is not (necessarily) evil, it's just not easy to run elsewhere.
Am going to see if I can get it running using proton or wine. If it does, I'll simply write up those instructions and we have a "portable" tool.

I'm not overly concerned about open/closed source, I simply want software to be available on all platforms with acceptable performance.
For games and work-related software that means: native or using a really good emulation layer.
For tools like this: Anything will do.

[drJulien]

Please share the code (and a compiled version) for others to use, if you don't mind.

.net-code is not (necessarily) evil, it's just not easy to run elsewhere.
Am going to see if I can get it running using proton or wine. If it does, I'll simply write up those instructions and we have a "portable" tool.

I'm not overly concerned about open/closed source, I simply want software to be available on all platforms with acceptable performance.
For games and work-related software that means: native or using a really good emulation layer.
For tools like this: Anything will do.

Thanks for your interest.
This is extremely rushed. And there's a lot of what some might consider useless fat around the bone. But I like to think big. if it's on a browser it's very portable and I've deployed this on linux mysql. Wait mysql? why do you need a database? the json file currently used is not ideal - I've seen how many ships you guys have!
Sorry I hijacked this thread, maybe it's the exact same project just a new version so I vote to keep discussing here.
https://github.com/docJulien/X4APS
see the readme for installation notes. very rushed. there is no "binary" gotta build it and research how dotnet standalone can be executed. im curious if visual studio code on linux can build this. otherwise use VS Community.

[Roeleveld]

Please share the code (and a compiled version) for others to use, if you don't mind.

.net-code is not (necessarily) evil, it's just not easy to run elsewhere.
Am going to see if I can get it running using proton or wine. If it does, I'll simply write up those instructions and we have a "portable" tool.

I'm not overly concerned about open/closed source, I simply want software to be available on all platforms with acceptable performance.
For games and work-related software that means: native or using a really good emulation layer.
For tools like this: Anything will do.

Thanks for your interest.
This is extremely rushed. And there's a lot of what some might consider useless fat around the bone. But I like to think big. if it's on a browser it's very portable and I've deployed this on linux mysql. Wait mysql? why do you need a database? the json file currently used is not ideal - I've seen how many ships you guys have!

Linux mysql? You mean Oracle mysql, I presume

As for database requirement, why not use "sqlite", that can easily be used, for single-access is more than enough and can handle the amount of data you're likely to encounter.

Sorry I hijacked this thread, maybe it's the exact same project just a new version so I vote to keep discussing here.

I agree, it's the same type of tool and the previous developer might awaken again and join in.
[/quote]
https://github.com/docJulien/X4APS
see the readme for installation notes. very rushed. there is no "binary" gotta build it and research how dotnet standalone can be executed. im curious if visual studio code on linux can build this. otherwise use VS Community.
[/quote]
You can't create a exe-file for me that I can test with wine?

Personally, I don't see the point in needing a login or even a web interface.
I wouldn't use a browser to go through the details stored on my own system.

[drJulien]

Linux mysql? You mean Oracle mysql, I presume
As for database requirement, why not use "sqlite", that can easily be used, for single-access is more than enough and can handle the amount of data you're likely to encounter

No I mean Linux (apache), Mysql. I've tested this thing on linux just now. sqlite is a bit slower. it should be used by changing the mysql adapter to the sql entity framework package on nuget and adjusting the connection string. I dont see a way of switching between both easily in a development project, sadly. it would also require testing and time.

You can't create a exe-file for me that I can test with wine?

Personally, I don't see the point in needing a login or even a web interface.
I wouldn't use a browser to go through the details stored on my own system.

you wanted portability, the web browser is the best. everyone has one and it's already open.
dot net (web) is run as a standalone app (kestrel) served by a number of ways including apache. https://docs.microsoft.com/en-us/aspnet ... etcore-3.1

My goal is to have this running on my PC while I play X4 and access it on my tablet in real time! I know it's a selfish set of criteria - the installation steps are pro level and dev is expert level but the results are there!

Even if it is on Lan, I want to enable security I don't want to be blamed for someone getting their pc crashed or data lost. But you're free to create a branch which strips off everything. And that branch if you start from the current status would still use json files. I put the logic in a separate project; you can branch out or create a separate project that accesses the "business logic" but uses c++ windows forms or anything you want, but would be part of the same solution (a visual studio collection of projects). I have zero experience making windows forms that work on wine and manipulate xml files.

[XTC0R]

Maybe worth checking this MOD as well:

viewtopic.php?f=181&t=431930

[drJulien]

Maybe worth checking this MOD as well:

viewtopic.php?f=181&t=431930

This is perfect for most people. A minimalized version.
But it seems to use tons of ram I wonder if it's fast. It seem to be parsing the whole save file into an object which our project here is skipping only to the log so it takes seconds (after my quick optimisation I did 2 days ago, even better performance to be expected in the future if I keep going)

[drJulien]

new version fixes the 10 minutes processing time after 50+ hours of gameplay.
processing is now down to 10 seconds.
further improved by storing the last import date and only importing new log entries.
https://github.com/docJulien/X4APS/issues?q=
you had to make me look at xml files, you maniacs.. I had to rewrite the whole code. I wish the OP would verify my work - I wonder why "container" was treated differently but I didnt see any example of this in my savegame.

now back to the game

[drJulien]

[drJulien]

new update in 4.0 promised to bring more economy stats in game.
So this could make this project irrelevant. but it's still nice to have this off-game or on a separate monitor/device. if they enable multi-monitor in the new version then it's game over from my project and I'm glad!

[Perry-Rhodan49]

Hi @drJulien, looks like a great addon. Unfortunately I can't get it running. Since they didn't enable 'multi-monitor', I dare to ask if you continue to work on this great tool, specifically on an installation package für windows and more statistics?

[drJulien]

Hi @drJulien, looks like a great addon. Unfortunately I can't get it running. Since they didn't enable 'multi-monitor', I dare to ask if you continue to work on this great tool, specifically on an installation package für windows and more statistics?

You can use visual studio community to make your package. Licensing and ruining my career is why I dont want to release a package. but please let's chat let me know how to reach you if you're still interested!