Happy 2022! Now, about those save times...

This forum is the ideal place for all discussion relating to X4. You will also find additional information from developers here.

Moderator: Moderators for English X Forum

iznogood
Posts: 11
Joined: Mon, 17. Dec 18, 14:41

Re: Happy 2022! Now, about those save times...

Post by iznogood » Sat, 8. Jan 22, 18:44

vmo wrote:
Sat, 8. Jan 22, 18:23
I don't disagree with making noise. Just with the idea that we can come up with a technical solution from the outside.
Heh, let's call that qualified noise :)

I think you get qualified noise when the ordinary noise gets shutdown with "no that's impossible, you have no idea of the cost it would incur".
Fact is, nothing much is truly impossible, and if a bit of noise here can allocate an egosoft dev for a week or two, who knows what might happen.

Personally what i object to is waiting, while my computer is barely doing anything. If i am to look at a pause screen, i expect my cpu, disk or some other part of the system, to be maxed out, to keep that pause to a minimum.
Last edited by iznogood on Sat, 8. Jan 22, 18:53, edited 1 time in total.

vmo
Posts: 8
Joined: Sat, 8. Jan 22, 10:51

Re: Happy 2022! Now, about those save times...

Post by vmo » Sat, 8. Jan 22, 18:50

See that's what I mean. I think there's been enough noise already that if all it took was a week or two of one engineer, they would have done it years ago. :)

vmo
Posts: 8
Joined: Sat, 8. Jan 22, 10:51

Re: Happy 2022! Now, about those save times...

Post by vmo » Sat, 8. Jan 22, 18:54

Anyway. I'm not here to tell anyone what they should or should not ask Egosoft for. Products absolutely get better with feedback, including the ones I've led.

Just pointing out that the technical "advice" we have from the outside almost certainly won't solve the actual problem because the problem is almost certainly way harder than we can imagine without understanding the system design of the software.

iznogood
Posts: 11
Joined: Mon, 17. Dec 18, 14:41

Re: Happy 2022! Now, about those save times...

Post by iznogood » Sat, 8. Jan 22, 19:23

vmo wrote:
Sat, 8. Jan 22, 18:54
Anyway. I'm not here to tell anyone what they should or should not ask Egosoft for. Products absolutely get better with feedback, including the ones I've led.

Just pointing out that the technical "advice" we have from the outside almost certainly won't solve the actual problem because the problem is almost certainly way harder than we can imagine without understanding the system design of the software.
It could if they made the save system pluggable. How cool would that be, if they gave us a config option and an interface, so we could create our own.

Rei Ayanami
Posts: 3333
Joined: Wed, 6. Nov 02, 20:31
x4

Re: Happy 2022! Now, about those save times...

Post by Rei Ayanami » Sun, 9. Jan 22, 19:54

iznogood wrote:
Sat, 8. Jan 22, 19:23
vmo wrote:
Sat, 8. Jan 22, 18:54
Anyway. I'm not here to tell anyone what they should or should not ask Egosoft for. Products absolutely get better with feedback, including the ones I've led.

Just pointing out that the technical "advice" we have from the outside almost certainly won't solve the actual problem because the problem is almost certainly way harder than we can imagine without understanding the system design of the software.
It could if they made the save system pluggable. How cool would that be, if they gave us a config option and an interface, so we could create our own.
True, but at the same time it would greatly compromise the current high savegame compatibility between game versions, especially when the savegame format changes in subtle ways and you didn't update your custom savegame-plugin yet.

xixas
Posts: 40
Joined: Sat, 1. Jan 22, 22:47

Re: Happy 2022! Now, about those save times...

Post by xixas » Mon, 10. Jan 22, 00:17

Falcrack wrote:
Sat, 8. Jan 22, 06:33
I predict CBJ or any other Egosoft dev would not touch this thread with a 10 foot pole, though I am pretty sure they are aware of it.
I tend to agree. If this were one of my games, I'd probably take the "sizzle or fizzle" approach and let the thread run its course with minimal-to-no direct interaction.
If the community comes up with something potentially useful on their own, then you still have the opportunity to jump in -- if not, you don't risk taking a stance that's hard to back out of or alienating anyone.
vmo wrote:
Sat, 8. Jan 22, 11:35
Thanks for letting me interject this little thought here, and thanks for keeping the conversation civil and respectful. Just want to afford the Egosoft team their due respect as well and note that with almost 100% certainty, they have already done a way better job of this than we could by pontificating in a forum with almost zero knowledge of the system design of X4. :)
And thank you for cordially voicing dissent -- that's how it's done :)

I certainly applaud EgoSoft's dev efforts to date. They've made more than one of my all-time-top-10 favorite games, and I believe on the whole they make great products.
As has been mentioned, they're a small team, and what they do with what they have is excellent work.
vmo wrote:
Sat, 8. Jan 22, 18:54
Just pointing out that the technical "advice" we have from the outside almost certainly won't solve the actual problem because the problem is almost certainly way harder than we can imagine without understanding the system design of the software.
vmo wrote:
Sat, 8. Jan 22, 18:50
See that's what I mean. I think there's been enough noise already that if all it took was a week or two of one engineer, they would have done it years ago. :)
In general, you're not wrong. Without "a million monkeys on a million typewriters" kind of luck, we can't expect to simply write something here that's going to be a plug-n-play fix.
We can extrapolate from our outside experience and what little insight we do have, though.
  • We have the save files in XML format, so we know what needs to be serialized, and we can approximately gauge how long it takes on varying degrees of hardware
  • We can easily obtain a list of dynamically linked libraries (via ldd and/or readelf on Linux, and dumpbin on Windows) to get a better idea of what's available
  • We can get a better idea of architectural intentions -- and pick up on some static libs we're familiar with -- by reading symbol tables (with tools like nm)
Some of my thoughts from a quick 2 min scan:
  • X4 uses zlib (dynamic) -- so suggesting pigz-style multithreading is fine -- though I doubt it'd grant more than a few seconds improvement on compression
  • X4 uses libxml2 (static) -- appears I guessed this one right :) ...so now we can potentially discuss known optimizations in this space
  • X4 doesn't appear to use Protobuf -- something of an industry standard at this point with XML/XPath-like utility for in-memory data -- potential alternative to current (apparently custom?) ECS model classes -- but this could require a major rework
I also took about 15 min and wrote a quick XML analyzer in Python to provide a general idea of tags, attribute types, where references are used, etc.
Performed the analysis on one of my 4.20 save files around some notable formatting used -- "[0xabcd]" format for entity/component references, "{int, int}" format for positional coords, etc.
I haven't bothered to analyze all the parent-child relationships yet (necessary for entity-component modeling) or common enums serialized as strings... but that shouldn't take long to add in.
  • 335 distinct tags (excluding xml and signature)
  • 856 attributes across them (some, like "id", are commonly used, but distinct for the purpose of simple deserialization)
Haven't decided what to do with the output yet.
Even though X4's not using protobufs, maybe I'll have the Python script autogen some protobuf messages (*.proto files) to compile in and deserialize into.
Then we could use those to serialize into a more manageable binary format so we can get a better idea of how much actual data (vs XML fluff) we're working with here and ballpark some realistic timings on what the high-level relational serialization looks like.

I haven't had much free time this week to think beyond that -- but seems like an ok (and shareable) starting direction.
Certainly open to alternate suggestions.

--

Edit: Repeatedly edited for stream-of-thought typos :)
Last edited by xixas on Thu, 20. Jan 22, 19:44, edited 1 time in total.

xixas
Posts: 40
Joined: Sat, 1. Jan 22, 22:47

Re: Happy 2022! Now, about those save times...

Post by xixas » Wed, 12. Jan 22, 04:26

I could have sworn I posted a link to said XML analyzer!
(sorry about that)

Since the last post, I added in enum estimations and nested relationships -- though I went a little too aggressive on the latter.
I'll clean that up in a v2 when I find a bit more time.

Anyway, output basically looks like this... thousands of lines of this...

Code: Select all

savegame (1)
  aidirector (1)
    entity (21090)
      : id (21090): id (21090)
      script (34029)
        : attention (112): enum (112) {visible}
        : id (34029): int (34029)
        : index (34029): int (34029)
        : label (33655): string (33655)
        : name (34029): string (34029)
        : order (7791): id (7791)
        : time (34029): float (34029)
        command (15288)
          : type (15288): enum (15288) {attackenemies, repair, freetrade, trade, dockat, explore, resupply, wait, patrol, escort, freemining, searchresources, buildstation, mining, support, investigate, attackobject, withdrawbattle, movetozone, protect, recon, follow, recycle, collect}
          param (5288)
            : type (5288): enum (5288) {component}
            : value (5288): id (5288)
        commandaction (21091)
          : type (21091): enum (21091) {standingby, repairchecking, searchingtrades, repairingto, attacking, flying, calculating, undocking, docking, searchingresources, buildingto, executingtrade, waitingdrones, waitingtodock, investigating, attackingto}
          param (190)
            : type (190): enum (190) {component}
            : value (190): id (190)
         counters (308)
          counter (389)
            : current (264): int (264)
            : max (384): int (384)
            : type (5): enum (5) {list}
            : value (5): int (5)
        vars (34021)
          value (584289)
            : name (584289): string (584289)
            : type (530717): enum (530717) {component, integer, list, build, group, time, table, float, hitpoints, macro, length, money, largeint, quadrant, order, position, rotation, dronemode, blacklistgroup, ware, xmlkeyword, flightbehaviour, keyword, string, trade, class, formationshape, assignment, attention, faction, angle, largefloat}
            : value (404816): string (404816)
Once I loosen up the nested relationship restrictions, it ought to provide a decent starter base for deserialization.

If anyone's interested in extrapolating and writing up a simple deserializer ... let me know :)
Otherwise, I'll get around to it when I have some more time.

af_2017
Posts: 696
Joined: Sun, 7. Oct 18, 19:55
x4

Re: Happy 2022! Now, about those save times...

Post by af_2017 » Wed, 12. Jan 22, 17:29

Current state of the saving/loading obviously has reasons and goals which egosoft defined.
And most likely they satisfied with it.
The point is that the time the process takes is still way too much.
Anything could be updated to work faster. The process ends somewhere at the stage when everything is already rewritten to asm with consideration of platform details (execution pipelines, cpu caches and such). Not sure though that egosoft will get there. And being at the stage is not really good from product's maintenance and extensibility.

Nobody asked but what I would do first with that:
- separate "(de)bug report" (which we have now) and "release"(binary data, TLV or something similar to support future/optional things) saves covered with unittests to ensure they contain equal data
- "release" saving should do no serialization at all, just write memory to file (right, some approaches in data design must be reviewed to make this work). Because serialization is a killer because most likely there are virtual methods, additional memory access to intermediate dynamically allocated buffers and that is multiplied by objects amount in the game.
- on loading use preallocated buffers (it is known how many objects in the save file) instead of allocating them dynamically
Multithreading is not required, sometimes adding it is a sign of solving consequences instead of root causes.
That definitely will help because that removes root problems (and will introduce another) %)

Good news here, whatever the problem is we've all seen steam deck egosoft use on some recent streams..
So I suspect soon or later we will get:
- more performance optimizations
- more save/load optimizations
- better gamepad support[?]
because the deck is "fixed" hardware (that is, "please use latest i9 and double your ram to play the game, thank you!" will not work) and if egosoft want to see their product on it, they will change things.
X4 is not a destination. It's a journey. Unfortunately in a wrong direction.

xixas
Posts: 40
Joined: Sat, 1. Jan 22, 22:47

Re: Happy 2022! Now, about those save times...

Post by xixas » Wed, 12. Jan 22, 21:12

af_2017 wrote:
Wed, 12. Jan 22, 17:29
...
Anything could be updated to work faster. The process ends somewhere at the stage when everything is already rewritten to asm with consideration of platform details (execution pipelines, cpu caches and such). Not sure though that egosoft will get there. And being at the stage is not really good from product's maintenance and extensibility.
...
  • "release" saving should do no serialization at all, just write memory to file (right, some approaches in data design must be reviewed to make this work). Because serialization is a killer because most likely there are virtual methods, additional memory access to intermediate dynamically allocated buffers and that is multiplied by objects amount in the game.
  • on loading use preallocated buffers (it is known how many objects in the save file) instead of allocating them dynamically
While I know I did mention asmxml, I don't think anyone wants to take it that far (least of whom, EgoSoft).

A straight memory dump to file both breaks portability across architectures and it can be difficult to properly remap such buffer dumps to objects on import.
As for a memory-mapped binary reference model, while it'd technically be possible to skip a block, write the binary data, then seek to top and write the required storage size, I'm not sure it'd do any good.
Depending on how much data we're talking about, there's no guarantee of being able to get a contiguous block to hold the entire data set on load.

I'm going for a similar notion, though, with the Protobuf dump to binary idea (still at least partially dynamically allocated, though).
As long as there's a parent-child relationship store (be it on the objects themselves or a separate relational map), such binary serialization is faster, not least because the objects can be dumped and loaded in any order.

Just spit-balling, not actually suggesting this, but...
An interesting middle ground, might be a quick binary dump to file followed by a re-serialization to XML on a background thread.
Of course, it's a gross misuse of disk I/O, but such thoughts sometimes lead somewhere else interesting :)

xixas
Posts: 40
Joined: Sat, 1. Jan 22, 22:47

Re: Happy 2022! Now, about those save times...

Post by xixas » Thu, 13. Jan 22, 08:27

Alright, I found some time this evening to update a version 2 of the XML analyzer, making it significantly more suitable to writing a deserializer.

In short, the new version:
  • prints flat format by default (there's a tree tree view option)
  • displays tags in the order they were encountered by default (there's an alphabetical sort option)
  • lists all known parents and children immediately under each tag (in flat view)
  • has an option to disable type consolidation -- in case you want to see, for example, how many ints vs floats were used for a given tag attribute
  • has an option to disable enumerations -- in case you'd rather not deal with enums
Sample Output:

Code: Select all

savegame (1)
  @ children: info (1), universe (1), economylog (1), modified (1), stats (1), log (1), messages (1), script (1), md (1), missions (1), aidirector (1), operations (1), notifications (1), ui (1), signature (1)
info (1)
  @ parents: savegame (1)
  @ children: save (1), game (1), player (1), patches (1)
save (1)
  @ parents: info (1)
  : date (1): int (1)
  : name (1): string (1)
game (1)
  @ parents: info (1)
  : build (1): int (1)
  : code (1): int (1)
  : guid (1): string (1)
  : id (1): string (1)
  : original (1): int (1)
  : seed (1): int (1)
  : start (1): string (1)
  : time (1): float (1)
  : version (1): int (1)
player (1)
  @ parents: info (1)
  : location (1): int_pair (1)
  : money (1): int (1)
  : name (1): string (1)
patches (1)
  @ parents: info (1)
  @ children: history (1), patch (2)
patch (5)
  @ parents: aidirector (1), patches (2), history (2)
  @ children: script (82)
  : extension (4): enum (4) {ego_dlc_split, ego_dlc_terran}
  : name (4): string (4)
  : version (4): int (4)
history (1)
  @ parents: patches (1)
  @ children: patch (2)
...
...
...
command (15288)
  @ parents: script (15288)
  @ children: param (5288)
  : type (15288): enum (15288) {attackenemies, repair, freetrade, trade, dockat, explore, resupply, wait, patrol, escort, freemining, searchresources, buildstation, mining, support, investigate, attackobject, withdrawbattle, movetozone, protect, recon, follow, recycle, collect}
commandaction (21091)
  @ parents: script (21091)
  @ children: param (190)
  : type (21091): enum (21091) {standingby, repairchecking, searchingtrades, repairingto, attacking, flying, calculating, undocking, docking, searchingresources, buildingto, executingtrade, waitingdrones, waitingtodock, investigating, attackingto}
counters (308)
  @ parents: script (308)
  @ children: counter (389)
counter (389)
  @ parents: counters (389)
  : current (264): int (264)
  : max (384): int (384)
  : type (5): enum (5) {list}
  : value (5): int (5)
operations (1)
  @ parents: savegame (1)
notifications (1)
  @ parents: savegame (1)
  @ children: type (1)
type (1)
  @ parents: notifications (1)
  : enabled (1): int (1)
  : id (1): string (1)
ui (1)
  @ parents: savegame (1)
  @ children: addons (1)
  : version (1): int (1)
addons (1)
  @ parents: ui (1)
  @ children: data (1)
signature (1)
  @ parents: savegame (1)
I think that's about everything required to get started, so if no one picks up the mantle, I'll try and find a bit of time to look into generating Protobuf messages this weekend.

Imperial Good
Moderator (English)
Moderator (English)
Posts: 4750
Joined: Fri, 21. Dec 18, 18:23
x4

Re: Happy 2022! Now, about those save times...

Post by Imperial Good » Thu, 13. Jan 22, 09:52

xixas wrote:
Wed, 12. Jan 22, 21:12
An interesting middle ground, might be a quick binary dump to file followed by a re-serialization to XML on a background thread.
If the system has enough memory it does not even have to be written to disc. If not then the page file would handle that. However this is in principle a worse implementation of Linux process forking that was already suggested, although it is at least Windows compatible.

TKz
Posts: 147
Joined: Sun, 22. Sep 13, 11:48
x4

Re: Happy 2022! Now, about those save times...

Post by TKz » Thu, 13. Jan 22, 10:22

Do we need faster saves :?

I think it would be mandatory if the game was unstable and we had to save every 10 minutes. However I have 700h on the game, and it crashed once (probably in 1.0 or so).
I let the game save automatically every hour (I think it's between 40 minutes and 80 minutes).

Of course I would be happy to have faster saves, but there are in my opinion many more important things to improve first (optimization in dense sectors with big factories, combat AI, pathfinding, mass apply of loadouts, fleets blueprints ...). It's all about prioritization, not everything can be done in a given timeframe.

af_2017
Posts: 696
Joined: Sun, 7. Oct 18, 19:55
x4

Re: Happy 2022! Now, about those save times...

Post by af_2017 » Thu, 13. Jan 22, 13:49

xixas wrote:
Wed, 12. Jan 22, 21:12
A straight memory dump to file both breaks portability across architectures and it can be difficult to properly remap such buffer dumps to objects on import.
Right, the save is not portable. That's why I suggested keep current one and name it "(de)bug report" save.
That is, if you want to share your save with other users then use "(de)bug report" save, which will save xml, as it does now.
For fast save/load on same PC portability is not required.
xixas wrote:
Wed, 12. Jan 22, 21:12
As for a memory-mapped binary reference model, while it'd technically be possible to skip a block, write the binary data, then seek to top and write the required storage size, I'm not sure it'd do any good.
Depending on how much data we're talking about, there's no guarantee of being able to get a contiguous block to hold the entire data set on load.
I think if object A has reference to object B, then object B should be saved in scope of the data where it is stored separately. As simple as that.
If we have a ship, which has 1000 ton of ore, then please lets save all ore from the game in one chunk of data, and all ships in another. It is expected that ships hold only identifiers for cargo, which in this case should be 1000 identifiers which points to the ore.
TKz wrote:
Thu, 13. Jan 22, 10:22
Do we need faster saves :?

I think it would be mandatory if the game was unstable and we had to save every 10 minutes.
Saving is a way to store your progress. There are other reasons to save results of your work regularly besides instability of any product.
For example, this game provides various scenarios to lose your progress very fast.
You, as experienced player, is aware of the scenarios, and will avoid the scenarios or will be well prepared.
But new players most likely will make all possible mistakes, and using "quick" save/load is a way for them to discover the game without losing priceless hardly gained staff.
And from this point of view long save/load hits new players best of all creating bad game experience (user is frustrated by unforeseen flow or its own mistake AND long save/load does not make the situation easier). But the aim should be opposite; any product should attract new audience.
X4 is not a destination. It's a journey. Unfortunately in a wrong direction.

Cdaragorn
Posts: 175
Joined: Tue, 20. Sep 05, 17:53
x4

Re: Happy 2022! Now, about those save times...

Post by Cdaragorn » Thu, 13. Jan 22, 17:39

Honest suggestion: stop using purely text formats to store and retrieve such large amounts of data.

If you switched to a format that can support random access to the data and doesn't force you to work with it in any given order you'd increase your speed by an order of magnitude easily. For instance, sqlite is an awesome storage format that's completely open and free to use with tons of tools to easily work with it and can even handle threaded access and provide amazing hardening against corruption.

I work with huge amounts of data every day and I've seen the difference just from using text formats like xml and json over a more powerful format like sqlite. My experience tells me this is a huge part of what slows down trying to serialize this data. And I'm sorry but "storing it all in memory first" doesn't solve this problem.
"All that is gold does not glitter; not all those who wander are lost.
The old that is strong does not wither, Deep roots are not reached by the frost." -- J.R.R. Tolkein

xixas
Posts: 40
Joined: Sat, 1. Jan 22, 22:47

Re: Happy 2022! Now, about those save times...

Post by xixas » Thu, 13. Jan 22, 19:37

Imperial Good wrote:
Thu, 13. Jan 22, 09:52
If the system has enough memory it does not even have to be written to disc. If not then the page file would handle that. However this is in principle a worse implementation of Linux process forking that was already suggested, although it is at least Windows compatible.
That's the spirit!
TKz wrote:
Thu, 13. Jan 22, 10:22
Do we need faster saves :?

I think it would be mandatory if the game was unstable and we had to save every 10 minutes. However I have 700h on the game, and it crashed once (probably in 1.0 or so).
I let the game save automatically every hour (I think it's between 40 minutes and 80 minutes).

Of course I would be happy to have faster saves, but there are in my opinion many more important things to improve first (optimization in dense sectors with big factories, combat AI, pathfinding, mass apply of loadouts, fleets blueprints ...). It's all about prioritization, not everything can be done in a given timeframe.
xixas wrote:
Wed, 5. Jan 22, 19:05
In that light, if we're to discuss it, I'd like to take the pressure off and say I think the discussion should be had for discussion's sake -- not necessarily for the sake of getting something done right now.
Nobody's arguing that save times are the most important thing, but they do still matter, and most of those other improvements aren't ones to which we as a community can contribute.
Hey, I'd love to work on improving pathfinding, but that's a tall order without access to the game source.
Data serialization's a general enough concept that it's reasonably easy to discuss based purely on principle and best practices.
af_2017 wrote:
Thu, 13. Jan 22, 13:49
Right, the save is not portable. That's why I suggested keep current one and name it "(de)bug report" save.
That is, if you want to share your save with other users then use "(de)bug report" save, which will save xml, as it does now.
For fast save/load on same PC portability is not required.
I see what you're saying now. I misunderstood the first post.
So yeah, I basically reiterated your point with the save-to-binary then serialize-to-xml comment.
Misunderstanding un-forked :)
af_2017 wrote:
Thu, 13. Jan 22, 13:49
If we have a ship, which has 1000 ton of ore, then please lets save all ore from the game in one chunk of data, and all ships in another. It is expected that ships hold only identifiers for cargo, which in this case should be 1000 identifiers which points to the ore.
Oh man, I do hope it's not that granular :lol:
af_2017 wrote:
Thu, 13. Jan 22, 13:49
And from this point of view long save/load hits new players best of all creating bad game experience (user is frustrated by unforeseen flow or its own mistake AND long save/load does not make the situation easier). But the aim should be opposite; any product should attract new audience.
/nod
Cdaragorn wrote:
Thu, 13. Jan 22, 17:39
Honest suggestion: stop using purely text formats to store and retrieve such large amounts of data.

If you switched to a format that can support random access to the data and doesn't force you to work with it in any given order you'd increase your speed by an order of magnitude easily. For instance, sqlite is an awesome storage format that's completely open and free to use with tons of tools to easily work with it and can even handle threaded access and provide amazing hardening against corruption.
That's why I think I'm going to start experimenting with Protobuf first.
Protobuf messages compile directly to C++ classes to get us closer to Imperial Good's way-back suggestion of an entity-component-based demo.
But more importantly, Protobuf binary serialization averages a literal order of magnitude (10x) faster serialization time over XML.
It also provides message class reflection, making it a bit easier to quickly additionally serialize to XML without writing a whole custom serializer -- so hopefully we can perform some side-by-side comparisons.

I'd really rather not write over 300 custom to-sql serializers :)
Still, if you've got an idea how to quickly automate that process from the base field data, it wouldn't hurt to experiment with sqlite too.
Though I'm doubtful it'd outperform any of the RPC-oriented protocol buffer formats (Protobuf, Thrift, Bond, etc), which are all specifically designed as high-performance XML serialization alternatives.
Cdaragorn wrote:
Thu, 13. Jan 22, 17:39
I work with huge amounts of data every day and I've seen the difference just from using text formats like xml and json over a more powerful format like sqlite. My experience tells me this is a huge part of what slows down trying to serialize this data. And I'm sorry but "storing it all in memory first" doesn't solve this problem.
Storing it all in memory first (snapshot style) would just alleviate the speed issue.
At that point, even if serialization took twice as long, if it ran at a lower priority than the rest of the game it'd be a much smaller impact on the average user.
Interesting side effect of that too... if the user quick-loads while serialization is still occurring, the snapshot's still available in memory and could just be hot-reloaded :)

User avatar
Matthew94
Posts: 832
Joined: Tue, 4. Jan 11, 01:59
xr

Re: Happy 2022! Now, about those save times...

Post by Matthew94 » Thu, 13. Jan 22, 21:03

Ehli wrote:
Thu, 6. Jan 22, 13:42
While I can appreciate the attempt to help Ego, IMO the OP has way too little information of the X4 code to be giving out advise like that. You can't assume ECS, it's a custom in-house made engine. It also feels like a border-line insult to the Ego developers; it has been said many times by Ego they very much looked into ways to pull this off for their engine.
That doesn't mean there weren't valid approaches they didn't think of or that they dismissed without fully appreciating. You exemplify this strange ignorance I see online all the time: "these people work in field X so that means they're the best in their field (and everyone else is equally as good) and anything they do is an optimal solution. If you think they could have done better then you're wrong and clearly don't understand the situation". Believe it or not but it's possible for people to be shit at their job and I really doubt Egosoft have the kind of pull to be hiring the cream of the proverbial crop of programmers.

Alan Phipps
Moderator (English)
Moderator (English)
Posts: 30367
Joined: Fri, 16. Apr 04, 19:21
x4

Re: Happy 2022! Now, about those save times...

Post by Alan Phipps » Thu, 13. Jan 22, 22:48

@ Matthew94: One of the nice things being repeatedly said in and about this thread is that the protagonists are being very respectful and polite about each others' views (and about the dev team) during their lively and *deeply* technical discussions. I don't think anyone here wants you interjecting with thinly-veiled insults about other posters and the devs while adding nothing to the thread topic, thank you.
A dog has a master; a cat has domestic staff.

xixas
Posts: 40
Joined: Sat, 1. Jan 22, 22:47

Re: Happy 2022! Now, about those save times...

Post by xixas » Fri, 14. Jan 22, 00:12

Matthew94 wrote:
Thu, 13. Jan 22, 21:03
That doesn't mean there weren't valid approaches they didn't think of or that they dismissed without fully appreciating.
I have no choice but to admit agreement with this initial statement, or my argument wouldn't have a leg to stand on.

Otherwise, however, I agree with Alan Phipps.
While I appreciate your defending my point of view, I see no reason to question anyone's development acumen.
I intended this thread to be a good-natured discussion between seasoned "greybeards" like myself. I didn't expect the attention it's received... and certainly not the negative bits.

If I didn't believe the dev team were a capable bunch I wouldn't have bothered to start the thread.
And if I didn't believe the X series draws an above-average-intelligence audience, I wouldn't have solicited assistance.

I'm here to address a problem I genuinely think is fixable -- or get solid answers as to why it isn't.
I'm not here to (nor will I attempt to) defend myself or my professional standing, and I'd prefer if no one else is made to feel that's a requirement.

Developers tend to be a... let's just go with "sometimes overly enthusiastic" bunch, and topics do get heated.
Let's just keep it hot for the right reasons :)

vmo
Posts: 8
Joined: Sat, 8. Jan 22, 10:51

Re: Happy 2022! Now, about those save times...

Post by vmo » Fri, 14. Jan 22, 01:35

I didn't intend to stay engaged in this conversation, but since you all are still having a good time with it... :)

The difficult thing is almost never serializing and writing the data to disk. The difficulty is almost always that the runtime organization of the data looks completely different from the on-disk representation. I would bet that converting from the runtime data model to the on-disk representation is the bulk of the work, which is why saving is CPU bound. The reasons are often along the lines of 1. the on-disk representation would result in abysmal runtime performance and 2. the runtime representation would result in dramatically larger on-disk size and could actually take longer to serialize than the current save times. Ehli's post on the other thread is the explanation that IMO is most likely the closest to the actual situation.

We can only guess at the runtime data model, which is why I said earlier that it's unlikely that we can offer any useful "help" to Egosoft on this matter.

xixas
Posts: 40
Joined: Sat, 1. Jan 22, 22:47

Re: Happy 2022! Now, about those save times...

Post by xixas » Fri, 14. Jan 22, 03:00

vmo wrote:
Fri, 14. Jan 22, 01:35
The difficult thing is almost never serializing and writing the data to disk. The difficulty is almost always that the runtime organization of the data looks completely different from the on-disk representation.
Indeed. But "my code is a mess, so it performs terribly" isn't an excuse I'd wager any of us are willing to pass up the chain to our superiors.
If that's the assumption -- and I'm not saying it is -- then there's no harm in us presuming to present a better model.
vmo wrote:
Fri, 14. Jan 22, 01:35
I would bet that converting from the runtime data model to the on-disk representation is the bulk of the work, which is why saving is CPU bound.
I, good sir, will gladly bet you a Coke.

Thing of it is, it doesn't really matter where the bottleneck lies -- only whether or not it's fixable.
We have a number of industry standards at our disposal, as does the EgoSoft team, and if we know how to wield them...
...well, perhaps we can't write a drop-in replacement, but we can illustrate potential in the hopes it reaches the right link in their proverbial chain.
vmo wrote:
Fri, 14. Jan 22, 01:35
Ehli's post on the other thread is the explanation that IMO is most likely the closest to the actual situation.
Aye... aye.
And if he or she had portrayed as much in the manner of portent to which they seem to believe to carry themselves, it might have been more well received.
But... frankly... it seemed like every time Ehli took a swing... Ehli punched Ehli in the nose... and I was more than happy to let that happen :)

Still, we're working in hypotheticals here.

For as much as I believe that Ehli knows the material, the assumption that I do not...
...or more to the salient point, that players have no right to question the current inner workings...
is equal parts hypothetical and ridiculous.
  • Imperial Good estimated that there's an ECS (entity-component system) at play, to which we have no direct working knowledge -- simply based on savegame tag notation -- and the assumption at least feels likely, even if they don't consider it a wholesale "ECS" in-house... though if it was a wholesale ECS, then the runtime data model should be inherently easy to serialize.
  • I estimated that libxml2 was the most likely serialization mechanism, and upon further inspection, was correct -- and I'm hoping I'm not the only one on this forum who knows how to use it... it's simple, but there are a number of potential performance hacks.
  • We've already determined that multi-threading savegame compression pigz-style is at least a viable option given the current library in use (zlib) -- though we've yet to determine the beneficial magnitude of such a change.
I can't say we're certainly going in the right direction.
It's a series of stabs in the dark -- best guesses -- but those best guesses are made with a general assumption of developer skill level, a toe-deep knowledge of available tools, some background in at least tangentially related fields, an internet full of resources and best practices, and, hopefully, some good intentions (yeah, yeah... go ahead and tell me what's paved with what... my sentiment stands).

Everyone chiming in appears to have a different background, and that's great.
But no one person's background -- unless it's as a developer at EgoSoft -- will properly qualify any of us to make hard assertions.
...and even then... wouldn't that just be asking someone stuck in the middle of the Minotaur's Maze for the quickest way out? ...a fellow traveler mired in quicksand to throw us a rope?

No, I don't know that we're moving in the right direction.
I do know that I don't shy away from "impossible" problems...
...and here we've got significantly more than an army of one to work with.

Save times have been bugging players for years, and I never underestimate passionate players.
vmo wrote:
Fri, 14. Jan 22, 01:35
We can only guess at the runtime data model, which is why I said earlier that it's unlikely that we can offer any useful "help" to Egosoft on this matter.
All we have to do is provide reasonable, viable, and (most importantly) working models to prove that our goals can be accomplished, at least under ideal conditions, and hope that they can make said conditions a reality.

But communication is paramount. Civil discourse has moved a mountain or two before :)
Last edited by xixas on Fri, 14. Jan 22, 03:39, edited 1 time in total.

Locked

Return to “X4: Foundations”