Happy 2022! Now, about those save times...

[Alan Phipps]

Re the American comment: let's not disappear down that rabbit hole. I agree that it was inappropriate and shouldn't have been said, but let's leave it at that. Back on topic please.

[BlackRain]

Without trying to sound too American (obviously I'm not)

What is that supposed to mean?

Yeah, what is that supposed to mean? lol. Okay, I won't.

[Ehli]

Re the American comment: let's not disappear down that rabbit hole. I agree that it was inappropriate and shouldn't have been said, but let's leave it at that. Back on topic please.

Ah, heh. That was referring to their cultural trait (self promotion being socially acceptable), e.g. https://theprofessorisin.com/2015/02/06 ... dont-brag/ My American colleagues think it's strange we don't emphasize our strengths during business talks, while e.g. my French colleagues think the opposite is not done. It was just there to indicate it's outside my comfort zone to write like this - nothing more.

[BlackRain]

Re the American comment: let's not disappear down that rabbit hole. I agree that it was inappropriate and shouldn't have been said, but let's leave it at that. Back on topic please.

Ah, heh. That was referring to their cultural trait (self promotion being socially acceptable), e.g. https://theprofessorisin.com/2015/02/06 ... dont-brag/ My American colleagues think it's strange we don't emphasize our strengths during business talks, while e.g. my French colleagues think the opposite is not done. It was just there to indicate it's outside my comfort zone to write like this - nothing more.

I see, I understand, those people are just the most visible (which would be very much in the nature of "self promotion"). However, I wouldn't say they are the majority. Most Americans aren't like that in my opinion, especially depending on what part of the country. There are many wholesome, humble, polite and respectful Americans too, it is just that they don't get any visibility. In the world we live in, those kind of people aren't interesting or/and don't make good stories.

[xixas]

Edit: This is a code-oriented post, I promise... just gotta skim down past the fluff
Here's the Code link from below for quick reference (Real-time DAG snapshotting, No Locks)

--

::yawn::
.
.
.
::stretch::
.
.
.

Oh man... as an American, I must admit, I quite enjoyed my 3 day weekend. Hope it was as nice for "y'all" over there "out yonder" on the other side of the pond

Though I have to say, I didn't expect to come back to a thread derailed by the moderators.
"While there's mice away, the cats will play," is that it?

I don't expect to change any minds here so I won't press further. Some things you learn from working on (and designing) big enough systems. I know that sounds like I'm trying to be a "sage", and there's no reason to believe some random dude on the internet, which I am at this point.

S'ok. Random dudes on the internet are what makes it the internet.
I was referring to EgoSoft as the "sages". But you do you. Pick your title. I don't mind in the least

Anyway, your work advice is totally sensible for anyone below SSE.
I only work on referral, and have for years. I'll give you the first week for free to test my mettle... but I don't think anyone's asked me a real coding question in an interview in 20 years.
Personal opinion -- any SSE+ who's not working purely on a referral model is doing something wrong.
Just my 2 cents.

--

Dear xixas,

If you'd started that way to begin with I think we'd have gotten off on a much better footing

Without trying to sound too American (obviously I'm not) I'm rather experienced with this problem space: I'm a principal architect working in the R&D of one the biggest European tech companies. I'm the lead of a high performance STM I designed and is used in production by a large amount of enterprises.
...
I programmed in C/C++ for the last 25 years, of which the last 15 as a full time job

That's great. So take a few minutes out of your evening to throw a couple code snippets together and illustrate what you believe to be the issues.

• I see you put effort in your code, the enthusiasm, time spent to make it better.
• Use modern C++ (17 or 20), not C++98.
• Using 98 like that would give you a hard time applying for a senior software engineer position in my country.
• This is why all of my answers were about architecture. How about thinking about that first?
• drop me a PM if you want a job in this field.

That's a pretty condescending list... and I'm going to ignore it, because this is your most polite post to date.
Truly -- if you're trying to be more polite, I'll try and respond in kind -- be more personable and what not... we'll see how it goes.

This stuff is much better done face to face in a room with a whiteboard.

I'll bring the beer

Look, I've been writing C++ for 22 years.
Before that it was mostly BASIC, COBOL, and Fortran.
I've got a background in aerospace -- mostly air traffic control and satellite/near-earth-object tracking -- so I'm pretty used to dealing with large quantities of real-time data.

I got into game dev because I didn't like working under clearance, I got tired of contracting, and I've been getting hounded for the last decade or so... and it sounded like fun! (and it is... most of the time)
And the world fell apart... so, working from home instead of a lab ain't so bad.

Anyway, I work in a lot of languages, so we can switch out if your hung up on my semantics -- I'm down for pretty much anything but Erlang, Go, or Rust
Seems like EgoSoft's a C++ shop, so I'm trying to keep the core there.
Tooling's a bit quicker in Python/JavaScript/Perl/etc. so I may switch it up a bit as we're scaffolding (if you decide to stick around, that is).

Unfortunately I feel your perception so far is entirely based on examples designed for discussion by a general audience -- and maybe a healthy dose of just thinking you know better.
If you try and focus on the concepts instead of the execution I think we'd get a lot further.

For future reference, I write most of my examples in C++11 (not 98), both because I regularly find myself working on 10 year old projects -- maintenance is 70% of the product lifecycle after all -- and because it'll compile on just about any system onlookers might have available (I'm looking at you, macOS).
The purpose is example, not production. But, I'm happy to step things up a bit if you wanna join the game

--

On to the code.

I would've suggested corrections, e.g. the CRTP ( https://en.wikipedia.org/wiki/Curiously ... te_pattern ) is ruined...

Please stop saying this.
A CRTP requires a derived class inherit in recurring template form -- thus the name -- e.g.

Code: Select all

class Derived : Base<Derived>

...which it does not. It's simple virtual inheritance.
There are no subsequently derived classes, just wrapped types.
Ergo, no CRTP.

Use unique_ptr, not raw pointers.

Hah. Ok, so you want to see why I didn't use std::unique_ptr?
I'll gladly take a few minutes and type out a direct translation.

Code: Select all

template <typename T> class Saveable : public virtual Updateable {
protected:
    mutable std::unique_ptr<T> a, b;

public:
    Saveable() { a = std::make_unique<T>(); b = std::unique_ptr<T>(a.get()); }
    Saveable(const T& t) : Saveable() { *a = t; }
    Saveable(const Saveable<T>& o) : Saveable() {
        a = std::make_unique<T>(*(o.a));
        if (o.a.get() != o.b.get()) {
            b.release();
            b = std::make_unique<T>(*(o.a));
        }
    }
    ~Saveable() { if (a.get() == b.get()) b.release(); }

    Saveable<T>& operator=(const T& t) { setValue(t); return *this; } // Direct assignment
    operator T() { return getValue(); }                               // Direct use as type
    T operator +() { return getSaveValue(); }                         // Retrieve save value with a '+' prefix

    void setValue(const T& t) { if (isSaving && a.get() == b.get()) { b.release(); b = std::make_unique<T>(); } *b = t; }
    T getValue() const { return *b; }
    T getSaveValue() const { return *a; }

    void update() const override { if (a.get() != b.get() && !isSaving) { a = std::unique_ptr<T>(b.get()); }}
};

That's not better. That's much worse. It's a mess filled with gets and spurious release calls.
And it performs exactly the same, since std::unique_ptr has no overhead once optimized.
Though it performs significantly worse unoptimized (so smart pointers aren't awesome for debugging speed).

Anyway, I didn't use them here because I'm specifically doing with pointers what smart pointers were designed to stop developers from doing.
Not because it's wrong, but because junior devs always spring leaks.

And for the record, I used virtual inheritance because, compiled, the vtable overhead is one assembly call.
For someone who doesn't even want to be writing code yet, you sure do seem like you want to jump to the end.
Optimization is usually the last step in code design... but we'll get to that in a minute.

Anyhow, as vmo said, big topics like these need architecture first. The coding samples you seem to be so fond of comes after you defined the problem space. This is why all of my answers were about architecture. How about thinking about that first?

Back in the day, before the dot-com bubble, I was a hacker for hire.

I think in code, topology, and on-the-ground logistics. Not design patterns, not "problem spaces", not DAGs.
I'm accustomed to zero-day timings and people who -do- instead of -ask- or -plan-.
I don't like diagrams, flow charts, or spread sheets. I don't like red tape. Though, you're right, I do like white boards
What I like most is to do the job -- to do it right -- but more importantly to do it right now.

Let's take the image of my favourite website: https://en.wikipedia.org/wiki/Directed_acyclic_graph

Well... I already said I don't like thinking in DAGs...
But sure, I'm a sport. Let's do so.

Let's use that image, with the following constraints/statements:
- All objects in the game world are part of the same acyclic graph, which IMO is fair to assume given Ego's replies in the original thread
- There's only 1 thread (which is logical) operating on this graph (Ego stated somewhere the main computation is single-threaded)
- This thread receives the game events which will modify sub parts of this graph, e.g. "C" will be deleted, "C" owns "D" and thus will delete along with it, both replaced by "F" and "G"

Oh good, we're getting into specific constraints now.
I can work with this.

Now apply all suggestions in this thread to this behaviour.

Sure, I can do that.

You will quickly see that taking a lock on a part will not work, because there are always other closures/subsets being edited. There's only ONE solution to getting the entire graph in a consistent manner and that's by putting the mutex around the entire closure.

There's only one way?

How about this completely different way? (compilable gist with comments, C++11, no dependencies)

You know, following such narrow constraints... and applying all the suggestions from this thread... I had to think about that one for a few minutes!
Thanks. It was a neat little puzzle to end a long weekend

Quick summary... and tell me if I missed any of the suggestions from this thread:

• It's a straight copy of the DAG from your link -- the first swap is even a C-owned-D to F and G pair... at your specific request
• Single producer/consumer model -- only one producer thread working on the graph -- though it works for any number of consumers you like
• Applying snapshots both incrementally at a low level (sub-vertex values) and last-minute at point-of-save on the graph itself
• Serializes the graph to XML (intentionally slowed down to two edges per second) -- prints the XML to stderr, so run it with 1>/dev/null if for some reason you just wanna look at XML
• Performs 3 serialization passes on snapshots at different points of play and then performs a single-thread stress test
• Not a single new or delete call -- It's not realistic to do it all on the stack in production, but works fine for a simple DAG like this
• No locks (didn't even include mutex)
• Just went with simple request-control sync via overlapping atomics
• No event queues to bottleneck
• Was originally going to, but passed on memory barriers -- just not worth the maintenance complexity for something this simple
• Didn't even use a condition variable -- though it may speed the transition-to-save ever so slightly if I had used one, it's not even worth the extra include
• When you say performance, you seem to mean speed (without any regard for memory consumption). I tend to work on a lot of low memory systems, so that's usually factored into performance for me, but I disregarded it here -- nearly double the memory usage for 800% increase in straight write performance but only during a save by dropping the pointer shifts and heap allocations -- though the new/delete model outperforms it in standard operation on a predetermined graph in both speed and memory efficiency... because they're never called except when a save occurs.
• I left in the vtable overhead because it was a lot simpler than multiple registrars, and it only runs once per snapshottable item per save -- so nearly nonexistent.
• Granted it's only rotating through 6 vertices and 8 edges, but the performance looks alright. One "iteration" translates to swapping a random vertex, zeroing all the edge counts in the graph, recounting and storing the edge count for each vertex, and snapshotting the entire graph for save. I'm getting about 600k iterations/second on a single thread (on my 10 year old test rig)

Took a bit to type it out, but I probably haven't written graph code from scratch in 10 years, so that was kinda fun
Anyway, give it a spin when ya have a few.
Maybe it'll open up another idea or two.

Well, that's what Ego is doing: it freezes the thread and traverses the entire graph.

As you said, there's no free lunch. This strategy offloads the majority of the snapshot management from the point of save across all the value writes.
Again, I think the new/delete method's more efficient, because most of the time you're not in a save state -- but it would bog the save state time significantly more than this method.
This way costs more RAM permanently, but write performance actually improves during save state, so if the RAM's there to spare it may be the better option.

This was my last-ditch effort to explain why this is a large issue...

I'd hate to see you give up just as you're starting to join in
Look, rocky starts aside, if you know the material as well as you say you do, what's it gonna hurt to write a couple tidbits and try out a few things?

Anyhoo, that's about all the time I have this morning.
If I forgot to comment on anyone else's specific notes, I'll get back to ya.

Oh, and quit raggin' on Americans. You're breaking the mods

Edited: A few times, to be more polite and on-topic. I don't want things like throwing around titles and experience to detract from the main point here (actual code)

[xixas]

Hey Cdaragorn, I missed your post this morning. My apologies.

I've looked at Protobuf before but didn't use it because it doesn't allow random access to the data. Basically it suffers the same weakness that any text format does because it's order dependent. If you can't truly split the data up you can't use good Data Oriented Design patterns to speed the data handling up.

For files, they do have a standard sequential read, that's true, because they're designed for RPC -- basically just reading a message stream in off the wire.

But each message is serialized independently, and as I alluded to in a prior message, there's no reason we can't leave an empty block at the beginning of the file, drop a size header on every message (tallying them as we write), and then just seek back to file head and write the total data size written block -- maybe leave a few blocks and store a small index -- for example where the closest message starts are located with the file size divided in 8 pieces. Then we could fire up multiple reader threads to decrease load time

Also there are ways to avoid having to write that much custom stuff but when you're talking about performance you're also going to have to accept handling more custom code. Still you can make it a lot simpler than it sounds.

Yeah... boilerplate never goes away.
But that's why I'm trying to automate scaffolding the "message" (serializer) classes.
You know, DRY principle and all that.

Storing everything in memory first is a nice theory but in practice when working with this much data that actually creates problems that prevent it from getting the speed you're expecting from it.
The best system I've worked with involved streaming the data out of that memory at the same time as you were putting it in so it never takes up too much memory. That gets back to the need for true random access.

We know the entire universe, as written to the save file, is already fully stored in memory (in a much more space-efficient manner).
So I don't think that's an issue... unless I'm misunderstanding what you're saying, in which case, please elucidate.

I'd love to help if you have some data examples to work with. I guess I could pull apart a save file to come up with an example. I admit I'm having a hard time having much oomf to try just because I don't feel like it's likely to go anywhere.

That'd be awesome.
I've had a couple crossover releases this past week and haven't had nearly as much time as I usually do, so some extra input would be fantastic -- and that goes triple for actual code!

Take a look at the X4 Savegame XML Analyzer I wrote last week.
It'll break down an unarchived savegame file into a flat tag/ownership/property/type list that you should be able to use as a base to write any automated scaffolding tools ya want.
If you need any more features, just let me know and I'll see what I can add in as time permits.

[xixas]

Complimentary Code Link -- DAG Snapshots - Double-Write vs New/Delete - Large Set Comparison

Well it's been a couple days... and for a change... the code sample I posted didn't get flamed with the fires of a thousand dying suns...
So I'll take that as a sign we're moving in a positive direction

In that light, I found a bit of time this evening to run some larger scale performance tests on the previous code set.
I must admit, I need a bit of a retraction on my opinion that the new/delete method supremely outperforms the double-write method in general play (non-save) operation.

After a good bit of testing, it does win out by a reasonable margin in non-save mode -- but not as much as I expected.
And it loses out in save mode by a larger margin -- as expected, but to a greater degree than I thought it would.

--

Tested on randomized graphs

• 1 million vertices
• ~2.5 million edges (random 1-4 forward edges per vertex)
• Full graph copy performed per "iteration"

(Note that these tests are save-ready, immediately serializable value writes)

Double-Write: Best (Save Mode) = 2,318,061 writes per second -- maxed at ~630 MB RAM
New/Delete: Best (Non Save Mode) = 2,049,558 writes per second -- maxed at ~522 MB RAM

New/Delete did out-perform Double-Write in non-save mode as expected (2,049,558 > 1,922,466) -- but not by as much as expected.
Double-Write did out-perform New/Delete in save mode as expected (2,318,061 > 1,727,155) -- but by a lot more than expected.

So... using this method of distributed writes to minimize pre-serialization save time, if there's extra RAM to spare, I gotta admit the Double-Write method wins out handily.

Test code and full results writeup available in the GitHub gist.

Raw Results:

Code: Select all

========== GENERATE VERTICES ==========
Vertex Count: 1000000
Time: 842.312 ms
========== GENERATE GRAPH ==========
Edge Count: 2500110
Time: 1695.74 ms
========== PERFORMANCE TEST ==========

-[ isSaving: false ]-----
Test Time:                  3.12099 s
Average Iteration:          3120.99 ms
Iterations per Second:      0.320411
Saveable Writes:            6000000
Saveable Writes per Second: 1922466

-[ isSaving: true ]-----
Test Time:                  2.58837 s
Average Iteration:          2588.37 ms
Iterations per Second:      0.386344
Saveable Writes:            6000000
Saveable Writes per Second: 2318061

========== GENERATE VERTICES ==========
Vertex Count: 1000000
Time: 1043.79 ms
========== GENERATE GRAPH ==========
Edge Count: 2499328
Time: 2293.16 ms
========== PERFORMANCE TEST ==========

-[ isSaving: false ]-----
Test Time:                  2.92746 s
Average Iteration:          2927.46 ms
Iterations per Second:      0.341593
Saveable Writes:            6000000
Saveable Writes per Second: 2049558

-[ isSaving: true ]-----
Test Time:                  3.47392 s
Average Iteration:          3473.92 ms
Iterations per Second:      0.287859
Saveable Writes:            6000000
Saveable Writes per Second: 1727155

Note: Test Time == Average Iteration because I reduced the loopCount to 1 to grab a quick capture -- increase for finer tuned results.

[Falcrack]

Speaking as an American who knows absolutely nothing about the technical aspects of this discussion, I seem to recall a previous discussion on save/load times where the save game needed to take longer because of some issue related to maintaining save game compatability with new patches. Is that part of the issue here?

[Xenon_Slayer]

A wild dev appears

Just wanted to say that there may be some improvements to savegame times through the 5.0 beta. Given that you're discussing it I thought I'd mention it. Given that it may behave differently on different systems we'd appreciate any observations you have. If you do have any feedback on 5.0, best to post that in the Public Beta forum.

Also, it's worth checking if your savegame folder is excluded from Windows file indexing. Here's a guide on how to check. Ditto for folder compression.

Carry on

[xixas]

For anyone following along that doesn't understand this "DAG" bit, or its relevance to save games:

One can visualize a Directed Acyclic Graph like a tree structure with interweaving branches.
You can only move forward through the graph by following its edges, though there may be more than one path to get to each vertex after the second level.

A vertex is an informational junction point in the graph -- for our purposes, it's where we store game datapoints (ship/weapon stats, station coordinates, etc).
An edge is a directional relationship between two vertices -- here it represents the this-owns-that or this-is-targeting-that nature between game elements.

A savegame is an XML representation of this sort of relational data model.
Though the in-game data may not be directed and reference identifiers are stored to jump the graph, it must still be structured as such for a nested format like XML.

These tests are attempting to determine if the save time on such a graph can be offloaded from point-of-save (i.e. reading the whole tree and data at once) to a supplemental memory write pattern that requires minimal-to-no delay before translating and writing to disk.
In an ideal version of such a system, a live rolling snapshot of the graph can be pulled at any time and a save can be performed in the background without pausing play.

[xixas]

A wild dev appears

Just wanted to say that there may be some improvements to savegame times through the 5.0 beta. Given that you're discussing it I thought I'd mention it. Given that it may behave differently on different systems we'd appreciate any observations you have. If you do have any feedback on 5.0, best to post that in the Public Beta forum.

Also, it's worth checking if your savegame folder is excluded from Windows file indexing. Here's a guide on how to check. Ditto for folder compression.

Carry on

Thanks for the update

I'm on Linux myself, but the Windows note is good for any Windows users that might be following along here.

I don't think I've found the time to play anything in the past month, so it's unlikely I'll hop into the beta.
But if anyone here does, I'd love to hear about any marked improvements.

I seem to recall a previous discussion on save/load times where the save game needed to take longer because of some issue related to maintaining save game compatability with new patches. Is that part of the issue here?

I don't recall coming across that comment, but I'm not sure why patch compatibility would extend the save time... unless there's heavy branching logic to support multiple savegame formats (?)

[xixas]

Complimentary Code Link: Tiny Space (github gist, C++11, no dependencies)

Hey all. I had some free time this evening, so I wrote up a mini space sim -- affectionately called Tiny Space -- to act as a base for some more realistic testing.
It's all rolled up into a single cpp for easy copy. I wrote it up pretty quick, so it still needs some cleanup, but the basics are there and I'd rather share sooner than later in case someone has a use.

Ships fly around, hopping sectors. Speed depends on ship class.
I'll probably add some simple combat, jump gates, stations, tasks, and basic path finding as time permits -- whatever will add some complexity for snapshot testing

Displays in-sector ship information, a sector map, and a global map showing how many ships are in each sector.

There are no controls. It's just a self-perpetuating sim. Quit with <Ctrl+C>.

Adjust refresh rate, ship count, etc. in main().
Notably, I find setting sectorSize = {20, 20} more comfortable to look at... but it takes up more screen real estate while I'm coding

Sample Display (v1, 10x10):

Code: Select all

> ( name:F001 code:AEI-563 loc:{-1.4, 1.7} dir:{-0.4,-0.9} class:Frigate   )
  ( name:C031 code:YHV-953 loc:{ 0.3, 1.3} dir:{ 0.3, 1.0} class:Corvette  )
  ( name:S059 code:RKC-685 loc:{-3.6, 3.3} dir:{-1.0, 0.1} class:Scout     )
  ( name:S062 code:CTW-818 loc:{ 4.4, 4.2} dir:{-1.0,-0.0} class:Scout     )
  ( name:F078 code:RTF-828 loc:{-1.7,-4.2} dir:{ 0.1, 1.0} class:Frigate   )
  ( name:F119 code:YZC-396 loc:{ 2.6,-1.9} dir:{ 0.3,-1.0} class:Frigate   )

                   +-[ 08D ]-------------------------+
                   |                                 |
                   |          .                      |
                   |                                 |
                   |                         .       |
                   |                                 |
                   |                                 |
                   |                .                |
                   |             ^                   |
                   |    .                            |
                   |                            .    |
                   |                                 |
                   +---------------------------------+

        A      B      C      D      E      F      G      H      I      J  
   .----------------------------------------------------------------------
01 |    3      5      7      7      7      4      3      6      5      5  
02 |    6      5      7      7      7      1      4      5      1      4  
03 |    7      3      2     11      5             6      2      5      4  
04 |    2      5     10      3      3      8      2      3      3      8  
05 |    2      3      5      7      7      8      7      8      7      7  
06 |    4      4      4      6      3      5      5      3      7      5  
07 |    6      2      3      5      3      1      8      5      3      2  
08 |    4      5      7  [   6 ]    3      4      5      8      3      6  
09 |    5      4      7      3      6      6      5      3      6      6  
10 |    9      7      2      5      6     10      6      7      6      4  

work: 0.084438ms  display: 0.598263ms

[xixas]

Misc update -- I found a bit of time last night to add a color UI and gate travel to Tiny Space v2.

If I can find a couple hours here and there in the evenings this week to get combat and stations in, it should have enough complexity to perform a basic mimic for snapshotting

New Options:

• --color - Enable color display (for terminals that support ANSI color codes)
• --no-jumpgates - Disable jumpgate travel and revert to the original fly-between-sectors style.

https://user-images.githubusercontent.c ... 17741b.png

[dmk]

If you do have any feedback on 5.0, best to post that in the Public Beta forum.

no significant improvement detected for 5.0b2, sorry,
27 sec for early game (20 min after start),
and 1 minute for late game.
save time.
p.s. may be 10% improvement if strictly use same unmodified games (i don't have them, so i used others unmodified saves for measurement),
but it feels same.

[xixas]

Note: I know it's been all week without an update, but the thread's not dead, I've just been busy

Added randomized jumpgates, stations, ship factions, and player-owned ships to Tiny Space across a few late-night sessions.
I just haven't quite had time to finish basic combat.
It's close -- just need to finish the respawn timers -- and maybe add a kill screen for the wait.

They're just dots on a screen, but I didn't want to write sub-par combat.
Military ships' main guns fire in a 90 degree forward arc, save for frigates, who have side-fire.
Beam weapons are damage over time, while others are per-shot damage.
Targeting is performed per-ship and independently per-weapon, with turrets having full freedom of rotation... so while a courier or transport may not select a primary target, it will rear-fire turrets while escaping.

My brain says that I'm missing a part-second weapon cooldown reset somewhere...
...and I haven't had time to write prioritized follow, strafe, or angle-of-engagement code yet, so the frigates are currently a bit gimped since they're not rotating for full engagement... but I'll get there in the next version.

Anyway, I expect to have an update tomorrow.

Current specs:

• 100 sectors
• 500 ships -- comfortably handles over 10k, but who wants to look at that many overlapping dots? Increase the count if ya want to. Won't hurt my feelings
• Constant position, trajectory, waypoint, and target storage (per ship and per-weapon) -- with system-unique IDs
• No OOS combat -- all combat happens in real time in all sectors -- though I'll gladly update this with remote delay timers if anyone thinks slower combat will increase real-time complexity

The complexity's s high enough now that I ought to be able to write a simple serializer and real-time snapshot scheme that approximates X's complexity.
Of course, scripting engine variability can't be taken into consideration in any external model... so, even though it hasn't been mentioned, if that's a developer concern, someone let me know and I'll find a way to fake it in the model.

For that matter, if anything -- save for render time, obviously -- requires additional complexity for argument's sake, let me know!

As an aside, mining is also in the works, but I haven't mainlined it... I haven't had time to set up a supply-demand chain yet.

Once v3 drops, I'll hop back on the real-time save mechanics and get this thread back on track

[xixas]

Tiny Space v3 is finally up
screenshot - C++11, no dependencies

New in this version (v3):

• Randomized jumpgate positions
• Added factions and player-owned ships
• Added stations, docking, and repair
• Added simple combat, killscreen, and respawn

I'll write up a separate gist with snapshots and XML-serialized saves over the next couple days as time permits.

[CBJ]

With 5.00 Public Beta 3 now available to those participating, now would seem like a good time to briefly respond to this thread. The TL;DR version for those for whose eyes glaze over when programmers start talking: saving is now a bit quicker!

For those interested in a more detailed explanation, the changes we've made are, in increasing order of significance:

1. We've improved the raw performance of the serialisation of numeric values with the help of fmt.
2. We've also improved the process by which savegame signatures are calculated, so that it's done during the serialisation process rather than as a separate step at the end. The performance effect of this is relatively small; the primary benefit is that it eliminates a very large spike in memory usage, which could previously occasionally cause the game to run out of memory and crash for some people.
3. This is probably the most interesting one for the OP of this thread, who proposed something similar: we've changed the interaction between two key processes: the process of extracting data that needs to be saved, and the process of serializing that data. These were already separate steps, but they were previously interleaved on the same thread; now the two can run mostly in parallel. Note that this doesn't change the fundamental fact that the game needs to be paused while the first of these two processes completes.

We do have a few other ideas up our sleeves for further improvements, but none that are likely to be possible before 5.00 hits full release, and none that are magically going to make it possible to save the game without a noticeable pause.

[chew-ie]

Awesome - many thanks for coming back to us CBJ

--- chew-ie, lurker

[Artean]

1. We've improved the raw performance of the serialisation of numeric values with the help of fmt.
2. We've also improved the process by which savegame signatures are calculated, so that it's done during the serialisation process rather than as a separate step at the end. The performance effect of this is relatively small; the primary benefit is that it eliminates a very large spike in memory usage, which could previously occasionally cause the game to run out of memory and crash for some people.
3. This is probably the most interesting one for the OP of this thread, who proposed something similar: we've changed the interaction between two key processes: the process of extracting data that needs to be saved, and the process of serializing that data. These were already separate steps, but they were previously interleaved on the same thread; now the two can run mostly in parallel. Note that this doesn't change the fundamental fact that the game needs to be paused while the first of these two processes completes.

Ah, I see. Did you ever consider using spectral morphology quantum encapsulated binary... holistic transitions instead? Just sayin'... but I'm sure your solution is just as good.

[alt3rn1ty]

With 5.00 Public Beta 3 now available to those participating, now would seem like a good time to briefly respond to this thread. The TL;DR version for those for whose eyes glaze over when programmers start talking: saving is now a bit quicker!

For those interested in a more detailed explanation, the changes we've made are, in increasing order of significance:

1. We've improved the raw performance of the serialisation of numeric values with the help of fmt.
2. We've also improved the process by which savegame signatures are calculated, so that it's done during the serialisation process rather than as a separate step at the end. The performance effect of this is relatively small; the primary benefit is that it eliminates a very large spike in memory usage, which could previously occasionally cause the game to run out of memory and crash for some people.
3. This is probably the most interesting one for the OP of this thread, who proposed something similar: we've changed the interaction between two key processes: the process of extracting data that needs to be saved, and the process of serializing that data. These were already separate steps, but they were previously interleaved on the same thread; now the two can run mostly in parallel. Note that this doesn't change the fundamental fact that the game needs to be paused while the first of these two processes completes.

We do have a few other ideas up our sleeves for further improvements, but none that are likely to be possible before 5.00 hits full release, and none that are magically going to make it possible to save the game without a noticeable pause.

Surprised me how well this works compaired to beta 2.
Circa 17 seconds for a well established game, not even time to make a cup of tea