CBJ wrote: Stuff
Here, you're effectively describing the most direct approach to a mesh computational model, generally used (in its more advanced form) in computational air/fluid dynamics (like finite volume variation). Of course, approaching such a problem head-on, especially when adding complexity layers, will quickly hit a wall in computation and storage capacity. In practical application, it is always optimized in a number of ways. Also, it is not really scalable without losing accuracy.
Unlike modelling some physical substance, however, we're lucky in several ways. First, our objects are not interconnected. Second, they are not random and are predictable in a number of ways. Third, their values can be both continuous and generalized when it comes to some properties. So, they can be both decoherent from most of the swarm and clusterised.
To think of it, my proposal is also similar to mesh computation, only being closer to finite element and replacing mesh with graphs. That opens up many possibilities.
For your first example, the space of the encounter itself. Storing 10km as 1000 cubes can and must be replaced with storing position in timed points of trajectories (since ship movement can be accurately described with those). So, instead of 1000 cubes each cycle, we only have a single point on a trajectory piece.
That, of course, needs a standard library of trajectory pieces and an assembly map (list or even stack, effectively) for those used by the ship in encounter. But it, unlike static 1000 cube resolution, can be optimized in known (and yet unknown, applicable only to the game mechanics) ways. We won't even need to run the whole bunch through high-resolution Fourier transformer each cycle of the simulation, just run it once per simulation session, saving power on pre-calculating end. Also, despite library volume and map length, both don't multiply with each layer above, first being static (being in fact the most basic layer) and second being per-entity.
Effectively, that turns 10 unvectored coordinates in 10x10x10 space into 1 vectored curve piece location, 1 curve type and 1 curve position, repeated 10 times if we're storing ships as separate entities, or 1 time if we're storing them as a group.
And I suppose the first piece (vectored curve start location) can be optimized upon pretty heavily, like storing relative positions of close vectors in flat or even linear space instead of 3D. I'm not into that math, but outside of that, encounter type should provide us some basic optimization in that regard. That also gives us ability to discern true 3D starts for encounters and 2D ones (since X is generally played out flat-like), further saving dimensions.
As for ship types, yes, clustering ships by capability is a way that seems good for optimizing. It's not like there is a drastic difference in performance between many similar ships, despite their slightly different handling.
Ships also usually have more or less standard loadouts, which can help with this generalization even in cases when loadout elements are missing due to action. That damage seems to be the main hog on our space. Lucky for us, these are quite granular and are generally static (like engine state which is either working, damaged and destroyed). Also, like flight curve path library, it is static and not multiplied per-entity in data volume assessment.
Crew orders, as I see it in application to encounter, can be clustered into just 3 states -- attack/escape, coordinated/uncoordinated, hull/subsystem. The first one would be interchangeable if there was a squadron AI. So there doesn't seem to be a need for 10 variations of those. Again, these can be stored in a static list for all the entities.
Of course, I'm missing a whole lot of optimization here, but in general, instead of 10^6 states per ship, we have following:
1. Encounter type. Number from 1D list, used once.
2.a. (raw) Per ship: curve piece vector in 3D, curve type (stack), curve point (1D, stack?)
2.b. (with optimization) Per large group parts: 1 vector 3D per group part, group part member curve displacement (1D), curve point (per ship).
3. Ship types (1D list)
4. Ship loadouts (2D table)
5. Crew orders (1D list).
So instead of monstrous multi-dimentional matrix that continues growing (it shouldn't do that this quickly, but that's not the point), we're down to only several actually moving parts. Effectively, a list/table of 5 values/lists + vector + 2D table per state. Each easily accessible from simple data structures, I might add. And I'm sure vector and table parts are heavily optimizeable, as well.
Also, if we'd have the full spectrum of simulation results, I'm pretty sure we could use much more heavy optimization with standard (or new, X4-specific) tools for optimization.
Also also, unlike discrepancies mentioned above, it would be significantly more true to behavior observed by players since it'd be a product of direct, not simplified simulation. Especially since I thought of this in light of simulating in-sector encounters.
/EDIT Some typos/corrections. Also also also, we're talking groups as single ships interacting with each other just now. But that's completely unnecessary in most points of simulation. It's usually small groups dishing it out against each other or even a single ship in a small amount of space. That, of course, increases the number of encounter parts, but also reduces volume for each encounter -- and that's what we need, since we're looking for a way do describe it in a shortest way possible.
