Super-sized complexes - without the LAG - for all

General discussions about the games by Egosoft including X-BTF, XT, X², X³: Reunion, X³: Terran Conflict and X³: Albion Prelude.

Moderator: Moderators for English X Forum

glenmcd
Posts: 920
Joined: Sat, 16. Oct 10, 11:07
x3tc

Post by glenmcd » Tue, 1. Mar 16, 10:51

Thank you.

>> I wasn't aware that increasing the amount of objects in a sector does that little to the framerate.
To some degree this is a hunch of mine but partly observation too. I'll come back with some numbers. Pretty sure they'll be an eye opener for some.

With MD it seems that there is enormous resistance against doing a rewrite or anything major on it. My contacts so far have been Owen and Bernd. So my post in the tech support thread was focussed on an Egosoft stamped workaround that would require only days to weeks and perhaps two months including testing before a release that would be save file compatible. Here's what I had in mind (for someone else to do lol):

1. Don't allow Convoy missions to spawn. No problem with save compatibility here. That gets rid of perhaps 50 percent or more of the framerate problem.
2. Get rid of "buy asteroid survey" offered from flying ship in space. We already can do this from a station mission, and it behaves itself. A mission that doesn't spawn doesn't bother us with cleanup issues.
3. Two generic mission types need to be fixed to respect quota. I don't really spend any significant time on MD stuff but I can't imagine this could be too hard. Surely must have been the intent. These mission types are ones that players would miss (freight scan, buy blueprints).
4. Set quota in globals.txt to 1, for all four items. 20 second job and except for those missions that ignore it, is working fine in my tests.
5. Something I was toying with today: At the moment, most missions cleanup after four sector swaps. With all above done, performance is pretty good, however halving this to two sector swaps would give significantly higher resistance on slow machines to ridiculous framerate drops when other stuff is added on: war sectors, Albion Gamma etc.

I see the GM stuff as being far easier and quicker to get to release stage compared to the full fix on Complex CCKs. Complex CCKs does have a viable vanilla workaround, GMs don't. GM fixes will be felt by all and in all games, new or mature. Those that thought performance was already okay will get a nice surprise. Players that build the biggest complexes and hit the CCK FPS wall are also those most likely to visit these forums and see the workaround - or be pointed to it.

I've been on Devnet a few years, but TBH haven't got much out of it. It seems that one voice isn't enough to get some things done, but many voices - which are found here in the open forum - are listened to.

I do have time on my hands at the moment, and would be willing to see what I can do with MD while retaining compatibility. Understand that the only edits I've done to MD files were done in a text editor and mostly restricted to flipping individual bits or bytes or commenting stuff out. Not exactly ideal for XML I imagine. I'm no noob so no need for detailed coaching, perhaps just mentioning the best app to use for editing XML and any "gotyas" I'm likely to come across should suffice.

You speak of "optimization potential" as something that's a big job. It doesn't need to be. I've found that if you spend more time working out the smallest changes possible to make the biggest impact possible, that it's the best way to go.

If you can point me in the right direction with MD I'll see if I can put my workaround into MD code. I dislike rewrites unless really necessary. If I can flip a bit and get a good result, that's what I prefer. I got the impression from my workaround above that especially given the maturity of X3, it may not be necessary with the MD.

glenmcd
Posts: 920
Joined: Sat, 16. Oct 10, 11:07
x3tc

Post by glenmcd » Wed, 2. Mar 16, 09:45

enenra wrote:I wasn't aware that increasing the amount of objects in a sector does that little to the framerate. That's certainly news for me.
As promised, here are some numbers comparing CPU load with and without ships throughout Universe.

Albion Prelude 3.1
All asteroids, planets and debris removed from universe.
Computer used was hand built with fastest available parts 3.5 years ago, not overclocked.

In UKS East of Unseen Domain. All framerate measurements in here, same position same rotation. I converted FPS to time between frames, assuming that most if not all of this was required to generate the frame.


Baseline:
Vanilla jobs.txt, around 8090 ships.

As framerate was >1000 without missions, in both tests I first generated 2 missions of same type (Passenger Transport). This brought the FPS down below 1000. Dozens of tests using same mission types confirmed repeatability of FPS.

Framerate with 8090 ships: 780 FPS = 1282 microSeconds / frame
Framerate with empty Jobs.txt (no ships): 882 FPS = 1133 microSeconds / frame
Difference is 1282-1133=149 microSeconds per frame.
That's 18 nanoSeconds per ship average added to each frame time.
At 60 FPS, it would take twice this overhead just to drop one FPS.
At 1200 FPS, it's enough to drop by 150 FPS to 1050 FPS.

Comparison with generic mission overheads:

Each generic mission that spawns ("on offer") with exception of Convoy types, adds roughly 79 microSeconds to each frame time. With two generic missions on offer somewhere/anywhere in the universe, you've already added more time to each frame than all the ships in existence at the start of an AP game, and that's throughout the entire universe. We're still a number of times short of the GM overhead likely experienced in my workaround linked to above. And this in turn is dozens of times less than that experienced in most AP games today, once the player has explored half the universe or more.

Others? I found generally that ships add more overhead to (OOS) frame time than asteroids, planets, debris and even stations. Without the CCK issue, player factories will not even come close to the overhead that 8090 ships add. And it's still hardly even worth a mention.

Summary: When it comes to OOS performance in TC/AP, only large complex CCKs and generic missions matter at the moment. Truly, just forget the rest until these are fixed.

MrFiction
Posts: 215
Joined: Sat, 7. Jul 12, 19:16
x3ap

Post by MrFiction » Wed, 2. Mar 16, 12:10

glenmcd wrote:...until these are fixed.
Not likely, time to move on to other games

User avatar
enenra
Posts: 7150
Joined: Fri, 8. Apr 05, 19:09
x4

Post by enenra » Wed, 2. Mar 16, 15:55

glenmcd wrote:Thank you.

>> I wasn't aware that increasing the amount of objects in a sector does that little to the framerate.
To some degree this is a hunch of mine but partly observation too. I'll come back with some numbers. Pretty sure they'll be an eye opener for some.

With MD it seems that there is enormous resistance against doing a rewrite or anything major on it. My contacts so far have been Owen and Bernd. So my post in the tech support thread was focussed on an Egosoft stamped workaround that would require only days to weeks and perhaps two months including testing before a release that would be save file compatible. Here's what I had in mind (for someone else to do lol):

1. Don't allow Convoy missions to spawn. No problem with save compatibility here. That gets rid of perhaps 50 percent or more of the framerate problem.
2. Get rid of "buy asteroid survey" offered from flying ship in space. We already can do this from a station mission, and it behaves itself. A mission that doesn't spawn doesn't bother us with cleanup issues.
3. Two generic mission types need to be fixed to respect quota. I don't really spend any significant time on MD stuff but I can't imagine this could be too hard. Surely must have been the intent. These mission types are ones that players would miss (freight scan, buy blueprints).
4. Set quota in globals.txt to 1, for all four items. 20 second job and except for those missions that ignore it, is working fine in my tests.
5. Something I was toying with today: At the moment, most missions cleanup after four sector swaps. With all above done, performance is pretty good, however halving this to two sector swaps would give significantly higher resistance on slow machines to ridiculous framerate drops when other stuff is added on: war sectors, Albion Gamma etc.

I see the GM stuff as being far easier and quicker to get to release stage compared to the full fix on Complex CCKs. Complex CCKs does have a viable vanilla workaround, GMs don't. GM fixes will be felt by all and in all games, new or mature. Those that thought performance was already okay will get a nice surprise. Players that build the biggest complexes and hit the CCK FPS wall are also those most likely to visit these forums and see the workaround - or be pointed to it.

I've been on Devnet a few years, but TBH haven't got much out of it. It seems that one voice isn't enough to get some things done, but many voices - which are found here in the open forum - are listened to.

I do have time on my hands at the moment, and would be willing to see what I can do with MD while retaining compatibility. Understand that the only edits I've done to MD files were done in a text editor and mostly restricted to flipping individual bits or bytes or commenting stuff out. Not exactly ideal for XML I imagine. I'm no noob so no need for detailed coaching, perhaps just mentioning the best app to use for editing XML and any "gotyas" I'm likely to come across should suffice.

You speak of "optimization potential" as something that's a big job. It doesn't need to be. I've found that if you spend more time working out the smallest changes possible to make the biggest impact possible, that it's the best way to go.

If you can point me in the right direction with MD I'll see if I can put my workaround into MD code. I dislike rewrites unless really necessary. If I can flip a bit and get a good result, that's what I prefer. I got the impression from my workaround above that especially given the maturity of X3, it may not be necessary with the MD.
Your suggestions certainly will get rid of the problems you've found but there is the issue of changing gameplay quite significantly by influencing the amount of missions available or completely removing a mission type. Those are not really the kind of solutions that we'd want to implement into a game, or only if there really is no other option.

1. The issue here lies with what the mission does. I haven't looked at its code outside of a cursory glance but there must be something that's going terribly wrong for it to have such an impact. You mentioned before that the ships of these missions take ages to despawn, but just them existing can't really be the issue considering other missions handle way more ships than these. I'd assume there's either something wrong where its ships don't get cleaned up properly or way too slowly, although again, ordering ships to fly to a station where they are then despawned shouldn't have as much of an impact.

On the top of my head there's a couple things that could be done if the issue lies with ships not despawning quickly enough: a) Make sure to not use really slow ships for convoys so that they can reach their destination in a more reasonable timeframe, be that the mission target or the station where they are to be cleaned up at. b) Spawn convoy ships docked at a station. That way they can just be destroyed if the player doesn't accept the mission and don't have to first fly to a station. c) Additional rule where if the player is not in the sector and there is no player owned object near (hence not in radar range), instantly despawn ships.

The issue might also be in the way that conditions are used inside the code. Sometimes there's delays missing inbetween checks, etc. This is something that has to be analyzed with the MD debugger.

2. Seems reasonable. Doesn't really result in a loss of content either. Though again I'd be curious what exactly is the issue here and whether it could be resolved without removing the ships completely. There ought to be something very wrong with its handling to cause such an impact.

3. To make a mission respect quota is indeed not a big change, but I fear the issue lies more in a buggy check rather than it not having the code for it. Not sure.

4. That would influence the amount of missions available quite drastically and therefore alter gameplay. Can't imagine any player doing a lot of missions would be happy about this. IMO not really a viable solution. Need to find and fix the bugs instead. And figure out why multiple instances of the same mission have such an insane effect. (likely outside of what can be affected in the MD itself)

5. From what I remember the cleanup is not necessarily dependant on sector swaps but on mission offer timeouts. Not sure though. In any case though that is certainly something that could be looked at since the gameplay impact won't be large.


Devnet lvl 3 is not much atm. Well, not since XR testing has been made available to everyone. What you might want to consider is signing up for lvl5 by signing the NDA. lvl5 gets more access into the development, earlier betatests, and is also able to contribute to patches etc.

What do you need regarding MD? I can help you set it up but it's really not much more than extracting the files. Pretty much any text editor will do that can read the xml stylesheets that you'll find in the same directory. I've been using Visual Studio which has worked well for me. I'm pretty sure Sublime works as well. Also: mdfiles.txt lists (if present) all files that are loaded by the game. That's pretty much it.

I agree that relatively small changes can make a big difference but it's really not good practice to remove functionality from the game in the process of doing so unless it can absolutely not be helped. ^^

Thanks for the object analysis btw. though I gotta say I was actually more thinking about specifically player ships, since they run on a different set of scripts etc. than the ships spawned by the Jobs file.

shanrak
Posts: 651
Joined: Wed, 25. Feb 09, 05:54
x4

Post by shanrak » Thu, 3. Mar 16, 19:41

So I've been making complexes using this method in my new game ever since I saw this. I have a 128 station energy loop in savage spur, and a few 80+ station complexes in other areas. So far load times seems to be great, haven't noticed much increase during jumping at all.

The only downside of this method is that its a huge hassle while connecting the stations:

[ external image ]

Connecting stations the old way was much easier :P

glenmcd
Posts: 920
Joined: Sat, 16. Oct 10, 11:07
x3tc

Post by glenmcd » Sat, 5. Mar 16, 12:27

shanrak wrote:The only downside of this method is that its a huge hassle while connecting the stations:

Connecting stations the old way was much easier :P
Thanks for the feedback @shanrak. I must agree with you that knowing which complex hubs to connect next is not straight forward. But here is an idea that may ease this task a little:

1. Joining factories initially is easy so do all of these first. One of these must be positioned carefully as it will become the one that ships will dock at (the only surviving complex hub).
2. Before you join any complex hubs together, rename ALL complex hubs, adding a string of digits to the start of current names, where the numbers are incrementing. So instead of all being "Your Complex Hub alpha", the first would be "100Your Complex Hub alpha", the second "101Your Complex Hub alpha", the third "102Your Complex Hub alpha" etc. If you plan a really huge complex, use a 4 digit sequence. The hub you select to be first in this list should be the hub that you positioned carefully in step (1) above. See below for reasons.
3. Because all hubs now have unique names, they will no longer jump around in sector lists as well as when asked which things to connect using CCKs.
4. Connect the two hubs at start of list of hubs.
5. The first in the list now will be the first hub you selected in step 4 above and it will contain 4 factories. Rename this hub, adding a "z" to the start of the name. So "100Your Complex Hub alpha" would change to "z100Your Complex Hub alpha". It will drop to end of list at this point.
6. Repeat steps 4 and 5. Eventually, all the hubs containing 2 factories will instead contain 4, and all will have names that start with "z". Next, it's actually just as simple as repeating steps 4 and 5 again until you get down to just one hub.

Ensuring that the correct hub becomes the final one:
This is easy. Choose which hub you want to be the one that you dock at, and make this the one that is first alphabetically. In above example, this would be named "100Your Complex Hub alpha". When you select hubs to connect, the first one you select survives, the second is consumed. If when you connect "100Your Complex Hub alpha" it's always the first selected, then it will survive all connects and thus be the last remaining hub. The only real gotya here is when you have an odd number of hubs to join in a particular level. This won't be the case if your initial number of factories is an even power of 2 (32 factories, 64 factories etc). No problem, just be aware as you approach the end of each level of connects.

Cursed Ghost
Posts: 637
Joined: Sat, 2. Oct 04, 22:44
x3tc

Post by Cursed Ghost » Sat, 5. Mar 16, 23:27

Personally, I suspect that the solution you offer would lose a great deal of realism that is present in the vanilla game.
Perhaps but I would think this is one of them situations where you would want to make a small compromise for better performance and better simpler game play for players because I know that is something that a lot of players complain about particularly those new to the series and even as a veteran of the series I have to say they have a point there are a lot of elements which could do with being simplified for the sake of improved game play

Yes it detracts a little from the realism but you have to ask your self the question is it not worth sacrificing a little realism for improved game play especially when those changes not only improve game play by removing unnecessary complexity which really doesn’t add very much to the game anyway but also removes the very performance issues you are positing about

Now I don’t know about anyone else but personally I’d say that’s a trade worth making as long as it’s handled properly

don't get me wrong i realize that its not always so simple and depending on the nature of the issue it could well be one of those things that would require a massive rewrite of code which simply isn't practical because of the time and cost involved having said that however if they just wrote clean efficient code and properly tested stuff to make sure it works correctly before releasing it then this wouldn't even be an issue in the first place

now i know how that sounds and I don't know maybe if its just me but i was always taught to take my time and do it right first time and don't release work until I'm satisfied it's done correctly.

admittedly in the real world where you have dead lines and schedules etc you don't always have that luxury having said that that's no excuse for taking short cuts and not doing what you are supposed to do this is why so meany games these days come out so bug ridden that they are completely unplayable Stalker Shadows of Chernobyl being a prime example that game was so bug ridden that it crashed continually was unplayable because it.

although to Ego's credit they are for the most part good at fixing stuff when it doesn't work right which is more than can be said for a lot of other game developers who simply wont fix things even when they know its broken because they are only interested ££££ and once they have it they don't care anymore.

Rabiator der II.
Posts: 1189
Joined: Mon, 14. Nov 11, 20:31
x3ap

Post by Rabiator der II. » Fri, 18. Mar 16, 10:18

@Glenmcd:
Thanks for putting in the effort to research this issue :).

And without having done the research myself, I strongly suspect that Egosoft could solve the performance problem in complexes by picking a better algorithm. Simply because the first idea that comes to mind (going through a linked list of the connected stations) should require time roughly proportional to the number of connected stations. That is without spending time on finding a better solution, such as aggregating all station in the hub into one super-station for resource calculations.

How Egosoft managed to end up with exponential time is beyond me.

Edit:
I've reread your post from the beginning, and if the vanilla algorithm looks up every factory in the linked list in a separate search, with going through the list again for the next factory, an O(n²) complexity of the algorithm would result. But that would be stupid indeed :headbang: .

Edit: corrected my Big O notation :oops:
Gazz in the LT forum:
In X3, piracy is not implemented at all. All the "pirates" that fly around are bands of roaming psychopaths that destroy everything they see without even trying to loot anything.

glenmcd
Posts: 920
Joined: Sat, 16. Oct 10, 11:07
x3tc

Post by glenmcd » Sun, 20. Mar 16, 21:31

Rabiator der II. wrote:I've reread your post from the beginning, and if the vanilla algorithm looks up every factory in the linked list in a separate search, with going through the list again for the next factory, an O(n²) complexity of the algorithm would result. But that would be stupid indeed
That's the only thing I can think of that would be happening. Although to us (outsider devs and ex devs looking on) it certainly looks like a stupid choice of algorithms, you have to consider that at the time of choosing, computers were much slower graphically, and nobody would have expected us to be building complexes as big as we do now. The graphic adapters of the day would have made frame-rates so low after say 200 factories were in sector, that building an even bigger complex would have been impossible. As graphic adapters are much more capable now, we've finally come to moving the stress onto other components in the X3 engine. Not just for some, but for all players that can afford very capable graphic adapters. In 2016, I'd say that would be the majority.

RAVEN.myst
Posts: 2585
Joined: Mon, 20. Jun 11, 13:16
x3tc

Post by RAVEN.myst » Mon, 4. Apr 16, 18:19

I'd like to add my voice to the many others thanking you, glenmcd, for your stellar (heheh!) work on this - it is greatly appreciated!

(Your research is particularly well timed for me, as I am now almost done with my latest visit to Rebirth-land and nearing my inevitable return to X3AP - armed with a constructively different way to do something!)

I hope that your efforts nudge ES in the right direction - you've proven that certain significant (to say the least!) performance improvements can be effected with relatively little investment of effort.
-
Boron passenger: "You must hurry - my testicles are drying out!"
-
Born on Lave, raised on Freeport 7...
-
The Write Stuff

bounty_hunter66
Posts: 550
Joined: Tue, 15. Aug 06, 13:36
x4

Post by bounty_hunter66 » Thu, 15. Jun 17, 09:04

How are you supposed to build like this?

Considering the fact that Complex Hubs have the extremely retarded tendency to misplace themselves when you build them too close to any object because they think they interfere with other collision boxes or whatever, but they aren't even remotely close to begin with(unlike normal stations witch do not give a damn where they are placed and can even be overlapped/stacked)

And considering the 20km maximum range for connections.

I'm having the most difficult time to build a measly 33 station complex with this method. Each time I connect Hubs, the next Hubs that will contain X^2 stations are farther and farther apart and go beyond the 20km range.

Snafu_X3
Posts: 4472
Joined: Wed, 28. Jan 09, 15:14
x3tc

Post by Snafu_X3 » Fri, 16. Jun 17, 01:06

The last hub you connect to will become the only hub for that plex. If building plexes, by all means connect them with hubs if necessary but then leave the last connection to the plex (your trade dock) until.. last!
Wiki X:R 1st Tit capping
Wiki X3:TC vanilla: Guide to generic missions, Guide to finding & capping Aran
Never played AP; all X3 advice is based on vanilla+bonus pack TC or before: AP has not changed much WRT general advice.

I know how to spell teladiuminumiumium, I just don't know when to stop!

Dom (Wiki Moderator) 8-) DxDiag

RAVEN.myst
Posts: 2585
Joined: Mon, 20. Jun 11, 13:16
x3tc

Post by RAVEN.myst » Fri, 16. Jun 17, 01:16

bounty_hunter66 wrote:How are you supposed to build like this?

Considering the fact that Complex Hubs have the extremely retarded tendency to misplace themselves when you build them too close to any object because they think they interfere with other collision boxes or whatever, but they aren't even remotely close to begin with(unlike normal stations witch do not give a damn where they are placed and can even be overlapped/stacked)

And considering the 20km maximum range for connections.

I'm having the most difficult time to build a measly 33 station complex with this method. Each time I connect Hubs, the next Hubs that will contain X^2 stations are farther and farther apart and go beyond the 20km range.
I recommend building in a structured matrix, and ideally somewhat flat - if you are not deep in one of the three dimensions, you will have plenty of space in reserve. I've built plexes with 64 and more stations this way, no problem. Keep in mind that when you link two hubs, the first one you select will be the one that stays, the second one selected will connect to it. This way, you can easily predict where your resultant hubs will be. When you do the "real" one (last, as Snafu pointed out), you choose it out of the two remaining hubs, and then link the other to it. However, I suggest selecting the "true" hub's location early, so that you can plan accordingly.

To stay within the requisite maximum distance, try to end up with a sort of regular alternating pattern of hubs (sorry, I can't really explain it better than that, at least not until I've slept :P ) - you end up with something almost resembling a caterpillar (depending on your 'plex layout), and then you join pairs of "legs" to each other, and then join those links to each other.

Best of luck!
-
Boron passenger: "You must hurry - my testicles are drying out!"
-
Born on Lave, raised on Freeport 7...
-
The Write Stuff

User avatar
RXJ32-11
Posts: 27
Joined: Mon, 24. Jan 05, 21:02
xr

Re: Super-sized complexes - without the LAG - for all

Post by RXJ32-11 » Mon, 11. Apr 22, 18:06

The link (THE DOWNLOADS (Benchmarks.txt, 24 save files for benchmarking Albion Prelude complexes) ) is offline. Can you reupload the files, please? Thank you for investing so much time in this complex issue.
Der PC rechnet mit allem, nur nicht mit seinem Besitzer.

Post Reply

Return to “X Trilogy Universe”