[TC] Very large complexes with fast sector changes may be possible

[glenmcd]

NOTE: Thread subject changed to reflect new direction, subject was "No reason to tow asteroids?".
So I finally decided I should do a gamestart in which I completed all main missions. I was dreading the thought of spending another week(!!) towing asteroids, lining them up, saving and then restoring when asteroids collided etc etc. But then I found the vanilla-compatible "tubeless complex mod" and I decided to try something new with it. New to me anyways.

I first built a mine on every asteroid in sector "Ore Belt". Bought 3 TL's full of Complex Construction Kits, and proceeded to build a complex hub towards the south east corner of this large, sprawling group of mines. I placed it so that it was able to connect to mines furtherest out from centre in that corner. I used more CCKs to connect to all mines within range. Then I built a second hub, barely within connection range (19.9km) of the first, and used CCKs to connect any mines in range to it. When this was complete, I joined the second to the first, selecting the #2 hub first. This basically made the first hub disappear and the second hub was connecting to mines up to 39.9km away. I built a third complex hub and then a fourth etc, until I had all mines connected to just one hub, which was somewhere close to where I wanted to place the end result hub. I then built a total of eleven SPP XLs at a position 39.8km directly above the hub to which all mines were connected. For the hub that connected the SPPs, I placed this 19.9KM directly below the SPPs, which placed it barely within range of connecting to the mines hub. The SPPs were actually all at the same location and orientation. Completely overlapping so that you couldn't tell there were any more than one of them by looking alone. This "overlap" build method is detailed in other threads. The theory goes that it means less framerate degredation in sector. For me, it makes station placement much easier, as I only need to be there looking for the first, thereafter I can be anywhere in the sector.

Then I built crystal fabs under the SPPs, twelve in each "overlap". Once all (219) necessary fabs were built, I used one more CCK to join up all mines with all factories, selecting the hub located higher in the sector first. This put the end result complex hub 20KM or so above the mines themselves, and 20KM below my factories. Without any tubes, it's not obvious that there's a complex in the sector at all, let alone realise that every single asteroid in the sector is now part of one. And you can't even see the factories some 40KM or so above the equatorial unless you approach the complex hub.

While I was extremely pleased that I ended up with a large complex capable of doing justice to the hub plot without having to tow asteroids, I'm having a rather severe problem with loading and sector change times. It's been so long since I built a large complex that I've forgotten how much they normally increase loading times. In any case, I can't understand why having large complexes in Ore belt and Barren Shores should make flying through a gate (any gate) take three times as long.
As I don't know the answers to these, I'm just mentioning them in case something I've done in the above makes your game as slow as mine has become. I would like to continue to build large complexes without tubes and without towing mines around. If that means sector entry times doubling, tripling and quadrupling, then perhaps I'll have to reconsider. I guess I can always destroy all fabs once the hub is complete?

EDIT 8th June 2011: As I completed the hub last night I've destroy the 500 stations over 2 complexes I built. Nothing has sped up by much yet but then the station debris is yet to be removed by GoD. As mentioned on page 3, super-imposing fabs doesn't generally lead to longer gate passing times. I made an observation a while back that "looking at" stations in sector that had been built super-imposed but not joined yet (which would normally run down their shields quickly) were actually recovering. I had concluded at that time that this was triggered simply by looking at their info. But I worked out today that it isn't looking at the info that makes the shields repair in sector, it's the use of SETA which seems to be working against the shield degradation. This effect is so pronounced, that when I compared the shield repair speed of being out of sector to that experienced of being in sector but with SETAx10 active, the SETAx10 repair speed was six times as fast! For a 100 MJ shield increase, this took 55 seconds OS (non SETA) and only 9 seconds IS with SETAx10 active. One would assume then, that both being OOS *and* with SETAx10 active that shield repair speed would be even faster again. With some fabs this is the case but with others (SPP XL's in particular) it's not. But in all cases, there seems to be no point in going OOS to recover shields so that you can continue to super-impose more fabs. Just SETAx10 for a while and keep stacking on more. Definately more convenient eh? This may be effected per system so do a save before you commit to this method..

[twilight_echoes]

For sure it's best not to have to move asteroids if at all possible. I have also just finished connecting up every silicon asteroid in Ore belt - though without the aid of the tubeless mod, and it isn't acting out too badly. Flying in sector can result in a bit of jerkiness, but nothing worse than that.

I have a total of around 900 stations now, nearly all of which are in complexes of 30+ stations, the largest being Barren Shores which is a self-sustaining 78 microchip hub plex. The stations that create most lag for me are the most disordered ones, before I'd learned to build in cubes. Load time and time to enter sectors is ok, but I wouldn't take combat missions in any of the sectors with megaplexes in. In fact I avoid going in-sector at all, however, there is a great deal of the universe where I've no factories and gameplay there seems just fine.

[glenmcd]

For sure it's best not to have to move asteroids if at all possible. I have also just finished connecting up every silicon asteroid in Ore belt - though without the aid of the tubeless mod, and it isn't acting out too badly. Flying in sector can result in a bit of jerkiness, but nothing worse than that.

I have a total of around 900 stations now, nearly all of which are in complexes of 30+ stations, the largest being Barren Shores which is a self-sustaining 78 microchip hub plex. The stations that create most lag for me are the most disordered ones, before I'd learned to build in cubes. Load time and time to enter sectors is ok, but I wouldn't take combat missions in any of the sectors with megaplexes in. In fact I avoid going in-sector at all, however, there is a great deal of the universe where I've no factories and gameplay there seems just fine.

With your ore belt complex, did you leave all mines in place (no towing) also? If so it looks like the tubeless mod across vast distances is causing the long gate passing times.

[twilight_echoes]

I left everything in its original position, but ended up with 4 complexes rather than just 1. A huge one in the top left, and 3 middling sized ones. I wasn't so bothered about having them all in a single unit, as I built them for silicon. Once that part of the HUB plot is over, I will add more fabs to do something else.

My Barren Shores complex is a single megaplex with no towing. It does lag in sector but doesn't lag the game OOS noticeably.

Perhaps computer specs play a part too - Im running Win 7 64 bit edn with 4Gb RAM on an Quad Core 9550 2.83Ghz.

[glenmcd]

I left everything in its original position, but ended up with 4 complexes rather than just 1. A huge one in the top left, and 3 middling sized ones. I wasn't so bothered about having them all in a single unit, as I built them for silicon. Once that part of the HUB plot is over, I will add more fabs to do something else.

My Barren Shores complex is a single megaplex with no towing. It does lag in sector but doesn't lag the game OOS noticeably.

Perhaps computer specs play a part too - Im running Win 7 64 bit edn with 4Gb RAM on an Quad Core 9550 2.83Ghz.

I did Barren Shores with no towing, noticed some longer gate times but not anywhere near as much as after I completed Ore Belt. My system is fairly similar to yours, Win7-64, 12GB on i7-950 3.06Ghz. But it's pretty easy to rule out system differences by exchanging saves. I've uploaded mine to http://www.glenmcd.com/X09.sav

Flying through a gate from that sector takes 50 seconds. Reloading that save (with same already loaded) takes 89 seconds. Loading from main menu takes 70 seconds. Oh, and good luck with the Xenon J a few hundred metres to your right hehe

EDIT: For comparison, I timed flying through the gate between Herrons Nebula and Cloudbase North West from a Humble Merchant start. Around 4 to 5 seconds. Quite a difference!

[glenmcd]

I've just used self-destruct on all of my complex hubs, effectively isolating the problem from the tubeless mod and the vast distances I was forcing it to cover. Gate passing and reload times didn't change. It isn't the number of factories so the only thing left is overlapping factory positions. If you're in sector and without CCKs joining them, overlapping factories destroy each other. Once you connect them all, or before that if you're out of sector, shields stay intact (and will recover if damaged before). I'll have to try this all again but without overlapping factory positions. I have used the overlap method before but not to this extent. I can live without it, will just take a bit longer to position each factory that's all. Thanks for your help twilight_echoes.

EDIT: Luckily I've got plenty of save files to select from. I did an extreme comparison test by timing how long it took to fly between the 2 unknown sectors near southern end of universe. After building all Barren Shores mines but before factories: 6 passes in 12 seconds = 2 seconds per pass. After building Barren Shores and Ore Belt complexes: 49 seconds per pass.

[TTD]

When I connected as many as possible in Ore Belt
I did tow the mines as far from the main map as I could and had my overlapping "New-Tech" complexes built as mini-plexes.I then built miniplexes out of the mines and finally connected them all up .bar one mini-plex.

This event is narrated in my Trader's Adventure.

The main thing to reduce IS FPS lag is to have as much of your complex as far away as possible so that the stations do not all have to be visible.
Thus I think this is where you are getting some of the problem,as the stations are all fairly visible when you enter your sector.

[VincentTH]

When I connected as many as possible in Ore Belt
I did tow the mines as far from the main map as I could and had my overlapping "New-Tech" complexes built as mini-plexes.I then built miniplexes out of the mines and finally connected them all up .bar one mini-plex.

This event is narrated in my Trader's Adventure.

The main thing to reduce IS FPS lag is to have as much of your complex as far away as possible so that the stations do not all have to be visible.
Thus I think this is where you are getting some of the problem,as the stations are all fairly visible when you enter your sector.

I build all of my mega-complexes behind the gates, that way, I never have it on my screen as I enter the sector, unless I want to dock there.

[glenmcd]

When I connected as many as possible in Ore Belt
I did tow the mines as far from the main map as I could and had my overlapping "New-Tech" complexes built as mini-plexes.I then built miniplexes out of the mines and finally connected them all up .bar one mini-plex.

This event is narrated in my Trader's Adventure.

The main thing to reduce IS FPS lag is to have as much of your complex as far away as possible so that the stations do not all have to be visible.
Thus I think this is where you are getting some of the problem,as the stations are all fairly visible when you enter your sector.

Thanks TTD. The problem was not IS lag, but the time taken to change sectors (any sector), be that by jumpdrive or flying through a gate. Reload times - regardless of which sector I was in when the save was done - is really long also.

I'm still researching this. Fortunately, I've got a huge series of backups to fall back on, and it's actually looking like something happens when the number of factories in a complex gets to somewhere around 100. I had a save with 96 fabs/mines joined, and then another with 112 joined. I used the time taken to fly 12 times between the two unknown sectors west of Unholy descent as a benchmark. I simply held down tab while in a maxed out Kestrel. Right from scratch until I had 96 factories joined with CCKs, the time was 26 seconds (12 gate crosses). I have a save for every 10 to 14 factories that I built. As it went past 100, suddenly the time for 12 gate passes jumped to 30, then 38 etc until at Complex completion (170 factories/mines), the time had more then doubled. So 96 stations connected and not even one second added, yet another 80 stations and 30 seconds are added. The Ore Belt complex is/was much larger again, with a total of 256 stations/mines. That's when the time blew out from 58 seconds (12 passes) to 588 seconds. I know all this sounds far fetched, that's why I'm still researching it. At the moment I'm building Barren Shores again, but splitting into two similarly sized complexes, so should be around 80 fabs per complex. I'll otherwise build in the same way (overlap included) and see how it goes.

[TTD]

That is interesting.

So is it the total number in a sector,or the number of stations in a complex,that is causing this?

On a similar note I read somewhere about a save taking over forty minutes to load.

[glenmcd]

That is interesting.

So is it the total number in a sector,or the number of stations in a complex,that is causing this?

On a similar note I read somewhere about a save taking over forty minutes to load.

Yesterday I built 191 stations/mines in Barren Shores, but have not used any CCKs at all as yet. For every ten stations built, I timed 24 passes of the gate between two unknown sectors near Unholy descent, and the time was 48 or 49 seconds for every test. Thus as far as gate passing times go, building stations does NOT make gate passing slower AT ALL. If anything, the 48 second times came more often than 49 towards the end. If the result from this provides a workaround to a common problem then I'll create a new thread to detail it. At the moment the possible causes are:

1. More than X number of stations (almost certainly 100) in a complex joined by CCKs
2. The use of tubeless mod in a 100+ station complex
3. Spanning extreme distances with invisible tubes
4. Spanning extreme distances with CCKs (standard visible ones)

I'm expecting and hoping that the answer is #1, as this means that a half reasonable workaround would be available (split complexes into <100 stations), and mods such as tubeless can continue to be used, as well as spanning long distances which allow for no-tow complexes that include mines. As the tubeless mod looks to be a graphics only mod (zero actual code), I highly doubt that it would have anything to do with an issue that only comes into being when 100+ stations are joined. If it was a smooth transition (each station beyond zero slows down gate passes) then that would be a different matter.

EDIT: Loading time (from a .sav file) is increasing linearly with number of stations, thus this is an independant "issue", one that can be accepted/ignored as there's good reasons for it. I'm focussing purely on the time to fly through a gate, or to enter a sector from a jump or to leave a sector. You can leave a sector without entering another by quitting back to main menu. This is part of the same issue and amounts to roughly 35% of the added time.

[nponoBegHuk]

I'm thinking about destroying all the stations and complexes I own (including chip complex on all silicon in Barren Shores and huge crystal complex in Savage Spur; except the missile complex in Merchant Heaven. Dealing with overall game lag, loading time increases, gate delays (and protecting CAGs/dealing with pirates attacking the station itself all the time) just for minor credit income is so not worth it, especially given that in TC you can make 20 billion in 5 seconds.

I'll just attach rastar/rimes/telad etc factories to support PHQ production queue, the complex is stalled due to being full on missiles all the time anyway and I CBA buying more freighers to keep them.

[vr01]

IMO it's the number of factories in the sector. A while ago I built a 706 station complex in the sector east of PTNI (add 705 CCK's to that as well). Towards the end of building it, the game load and sector change times blew out to 10 minutes...

At that stage of the game I already had over 1100 stations before building the above complex. (Barren Shores, Antigone Memorial, Ore Belt complexed amongst others...)

Tubeless mod made no difference to the load times but did help the in sector FPS.

I ended up going back to a pre-build save and building a much smaller version of the same complex. No discernible changes to the load times.

[David Howland]

The thought of you guys destroying all that good work because the hub is done breaks my heart. Although I only aspire to your levels of building, I did over 18 months ago nearly give up playing TC due to being stuck on the Ore Belt mission of Treasure Hunt due to game freezes and crashes.
I did however get through it by a month of constant saves, crashes and reloads.
Now although Ore Belt is my main equiping complex for new ships, I just avoid going there. There is a Shrike on patrol there to take care of pirate attacks and the only reason I would visit my larger complex sectors is to do some station building. Note most of these huge complex sectors are away from Xenon regions and only two (Ore Belt and Asteroid Belt) made plots difficult. Also I have found the Hub complexes, particularly chip plants, very useful for supplying my PHQ! So please think twice before destroying all that good work.

[nponoBegHuk]

Building itself is fast - at 4-6 stations per minute there's no problem deploying 200 of them in an hour. The problem is the asteroid towing which takes 3x the time needed to build everything else for the complex.

[Hubert99]

Plus chip complexes generate insane amounts of credits. I have a 10 and an 11 chip complex in Ore belt, but in fact it is 3 complexes and 3 unlinked silicon mines. One complex makes energy, one is self suffcient, one gets the energy from the first. The self sufficent one actually isn't entirley self suffcient as it relies on the three separate silicon mines and some logistic ships. Plus I have 3 ore mines, left over from the ore part of the plot, they have each a SF CAG assigned and it makes good money selling to the nearby sectors.

The only problem is not the lag as I did tow some roids and the complexes are in three far away corners of the sector, behind the gates, but rather, I messed up my energy complex, and sometimes the energy CLS ship explodes as she undocks. Don't know why as the hub has unrestricted access. Anyway, I just don't go there anymore, or I have to dock that silly ship. It's a mistral SF ...

[David Howland]

Yep I would always advocate using Self Sustaining complexes where roids will allow. Its well worth the extra time and credits setting it up, to know that when it is set up there are no supply worries and all you have to do is defend the actual station from enemies!

[glenmcd]

I've spent a lot of hours researching this issue now. Well maybe not THAT many but they've been quite painful - building something like 600 fabs with 550 or so CCKs, flying through gates a thousand or so times, timing stuff..

I haven't got a conclusive result. But I do have fairly strong indicators to a couple of things that are causing long gate passing times. "Norman" probably won't like this

1. The thing that most strongly indicates that it's causing long gate passing times is overlapping factories. TTD documented/invented these some years ago and documented same. On the surface they are exceedingly useful in speeding up the building of complexes. Place one fab carefully, dump a bunch of them over the top of that one and race to get them all connected with CCKs before the shields drop to zero and the overlap starts eating into the hulls. As TTD discovered, for each additional fab you overlap, the eating away of shields accelerates. It looks to me that where shield eating speed for 2 fabs is 1+1, 3 fabs is 1+1+2, 4 fabs is 1+1+2+3 etc. I managed to get 17 Sun oil refineries overlapped and successfully connected them with CCKs. I did this in two sessions, and in between I went out of sector and used SETAx10 while the shields got back up to 100%. I normally keep the overlap count down to 12 or less. At the time of doing this my fab count within the Barren Shores complex increased to 95 - very close to the 100 at which I mentioned before things starting going wrong. I built my complex with the mines first (no overlap possible), then SPPs, then crystals, then sun oil refineries, then flower farms, then chip plants. The counts of these fabs are progressively increasing - 6 SPPs overlapped, then 11 Crystal fabs overlapped, then 28 sun oil refineries overlapped and eventually 71 chip plants. With only a small number of fabs overlapped, the gate passing delay times seems to be insignifican to the point of being unmeasurable. But as I overlapped more and more fabs, the gate pass times starting really blowing out. Instead of a couple more seconds it began adding 20 seconds and more at a time. So somewhere in there as you add yet one more overlapped factory, the execution time is increased exponentially rather than linearly. And because of this, to test properly it requires building more than one massive complex in a particular way, then starting again and build it another way etc. It would take weeks to do so and I'm just not motivated enough to go quite that far. My entire reason for going back to complex building in the first place was to complete all missions in one game start so that I could get all game starts. I feel like it's not worth it, and I can always use the registery hack to get the game starts. Maybe I'll change my mind next week who knows.

2. The second factor which seems to have an effect on gate passing times is the use of CCKs, possibly in conjuction with the no tubes "vanilla mod". I haven't tested without the no tubes mod so I don't know whether that has anything to do with it. In any case. the effect seems linear (#1 effect is exponential) and is not quite insignificant but not far off. A complex with 192 fabs increased gate passing time by ten percent. Considering that I got into this because gate times had grown to 25x as long it's hardly worth even mentioning. It is interesting to note that building factories alone (without using CCKs, no tubes mod or not) did not add to gate passing times anything that could be measured (around 2 percent in my tests as I'm only timing to the whole second).

I'd like to summarise by saying that the above "results" are IMO not conclusive and if anyone else takes the research further (which I'd be happy to see) they could well discover stuff that would make the above look somewhat inaccurate, or at least incomplete. For now, if I do build more complexes then I'll be using no tubes mod, not overlapping, and keeping fab count per complex down to 90 or so. I'd rather spend the time once buying and outfitting ships to go between the smaller complexes than waiting ten times as long just to fly through each and ever gate.

[Gazz]

That is the reason why complex mods exist.

The number of objects in a sector is an issue. Whether those are factories or "only" CCK, they are all objects.
They all need to be loaded and checked for possible collisions.
They need models and textures, too. With XP's 2GB limit, X3 will eventually crash upon entering such a sector.

There are various approaches of mods and scripts but they are all centered around reducing the number of objects in the sector and/or reducing the models and thereby load times.

[TTD]

And here's a more direct link to my experiments.
http://forum.egosoft.com/viewtopic.php? ... 11#3149111
Part way down were it says

Universal Fax to all Station Managers in the employ of Norman Fudge...

Please note,

I was not the first to come up with the idea.
There were those before me who,like many scientists of today,were ridiculed or disbelieved.
I just put it into a story where I could test the theory.
Note also that this method may not work in non-TC productions.