X3:TC Performance Issues

[Alan Phipps]

Maybe it is time to start thinking outside the box then. Are you sure (ie have you looked) that you don't have an in-game bottleneck issue such as hundreds of NPC traders all trying to get to the same station in a sector at the same time? (I know the 'Billion Barracudas Bug' is an X3AP issue but there are a few threads about hugely clogged NPC and Player stations in vanilla X3TC too.)

Your odd fps behaviour and cpu core overload might just be the result of a script logjam due to a rare set of in-game circumstances. It is at least worth a check around your Universe to see if there are any crowd-fests.

[cakes]

Actually Alan, re-reading the original post again, djnoob states the issue exists in TC and AP, albeit worse in AP.

[djnoob]

Maybe it is time to start thinking outside the box then. Are you sure (ie have you looked) that you don't have an in-game bottleneck issue such as hundreds of NPC traders all trying to get to the same station in a sector at the same time? (I know the 'Billion Barracudas Bug' is an X3AP issue but there are a few threads about hugely clogged NPC and Player stations in vanilla X3TC too.)

Your odd fps behaviour and cpu core overload might just be the result of a script logjam due to a rare set of in-game circumstances. It is at least worth a check around your Universe to see if there are any crowd-fests.

well since this thread started, I did a clean install of the game. So recent testing has been from a complete vanilla game with a brand new start.

It's interesting too because I'll go in, start a new game, then hit seta for a few minutes. As the world gets populated I slowly watch my gpu usage and frame rate slowly drop. After a few minutes I go from full gpu usage to about 30% usage. Ultimately this is just a cpu bottleneck (as well as a game not optimized for current hardware/software) where the only way around it is going to be to overclock the cpu.

The part that kind of drives me crazy is the hitching. It's almost as if my whole system just stops. I see a reduction in cpu/gpu along with a drop in fps and everything on the screen stops for a moment. I just have no idea as to what could be causing this.

[Alan Phipps]

Sorry to be rather pedantic, but when you say a clean reinstall, do you mean that you manually deleted the entire Steam X3TC gamefile folder instead of or after using the relevant Steam 'Delete Local (Game) Content' and before reinstalling or verifying?

Tha manual folder delete is necessary to remove any corrupt or additional 3rd party gamefiles that the uninstaller itself cannot remove or does not know about. Sorry if this is too obvious a question but we are clutching at straws here!

[Sam L.R. Griffiths]

In my current game I am running some mods atm and I'm getting far enough along with it where the fps drop of a longer game is reaching the point of where a big battle is likely to be unplayable. Plus with the lower overall fps, the hitching/stuttering is becoming ever more noticeable.

I appologise if this has been covered/asked already, but exactly what scripts/mods are you using in both your TC and AP games?

[djnoob]

Sorry to be rather pedantic, but when you say a clean reinstall, do you mean that you manually deleted the entire Steam X3TC gamefile folder instead of or after using the relevant Steam 'Delete Local (Game) Content' and before reinstalling or verifying?

Tha manual folder delete is necessary to remove any corrupt or additional 3rd party gamefiles that the uninstaller itself cannot remove or does not know about. Sorry if this is too obvious a question but we are clutching at straws here!

LOL that's a fair and valid question. I deleted the folder. Actually both the folder in steam and my save games in my documents.

I'm going to do a bit more testing with some other games to see if I get that hitching with other games.

[djnoob]

In my current game I am running some mods atm and I'm getting far enough along with it where the fps drop of a longer game is reaching the point of where a big battle is likely to be unplayable. Plus with the lower overall fps, the hitching/stuttering is becoming ever more noticeable.

I appologise if this has been covered/asked already, but exactly what scripts/mods are you using in both your TC and AP games?

yes it's been covered. I'm currently running vanilla.

[cakes]

Have you disabled evga precision and tested?

Looking at your original post, you mention "tweaking" the Win7 processes to a minimum. Did this extend to tweaking services too? Depending on what you did, how about returning the OS to a non-tweaked state and retesting?

Furthermore, is the steam installation on your SSD or your HDD? I know it sounds counter-intuitive, but if it's on your SSD, try a wholesale copy and paste onto your HDD and run it with the no-steam .exe that's available on this site.

As with everything else in this saga, we're still trying to narrow it down. As for throwing in the towel, I'm sure we all appreciate how frustrating this is.

[djnoob]

Have you disabled evga precision and tested?

Looking at your original post, you mention "tweaking" the Win7 processes to a minimum. Did this extend to tweaking services too? Depending on what you did, how about returning the OS to a non-tweaked state and retesting?

Furthermore, is the steam installation on your SSD or your HDD? I know it sounds counter-intuitive, but if it's on your SSD, try a wholesale copy and paste onto your HDD and run it with the no-steam .exe that's available on this site.

As with everything else in this saga, we're still trying to narrow it down. As for throwing in the towel, I'm sure we all appreciate how frustrating this is.

1) Yeah I tried it without precision running.

2) When I first did the win7 install I tested before I tweaked anything. Plus, those handful of tweaks would have nothing to do with how the game runs.

3) Steam is installed my ssd. LOL you're right is counter-intuitive but it's actually not a bad thing to try. The game does do a crap ton of file read/writes. Maybe there is some issue regarding the sata3 controller.

4) I did do some testing with other games. Skyrim, borderlands 2, black mesa source, none of them had the hitching. Even with BMS, core zero stays pegged at 100% and no hitching.

5) I appreciate all of the input from everyone. I've personally helped a ton of peeps with pc issues. Sometimes it just takes a meeting of the minds to sort it out.

[cakes]

The reason I was suggesting using your HDD as a test, was primarily due to just noticing you having a Marvell SATA 6G controller, which I assume you have the SSD connected to.

Re-reading your original post and dxdiag for the umpteenth time now , I seem to recall issues with the 91xx series having issues with stability, smaller size write performance, failing to issue TRIM commands and consequently ...... causing stuttering!

As you're using an X58 series chipset, I'd recommend plugging the SSD into your intel ICH10R SATA ports. I realise the max bandwidth on the channels will be lower, but it will probably increase your real world SSD speed, as most transactions will be smaller read/writes.

You may find a newer driver/firmware from Marvell directly, but I seem to remember there was some inherent issues in the 91xx series controllers. I'm using the intel controller on my setup with SSD's, to good effect (mines a Z68 series chipset however).

[djnoob]

The reason I was suggesting using your HDD as a test, was primarily due to just noticing you having a Marvell SATA 6G controller, which I assume you have the SSD connected to.

Re-reading your original post and dxdiag for the umpteenth time now , I seem to recall issues with the 91xx series having issues with stability, smaller size write performance, failing to issue TRIM commands and consequently ...... causing stuttering!

As you're using an X58 series chipset, I'd recommend plugging the SSD into your intel ICH10R SATA ports. I realise the max bandwidth on the channels will be lower, but it will probably increase your real world SSD speed, as most transactions will be smaller read/writes.

You may find a newer driver/firmware from Marvell directly, but I seem to remember there was some inherent issues in the 91xx series controllers. I'm using the intel controller on my setup with SSD's, to good effect (mines a Z68 series chipset however).

Cakes you're bang on with this. I do remember as well there being issues with the marvel controller. I just looked and there's a firmware update for the controller. The result is.... I've got a long night ahead of me. I'm going to run the firmware and then reinstall win7 (again). I'll test after that's done then if all else fails I'll try it with the game on another drive.

[cakes]

How about simply connecting it to the intel controller sata port? Unless you're dead set on a rebuild of course

[djnoob]

well so I did another clean install of win7 and this time I just disabled the marvell controller and am running the ssd on sata 2 port. Looking through the forums it was suggested to dump the marvell controller all together.

So then I started up the game, ran seta, about min in, bam! Another fat hitch. So that's not it.

I will say this though, it could be just random chance but after all this tweaking and trying different things out, the hitching seems to be happening less often. But again, that could just be random chance.


uuugh. sigh.

[djnoob]

How about simply connecting it to the intel controller sata port? Unless you're dead set on a rebuild of course

I thought about that but when I did this last install I didn't have ahci enabled in the bios. So after installing I had to do a reg edit to ensure that was working properly. Seeing as that happen, just in the off chance something wasn't right I figured a clean install was the way to go just to be sure there weren't any issues there.

[cakes]

I suppose the next test would be to copy the X3 directories to your HDD as in my earlier post. That way I suppose we could rule out the SSD as a factor in the issue.

Incidently, are you doing alot of benchmarks of your SSD? I know there's alot of warnings out there against doing this. Just a thought.

On a slightly different line, just some basics really, have you tried replacing the data and/or power cable to the SSD? I know it's probably not the issue, but I've seen transient errors like this before and they sometimes are simple things like cabling, psu issues with a certain power connector or power channel.

All in all X3 is utilising something that isn't apparent within other software. Apart from threatening your PC with a hammer, I'm officially stumped on this one.

[zhaarteth]

I've been following this thread since a couple days after the first post just to see where it'd go before I posted my own thread. But now, everything posted so far here leads me to believe that this is the same issue I've had since I got the game in early December. My hardware is a bit different, the only real similarity being the Marvell controller, which I'd temporarily disabled to see if that'd fix it (it didn't).

Before I go and make my own thread (and post an updated DxDiag and such) I want you (djnoob) to do one thing that'd confirm this is the same issue that I'm having: Download Process Explorer, and upon running the game, right click its process (within the ProcEx window) and select Properties, then navigate to the Performance tab. Have it prepared on your secondary monitor so you can correlate it with the hitches.

When the worst of these hitches happen for me, the Read Bytes Delta always goes into the upwards of 40MB/s. Given that the issue in AP is far worse, you might get quicker results. If you can verify this, then we're likely having the same issue on two different machines. Here's what mine looks like when this happens. Whether two minutes or two hours in-game, the worst of these hitches are always in high-density areas with a lot of ships, stations, and asteroids.

I think it's important to note that there is minimal-to-no activity on the IDE indicator LED when the Read Bytes Delta skyrockets, suggesting that there isn't a PF read/write issue nor is the game attempting to write/gather data from the disk. The only other time I ever notice this sort of thing happening (high RBD) in other games is during loading screens or when a game is loading absurdly-large textures (e.g. a poorly-optimised mod for a Bethesda game). But even when high RBD occurs in other games, there is disk activity that goes along with it. So I suspect that whatever it's doing, the problem is with active data and nothing (or close to nothing) which is stored to ROM.

Modified or Vanilla, the labile performance is the same. Also I don't have a Solid State Drive. However I have tried nearly everything stated in this thread among others (where applicable, aside from a complete system wipe which is on my LT agenda anyway) and this odd lag issue remains unchanged. Again, literally everything stated in this thread so far has me convinced this is the same issue. The FPS drops when using menus (even at the ingame start screen), and the lag spikes (hitches) get worse with the more stuff there is onscreen. Just for GP, I verified that BMS runs flawlessly.

Some other things that I've tried:
*Setting Win7 theme to Basic.
*Razer Game Booster
*Temporarily disabling all anti-malware applications.
*Temporarily disabling power-related options in BIOS (such as SpeedStep and CxE)

[Alan Phipps]

@ zhaarteth: Thanks for this and it will be interesting to see what djnoob makes of it.

That said, you should be having significantly less good X3 gameplay than him anyway because of your 32 bit OS that will limit the amount of your 4 Gb RAM available to the game exe (which can use up to 3.5 Gb usefully) after the graphics card VRAM addresses, shared graphics memory and OS/services/drivers/other applications have all had their slice of available RAM. That you have very similar symptoms might actually point to a memory management/addressing issue on djnoob's system which has an x64 OS with 6 Gb RAM.

[zhaarteth]

@ zhaarteth: Thanks for this and it will be interesting to see what djnoob makes of it.

That said, you should be having significantly less good X3 gameplay than him anyway because of your 32 bit OS that will limit the amount of your 4 Gb RAM available to the game exe (which can use up to 3.5 Gb usefully) after the graphics card VRAM addresses, shared graphics memory and OS/services/drivers/other applications have all had their slice of available RAM. That you have very similar symptoms might actually point to a memory management/addressing issue on djnoob's system which has an x64 OS with 6 Gb RAM.

Not a problem.

As for part deux, I think you're about half right. When avoiding any combat or high-density areas I am getting an average of 80+ FPS after certain tweaks (main in-game menu gives about 92 on average, going down to the upper seventies when using the menu or mid-80s when just moving the mouse). However the hitches are indeed far worse for me, as they also cause parts of the interface to disappear. Actually, one of my saves (modified) are in a battle with a pirate squadron, and I net ~30 FPS in between the hitches. The only way I can know this is due to having the Cheat Package installed so that I can monitor frame rate as well as RBD behaviour during these hitches. Without it, my ability to monitor this stuff would be quite limited. As previously mentioned, the hitches are bad regardless of having extra scripts. This is what leads me to believe it's the same issue; while I get understandably lower performance, and the hitches are indeed worse for me, the symptoms and analysis so far in this thread are indeed quite similar.

All that said though, I am planning on obtaining a new hard drive and swap RAM with my previous mobo (6GiB but slightly slower clock) and eventually install 64-bit Win7Pro. I was going to wait until then before I posted my issue (if it persisted), but this thread rather surprised me. ETA on the hardware and OS changes is mid-Feb, so we'll see.

In the meantime I have now been awake for 29 hours (GMT-8) fiddling with different settings to see what might improve/address the issue OS-wise. Haven't found anything yet. Hopefully by the time I get back djnoob or someone else will have more data.

TTFN

[djnoob]

I've been following this thread since a couple days after the first post just to see where it'd go before I posted my own thread. But now, everything posted so far here leads me to believe that this is the same issue I've had since I got the game in early December. My hardware is a bit different, the only real similarity being the Marvell controller, which I'd temporarily disabled to see if that'd fix it (it didn't).

Before I go and make my own thread (and post an updated DxDiag and such) I want you (djnoob) to do one thing that'd confirm this is the same issue that I'm having: Download Process Explorer, and upon running the game, right click its process (within the ProcEx window) and select Properties, then navigate to the Performance tab. Have it prepared on your secondary monitor so you can correlate it with the hitches.

When the worst of these hitches happen for me, the Read Bytes Delta always goes into the upwards of 40MB/s. Given that the issue in AP is far worse, you might get quicker results. If you can verify this, then we're likely having the same issue on two different machines. Here's what mine looks like when this happens. Whether two minutes or two hours in-game, the worst of these hitches are always in high-density areas with a lot of ships, stations, and asteroids.

I think it's important to note that there is minimal-to-no activity on the IDE indicator LED when the Read Bytes Delta skyrockets, suggesting that there isn't a PF read/write issue nor is the game attempting to write/gather data from the disk. The only other time I ever notice this sort of thing happening (high RBD) in other games is during loading screens or when a game is loading absurdly-large textures (e.g. a poorly-optimised mod for a Bethesda game). But even when high RBD occurs in other games, there is disk activity that goes along with it. So I suspect that whatever it's doing, the problem is with active data and nothing (or close to nothing) which is stored to ROM.

Modified or Vanilla, the labile performance is the same. Also I don't have a Solid State Drive. However I have tried nearly everything stated in this thread among others (where applicable, aside from a complete system wipe which is on my LT agenda anyway) and this odd lag issue remains unchanged. Again, literally everything stated in this thread so far has me convinced this is the same issue. The FPS drops when using menus (even at the ingame start screen), and the lag spikes (hitches) get worse with the more stuff there is onscreen. Just for GP, I verified that BMS runs flawlessly.

Some other things that I've tried:
*Setting Win7 theme to Basic.
*Razer Game Booster
*Temporarily disabling all anti-malware applications.
*Temporarily disabling power-related options in BIOS (such as SpeedStep and CxE)

A couple of things that you mentioned are very interesting.

1) You mentioned bethesda games and after dealing with the frustration with x3 I decided to step away and go play Fallout 3. Well guess what.....the same type of issue presented itself. I'll be playing and the game will be smooth as butter when all of a sudden I'll see a drop in fps. It happens quite random but there is a correlation which is my 2nd point.

2) As you mentioned, there does seem to be a link between the lag spikes and the game having to load in textures. It's very obvious in FO3. With x3, as you described, the issue is more prevalent in systems where there are a lot of things going on and a lot of stations/asteroids/ships, etc. Something I've also seen people describe is getting lag spikes when capital ships come into the system. There's also the common issue with such things as complexes where just looking at a even a medium sized complex can cause a massive reduction in framerate regardless of graphics settings.

I'll go ahead and give that tool a try and see what happens and then report back.

[djnoob]

I need to do some more testing but I did see one instance where there was a jump in the RBD that correlated to a hitch but it was a LOT smaller than yours. It only jumped to 500+ kb (yes that's with a k). I'd seen a couple of times where it jumped to a few MB but didn't see a hitch.

I also have seen a new issue. I did as I have done previously in my testing, started up a brand new game and then just ran seta. Well after about 30-45 min my game had stayed running around 60 fps with 10x seta running, then all of a sudden it dropped down to about 25fps and never went back up. Even when I turned seta off and my fps still did not go back up. I saved the game, then restarted x3 all together and went back to that save and still my fps was only around 25. There was no corresponding increase in the RBD, IO, or disk activity.