[SCR] OK Traders v1.7.1 (04-12-14)

The place to discuss scripting and game modifications for X³: Terran Conflict and X³: Albion Prelude.

Moderators: Moderators for English X Forum, Scripting / Modding Moderators

Post Reply
Shimrod
Posts: 907
Joined: Tue, 18. Feb 03, 01:43
x4

Post by Shimrod » Thu, 27. Nov 14, 20:17

I've installed XTC 2.2 with hotfix 22. I created 50 docked drakes and broadcast OK to them.

The traders started out selling all the equipment I spawned on them that they don't need, like excess energy and ammo. During this time I get no apparent slowdown at 10x seta, nor while fighting 20 xenon M spawned around my tapian. This tends to suggest the monitoring task running on each trader isn't what's taxing the cpu.

Still no appreciable slowdown after I left it on seta for a while. I've been observing an ATF vidar fighting 10 xenon fujin at seta 10x while rotating the camera and its pretty smooth.

This is on i7-2600k with 16gb ram, win7 x64.

All I can say at this point is, make sure ok's debug mode is disabled as the file logging has a big overhead, and minimize the number of economy boost mode traders.

User avatar
dizzy
Posts: 1019
Joined: Sun, 26. Sep 10, 06:00
x4

Post by dizzy » Fri, 28. Nov 14, 01:30

I do have a couple of Economy Boost mode traders. Let me disable that and see how it goes. I could do some other tests too such as suddenly telling all my traders to stop doing anything they're currently doing and see if the lag stops.

I was kinda busy with other things (getting Xtended statistics working and such). Thanks for looking into it, now I feel bad if I sent you on a wild-goose chase :) I'll let you know if I have any new information.

BTW, profit wise it has definitely gone up compared to using UTs. I make about 25 mil per game hour now as opposed to 10 that was the average before.

albazeus
Posts: 76
Joined: Fri, 21. Feb 14, 17:21
x3ap

Post by albazeus » Wed, 3. Dec 14, 01:36

I recently upgraded to version 1.7 and I am experiencing lot of lags.
I have 20 free traders and other ~20 homebased. I don't know what causes this lag but if I uninstall the script the lags disappear.

What should I do to debug it? I post the log of a homebased trader:

Code: Select all

OK Traders Version=1.7.0, Internal=1007000
[1370424 5  Monitor Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Terminating monitor; cmd script not running
[1370434 5  Cmd     Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Start: Directive=0, Version=1007000
[1370434 5  Monitor Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Start: Version=1007000
[1370434 5  Cmd     Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Select.Mission.Homebased: Trade opportunity: ware=Mosquito Missile, dest=Boron Equipment Dock(Depths of Silence), mode=2, price=168
[1370434 5  State   Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Cmd=Sell, Dest=Boron Equipment Dock(Depths of Silence), Ware=Mosquito Missile, Price=168, Environment=Depths of Silence
[1370434 5  State   Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Cmd=Sell, Dest=Boron Equipment Dock(Depths of Silence), Ware=Mosquito Missile, Price=168, Environment=Depths of Silence
[1370480 5  Monitor Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Terminating monitor; cmd script not running
[1370497 5  Cmd     Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Start: Directive=0, Version=1007000
[1370497 5  Monitor Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Start: Version=1007000
[1370497 5  Cmd     Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Select.Mission.Homebased: Trade opportunity: ware=Mosquito Missile, dest=Boron Equipment Dock(Depths of Silence), mode=2, price=168
[1370497 5  State   Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Cmd=Sell, Dest=Boron Equipment Dock(Depths of Silence), Ware=Mosquito Missile, Price=168, Environment=Depths of Silence
[1370497 6  Action  Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Cmd.Sell.Ware: Dest=Boron Equipment Dock(Depths of Silence), Ware=Mosquito Missile, Price=168, Amount=5
[1370497 5  State   Depths of Silence      /MISSILES/ ($OK01) =OK= Baldric                     ] Cmd=Sell, Dest=Boron Equipment Dock(Depths of Silence), Ware=Mosquito Missile, Price=168, Environment=Depths of Silence
This trader is going to sell 5(!) mosquitos. And I've also seen trips for only 1 mosquito. It doesn't even pays the Energy Cells used!
I don't know if it's intended, but I don't think it should.

User avatar
dizzy
Posts: 1019
Joined: Sun, 26. Sep 10, 06:00
x4

Post by dizzy » Wed, 3. Dec 14, 01:52

Try to stop all the homebased traders and see if the lags remains. I too noticed lag but I had a much larger number of traders (70+ free, 20+ homebased) so it may have been that. Plus I was playing with Xtended Mod which even with OK Traders uninstalled seems slower than XRM/vanilla.

However, when I was doing some tests last night (debugging an Xtended Mod bug) and stopped all my traders dead in their tracks, that definitely suddenly removed most of the lag. So it was definitely trader related but I haven't followed yet to see exactly what (if it's free roaming vs homebased, if it's about their number and how much is too much) since like I said I was looking into another bug. But while looking into that bug I did restart about 20 free OK traders while I didn't touch my homebased ones and the game didn't feel laggy.

Unfortunately, if it turns out that homebased traders are buggy that would be pretty bad because my main use case for OK Traders is a CAG replacement compatible with Xtended (as vanilla CAG isn't and requires fixes).

albazeus
Posts: 76
Joined: Fri, 21. Feb 14, 17:21
x3ap

Post by albazeus » Wed, 3. Dec 14, 02:02

I will do some tests but I think you're right.
I noticed that the more products the homebase has, the more lag occurs.

Shimrod
Posts: 907
Joined: Tue, 18. Feb 03, 01:43
x4

Post by Shimrod » Wed, 3. Dec 14, 02:39

This trader is going to sell 5(!) mosquitos. And I've also seen trips for only 1 mosquito
The 'Min Stock Level Trade %' global setting is intended to avoid this behaviour, by requiring a minimum percentage stock level before a trade can trigger. The default is 10%. Check this hasn't been set to zero. If traders are ignoring this setting in 1.7.0 it's possible the 1.7.0 'Stock Limits' feature has caused a regression in this area.

To get insight into it via logging:
- Set the trader as OK's logging target but don't enable logging just yet.
- Order the trader to dock at homebase, thus cancelling the OK trade command, and remove tradeable wares from the inventory.
- Enable level 10 logging and start the OK trade command on the trader
- As soon as the ship picks a mission and starts moving, switch off logging and examine or PM me the log file. Of course, the log will only be interesting if the mission it selected reproduces the problem.

This will capture the most verbose logging of the traders decision making, which should hopefully suggest why it has loaded a small amount of mosquitoes.

Another thing to check is whether the equipping menu setting for missiles is enabled and somehow interfering with trading missiles from a homebase.
I recently upgraded to version 1.7 and I am experiencing lot of lags. What should I do to debug it?
Look for traders which are not moving and may be stuck in a loop. Perhaps their action text toggles between one thing and another while sitting idle. If suspicious traders exist then try enabling logging on them and see what they're up to. Stick to low logging verbosity first (5,6,7) and only crank it up if necessary, as its much harder to see the wood for the trees with high verbosity.

Also note that enabling logging in itself causes a lot of lag, so be sure that logging is disabled.

albazeus
Posts: 76
Joined: Fri, 21. Feb 14, 17:21
x3ap

Post by albazeus » Wed, 3. Dec 14, 12:38

Shimrod wrote:The 'Min Stock Level Trade %' global setting is intended to avoid this behaviour, by requiring a minimum percentage stock level before a trade can trigger. The default is 10%. Check this hasn't been set to zero. If traders are ignoring this setting in 1.7.0 it's possible the 1.7.0 'Stock Limits' feature has caused a regression in this area.
Omg, I totally misunderstood the meaning of this option! I thought it was meant to prevent the homebase to run out of products. Something like: 'don't sell this ware if the stock level is < than this %'.
So the right interpretation should be: 'sell this ware if you can sell at least % stock level'.
And now that I think about it, there's another option that works together with complex ware limits, if I remember correctly.

Ok, I think you found the problem. I'll check it tonight.

Shimrod
Posts: 907
Joined: Tue, 18. Feb 03, 01:43
x4

Post by Shimrod » Wed, 3. Dec 14, 13:48

There are some docs in the 3rd post describing a few of the features. The relevant part:

Stock Limits

The homebased trader menu contains a Stock Limits section. This allows a minimum and maximum threshold to be set for each of the homebase wares. These settings apply to the homebase rather than per-trader so only need to be set once.

Min Stock % is used to set a floor amount traders won't sell into This is useful for trading intermediate wares. The default of 0% means traders will sell everything.

Max Stock % is used to set a ceiling where traders won't buy stock above that level. This works in the same way as a dockware manager limit except it is percentage based. The default of 100% means traders will buy the ware to full stock level.

The menu also shows any dockware manager limits in a read only fashion. If a dockware limit is set the minimum absolute stock level between the OK setting and dockware setting will apply.

The global settings page also contains a Min Stock Level Trade % setting. This is used to create a buffer which must be exceeded before a homebased trader will trade a ware. The purpose of the setting is to avoid traders performing small trades, such as immediately re-buying the jumpdrive energy they refuelled with. This setting only applies to triggering a trade run; the trader will still use the full range of the stock level, or as limited by Min and Max Stock Level % settings described earlier.

albazeus
Posts: 76
Joined: Fri, 21. Feb 14, 17:21
x3ap

Post by albazeus » Wed, 3. Dec 14, 16:34

Hi Shimrod,

sorry, my bad. I didn't read that part.
I think it should be possible to set something less (1%, 2%, etc..).
Why did you choose the stock level of a ware to determine if a trade should be made? Shouldn't be more logic to set how profitable the trade may be, setting, for example, >1000 credits?

Anyway, that wasn't the problem. It was set to default value (10%). I'll send you the log.

About the lags: after some testing, I am pretty sure the problem are the free traders, not the homebased. I'll send you this log too.
After stopping all the traders (free/homebased), if I start a free trader the game lags (with or without debugging on). And lags every time the trader calculate the route (only when buying, not selling).
If I start the same ship as a homebased trader, everything is fine (no lags).

Logs are here

Shimrod
Posts: 907
Joined: Tue, 18. Feb 03, 01:43
x4

Post by Shimrod » Wed, 3. Dec 14, 18:16

Regarding the mosquito problem, I saw 2 problems in the log.

The first is a bug causing homebased traders to search for stations using 'best chance' instead of 'best price', introduced in recent jump fuel optimization changes, so the best price sale may not get selected within the full homebase jump range. Associated trace:

Code: Select all

Find.Station: Ware=Mosquito Missile, Mission=2, Origin=Unknown Sector 8,16, Max.Jumps=50, Find.Mode=1

where

$Find.Mode.Best.Price = 2 
$Find.Mode.Best.Chance = 1 
$Find.Mode.Nearest = 0
But the core problem is that mosquitoes are the highest priority to sell (99% stock) yet the 'best' place to sell them only has free space for 2 more. The trade goes ahead regardless. Relevant traces:

Code: Select all

-- Mosquitoes are highest priorty to trade as stock level is 99%
Get.Ware.Priority(Sell Mosquito Missile): Stock=16591, Limit.Min=0, Max.Actual=16660, Priority=99

-- The top place to sell the ware (based on the unfavourable 'best chance' search) is Boron Equipment Dock(Great Reef), for the average price 168
Find.Station: Ware=Mosquito Missile, Mission=2, Origin=Unknown Sector 8,16, Max.Jumps=50, Find.Mode=1
              Flags=0, Limit=null, Excludes=ARRAY ( Void of Opportunity, Vestibule of Creation, Veil of Delusion, Uranus 3, Uranus 2, Uranus, Unknown Sector 8,5, Unknown Sector 8,4, ... ), Price=168
Find.Station: exit rc=Boron Equipment Dock(Great Reef), tries.remain=147
Cmd=Sell, Dest=Boron Equipment Dock(Great Reef), Ware=Mosquito Missile, Price=168, Environment=/Missiles/ Complex Hub(Unknown Sector 8,16)

-- The equipment dock doesn't buy infinite amounts of wares and only has free space for 2 mosquitoes
Test.Buys.Infinite.Wares: Station=Boron Equipment Dock(Great Reef), Ware=Mosquito Missile, Result=0
Lib.Config.Get.Stock.Limit.Max: Ware=Mosquito Missile, Station=Boron Equipment Dock(Great Reef), Max.OK=null, Max.Dockware=null, rc=3332
Lib.Config.Get.Stock.Limit.Free: Ware=Mosquito Missile, Station=Boron Equipment Dock(Great Reef), Have=3330, rc=2

-- Trading begins
Cmd.Sell.Ware: loading 2 units of Mosquito Missile
The first issue could be resolved by making the trader always select 'best price' if it has a jumpdrive, without checking its jumpfuel.

The second is more interesting as this is really a systematic problem with selling to docks, where the price doesn't decrease relative to stock level. And I suspsect the 'best price' script commands won't take into account stock level in docks. Some ideas to increase intelligence here:

1. If the homebased or free trader station search encounters a dock, for any buy or sell mission, continue to search until we find all stations trading at the same price. Then sort them by stock level and select the best one based on stock level.
2. Somehow factor in availability into homebase [buy] ware priority, and market saturaton into the [sell] ware priority, in a way that fairly balances the need to buy against selling.

The reason stock level of ware is used for priority is simplicity, and balance between buying and selling. We may be able to introduce more intelligence by ordering sold wares buy potential profit as you suggest, but we need to somehow still balance buying against selling and stock level is a good equalizer.

Shimrod
Posts: 907
Joined: Tue, 18. Feb 03, 01:43
x4

Post by Shimrod » Wed, 3. Dec 14, 21:35

Albazeus - is 'Argon Equipment Dock(Legend's Home)' a player owned station in your game?

albazeus
Posts: 76
Joined: Fri, 21. Feb 14, 17:21
x3ap

Post by albazeus » Wed, 3. Dec 14, 21:58

Shimrod wrote:Albazeus - is 'Argon Equipment Dock(Legend's Home)' a player owned station in your game?
yes

User avatar
dizzy
Posts: 1019
Joined: Sun, 26. Sep 10, 06:00
x4

Post by dizzy » Wed, 3. Dec 14, 22:34

In my game (where I was experiencing lags) I also have personal equipment docks (POED, xtended mod) plus a personal shipyard (POSY) and a complex hub. The POED had one homebased trader with a filter to collect military shields of any type from anywhere in the universe at any price while the complex hub had 10 homebased traders to sell 2 products and one intermediary product. Most of them are idling landed in "OK Trade" at some point, I guess they were too many for the needs of that complex.

Shimrod
Posts: 907
Joined: Tue, 18. Feb 03, 01:43
x4

Post by Shimrod » Wed, 3. Dec 14, 23:05

Albazeus' free trader log shows a bug in station ware limit config introduced in 1.7.0 which looks like it may be triggered when the player removes a ware entry from a dock. Lib.Config.Get.Station.Config attempts to remove ware config no longer present, but the arrays (wares, minimums, maximums) end up getting out of sync, and will continue increasing in size periodically.

This may be the source of the slowdown as the latter 2 arrays have increased to 269k elements due to the bug, and these get sorted.

Code: Select all

rc=ARRAY ( 1007000, ARRAY(73), ARRAY(269492), ARRAY(269492), null )
I'm planning a 1.7.1 with a fix for this plus the homebased trader 'best chance' station finding issue, hopefully this evening. The fix will delete and recreate corrupt config so will need to reconfigure any limits.

albazeus
Posts: 76
Joined: Fri, 21. Feb 14, 17:21
x3ap

Post by albazeus » Wed, 3. Dec 14, 23:33

Shimrod thanks for your effort.
For what it's worth, if I set that equipment dock as homebase for a trader, and start that trader, the game lags. Just to confirm what you said.

User avatar
dizzy
Posts: 1019
Joined: Sun, 26. Sep 10, 06:00
x4

Post by dizzy » Wed, 3. Dec 14, 23:42

Yes, I have used Dockware Manager to periodically clean up the wares on my POED while the homebased OK Trader was servicing it. It would be great if this was the bug and now things shouldn't be laggy.

Also, since we are on this subject, I noticed that OK Traders assigned to my complex will stop what they are doing after I add new factories to the hub. Even if when adding new factories I make no change whatsoever to the configuration of products/resources/intermediary products (ie I add factories of a type that I already have in the complex). The homebased OK traders just end their current command and then stop doing anything and I always have to remember to restart them. I'd appreciate at least a message in the log otherwise I'm losing profitssss :)

Shimrod
Posts: 907
Joined: Tue, 18. Feb 03, 01:43
x4

Post by Shimrod » Thu, 4. Dec 14, 02:04

I've uploaded 1.7.1.

1.7.1
- Fix a bug which could lead to station ware limit settings corruption, causing traders to gradually use more CPU. If corruption is detected in 1.7.1 the station ware limit config will be reset to default.
- Fix a bug where homebased traders with jumpdrives would perform a 'best chance' station search instead of 'best price'.
- When evaluating docks to buy or sell at, examine other elligible docks trading at the same price to determine a more favourable stock level.
I noticed that OK Traders assigned to my complex will stop what they are doing after I add new factories to the hub
When I last investigated this, the game itself was terminating the trader's command script, rather than OK trade actually stopping. Script debugger shows no lines of script executing after modifying the complex. IIRC happens for standard buy/sell traders too. So there doesn't seem to be much I could do about it, outside of introducing a controlling intelligence say to the homebase itself to monitor and take ownership if the command stops. The broadcast option should hopefully help a bit with bringing them back up though.

User avatar
dizzy
Posts: 1019
Joined: Sun, 26. Sep 10, 06:00
x4

Post by dizzy » Thu, 4. Dec 14, 03:10

Yes, it's very easy to bring them back on once I realize they are stopped again. But sometimes it takes a long time for me to realize that o.O

Sad to hear it's the game doing something this dumb. Having an external "babysitter" task restarting OK Traders would be tricky because you wouldn't know if the trader was stopped because of this broken engine logic or because the user just stopped it through other, more explicit means. I guess you could try to detect this by caching the configuration of connected factories and every time the babysitter task detects that a homebased to a hub trader has been stopped, it can check to see if the configuration is different from the last cache if so it's likely stopped because the engine did so, however, it would mistakenly restart if the user stops a homebased OK trader and immediately changes the hub configuration so the babysitter task misses the explicit stop from the user.

Thanks for the fixes, I'll try them out! Do you think that this bug could be related to some of the random missions in Xtended being offered by people who's name shows as "ReadText30-... ReadText34-..."[1]? (that's the bug I'm getting in Xtended and was investigating at the same time)

Also, do we need to stop and restart our OK Traders for the new logic to kick in?

[1] http://postimg.org/image/szo77sj8p/

Shimrod
Posts: 907
Joined: Tue, 18. Feb 03, 01:43
x4

Post by Shimrod » Thu, 4. Dec 14, 11:56

The scripts are designed to restart themselves after the setup script advertises the new version number through the global variable.

It will certainly be useful to hear if the lag is fixed after applying the patch, ensuring logging is switched off while observing.

One way to see if it corrects the problem is, in the context of albazeus' free trader log,
- Before applying 1.7.1, enable level 8 logging on the free trader from which log was previously produced (($OK01) Mercury Hauler), and save the game.
- Copy in 1.7.1 and reload, check the log for the following traces from Lib.Config.Get.Station.Config:

Code: Select all

Lib.Config.Get.Station.Config: Clearing corrupt station ware config for %s
Check there are no traces about removing a null ware:

Code: Select all

Lib.Config.Get.Station.Config: Removed entry for ware null station Argon Equipment Dock(Legend's Home)
Finally be sure to disable the logging again before saving.
Do you think that this bug could be related to some of the random missions in Xtended being offered by people who's name shows as "ReadText30-... ReadText34-..."[1]? (that's the bug I'm getting in Xtended and was investigating at the same time)
According to zazie who encountered the same readtext symptom, he had no scripts or mods installed outside of XTC, so OK trade could not have been the root cause in that instance.

Note that the OK bug was introduced in 1.7.0 released on 22-08-2014.

albazeus
Posts: 76
Joined: Fri, 21. Feb 14, 17:21
x3ap

Post by albazeus » Thu, 4. Dec 14, 12:35

Shimrod,

thanks! The new version works perfectly. No lag so far! No more trip for 1 mosquito, only lots of profittttssss! :D

There's something I don't understand though: how does a trader manage the missiles he uses for defense?
I mean, I equipped every TS with 20 mosquitos and configured the 'Automated Missiles Resupply' accordingly.
Now every OK trader I checked has sold/lost the mosquitos, both the free traders and the homebased.
Let's take for example a trader homebased to a missiles complex (that produces and sells mosquitos):
- if the trader has the mosquitos in the cargo bay, he will drop them at the complex
- if the trader has no mosquito, he will not refuel at the complex (I think he refuels and then drops them)
This happens when 'Missiles' in 'on' global settings. When off, he just sells the mosquitos.
:?
Can look at this? Do you need logs?

Post Reply

Return to “X³: Terran Conflict / Albion Prelude - Scripts and Modding”