[SCR] OK Traders v1.7.1 (04-12-14)

Shimrod · Post by **Shimrod** » Fri, 19. Jul 13, 10:01

Logging is output to: C:\Users\<user>\Documents\Egosoft\X3AP\log09055.txt

hourheroyes · Post by **hourheroyes** » Fri, 19. Jul 13, 14:38

Ok, I caught one in the act--It bought a bunch of EBC's and only sold 2.

Here's the buy:

[186378 9 Monitor Free Trader YTSLQ-91 ] ------------------ Monitor ------------------
[186378 5 Monitor Free Trader YTSLQ-91 ] Check.DockingComputer: Using docking computer to dock at Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), range 2129
[186378 5 Cmd Free Trader YTSLQ-91 ] Start: Directive=1, Version=1004001
[186378 8 Cmd Free Trader YTSLQ-91 ] Perform.Mission: type=1, dest=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), ware=Energy Bolt Chaingun, price=131604, cur=Age of Oedipus
[186378 9 Lib Free Trader YTSLQ-91 ] Get.Reserve.Jump.Energy: rc=1130
[186378 9 Lib Free Trader YTSLQ-91 ] Get.Topup.Jump.Energy: rc=120
[186378 7 Lib Free Trader YTSLQ-91 ] Get.Amount.To.Buy: Ware=Energy Bolt Chaingun, Dest=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), Dest2=null, rc=36
[186378 6 Action Free Trader YTSLQ-91 ] Cmd.Buy.Ware: Ware=Energy Bolt Chaingun, Dest=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), Dest2=null, Price=131604, Amount=36
[186378 9 State Free Trader YTSLQ-91 ] Trade.Cache.Remove: Removed ship and cache entries. Mission=1, Station=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), Ware=Energy Bolt Chaingun, Key=1_PSTFD-58_8_5
[186378 9 State Free Trader YTSLQ-91 ] Trade.Cache.Add: Added cache and ship entry. Mission=1, Station=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), Ware=Energy Bolt Chaingun, Key=1_PSTFD-58_8_5
[186378 5 State Free Trader YTSLQ-91 ] Cmd=Buy, Dest=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), Ware=Energy Bolt Chaingun, Price=131604, Cur=Age of Oedipus
[186378 9 Lib Free Trader YTSLQ-91 ] Get.Reserve.Jump.Energy: rc=1130
[186378 9 Lib Free Trader YTSLQ-91 ] Get.Topup.Jump.Energy: rc=120
[186378 9 Lib Free Trader YTSLQ-91 ] Get.Reserve.Jump.Energy: rc=1130
[186378 9 Lib Free Trader YTSLQ-91 ] Get.Topup.Jump.Energy: rc=120
[186378 7 Lib Free Trader YTSLQ-91 ] Get.Amount.To.Buy: Ware=Energy Bolt Chaingun, Dest=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), Dest2=null, rc=36
[186378 5 Action Free Trader YTSLQ-91 ] Buy.Ware: Ware=Energy Bolt Chaingun, Station=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), Wanted=36, Got=36
[186378 6 Action Free Trader YTSLQ-91 ] Cmd.Buy.Ware: Ware=Energy Bolt Chaingun, Dest=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), rc=36
[186378 9 State Free Trader YTSLQ-91 ] Trade.Cache.Remove: Removed ship and cache entries. Mission=1, Station=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), Ware=Energy Bolt Chaingun, Key=1_PSTFD-58_8_5
[186378 8 State Free Trader YTSLQ-91 ] Cmd=None, Cur=Age of Oedipus

This is what the selling part looked like:

[186616 5 Monitor Free Trader YTSLQ-91 ] Check.DockingComputer: Using docking computer to dock at Teladi Space Equipment Dock(Home of Opportunity), range 2184
[186616 5 Cmd Free Trader YTSLQ-91 ] Start: Directive=1, Version=1004001
[186616 8 Cmd Free Trader YTSLQ-91 ] Perform.Mission: type=2, dest=Teladi Space Equipment Dock(Home of Opportunity), ware=Energy Bolt Chaingun, price=151144, cur=Home of Opportunity
[186616 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Energy Bolt Chaingun, rc=0
[186616 9 Lib Free Trader YTSLQ-91 ] Test.Buys.Infinite.Wares: Station=Teladi Space Equipment Dock(Home of Opportunity), Ware=Energy Bolt Chaingun, Result=0
[186616 6 Action Free Trader YTSLQ-91 ] Cmd.Sell.Ware: Ware=Energy Bolt Chaingun, Dest=Teladi Space Equipment Dock(Home of Opportunity), Price=151144, Amount=36
[186616 9 State Free Trader YTSLQ-91 ] Trade.Cache.Remove: Removed ship and cache entries. Mission=2, Station=Teladi Space Equipment Dock(Home of Opportunity), Ware=Energy Bolt Chaingun, Key=2_TDOJG-44_8_5
[186616 9 State Free Trader YTSLQ-91 ] Trade.Cache.Add: Added cache and ship entry. Mission=2, Station=Teladi Space Equipment Dock(Home of Opportunity), Ware=Energy Bolt Chaingun, Key=2_TDOJG-44_8_5
[186616 5 State Free Trader YTSLQ-91 ] Cmd=Sell, Dest=Teladi Space Equipment Dock(Home of Opportunity), Ware=Energy Bolt Chaingun, Price=151144, Cur=Home of Opportunity
[186616 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Energy Bolt Chaingun, rc=0
[186616 5 Action Free Trader YTSLQ-91 ] Sell.Ware: Ware=Energy Bolt Chaingun, Station=Teladi Space Equipment Dock(Home of Opportunity), Wanted=2, Sold=2
[186616 9 Lib Free Trader YTSLQ-91 ] Get.Reserve.Jump.Energy: rc=1130
[186616 9 Lib Free Trader YTSLQ-91 ] Get.Topup.Jump.Energy: rc=270
[186616 6 Action Free Trader YTSLQ-91 ] Cmd.Sell.Ware: Ware=Energy Bolt Chaingun, Dest=Teladi Space Equipment Dock(Home of Opportunity), rc=2
[186616 9 State Free Trader YTSLQ-91 ] Trade.Cache.Remove: Removed ship and cache entries. Mission=2, Station=Teladi Space Equipment Dock(Home of Opportunity), Ware=Energy Bolt Chaingun, Key=2_TDOJG-44_8_5
[186616 8 State Free Trader YTSLQ-91 ] Cmd=None, Cur=Home of Opportunity

and here is it going through its checklist of things to sell.

[186689 9 Cmd Free Trader YTSLQ-91 ] -------------------- Cmd --------------------
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserve.Jump.Energy: rc=1130
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Topup.Jump.Energy: rc=420
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Cyclone Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Thunderbolt Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Energy Bolt Chaingun, rc=0
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Rapier Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Blizzard Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Mosquito Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Rudder Optimisation, rc=10
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Engine Tuning, rc=10
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Advanced Scanner, rc=1
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Military Scanner, rc=1
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Rapier Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Blizzard Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Cyclone Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Thunderbolt Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Mosquito Missile, rc=3
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Particle Accelerator Cannon, rc=4
[186689 7 Monitor Free Trader YTSLQ-91 ] Check.Should.Sleep: docked
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Mass Driver, rc=8
[186689 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Phased Repeater Gun, rc=4
[186691 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Energy Bolt Chaingun, rc=0
[186691 9 Lib Free Trader YTSLQ-91 ] Find.Station: exit rc=null, tries.remain=149
[186691 9 Lib Free Trader YTSLQ-91 ] Get.Best.Ware.To.Sell: Nowhere to sell ware Energy Bolt Chaingun
[186692 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Ion Disruptor, rc=4
[186692 9 Lib Free Trader YTSLQ-91 ] Find.Station: exit rc=null, tries.remain=149
[186692 9 Lib Free Trader YTSLQ-91 ] Get.Best.Ware.To.Sell: Nowhere to sell ware Ion Disruptor
[186692 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Concussion Impulse Generator, rc=4
[186693 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Ion Pulse Generator, rc=4
[186693 9 Lib Free Trader YTSLQ-91 ] Find.Station: exit rc=null, tries.remain=149
[186693 9 Lib Free Trader YTSLQ-91 ] Get.Best.Ware.To.Sell: Nowhere to sell ware Ion Pulse Generator
[186694 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Fusion Bomb Launcher, rc=0
[186694 9 Lib Free Trader YTSLQ-91 ] Can.Trade.At Headquarters(Xi): cannot sell at player stations
[186694 9 Lib Free Trader YTSLQ-91 ] Find.Station: exit rc=null, tries.remain=148
[186694 9 Lib Free Trader YTSLQ-91 ] Get.Best.Ware.To.Sell: Nowhere to sell ware Fusion Bomb Launcher
[186695 9 Lib Free Trader YTSLQ-91 ] Get.Reserved.Amount: Ware=Cluster Flak Array, rc=4

Is this script supposed to be able to exploit the infinite selling thing for equipment docks? It looks like it's buying way more guns than it can sell but doesn't realize that until after they're bought=

Shimrod · Post by **Shimrod** » Fri, 19. Jul 13, 15:35

That's good data, thanks. The traces show the station being sold to won't buy an infinite amount.

Code: Select all

[186616 9 Lib Free Trader YTSLQ-91 ] Test.Buys.Infinite.Wares: Station=Teladi Space Equipment Dock(Home of Opportunity), Ware=Energy Bolt Chaingun, Result=0

In fact we can see it only consumed 2

Code: Select all

[186616 5 Action Free Trader YTSLQ-91 ] Sell.Ware: Ware=Energy Bolt Chaingun, Station=Teladi Space Equipment Dock(Home of Opportunity), Wanted=2, Sold=2

Yet Get.Amount.To.Buy reported 36, so this amount was bought.

Code: Select all

[186378 7 Lib Free Trader YTSLQ-91 ] Get.Amount.To.Buy: Ware=Energy Bolt Chaingun, Dest=Holy Dimensions Weapons Dealer L Alpha(Age of Oedipus), Dest2=null, rc=36

The key thing about the last entry is that the Dest2 parameter, which is used to hint to the routine where the ware would be sold on to, is null. So the amount that Dest2 could consume is not taken into account.

Code: Select all

if $Dest2
* Adjust for how much a designated consumer station can consume
skip if [THIS]-> call script 'glen.trade.ok.lib' : Func=$Lib.Test.Buys.Infinite.Wares Arg1=[THIS] Arg2=$Ware Arg3=$Dest2 Arg4=null Arg5=null Arg6=null
$p3 = $Dest2-> get free amount of ware $Ware in cargo bay
end

So this certainly looks like a bug, and most likely a regression introduced by the 1.4.0 design changes.
... yep, I think the docking computer's mechanism of restarting the command script has cleared the Dest2 variable, which isn't persisted in the ship state like other mission variables. I'll should be able to fix this readily enough.

Is this script supposed to be able to exploit the infinite selling thing for equipment docks?

Yes, however infinite buyer detection is based on how it works in XTC, which is the environment under which the script was initially written. In XTC if the dock buys an infinite amount of a ware, then it's stock will never change from 50% (though in some cases the stock doesn't begin the game at the 50% level).

A script can detect this by spawning a docked ship of same race as the station, of same class as the trader, adding 1 unit of ware to the ship, selling it, checking if the stock level has changed, removing the sold ware if it did change, then destroying the docked ship. In XTC, docks will typically refuse to buy wares that are not listed, so you cannot sell infinite amounts of anything not listed like can be done in vanilla.

I have not yet encountered a dock in vanilla which consumes infinite amounts of wares listed in the dock's tradeable ware list. IIRC instead you can simply sell infinite amounts of shields, lasers and missiles which aren't in their ware list.

Unfortunately if a ware isn't listed on a station it's difficult for a trade script to know that it can actually sell it there.

I can potentially extend the infinite ware selling to vanilla also, by searching through docks and performing similar tests to the XTC detection. Though everything depends on whether 'sell ware' returns a nonzero value in this case. This may simply be a hardcoded feature that only players interactively selling wares can exploit.

Or altertnatively the script could simply make an assumption that, if not XTC, that every EQ dock can consume infinite amounts of shields, lasers and missiles, that aren't in their ware lists. Or perhaps at game load time the script can search for a single argon EQ dock and perform this test, and set a flag.

I might well experiment with this in future, but for the time being, infinite ware selling will probably only work on XTC. I've made a prominent note of this in the info post to avoid future confusion.

hourheroyes · Post by **hourheroyes** » Fri, 19. Jul 13, 18:54

Thanks dude, that's a great writeup. Glad I could actually help quash a bug

Vayde · Post by **Vayde** » Sat, 20. Jul 13, 00:11

Hi Shimrod.

With regards to stalled stations, these don't change rapidly and could probably be polled and then added to an array or look up table. Would using this method help performance?

Shimrod · Post by **Shimrod** » Sat, 20. Jul 13, 01:23

I'm experimenting with this at the moment. In a new XRM gamestart the following finds 1639 stations.

Code: Select all

$Stations = get station array: resource={Energy Cells} include empty=[TRUE]

Determining if they're stalled is a case of:

Code: Select all

$idx.Station = size of array $Stations
while $idx.Station
dec $idx.Station
$Station = $Stations[$idx.Station]

* Test station qualifies for eco
skip if not $Station-> get production status: as percentage=0
continue
skip if not $Station-> is of class [Dock]
continue

Following on from this the code will do various suitability tests - tether range, can trade there, blacklist. If it qualifies it'll gets and sort primary resources by stock level, iterate them skipping those already being traded, and seeking a place to buy that ware at.

I plan to select the first qualifying match, which means this will only partially traverse the 1600 - unless the universe has been fully eco'd.

Caching would undoubtedly see a performance with many eco boost traders, though there's a downside that the full 1600 elements would need iterated to fully populate the cache, making it slower for a single trader, or a small number of traders that visit the cache infrequently.

Coordinating multiple eco traders concerns me also. This is focused on selling a ware but the trader must first travel and buy it. Picking the first in the list and testing only the current trade being performed by a trader, would mean multiple eco traders potentially buying the same ware with intent to sell at the same place. So this suggests the mission coordination system must take into account the secondary destination as well as first. This should help improve things in the general free trading case, but that's not such a significant problem as for eco traders.

Delerion · Post by **Delerion** » Sat, 20. Jul 13, 04:09

Thanks for the best trading script i've used.

I've got the same bug mentioned earlier, my traders buy copious amounts of weapons and only sell a small amount of them making pretty big losses for now, should i downgrade for now or is the fix easy?

I'm using XRM mod in case if that's the cause.

Shimrod · Post by **Shimrod** » Sat, 20. Jul 13, 10:21

The fix for that is coded but untested. I hope to release a fix this weekend, ideally today, but much depends on how testing goes. I'm looking to do something about economy boosting trader performance as well.

alt3rn1ty · Post by **alt3rn1ty** » Sat, 20. Jul 13, 13:13

Shimrod - Did a bit of logging, and zipped it up ..

http://rapidshare.com/files/2683859199/ ... enaries.7z

Hit the blue download button

Free Trader YTMHT-37 is the serial mercenary trader

This is a continuation of the AP game mentioned previously, with 8 homebased and 11 free traders so quite a bit of data in this ( I let them all do a few trades = 37mb log file with default logging settings )

I didnt use seta to ensure nothing could potentially be skipped

Edit: As previously mentioned I am using CMOD4.16 which has been updated for AP, also using the Alpha PPC pack for CMOD ( replaces CIGs ) .. Mentioning this if you see any wares you would not expect in vanilla AP. No other major mods to affect this log, I have your smart turrets / a gate model replacer / low poly roids / DrBullwinkles Detector / grax rock remover - Thats all.

( I use XTC 2.1 in a different game setup, but not this one. In the XTC setup its too early with just one trader to be acquiring any useful feedback )

Shimrod · Post by **Shimrod** » Sat, 20. Jul 13, 14:16

Thanks, I've got a fix in for the marine & mercenary bug. Doing a bunch of testing and log analysis at the moment to identify any odd behaviours.

I've had 10 eco traders running, mix of tethered and not, without significant lag. Going to ramp up to 20.

Shimrod · Post by **Shimrod** » Sat, 20. Jul 13, 14:59

1.4.2 uploaded. Seems stable from this morning's testing, and I don't want to prolong current bug related trouble.

Traders on buy-missions now track their expected sale destination more closely and keep it updated when rerouting trade. Some benefits of this include:
- After buying a ware traders don't need to search for a place to sell it, they'll start off on the selling leg of the mission pretty quickly.
- Competition cache entries are created for both the buy and the sell legs of the mission, to better ensure traders follow distinct missions.
- Less work performed when determining if a buy-mission is still viable, as this can test the expected secondary destination is still able to buy it, avoiding searching unless a new dest2 needs to be found.

Economy boosting traders now have their own algorithm for selecting a trade which is less resource intensive than when they reused the general free trading algorithm.

1.4.2
- Several performance enhancements, notably for economy boosting traders.
- Fix: exclude marines and mercenaries from being traded (always been present)
- Fix: free traders with docking computers would buy more wares than the destination station could buy. (1.4.0 regression)

hourheroyes · Post by **hourheroyes** » Sat, 20. Jul 13, 15:56

This is awesome news to wake up to. thanks for being the man, Shimrod

Vayde · Post by **Vayde** » Sat, 20. Jul 13, 18:55

With the Eco traders, do you still try to maximise profit? I ask because I would forgo profit for stimulating the economy. It's a fair trade off imo.

alt3rn1ty · Post by **alt3rn1ty** » Sat, 20. Jul 13, 22:30

Many thanks Shimrod - First load after installing 1.4.2 Free Trader YTMHT-37 went straight into buying a Tractor beam instead of mercenaries

Will start introducing Eco's one by one now and see how far I get in this old dog of a machine

Shimrod · Post by **Shimrod** » Sat, 20. Jul 13, 23:50

With the Eco traders, do you still try to maximise profit?

No, the new algorithm picks the first stalled station it finds with a ware it can trade, rather than evaluating all stations, sorting potential profit and selecting the highest, as is done for normal traders.

The trades should generally be profitable as:
- The trader considers the stalled station's wares in ascending stock level order, so it should typically be bought at high price.
- The normal free trader algorithm is employed to find a station to buy the ware at, so this will find the lowest price available.
- There's a safeguard in the monitor thread to prevent buying if it would be sold at a loss.

Shimrod · Post by **Shimrod** » Sun, 21. Jul 13, 01:57

Uploaded another patch version. I noticed a trader had strayed into a Xenon sector, and found the refuelling check didn't get a chance to run on the selling leg of a free trader's buy-mission.

Someday 1.4 will be stable, then I'll break it again in 1.5 by adding more features.

1.4.3
- Fix: traders might not refuel before selling a ware outside their jump fuel range.

DrBullwinkle · Post by **DrBullwinkle** » Sun, 21. Jul 13, 02:19

Why have a separate trader type for Eco-trades? Why not have your regular traders look for an eco trade once every 5-10 trades?

Shimrod · Post by **Shimrod** » Sun, 21. Jul 13, 03:03

I can certainly consider a hybrid behaviour for the future.

alt3rn1ty · Post by **alt3rn1ty** » Sun, 21. Jul 13, 03:12

Would the hybrid idea not have a chance of all Traders potentially randomly synchronizing and becoming eco traders all at the same time, depending on the amount and machine capabilities you could potentially have a slideshow situation again.

Maybe it would not be so bad now with the recent optimisations, but I experienced it with little else going on in sector, I cant imagine what it would be like if a bit of a war was going on at the same time all your traders went into eco mode simultaneously with no way of stopping them doing that.

Shimrod · Post by **Shimrod** » Sun, 21. Jul 13, 03:22

The new eco mission selection algorithm can still be used in the hybrid case, just needs to run this instead of normal mission selection when the time comes to make an eco trade. No evidence yet that the new algorithm is as hard on the CPU, and I think there's room yet for optimization since 1.4.2 just has the first cut of it.

Its unlikely that free traders all plan missions at the same time, perhaps only if a patch should require them to reset their mission state and replan, but I wouldn't expect it to happen in the normal course of the game.

The work with a hybrid approach is in tracking this new state to the trader:
- Counter for tracking whether a trade should be an eco trade
- Flag attached to the trader's current mission to indicate that it is an eco mission, so that if a replan occurs this takes stalled stations into account when finding a new destination to sell the ware.

State must persist between command script restarts, which are initiated by the monitor task to change the current action, and when using the docking computer (put into environment) because the change of environment can cause a collision avoid interrupt script to hang in task 0.

I personally prefer dedicated modes as in particular at the start of the game, all I want traders to focus on is profit. However if a hybrid approach is preferred by some other segment then I'm happy to consider multiple options.

egosoft.com

egosoft.com

[SCR] OK Traders v1.7.1 (04-12-14)