[SCR] [07.02.11] Trade Overview 2.05 by ScRaT

ScRaT_GER · Post by **ScRaT_GER** » Sat, 29. Jan 11, 00:01

Hi Excelsior,

thanks for the reports. I just looked into the whole script again and found some other stuff.

The main problem atm is summarizing the trades correctly in the logbook. The overall data is always correct, as far as I can see, only the display in the logbook isn't.

The biggest flaw is a trading case, which I simply forgot to consider:
Let's say I'm buying 200 E-Cells from one station, then 100 E-Cells from another. This is all shown correctly in the logbook.
Then I go ahead and sell 200 E-Cells to a station - and bam! - this is the point where the logbook gets confused. Because to merge flights (so one buy- and one sellrun into a single trade) it will look for the latest trade. If you sold less in the current sellrun than you bought in the latest buyrun, a dummy entry is created with the left-over cargo (this is what is done in the code you posted).
However if you sold more in the current sellrun than you bought in the latest buyrun this is not caught, so selling your 200 E-Cells will be related to buying 100 E-Cells, which is obiously wrong.
Unfortunately there are still the 200 E-Cells from the first buyrun floating around as an uncompleted trade. So when you now sell the remaining 100 E-Cells the logbook will again go ahead and search for an uncompleted flight. It find the 200 E-Cells, notices that this is more than we sold now (100 E-Cells), and creates a dummy entry. Thus you end up with a completely screwed up logbook...

So, this is really a problem I HAVE to solve before I upload a next version.

I'm also trying to take into account ship upgrades in order to know exactly how much profit my MK3s does.

This is currently filtered out on purpose. Should it be configurable?

BTW, I translated your t files in french language.
Ask if you're interested.

Yes, of course I am. Just upload them somewhere and post it here or send me a PM, so that I can add it to the main post (and of course into the download when the next version is up).

Concerning the next version:
I will not make any promises, but I hope that I can get to work on this project again in the next weeks. Please excuse the long period of inactivity with this script.

Greetings,
ScRaT

Excelsior82 · Post by **Excelsior82** » Sat, 29. Jan 11, 01:31

ScRaT_GER wrote:
I'm also trying to take into account ship upgrades in order to know exactly how much profit my MK3s does.
This is currently filtered out on purpose. Should it be configurable?

If you're not against that, I think it would be useful, or this could be an add-on to the script (like a mod for the script). ATM I use Trade Overview to review my MK3 traders and this is why I wanted to know how much they cost in upgrades terms and in salary (thus I had to change some egosoft scripts).

I think the problem shall be simplified. Let's consider all options :

1) You buy a certain amount and sell it elsewhere : Simple case very well supported now
2) You buy a certain amount and sell less : When the code will be updated, it'll be good also.
3) You buy and collected a certain amount of ware and, so you sell more than what were bought : That is the FLAW.

(I'm gonna try to explain a solution, but english is definitely not my best language to explain clearly what I have in mind, so let's have an example. I bought 100 E-Cell and found 100 others in space. So I sold 200 E-Cell)
A solution would be to find the last buy of that ware (100) and manage the trade up to the bought amount (100).
We would have one line stating we exchanged 100 E-Cell.

Now we still have 100 E-Cell to process and I think of 2 possibilities :

Overkill possibility (possibly greedy in processor time) : We continue to go up in the logbook until we have spent the 100 spare E-Cell from previous buys. (Example : 3 hours ago, I have bought 250 E-Cell, transforming that buy line in a 150 E-Cell buy and a 100 E-Cell trade). Even if that is not a trade run "stricto sensu", it's like we trade them. If we don't find enough lines of bought E-Cell, we process the remaining E-Cells like a sell, not a trade.

Lazy solution : We process the 100 spare E-Cells as an independent sell. We won't have beautiful stats on how much profit we made on that trade run but we don't make a heavy process for that

I think it's a case of conscience, either we choose to be overkill and process at full scale, or we consider these cases too scarce to make big process.

IMHO, the lazy process would be good enough as this is an informational and non-critical script.

(I hope that was clear enough. BTW, I know I used extensively the "we", that's just to make me understand easily, otherwise I would construct my sentences like a 6 years old kid.)

I'm gonna send you a MP with an url for the translation files (I will check them another time before).

Oh, and before I forget : I've noticed you were still counting stazuras as 96*96 seconds, but it seems that TC has stazuras that are an hour length. (Correct me if I'm wrong).

ScRaT_GER · Post by **ScRaT_GER** » Sat, 29. Jan 11, 09:39

Oh, and before I forget : I've noticed you were still counting stazuras as 96*96 seconds, but it seems that TC has stazuras that are an hour length. (Correct me if I'm wrong).

Yes, you're right. I took that from the manual, but it is actually not really in the game.

I'm going to change that.

3) You buy and collected a certain amount of ware and, so you sell more than what were bought : That is the FLAW.

Yes, exactly, although I don't think that selling a collected ware should be marked as a trade in the logbook, since it isn't. It's just a sell.

However if I buy the wares in two steps and then sell more than I bought the second time, the logbook will be faulty (as I explained), and I think this is the case to really solve. Because here I really want the buy- and sellruns to be merged into a trade, since this was a real trade (unlike collecting a ware and selling it).
But I think reorganizing the code a bit and using some recursion could solve the problem elegantly.

If you're not against that, I think it would be useful

No, I'm absolutely not against that. It's actually quite logical to include that, maybe even with a whitelist and blacklist for wares.

Greetings,
ScRaT

Excelsior82 · Post by **Excelsior82** » Sat, 29. Jan 11, 15:03

ScRaT_GER wrote: Yes, exactly, although I don't think that selling a collected ware should be marked as a trade in the logbook, since it isn't. It's just a sell.

However if I buy the wares in two steps and then sell more than I bought the second time, the logbook will be faulty

You're right, but the script has no means to know if the trade you're selling has be bought before or collected. It can only look if it has been bought some time ago, that's why I took this example. For the case where you'd have bought the ware from 2 different stations, it's indeed more important to merge them into a single trade operation.

So the only solution is the overkill one : Regress the full logbook until there's no more buys of the ware, and if there's still some wares that have been sold without a buy, then treat it like an independent sell.

ScRaT_GER · Post by **ScRaT_GER** » Sat, 29. Jan 11, 17:02

So the only solution is the overkill one : Regress the full logbook until there's no more buys of the ware, and if there's still some wares that have been sold without a buy, then treat it like an independent sell.

Yes, exactly.

Thanks for the french version, btw..
Edit: Added it to the start post.

Greetings,
ScRaT

Excelsior82 · Post by **Excelsior82** » Sat, 29. Jan 11, 23:34

While I'm in your script's code, I found a new problem for profits per stazura calculation.

It's not a bug, but caused by the way behave X3 (if you know C or Pascal code, you'll catch it easily)

ATM, the profit/staz is calculated with the formula

Code: Select all

$profit.staz = $profit / $time * 60 * 60

How does X3 to compute the number ? Like many compilers/interpreters : We only have multiplications and divisions, which have the same priority so it counts from left to right (i'll use a fake $result variable to simulate how a processor store its data) :
$result = $profit / $time
$result = $result * 60
$profit.staz = $result * 60

The problem is that if $profit / $time < 1, $result = 0, cause both $profit and $time are an integer (I don't know if there are real numbers in X3).
And so on, $profit.staz will be 0.

EDIT : In the same way, your result will always be rounded (up or down, not sure) to an integer, making your profit.staz accurateness to a multiple of 3600.

To counter that imprecision, you should calculate profits per stazuras that way :

Code: Select all

$profit.staz = $profit * 60 * 60 / $time

Now, profits per staz will be more accurate

BTW : I'm working also on taking account of previous buys. I'll let you know when I'll have fully functional script and I'll submit it to you cause as I don't know exactly how things works in the first time, I try to avoid changing existing variables

ScRaT_GER · Post by **ScRaT_GER** » Sat, 29. Jan 11, 23:56

While I'm in your script's code, I found a new problem for profits per stazura calculation.

Good catch! Nowadays I'm well aware of the flaws of the X3 scripting engine (actually MSCI scripting brought me to programming languages like C++, Python, Haskell, etc.), but I guess this code dates back to one of the earliest versions of the script.

I guess if I had to rewrite the script from scratch, I'd do many things differently.
However I'm not unsatisfied by the structure of the code. Even though I hadn't looked into it for month I got along quite fast. Still, restructuring and refactoring it might be a good idea.

BTW : I'm working also on taking account of previous buys. I'll let you know when I'll have fully functional script and I'll submit it to you cause as I don't know exactly how things works in the first time, I try to avoid changing existing variables

Ah, good to know. Further additions to the project are of course welcome.

In the same way, your result will always be rounded (up or down, not sure)

Down. Integer division simply cuts the decimal place, so 5/10 = 0.5 => 0.

Greetings,
ScRaT

Excelsior82 · Post by **Excelsior82** » Sun, 30. Jan 11, 14:40

I'm also think the structure itself (I only looked at the Logbook sub) is not that bad.

EDIT : You can see some screens I made there http://excelsior89.free.fr/X3TC/TROV/
On these screen, here's what I 've done :
Bought 1 MJ Shield with the disco, then sold 2
Bought 2 * 30 Cahoona meatsteak with the mercury and sold 35 to another station.
EDIT 2 : Yeah, sorry, screens are with my french t file, but If you look for amounts, prices and time where movements where made, you should get it

At the moment I have something that seems to work, but I'm not sure that's 100% sure cause there's still some things I didn't catch.

Eg : I don't understand what are the purpose of the following arrays
$trov.incomplete
$trov.incomplete.ref.1
$trov.incomplete.ref.2
$trov.incomplete.dummy

I've understood they are used to remember which trade is not completed, but not how it's done.

I think it's important because here's what's done when we look for an incomplete trade :

Code: Select all

$dummy = get index of $ware in array $trov.incomplete offset = -1 + 1
if not $dummy == -1
  * We found an incomplete trade
  $index = $trov.incomplete.ref.1[$dummy]
  remove element from array $trov.logbook at index $index
  remove element from array $trov.incomplete at index $dummy
  remove element from array $trov.incomplete.ref.1 at index $dummy
  
  $index = $trov.incomplete.ref.2[$index]
  
  remove element from array $trov.incomplete.ref.2 at index $index

It appeared to me that I had to comment out the line "$index = $trov.incomplete.ref.2[$index]" in order to look for olders incomplete trade, but I didn't get why.

Do you remember why things are done that way ?

ScRaT_GER · Post by **ScRaT_GER** » Sun, 30. Jan 11, 15:33

Eg : I don't understand what are the purpose of the following arrays

I also had to wrap my head around these again. Additionally there's a bug in the code you posted. In my version I rearranged the code a bit and changed variable names (and corrected the bug), so that the code get clearer. Here's how I've arranged it:

Code: Select all

330   $dummy = = get index of $ware in array $trov.incomplete offset=-1 + 1
331   
332   * Found an incomplete trade
333   if not $dummy == -1
334   |
335   |* index into logbook
336   |$indexLogbook = $trov.incomplete.ref.1[$dummy]
337   |* Remove from logbook (compund entry is created later)
338   |remove element from array $trov.logbook at index $indexLogbook
339   |
340   |* index into local array trov
341   |$index = $trov.incomplete.ref.2[$dummy]
342   |
343   |* remove incomplete trade from list of incomplete trades
344   |remove element from array $trov.incomplete.ref.2 at index $dummy
345   |remove element from array $trov.incomplete at index $dummy
346   |remove element from array $trov.incomplete.ref.1 at index $dummy

To understand what this code does, I'll first explain the single arrays:

trov.incomplete

holds wares whose trades are not completed

trov.incomplete.ref.1

holds indices into the ships logbook array, the indices are the same as for trov.incomplete. So the ware of an incomplete trade (i.e. when a ships buys a ware from a station) is appended to the array trov.incomplete, and the corresponding logbook index is appended to the array trov.incomplete.ref.1

trov.incomplete.ref.2

holds indices into the local array trov, which simply holds the raw data of each buy and sell. As trov.incomplete.ref.1 the indicies are the same as for trov.incomplete

trov.incomplete.dummy (I ran out of names...)

holds "dummy" incomplete trades. So when you buy a ware and then sell less afterwards, this will create a dummy incomplete trade, which doesn't have a real counterpart in the local array trov. The indices into this array are stored as negative numbers in trov.incomplete.ref.2, which is used to distinguish between real and dummy incomplete trades.

So the code works like this:
If an incomplete trade with a matching ware is found, we first remove this incomplete trade from the objects logbook, then get the index into the local array trov (which holds the raw data) and then remove this incomplete trade entirely. The index into the local array trov is then later used to merge the incomplete trade with the current trade into a complete trade.

Actually this design with all the different arrays is really horrible in my opinion. It's hard to understand, hard to maintain and hard to extend (as we see now).

Maybe we should discuss this stuff in private (via PMs) and not spam the forum.

Atm I can't spend much time working on a better solution, but in a week I'll have some more time.

Greetings,
ScRaT

ScRaT_GER · Post by **ScRaT_GER** » Mon, 7. Feb 11, 23:10

New version finally arrived: Trade Overview v2.05

I'm happy to announce the release of a new version. After more than one year of silence and with the help of Excelsior82 a new version if finally ready.

Actually it only resolves one major problem the old version always had: The logbook didn't merge trades correctly and calculated wrong numbers. This is now solved.

I know this is a rather "small" bugfix, but Excelsior82 and I already planned to continue to work together on this script. So if you have any ideas, now's the time to tell us.

As always the current version can be downloaded in the start post. It also includes the french version, translated by Excelsior82.

To install the new version, simply proceed as explained in the start post.

So stay tuned for the next release and please report any errors you encounter.

Greetings,
ScRaT

TouchMyNipple · Post by **TouchMyNipple** » Wed, 23. Feb 11, 23:23

Hi! I realy love the script, but the new version returns "null" values for profit, bought and sold wares.
Any ideas why that happens?
i have a mk3 trading system and the TS is in sector trading mode. Maybe that is caused due to lack of TS mk 1-2?

ScRaT_GER · Post by **ScRaT_GER** » Wed, 23. Feb 11, 23:30

Yes, I already silently uploaded a new version (v.2.052). The problem is a missing library script (lib.scrat.number.to.string.safe).

Simply download the new version and install it as always.

Greetings,
ScRaT

paulhamm · Post by **paulhamm** » Thu, 24. Feb 11, 17:40

Nice to see you back ScRaT_GER.

Couple of ideas, may not be practical. The ability to select a ship in the overview and enter it's command menu or view sector would be really great. Useful for ships that have stalled, or having some problem.
The ability to graph over time profitability. Would likely need to be external via a web browser. Very useful for determining how well you have dispersed your traders.

Excelsior82 · Post by **Excelsior82** » Thu, 24. Feb 11, 21:18

paulhamm wrote:The ability to select a ship in the overview and enter it's command menu or view sector would be really great. Useful for ships that have stalled, or having some problem.

You bet it. I've thought about that surely. I'm not sure if it would be easy or very hard to do that, but I've a feeling it would be a pain in the *ss.
However, if you know a script which enable that feature, I'd download it right now to look at it.

paulhamm wrote:The ability to graph over time profitability. Would likely need to be external via a web browser. Very useful for determining how well you have dispersed your traders.

Woah... That is something too. It would mean creating a xml file to save the data of the trades and then an external tool to generate the web page, or something like that. That's interesting however, but I'm not sure it would be very useful (compared to the sum of work).

I don't know what Scrat would think of that, but I'm sure I wouldn't do that part

ScRaT_GER · Post by **ScRaT_GER** » Thu, 24. Feb 11, 22:03

The ability to select a ship in the overview and enter it's command menu or view sector would be really great. Useful for ships that have stalled, or having some problem.

Unfortunately there is no command to simply open the sector map. There are only commands to recieve user input (i.e. a ship from the sector map), but choosing a ship in this command doesn't open the ships control menu.

The ability to graph over time profitability. Would likely need to be external via a web browser. Very useful for determining how well you have dispersed your traders.

I've definitely thought about that, but more like an ingame graph plotter. But the only possibilties I see would be to use ASCII characters or (which I find more interesting) empty sectors, where you place the objects so that they appear as a graph and then open the sector map.

Woah... That is something too. It would mean creating a xml file to save the data of the trades and then an external tool to generate the web page, or something like that. That's interesting however, but I'm not sure it would be very useful (compared to the sum of work).

Yes, I'm not sure either. There is an Egosoft trade script, which can do that though.

Greetings,
ScRaT

HotSake · Post by **HotSake** » Fri, 25. Feb 11, 05:12

ScRaT_GER wrote:
Woah... That is something too. It would mean creating a xml file to save the data of the trades and then an external tool to generate the web page, or something like that. That's interesting however, but I'm not sure it would be very useful (compared to the sum of work).
Yes, I'm not sure either. There is an Egosoft trade script, which can do that though.

Oh? A vanilla script or something from the bonus pack? Do you remember the name? I've been keeping an eye out for anything involving graphical displays, since the restriction to text-only menus is currently my biggest pet peeve with the script/mod framework.

ScRaT_GER · Post by **ScRaT_GER** » Fri, 25. Feb 11, 10:46

I'm talking about the MK3 Trading Log. It's still only text, but viewable in a browser and clearer.

Greetings,
ScRaT

redwards · Post by **redwards** » Fri, 4. Mar 11, 06:12

May have found a bit of a bug here. I had a 1MJ shield factory that I decided to quadruple into a complex. I had the original factory tracked in trade overview, and now that it's part of a complex I'm unable to remove it.

This minor glitch aside, I absolutely love this mod. Thank you.

0o0o0 · Post by **0o0o0** » Wed, 23. Mar 11, 06:40

Great script! I Wanted to update to 2.052 but I'm having trouble with the download link. Anybody else have a problem getting it?

ScRaT_GER · Post by **ScRaT_GER** » Wed, 23. Mar 11, 11:03

Just tried it, works for me.
What exactly is the problem with the link?

Greetings,
ScRaT

egosoft.com

egosoft.com

[SCR] [07.02.11] Trade Overview 2.05 by ScRaT