What do we know about modding X-Rebirth?

[Observe]

Short answer: Not much.

------------------------------------------------------------------------
General:

TNBT will soon be officially unveiled!!

Interview with Bernd Lehahn

Egosoft's Art Director Alexander Preuss guest of honor.

------------------------------------------------------------------------
Modding:

Here are a few crumbs from which we can conjecture:

I noticed a post by Apoch (EGOSOFT) which indicates TNBT will include Mission Director and other use of XML language:

There are good reasons behind the technical decisions made in the X3 engine tech, and likewise behind the choices we've put into TNBT. One of those choices is in fact that we're using not one but 3 XML-based languages for game control, based on the Mission Director of TC, the Cutscene Director of TC, and a third which was unveiled at GDC 2011 last week

Apoch did say not to make too much out of his statement.

Referring to GDC 2011, we have this from Egosoft developer Mike Lewis (aka Apoch).

There he talks about Domain-Specific Languages and AI with some code for:

Example Cutscene from [Unannounced]
Example AI script from [Unannounced]

I think it's fair to say "[Unannounced]" refers to TNBT.

The AI script is XML, written by Apoch (Lewis), and it has to do with a dogfight with player.ship.

Statement from Bernd in support of modding (not TNBT specific)

------------------------------------------------------------------------

Do we have any other clues at this time?

[Litcube]

Nice find!

That dogfighting script has me worried. I hope they're not going all external XML based scripting.

[Observe]

That dogfighting script has me worried. I hope they're not going all external XML based scripting.

Yes, that will likely cause some wringing of hands and gnashing of teeth.

[Sam L.R. Griffiths]

That dogfighting script has me worried. I hope they're not going all external XML based scripting.

Yes, that will likely cause some wringing of hands and gnashing of teeth.

Ok... I'll bite... why?

If this is the case, I am sure existing tools like Exscripter could be modified to export existing scripts in the new format (might even be something I would consider for Changeling).

[Litcube]

Ok... I'll bite... why?

The ability to instantly see/test script changes in the game environment is an invaluable asset.

[Sam L.R. Griffiths]

Ok... I'll bite... why?

The ability to instantly see/test script changes in the game environment is an invaluable asset.

Ok, I can kind of see the rationale. But I believe X has been rather unusual in having such a facility and that most other games require use of extra-game tools to build creative content. Perhaps I am desensatised to it because I have been developing mostly in compiled languages for over 13 years.

[Trickmov]

Perhaps I am desensatised to it because I have been developing mostly in compiled languages for over 13 years.

Yep - but would you like to do that without a debugger, forcing you to leave your programming tool and start your compiled program to see, if your change works as intended (and that for every minor change)?

Ok, the SE has also no debugger, but you got the instant testing ability instead.

I think this is partly a problem of the MD in TC - the effort you have to put into it is rather harsh, imo.

[InvrSmall]

Thank you for finding and posting this little Jem. MD and Cutscene editors! This just fills me with great joy as I have been playing with both for the past couple of months and was wondering if there was any point in continuing any further.


I just hope the Cutscenes get a bit of documentation this time as trial and error to get fluid transition between camera shots is very trial and error at the moment. I've had a look at his example code and I will be trying out some of it in the current X3 build as there are commands there that I had no clue to how they were used and that small cutscene example has just opened it up.

I posted a thread on it a while ago with a working stylesheet but I don't think anyone is playing with it at the moment or it just dropped off the page too quickly to be noticed.

But at the end of the day, some find the MD fustrating in the same way that I find the Script editor a total pain in the arse.

[Sartorie]

Interesting presentation you found there and quite good points made in it too.

However XML ist just a storage format which aims to preserve human readbility, it is definitely not something anyone should be programming in with a text editor. One of the basic ideas behind the MD (and as it seems they continued to follow that) was to make it easier to write things by supplying higher layers of abstractions for the commands etc. ... the problem is how much fine control they are willing to cut away to simplify the "language".

With X3 scripts only the very basic commands are hardcoded (and I wish they weren't ^^) while in their new approach much more functionality is hidden within the hardcode - this may work out to be a great advantage for new scripters (or how the powerpoint puts it : "Designers can directly write code") if the hardcode works perfectly well (avoidance or wingmans anyone ?) but of course this sacrifices a great deal of control to seasoned scripters.

In the end what it really comes down to are two things : a) explicit and in-detail documentation (this becomes extremly more important the higher the abstraction level of the commands get) b) proper tools available (this includes some kind of ingame debugging tool).

For solving a limited set of problems and only allowing basic language constructs a custom interpreted language is probably a very performant approach ... however I would have preferred they went with one of the established ones e.g. Python and then setup their language on top of it so those interested in going deeper into the code could still do so.

[Sam L.R. Griffiths]

Perhaps I am desensatised to it because I have been developing mostly in compiled languages for over 13 years.

Yep - but would you like to do that without a debugger, forcing you to leave your programming tool and start your compiled program to see, if your change works as intended (and that for every minor change)?

To be quite frank, somewhere in excess of 50% of my professional career has had to rely on log files. Real Time debuggers are only useful to a point. But that is another topic entirely

When it comes down to it in the modding context, probably the best answer would be an out of game tool that allows you to test things off-line in a very controlled environment. When testing materiel in the game, there are other potential factors outside of your control that may influence testing (EDIT: Especially since the indications are that the TNBT may be properly multi-threaded - c/f the reference to Epoch). However, there is no guarantee that MCSI will disappear, it may be sticking around (but we will have to wait till the TNBT gets formally announced/released).

[Trickmov]

To be quite frank, somewhere in excess of 50% of my professional career has had to rely on log files. Real Time debuggers are only useful to a point. But that is another topic entirely

Yes, agreed... but if this debugger is missing completly, you would get no warnings, if you missed a bracket or if there is somewhere an infinite loop, or just wrong naming of a variable.
Of course testing itself doesn't vanish by that, but a debugger eleminates at least those "first-line-bugs"

When it comes down to it in the modding context, probably the best answer would be an out of game tool that allows you to test things off-line in a very controlled environment.

Yes, that and a debugger

[Sam L.R. Griffiths]

To be quite frank, somewhere in excess of 50% of my professional career has had to rely on log files. Real Time debuggers are only useful to a point. But that is another topic entirely

Yes, agreed... but if this debugger is missing completly, you would get no warnings, if you missed a bracket or if there is somewhere an infinite loop, or just wrong naming of a variable.
Of course testing itself doesn't vanish by that, but a debugger eleminates at least those "first-line-bugs"

That is less of a debugger and more of a syntax checker, which is a base feature of any decent XML editor.

[Sartorie]

However, there is no guarantee that MCSI will disappear, it may be sticking around (but we will have to wait till the TNBT gets formally announced/released).

I hope not ... having 2 concurrent system which are not able to properly talk to each other both with their own unique approach and capabilities (and bugs !) is one of the worst possible approaches they could (again) take

I may not be a fan of where they are heading to but I really hope they concentrate on one system only and do that one properly ...

[Trickmov]

@Roger:
Don't try to nail me down to two of these three examples (which belong also to a debugger functionality) - the sense of a debugger is to test the working of the code itself (by different manners), while the testing itself is mainly to check, if your code does, what you intended to do.

Both things are necessary for good programming, and the MD is missing the debugger completly currently, while the SE has at least a direct testing possibility.

[Trickmov]

However, there is no guarantee that MCSI will disappear, it may be sticking around (but we will have to wait till the TNBT gets formally announced/released).

I hope not ... having 2 concurrent system which are not able to properly talk to each other both with their own unique approach and capabilities (and bugs !) is one of the worst possible approaches they could (again) take

I may not be a fan of where they are heading to but I really hope they concentrate on one system only and do that one properly ...

Judging from the quotation of the OP, they still have three different XML-systems.... probably the third is something like the SE.

But I would also prefer only one system

[Sam L.R. Griffiths]

@Roger:
Don't try to nail me down to two of these three examples

Sorry about that, I thought I had cancelled the posting of that one...

[OT]But to kind of put my comment into context, there are alternatives to real-time debugging (which seems to be what you are lobbying for) that can be equally effective if not more so (real-time debugging is a fire-fighting tool not a fire-prevention tool). Not that I am either against or in support of either, but anyway such discussions are best discussed elsewhere. [/OT]

[Trickmov]

but anyway such discussions are best discussed elsewhere.

not sure about that - I mean TNBT is still in development and perhaps such discussions change their opinions towards modding tools - currently I don't see a better fitting place for that.

But I think we (most of us at least) agree that the current structure ES used in TC is problematic in a few points:

a) documentation
b) debugging
c) limited/prevented communication between different languages

creating big projects isn't that easy by that - for instance currently there is a severe lack of MD-scripters, because nearly nobody liked that or it's approach.

[gmargaro]

<mylife part>
I am not actively scripting nor using MD (only for very simple things, like a givemoney script or to translate some scripts not using t-files), but i have a somewhat good background in programming (C, C++, java, ...).
</mylife part>

I think that to have a good community of modders, we need 2 sort of tools :
- simple tools that a base user can use to change simple things directly ingame (sort of wysiwyg tools) ==> visual basic for exemple.
- more complex tools / languages for experimented users/programmers to make great and complex mods/scripts (se/md, with debugger) ==> C++ / C#

This 2 sort of tools need to be able to communicate / convert from one to the other (simple to complex, the other way is, in my opinion, too difficult to do), for them to be truly useful.

[Observe]

More M. Lewis from this thread re. XML approach.

XML is a data markup system, not a programming language, let alone an interpreted programming language. The only thing "interpreted" in X3 is the script engine and even that is semi-compiled into a bytecode-like format for execution.

The XML input mapping is for data purposes only. The actual mapping code is a thin layer of C++. The slow response times are due to the lack of thread-independent input buffering in the X3 engine core, and have nothing to do with XML or anything of the sort. It's the same reason why the mouse cursor gets jittery at low framerates.

Python is pathetically unscalable and a terrible language for game scripting.

Lua is better on the scaling front but has dynamic typing and some other quirks that make it a bit of a liability in large systems; it's fine for some "glue" code here and there but for the most part we get much better mileage out of building our own tools.

There are good reasons behind the technical decisions made in the X3 engine tech, and likewise behind the choices we've put into TNBT. One of those choices is in fact that we're using not one but 3 XML-based languages for game control, based on the Mission Director of TC, the Cutscene Director of TC, and a third which was unveiled at GDC 2011 last week.

Rest assured that these systems are not performance problems.

Last but not least: nobody serious programs games in assembler anymore. The last title with any substantial assembler usage in it that I know of was Roller Coaster Tycoon (the original), and aside from that, low-level programming is reserved for high-end graphics effects and core math libraries. The rest is done in high level languages for very good reasons, which should be familiar to anyone with even a passing background in software engineering principles.

[Sam L.R. Griffiths]

<mylife part>
I am not actively scripting nor using MD (only for very simple things, like a givemoney script or to translate some scripts not using t-files), but i have a somewhat good background in programming (C, C++, java, ...).
</mylife part>

I think that to have a good community of modders, we need 2 sort of tools :
- simple tools that a base user can use to change simple things directly ingame (sort of wysiwyg tools) ==> visual basic for exemple.
- more complex tools / languages for experimented users/programmers to make great and complex mods/scripts (se/md, with debugger) ==> C++ / C#

This 2 sort of tools need to be able to communicate / convert from one to the other (simple to complex, the other way is, in my opinion, too difficult to do), for them to be truly useful.

Agreed in part...

To make the simple communicate with the complex and visa versa should be a simple matter of either (a) having a common underlying architecture and standard conventions to move between the two or (b) adopting a standard underlying global variable data heap which both can access or (c) having seperate architectures and use a form of event driven system to synchronise data between the two different heaps.