Anyone thinking of writing a MD coding program ?

[apricotslice]

I've been playing around with an MD file in the last few days, and I've cemented my previous conclusions.

I will never be able to write MD code, although I can minimally alter it.

Now, since Rebirth wont have a script editor or scripts, just a version of the MD, it seems to me that you clever program writers (Yes, Cycrow, looking at you ! lol ) should be starting to think about writing a tool to automate the writing of MD code.

It needs to be an intuitive program, that allows the user to specify what they want to do, and it codes it for you, after taking you through a list of ifs, buts and maybe's.

So, you want to add a station, you click on add object, select station, select where to put it, select what wares to go on it, and the program writes the code.

You want a wing of ships, it lets you select them all, where, fittings etc. and then writes the code.

What I really am getting at, is that everything we currently have a script for, needs to be replaced by MD code, and we need an intuitive tool for writing the code, without needing to know anything about MD code at all.

eg. Claiming a ship. How do you code that in MD ? I really want a tool where I can generate that code just by logically selecting what I want to do and letting the program do it.

Anyone thinking about this yet ?

Seems that we should be working on replacing scripts in AP now, in preparation for Rebirth.

Comments ?

[SpaceAcid]

we are on the same page and I am glad someone brought up the subject.

learning to script in any game these days that actually has mod support
first you need to relentlessly try to learn the entire game engine file
structure from the ground up ..endlessly searching for source docu on all
aspects ..tags, their funtions etc.

I have arrived at the same conclusion myself after being overwhelmed
by utter nonsense confusion during the process...uh yeah, no thanks
ile just play & go back to learning something simpler minded like Nuclear Fusion NASA Physics.

decompile this.. extract that.. use this tool for this run the game &
change your name to this then run the script editor then hop up and
down on one leg while touching your nose....AAAAHHH!

until an Official Release of an SDK for dummies ..I give up.

Egosoft really needs to do similar what TaleWorlds did with Mount & Blade enabling the simpler
smarter use of an editable updatable per release source using an engine that can decipher via
.py source scripts, & some sort of legal OpenSource CVS for moderators to access

why they made it so difficult to modify your own game is beyond me.

[jlehtone]

I've been playing around with an MD file in the last few days, and I've cemented my previous conclusions.

What toolset have you used for staring at the file?

[Drewgamer]

we are on the same page and I am glad someone brought up the subject.

learning to script in any game these days that actually has mod support
first you need to relentlessly try to learn the entire game engine file
structure from the ground up ..endlessly searching for source docu on all
aspects ..tags, their funtions etc.

I have arrived at the same conclusion myself after being overwhelmed
by utter nonsense confusion during the process...uh yeah, no thanks
ile just play & go back to learning something simpler minded like Nuclear Fusion NASA Physics.

decompile this.. extract that.. use this tool for this run the game &
change your name to this then run the script editor then hop up and
down on one leg while touching your nose....AAAAHHH!

until an Official Release of an SDK for dummies ..I give up.

Egosoft really needs to do similar what TaleWorlds did with Mount & Blade enabling the simpler
smarter use of an editable updatable per release source using an engine that can decipher via
.py source scripts, & some sort of legal OpenSource CVS for moderators to access

why they made it so difficult to modify your own game is beyond me.

I've stayed away from MD for this exact reason lol

PS: At least we CAN mod the game ^_^

[ScRaT_GER]

I already thought about something like that. Unfortunately it's really hard to say how hard developing some sane language, which compiles into the XML language X:R uses, will be.
Maybe, once the game is released, the people interested in modding and with programming capabilities could group and collaborate.

since Rebirth wont have a script editor or scripts

Rebirth still has scripts - the difference is that they are written in XML. So when you'd write "command" in X:TC you'll write "<command/>" in X:R.
Yes, this is a lot more verbose and cumbersome (especially writing expression like "a=x+y" seems to be a real PITA in the XML language from what we have seen in the preview file).
But conceptually it's still very much like MSCI, just with angle brackets everywhere.

Greetings,
ScRaT

[DrBullwinkle]

a tool to automate the writing of MD code.

We don't have a code generator for the script editor, but plenty of scripts get written. The only thing special about the script editor is that it enforces proper syntax. You still have to write the scripts.

Same idea with MD, except that MD works with standard editors. That part is a Good Idea.

As jlhetone suggests, it is easier if you use a proper xml editor. The commonly-recommended freebie is Microsoft Visual Web Developer Express. It will not write the code for you, but it makes MD code slightly easier to read.

The director.htm, of course, gives you a reference to the available commands. They are fairly easy to understand once you get familiar with the way they work. It is part of the package on the Mission Director Basics and Installation page.

Tip: Brush the dust off of your mothballed copy of Internet Explorer for reading the director.htm. It depends on an ActiveX control.

A more important missing piece seems to be a good manual. It would be nice to have something analogous to the MCSI Programmer's Handbook, which would explain the structure of various kinds of MD scripts.

Unfortunately, people who write good manuals are rare. They have to be fluent at programming as well as a human-readable language -- an uncommon combination. Kurt Fitzner did a brilliant job on the MCSI Handbook. I hope that he got paid for the job, because the royalties probably stink.

The preview file that ScRat_GER just posted is well-documented, at least. Examples like that will ease the learning curve. Now, if only we could convince other script writers to document their code in a similar manner...

[jlehtone]

Looking at the raw MD-file is like looking at a raw MSCI-file:

Code: Select all

<?xml version="1.0" standalone="yes" ?>
<?xml-stylesheet href="x2script.xsl" type="text/xsl" ?>
<script>
<name>foo.bar</name>
<version>1</version>
<engineversion>40</engineversion>
<description>Does nothing!</description>
<arguments>
</arguments>
<sourcetext>
<line linenr="001" indent=""><text>return </text><var>null</var></line>
</sourcetext>
<codearray>
<sval type="array" size="10">
  <sval type="string" val="foo.bar"/>
  <sval type="int" val="40"/>
  <sval type="string" val="Does nothing!"/>
  <sval type="int" val="1"/>
  <sval type="int" val="0"/>
  <sval type="int" val="0"/>
  <sval type="array" size="1">
    <sval type="array" size="3">
      <sval type="int" val="103"/>
      <sval type="int" val="0"/>
      <sval type="int" val="0"/>
    </sval>
  </sval>
  <sval type="int" val="0"/>
  <sval type="int" val="0"/>
  <sval type="int" val="0"/>
</sval>
</codearray>
</script>

However, looking at the same MSCI-file with IE (or ingame editor), which makes use of the 'x2script.xsl' file to render content, much less is seen:

Script foo.bar
Version: 1
for Script Engine Version: 40

Description
Does nothing!

Arguments

Source Text
001 return null

[apricotslice]

I've been playing around with an MD file in the last few days, and I've cemented my previous conclusions.

What toolset have you used for staring at the file?

At the moment, using Dreamweaver.

I actually have the same problem with html. All the programs for writing web pages are based on knowing html code.

When in fact the best editor ever imo, was netscape communicator 7.2 which had a word processor like interface, with the code underneath. You just wote a word processor like document, and the program supplied the code.

Which was fine up until one wanted to do iframes and stuff, which it wont do because its too old now.

But none of the decent web programs followed the same approach. They all assume your writing code and know what your doing.

Its the same with scripts. I want to place a station in a certain place and put stuff on it. Why do I need to know xml ?

Why cant someone write a program that lets people with no programming ability just tell it to do things in a particular order and it generates the code for you ?

Rebirth still has scripts - the difference is that they are written in XML. So when you'd write "command" in X:TC you'll write "<command/>" in X:R.
Yes, this is a lot more verbose and cumbersome (especially writing expression like "a=x+y" seems to be a real PITA in the XML language from what we have seen in the preview file).
But conceptually it's still very much like MSCI, just with angle brackets everywhere.

Then I hope someone is writing a conversion program. Input a TC or AP script and output a XR xml.

An automated tool for converting scripts would seem essential.

[Sam L.R. Griffiths]

I have written a basic tree style XML editor in C# as part of Changling, and intend to improve on it in the future (RL time has not been on my side the past few months - Changling is not dead just sleeping). In addition, auto generation of certain patterns of MD code (relating to starts) is planned.

There was someone working on an MD editor written from scratch in C/C++, but I do not know how far they have got.

[Ketraar]

This subject keeps popping up periodically and still has the same basic and mostly wrong assumptions. So here it goes again.

1. You write MD code pretty much the same way you do MSCI Scripts using the in-game SE. If you use an editor that can read schema files you can get the "blocks" of code to use by pressing ctrl+space and it will give you options to choose from. That's pretty much the same concept as in SE, with the added benefit of it being contextual, so only applicable options are shown.

Obviously it will not give you options everywhere, because sometimes the possibilities are almost endless, thus it makes no sense to give you a list to choose from that never ends. That's where the director.html comes in for reference.

2. Any program to help code MD stuff easier, will not make it understand better. You still will have to come up with mission structure yourself and find out when what has to happen, depending on what you want. No way you get a program to do that for you, the MD possibilities are just to freaking huge for that.

As reverse example I tell you that I just can't cope with how the SE works, exactly because how you have to select stuff from lists I was never able to figure out how to use it. Would the MD have something like that I would have never got around to use it. Think the Exscriptor is quite popular too for the same reasons probably.

3. The one thing one has to do to be able to wrap his head around the MD is not how you input the code, but to understand how it "thinks". Its to understand that it does stuff in sequence and it does stuff based on game events. Once passed beyond this hurdle it becomes increasingly easy to code MD stuff, sure one will have to look up variables and commands once and again, no way to memorize them all, but that's normal.

Last but not least, its important to "detach" yourself from SE habits, and not address the MD like you do the SE. It is NOT SE, don't try to make it be like it. That's probably the biggest handicap people that used to code stuff in SE have, compared to those that went "straight" to MD (just a personal feeling here). MD is for missions and its really good for it.

Just start by doing small stuff, code a ship going from A to B, maybe sending an message. Read the X3R Manual, it still applies to 99% of what you have now. Sure it will not tell you how to make good missions, but it will teach you how it "thinks" so you can come up with the missions.

And there is always the good old forums where one can ask stuff when stuck. Obviously the patience to go through ANY learning process will have to come from you.

MFG

Ketraar

PS.: Passing the word of MD since 2008

[apricotslice]

1. You write MD code pretty much the same way you do MSCI Scripts using the in-game SE. If you use an editor that can read schema files you can get the "blocks" of code to use by pressing ctrl+space and it will give you options to choose from. That's pretty much the same concept as in SE, with the added benefit of it being contextual, so only applicable options are shown.

Errrrr...... what is the "like the in game SE" you refer to ?

Does it have a program name ? I cant say I've heard of it. I thought MD got written in xml editors.

I'm not comparing MD with SE. I HATE the SE, although I have managed over the last 8 odd years to be able to turn out something pretty simple using it.

I am a programmer. But MD does my head in. There is so much indented redundant code in it that being 1 "cue" out buggers up the whole thing. (Admittedly a dot in the wrong place buggered up cobol, but at least the logic was obvious.)

Call me old fashioned, but the old fashioned If...Then...Else construct makes far more sense to me than MD does. The SE at least has those constructs and obeys basic programming structure, albiet in a very difficult to use program.

Hence me wanting a higher level. MD is code I will never get my head around, so hence a program to generate the code from what you want to do.

[Xenon_Slayer]

PS.: Passing the word of MD since 2008

Amen brother!

I'm not sure what a non-xml MD editor would look like, but it stands to reason that it wouldn't need to be bespoke to the MD. All of the rules for what is capable with a MD file come from the .xsd schema file. So, you would need a program which reads that file and gives you editing options.

I have no idea if there is a program out there which does something like that. I'm very happy with editing the XML directly with Visual Studio so I've hardly looked.

One thing which is not possible from schema validation are the variables. You always have to look up what variables are possible and be wary of missing brackets.

[apricotslice]

I have no idea if there is a program out there which does something like that. I'm very happy with editing the XML directly with Visual Studio so I've hardly looked.

Ok, that at least gives me an idea of whats being used.

I have Visual Studio loaded because of programs which require its presence, but using it is above my "pay grade".

I dropped out of mainsteam programming well before VS showed up and never made the jump. Probably never will either. Like the graphical stuff. That was all after my time as well.

[Ketraar]

Sorry I missed to mention the program and like X_S I use the free from Microsofts download section available Visual Web Developer 2010 Express Edition.

Also not sure if that was clear, you need to be editing the MD files in same folder as the director.xsd for the VWD to read them and thus give you the options. (it should all be mentioned in the X3R MD introduction topic)

Also note: I'm no where close to a programmer and got around it, then anyone else can too. But unless some one invents the Matrix Download-stuff-to-Brain app, you wont get around having to spend tedious and frustrating periods of try&error, and the occasional asking anyone that might know. Its called learning.

MFG

Ketraar

[Aydan]

The program you'd like to see, apricot, is probably very hard to do. The reason isn't the code as such, but the way the MD thinks, like Ketraar has already mentioned it. The way you specify a certain event in the SE is building up on given existing objects or variables. Before you use them, you define them, and it's completely up to the scripter how detailed that may be. Assuming you'd like to do a mission in SE, now you'd go and build up your code step by step, and each step based on the one before... normally

The MD doesn't allow this. You need to deconstruct your idea first, because the cues are run in exactly the order specified. So you'd need to have the complete idea of the mission first, and then cutting it down into little pieces, neatly arranged as cues. It requires a certain kind of thinking which most of the common scripters aren't used to. To me, the SE is always a constructive way of coding, while the MD is a deconstruction. The problem is, that you can't really tell a program the whole idea and it's breaking it down into arranged MD cues. If you'd tell said program step by step what you'd like to do, it'd build an MD code for every step. That would be problematic, as it'd need to merge all these steps later on into a single file (or some ) It'd need to be intelligent enough to decide which checks in the conditions are nescessary and which may be redundant. In the end, it'd be probably performing a lot more checks, decreasing the FPS for example. The input method would be to tell the program your idea in general, and to let it add in your steps afterwards, or to collect all the steps first, merging them together later. Either way, i don't think it'd work.

The way to think in the MD is probably the reason why it isn't generally accepted by the modding community. It's easier to find a dog with six legs than a MD coder doing more than only a few lines of code The ES announcements are always very cheerful, and if it's announced that Rebirth only uses MD code... let's say, i'd smile a bit, feeling reminded of Reunion, where there were similar announcements for TC, if i'd see a folder named "scripts" in the Rebirth installation. To me, there's no need to learn the MD xml scheme, as everything can be achieved by using the Script Editor.

[Sam L.R. Griffiths]

The program you'd like to see, apricot, is probably very hard to do. The reason isn't the code as such, but the way the MD thinks, like Ketraar has already mentioned it.

Not really hard, just time consuming. Most concepts that many modders want to put into MD terms can be expressed as simple patterns of code. The key point will be in identifying those patterns and then presenting the concept to the user in simple terms (hiding all the XML detail).

There will still be cases where modders will have to get down and dirty with the XML but if the tool is designed correctly, the user/modder will be able to design their own patterns (or import other people's) and re-use them via an interface they will feel more comfortable with.

Writing code in MD/MCSI is fine for us programers who love to understand the ins-and-outs, but for the average modder they just want the results in the simplest way possible without having to understand the black-arts of bits-and-bytes/data-structures-and-algorithms.

To me, there's no need to learn the MD xml scheme, as everything can be achieved by using the Script Editor.

Not exactly true, it largely depends on what you are wanting to achieve. There are somethings that only work in MD, other things that only work in MCSI, and others that require a mix of the two.

[apricotslice]

Its been stated that there will not be a script editor in XR. However, it has also been stated that there will be scripts, but written in conventional xml, more like the MD.

It remains to be seen if the new scripts are better or worse than the SE.

I dont see the issue of "mentality" for the MD.

All programming needs structure and this is based on logic. Basic programming skills include designing a program in psuedocode before you actually start to code it. So before one starts to write MD code, one should already have a structured design of what the MD is supposed to do.

My issue is with the coding of it itself, not the design process. And my experience with it so far, shows that the coding flow is logical, albiet with so much redundant repetitious "cr*p" in it as to render it extremely confusing. Then again, html is the same way, overly verbose. Ditto php and xml. No wonder I never made the jump to them. In comparison, cobol was so much simpler to write, and if you used meaningful names, the whole code was readable and should be understood by a non-coder. Thems was the days. *sigh*

Nothing thats been said so far as a reason for not doing what I'm asking for is something I see as an issue to doing it.

Back in the day, I was noted for doing the impossible. Surely there are still some programmers around for which the impossible is merely a good challenge ?

Seriously, if I sat down and thought about the mechanism for a while, I could design an interface. I just cant code it. So if someone who can code wants to collaborate, I'm happy to.

[DrBullwinkle]

The good news, Apricot, is that you have attracted the attention of some heavy-hitters to your request.

I don't think that anyone is saying "no". What I hear, mostly, are explanations about why it has not already happened. Which was your original question, wasn't it?

Will it help to divide your question into smaller, easier-to-respond-to pieces?:

1) XML is verbose. That is because XML, itself, is designed to be an interchange language; for communication between various kinds of data consumers and producers (mostly programs). Think of XML as an open-source database standard.

The advantage of a standard format, such as XML, is that you can use a wide variety of standard tools with it.

However, xml was never intended to be edited with a text editor (other than for small jobs).

2) A script language based on xml makes sense. MCSI is such a language, although it uses a non-traditional schema (or .xsd). Still, it looks fine if you open an MCSI script in a browser. That is the beauty of xml -- it can be viewed, edited, and used by any xml-aware software.

3) MD-language appears to be a more modern approach to MCSI. If not more modern, at least "alternative". Among other things, the MD language appears to be event-driven rather than procedural. That means that it may take some time to wrap one's head around the different approach. However, an event-driven language is ideal for missions, so it makes sense.

4) One step towards your request is an editor that allows one to script in the MD language without seeing the tags. Like the Script Editor or Exscriptor. It sounds, from this thread, as though that part is (or will be) covered. Did I get that right?

5) The second part of your request is to simplify the MD language by creating a code generator for it. OK, but keep in mind that the MD language is *already* a simplification. It is designed for people to write missions for the game without having to be one of the engine developers to understand it.

6) As Roger points out, code generators are not all that hard (for programmers who are good at abstraction -- which is not all programmers). I have written a couple. The main thing that I have learned is that code generators are always limited, and somebody will always want hand-tuned code. But tuning the code produced by a code generator may not be optimal (meaning that code generators sometimes produce code that is extremely hard for a human to read).

Roger's suggestion helps somewhat (that the code generator should allow the addition of new patterns). Still, though, this is a job that will require maintenance for as long as the MD language is in use. (Poor Whimsy, who did such a nice job with Exscriptor, now finds himself burdened by the maintenance task.)

A community-based, open-source, project would help to reduce the maintenance burden.

7) Would such an effort be better invested into an open source editor, like an "Exscriptor-MD"? Similar skill sets are required, but a good editor can be used by all, whereas a code generator is more limited in its appeal.

Add a snippet library to an editor and you get most of the benefit of both approaches.

[apricotslice]

At this point, I'd like to ask a couple of questions.

How many experienced MD coders do we have ?

How many of them are available to help people learn ?

How many are available to take on projects ?

I've had a request to do an Apricot Mapping Service Plot and I'd love to collaborate with an md coder to do it.

The main question is, is there anyone available and interested ?

[Sam L.R. Griffiths]

4) One step towards your request is an editor that allows one to script in the MD language without seeing the tags. Like the Script Editor or Exscriptor. It sounds, from this thread, as though that part is (or will be) covered. Did I get that right?

If you are referring to my tree-based XML editor in Changling then yes. It still requires knowledge of the XML schema in it's current form but it does hide the verbosity of the XML structure behind a GUI front end (looks a bit like regedit in some ways). I have written it in such a way as it can be made standalone, will consider doing that in the near future (time allowing).

5) The second part of your request is to simplify the MD language by creating a code generator for it. OK, but keep in mind that the MD language is *already* a simplification. It is designed for people to write missions for the game without having to be one of the engine developers to understand it.

Problem with programming/scripting languages is they are almost invariably an abstraction from one concept into another and will not suit everyone some will prefer the abstracted concept and others will want it abstracted even further.

6) As Roger points out, code generators are not all that hard (for programmers who are good at abstraction -- which is not all programmers). I have written a couple. The main thing that I have learned is that code generators are always limited, and somebody will always want hand-tuned code. But tuning the code produced by a code generator may not be optimal (meaning that code generators sometimes produce code that is extremely hard for a human to read).

Roger's suggestion helps somewhat (that the code generator should allow the addition of new patterns). Still, though, this is a job that will require maintenance for as long as the MD language is in use. (Poor Whimsy, who did such a nice job with Exscriptor, now finds himself burdened by the maintenance task.)

A community-based, open-source, project would help to reduce the maintenance burden.

7) Would such an effort be better invested into an open source editor, like an "Exscriptor-MD"? Similar skill sets are required, but a good editor can be used by all, whereas a code generator is more limited in its appeal.

Add a snippet library to an editor and you get most of the benefit of both approaches.

I do plan to make Changeling open-source eventually but I want to get the baseline concepts solid first. Saying that, I am open to support from other C# programmers if they are willing to help during the current phase.