[TOOL] X4C - A translator for X4 XML files to and from a form that's easier to work with, and easier on the eyes.

The place to discuss scripting and game modifications for X4: Foundations.

Moderators: Moderators for English X Forum, Scripting / Modding Moderators

Post Reply
StopBanningMe7_III
Posts: 47
Joined: Wed, 24. Feb 16, 05:23
x4

[TOOL] X4C - A translator for X4 XML files to and from a form that's easier to work with, and easier on the eyes.

Post by StopBanningMe7_III » Fri, 18. Jun 21, 09:54

Many, many moons ago, when X4 was released, I started working on a program to convert the ugly XML based scripting language it uses to something a little more palatable. A few hundred hours of work later I posted an early attempt on this forum, which proceeded to draw absolutely no attention whatsoever. So I gave up. At least, for a while. Every now and then I've revisited the idea but never quite seem to finish it. In short, this is a post I've been putting off for years at this point.

I have what would be generous to call a "prototype" of the translator working. Much more has been written but not incorporated in a meaningful way. It's become rather obvious that I won't ever get the motivation to finish unless I know that at least someone out there thinks this is a thing worth doing. I really didn't want to post anything until I had something actually impressive to show off but it's not meant to be.

Right, enough complaining. This program simply reads an X4 XML md or ai script and translates it into a more C-like syntax. Of course, it can also do the opposite. That was the tricky part. Writing the grammar for this language took much longer than it really ought to have, primarily because when I started off I didn't know how to write a grammar at all. I believe I've nailed it now, or at least nailed the expression syntax, which is the only tricky bit of the language. I've called the translated script `x4c`. The program can convert from XML to x4c and back again with only tiny changes in formatting, so at least it doesn't break things.

The plan is to do significantly more than this. I'd like to make it easier to write mods for this game. The language is just so verbose and hideous that I can't deal with it. Right now only simple condition statements are actually changed dramatically. All other XML statements are just changed to a simpler form. If there's any interest I can greatly expand on this.

Pictures do speak louder than words sometimes. These screenshots, though, are not the output of the current form of the program. These are from an earlier version, but it wouldn't be terribly hard to get things looking this way again. Unfortunately, the less simple part would be dealing with editor support. These screenshots are of the colour scheme I wrote for Vim. I don't know how many of you use Vim, but I'm going to guess the majority do not.

https://imgur.com/a/XVTNPga

I could explain things more but perhaps I'd best leave this simple for now. The point is this: is there anyone who thinks this is a good idea? Would anyone potentially write code in this modified form? How completely am I wasting (a lot of) my time?

If you want to play around with it, here's a pre-compiled binary for Windows 10. Pass it an XML file and it'll spit out x4c, pass it an x4c file for the opposite. There are option switches but they're a mess. And yes, it's command line only.

https://mega.nz/file/b0QGTZyb#FxFsFItkP ... 7VLDBFj_uI

Here is the source code for that binary. It's mainly in Go, with the grammar written in Antlr4.

https://github.com/roflcopter4/x4c-go

I've also written versions in C and C++, and there's much more Go code not currently uploaded to git.

Jeeze. After all this time, this is rather underwhelming. But please do let me know what you think. The truth, preferably. If you think my ideas are bad and I should feel bad, tell me.

EDIT: Oh, something I forgot to mention. First time you run the program it'll complain about an uninitialized ini file, and make one in the current directory. You need to provide the location at which you've extracted all the stock script files. In particular, the directory must contain the sub-directory "libraries" which has the critical XSD files.

SirNukes
Posts: 546
Joined: Sat, 31. Mar 07, 23:44
x4

Re: [TOOL] X4C - A translator for X4 XML files to and from a form that's easier to work with, and easier on the eyes.

Post by SirNukes » Sat, 19. Jun 21, 01:56

I think just about everyone doing a lot of scripting has gotten accustomed to xml and has xsl support set up (for documentation, autocompletion, error checking), reducing the usefulness of a basic translator.

StopBanningMe7_III
Posts: 47
Joined: Wed, 24. Feb 16, 05:23
x4

Re: [TOOL] X4C - A translator for X4 XML files to and from a form that's easier to work with, and easier on the eyes.

Post by StopBanningMe7_III » Sat, 19. Jun 21, 04:47

SirNukes wrote:
Sat, 19. Jun 21, 01:56
I think just about everyone doing a lot of scripting has gotten accustomed to xml and has xsl support set up (for documentation, autocompletion, error checking), reducing the usefulness of a basic translator.
I think you meant "eliminating the usefulness of a basic translator". This is why I didn't want to post this in this unfinished state. I had every intention of adding support for the XSL files and so forth, which would theoretically also enable auto-completion. The most notable problem there is that I only know how to generate that for Vim. Probably a big problem, actually.

FWIW it can already do error checking both via schema validation and more simply by checking expression grammar.

I also intended to add macro processing and automatic generation of patches instead of xml files when appropriate. Key word: intended. If nobody would use it there's not much reason for me to bother.

arshiba
Posts: 48
Joined: Thu, 1. Jul 04, 15:30
x4

Re: [TOOL] X4C - A translator for X4 XML files to and from a form that's easier to work with, and easier on the eyes.

Post by arshiba » Sat, 19. Jun 21, 11:37

You want to create a language that egosoft has canceled (remember scripts in X2/X3) in favor of a simpler option (extend xml-based MD onto AI). And I know the scripts were also based on xml, but it was a storage form, not a presentation. Little nostalgia-example:

Code: Select all

149       $sat =  create ship: type=Navigation Relay Satellite owner=Player addto=$xsector x=0 y=2000 z=0
150 @     = wait 100 ms
151       $sat -> destruct: show no explosion=[TRUE]
152       $stage = 3
153       set global variable: name='NivRush.Mission.2.Stage' value=$stage
The main question "Why?". We can only guess, but I think the reason is an EDITOR. It became too difficult to keep developing the built in editor. It is easier (cheaper) to use publicly available tools and what could be simpler than XML and its developed infrastructure.
You've already spent hundreds of hours on this transplainer, but I'm afraid this is just the tip of the iceberg. To make it useful, you need to spend thousands more.
My suggestions and issues:
1. Complexity. In the current implementation, it is impossible to replace MD / AI with X4C. We must learn both.
2. Editor. To write your own from scratch is madness. Write a plugin for an existing IDE, but which one: MSVS Code, Qt creator (my preference), notepad++, eclipse, or any other of dozens of cool editors?
3. Syntax. Why C-like. Why not what were been used / is being used in the game. The above script example or lua-like would be more successful candidates.
4. Editor's smart capabilities. I am currently using msvs code + red hat xml extension. This already gives the aforementioned attribute autocompletion, tag name suggestions according to the context, error checking in the xml structure. But what I am missing is:
- an autocompletion for the names of my variables: $varN<ctrl+space>ame
- an autocompletion for the names and descriptions of object properties: $ship.<ctrl+space> => list [..., dock (Dock where this ship is docked), ...]
- references for other files: ware.<ctrl+space> => list [..., energycells, ...]
- an expression syntax check
- a hint for the text of messages: {20202,101} => "Argon"
- a quick search for examples of using the action: find_closest_resource <ctrl+s> => list [encounters.xml "<find_closest_resource refobjec...", rml_find_resources.xml "<find_closest_resource sect..", ...]
5. Debugging. I can only dream of step-by-step debugging. But now looking at the debug log I will see file and line number of the converted text, and not my source.
6. Debugging / static analysis. Ability to dry run code for the simplest (stupidest) errors like

Code: Select all

<cue name="Demo1">
  <delay exact="1s"/>	<!-- source of problem -->
  <actions>
    <find_station name="$station" space="$sector" />
  </actions>
  <cues>
    <cue name="Demo2">
      <conditions>
        <event_object_destroyed object="$station"/>	<!-- ERROR: variable not initialized during evaluation -->
      </conditions>
    </cue>
  </cues>
</cue>
PS. Implementation of #2, #4, #6 will be able to compensate #1, #3, #5 and then your tool can become useful. But not in its current form :(

Post Reply

Return to “X4: Foundations - Scripts and Modding”