I use Blender 2.8, in case your Blender/3D Max has different version something could work wrong.
1. prepare fresh template
We need to copy vanilla ship and make sure it is present in the game. There are could be many errors why a ship or texture is not visible, so we will deal with the problems one by one. As the template for our ship I'll take ship_tel_s_scout_01, it's teladi scout. Once we are sure that copy of this ship with different name could be bought at Argon wharf (I have save file where I docked there and I have a lot of money), we move to the next step.
2. prepare scene

Let's delete everything we don't need. We don't need fx_licence, anim_mainXlod1, anim_mainXlod2, anim_mainXlod3. It's best to delete them by right click, select Hierarchy, press Del button. Also we should hide anim_mainXcollision, anim_mainXlod0XgenericXgeneric_p1_fx, anim_mainXlod0XgenericXgeneric_p1_hullpattern, anim_mainXlod0XgenericXgeneric_p1_hulltexture for now.
We should import our custom model to the scene, in this example it is X3 Nova, split it by material and place it around cockpit. It will be the position of hull in the game.

After this we can safely delete anim_cockpit node and everything in the cockpit except anim_cockpit_frame/detail_xl_glass_outside.
3. Working with mesh
We need to make visible anim_mainXlod0XgenericXgeneric_p1_hullpattern and make some copies of it by pressing shift+D and hide copies for now. Now we select anim_mainXlod0XgenericXgeneric_p1_hullpattern, press Tab, press A, press Del, select vertexes, press Tab. This will delete all vertexes from the mesh but keep mesh itself. After that we select hull of X3 Nova, by holding ctrl we also select anim_mainXlod0XgenericXgeneric_p1_hullpattern and press ctrl+J. Mesh will have 2 materials, we'll keep vanilla material for now and remove another one.

For every part (unique material) of our custom ship we need 1 copy. Merging of a mesh into empty vanilla geometry has to be performed for every part or you'll find your ship very distorted.
In the _data folder of out template ship we need to delete files anim_main-lod0.xmf, anim_main-lod1.xmf, anim_main-lod2.xmf and anim_main-lod3.xmf. Now we need to export out model and copy generated anim_main-lod0.xmf into _data folder of template ship, make 3 copies of it and rename them to lods 1/2/3 accordingly.
And finally we run the game with enabled log to file and see what we got, but before that in macro file we should change "ship_s" to "ship_m" because X3 Nova is huge and won't fit into S dock.

I got this, game renders geometry, it's success. After that we need to get back to Blender and change name of material and lod. I won't cover creating custom material here but be aware that bugged material makes geometry invisible/black/grey/pink. We need to remove everything we don't need like anim_mainXlod0XgenericXgeneric_p1_hulltexture in my case, and place objects where we want - ladder, thrusters, engines, shields. Repeat lod generating and replacing procedure.
When we are sure that game renders custom materials, we need to work on collision. As before, we delete all vertexes from anim_mainXcollision, make copies of all parts of our custom hull and merge them into collision model, and remove all materials from it except one. We don't want to copy everything generated into _data folder of a template ship, don't mess with gears, ladder and everything you didn't change, just change its coordinates if you need or remove from component file.
4. applying paint jobs.

We need to deal with UV-maps and vertex colors. First, make sure names maps and color channels are unique in the scene. If you don't want to support paint jobs, then don't create second map and vertex color and paint everything in "vertex paint mode" in grey, it will affect your texture. If you want to add vertex jobs to your ship, you should color it in 2 channels.
1st channel. color 787878 means pure texture without paint job. color 2B2E33 is for pipes and other dark places without paint job. color BFFFD0 will be used for moderate saturation of paint job. color FF0101 is for saturated paint job.
2nd channel. color 010101 means no paint at all. color 010101 means paint job will be applied.
When I need to paint something, I split geometry and paint it separately by pressing shift+K and merge them afterwards.
Dock position
You probably noticed that our Nova got stuck into docking bay. We can fix it
Code: Select all
<connection name="con_dockpos" tags="dockpos ">
<offset>
<position x="0" y="-5.62136" z="87.68802"/>
<rotation yaw="0" pitch="6" roll="0"/>
</offset>
</connection>