Fact is, X4 is smaller than X3 in every department.
- Galaxy is smaller, both factually (yeah, sectors can "expand"... into virtual nothingness, so the feature is moot by definition) and gameplay wise: due to highways, you can traverse the galaxy from one corner to the other in a matter of minutes. Sure, in X3 we had jumpdrives for an equivalent result, but the actual initial effort to map the 232!! (in AP) sectors, quite a few of them pretty well hidden, was massive and kept you occupied for a good chunk of the early to mid game.
- There was a lot more to do in X3: way more ship types and classes to fly and experiment with (all of them much more diverse), more races, missions, more of everything. Prices - and the time it took to make a profit - were way higher too, it took much longer to get to the capital ships (real capitals, not the ones that fly almost like medium sized ones here), but there also was enough content and diversity to keep you occupied during the "grind".
- There were plots. Granted I don't miss them per se - they often were pretty much glorified tutorials and also often were pretty bad overall - they were yet another type of content to interact with while you slowly progressed. Scrapping the developing intensive plots would have been mighty fine if that meant using those resources and that time to offer more - and more involved - game generated content. But... they didn't.
When it comes down to it, X4 is a shell. A foundation, yeah: a platform that - when one day it will hopefully work in its base mechanics, many of which are currently still broken - could grow to offer a comparable amount of content. Right now it doesn't. The major reason why it's hard to really spend hundreds of hours in it is that there's hardly reason to: for instance, one of the biggest pushes to progression in X3 was getting that bigger ship, or that ship of another race you had to build relations with, to try the new toys it came with, different weapons and such. First to fly them personally, then to choose how to build (and flavour) your fleets. Would you mix and match? Would you go with the Terran ships to see your fleets use their exclusive weapons? Or would you flavour your fleet with OTAS ships only? And so on. You could spend hours just trying to find (or cap) that unique ship variant with a special loadout or look to then mass produce it at your HQ. In X4 all ships mount the same few weapons, nothing really relevant changes between ship classes or manufacturing race: as such, nothing really changes in your gameplay apart from having a bigger cargo, more or fewer turrets, and more or less sluggish flight, everything is pretty much the same. As such, nothing really keeps you invested for the long term.
Obviously, the dynamic faction wars and conquests are where we could see X4 beat X3: it's a feature X3 only had in a couple mods. Wars and player conquest can be a long term goal that makes you feel invested and push you through the grind. When they'll work. They are just barely beginning to, up until 2.0 they simply were not really implemented (even though they kept assuring they were...). And this is the other problem with X4: what little content actually is in the game, often doesn't really work yet. We'll see when they'll be finally in as they should. Even though, I've got to say, "wars and conquests" lose quite a bit of appeal when factions have half a dozen sectors tops, instead of several dozens. There's the risk they are going to feel more like neighborhood squabbles rather than "epic" wars. I seriously hope when the race expansions hit, they won't just add 5 sectors per race and put the Split there, the Boron here, but also add quite a bit more unrelated sectors all around to increase the scope of the galaxy. Some of the mechanics introduced in X4 can grow to be fairly interesting, but right now X4 suffers from an ironic malady: the foundations it's built upon are way too small to support them and let them reach their full potential.