I think you're always going to get that, to some extent.
Probably. But I think it ends up being about the mechanics, which is more akin to a video game, which I don't want. I think the key as DM is to make the obstacles less straight forward. Not to foil their builds just for the sake of it. But to make the obstacles challenging and interesting. Fights don't always occur on level ground. Put them in hip deep water and muck. Have mobs of archers. Put them on a narrow path on the side of a cliff, where falling off is a real possibility.
I'm of 2 minds about that. I do find that has a nostalgic charm to it--but it was also an unnecessary bit of complexity.
It is not about nostalgia. It is about making sense. Good fantasy and sci-fi are less about the technology and magic and more about seeing how people respond to them. Every good movie/ story has to have realistic elements in order for you to maintain your suspension of disbelief.
So from that perspective, I don't think it is unnecessary. Nor was it all that complex. We'd tally initiative scores, start with the lowest and count up. I would note whether there was a difference of 10, and if so, anyone inside that range got an extra attack.
Complexity only matters if it slows the game. If that complexity is transparent to the players, I have no problem with it. If it makes actual play a grinding chore, then it must go.
Whenever you add realism, you necessarily give up simplicity; conversely, in order to gain simplicity you must add abstraction (the hit point system is a perfect example).
You use two phrases - complexity and abstraction - in a way I don't think you meant to. Complexity was addressed above. Abstraction has come to mean something akin to "vague". But it is the theoretical core idea(s) of a thing.
So, if we take that meaning, nothing about d&d is not
an abstraction. Making it more realistic does not actually reduce the abstraction of it. If we understand what the point of our game is, then we can go from there and decide whether more realism is desireable
and if so, we have to do it in a way that does not slow down the game.
I have never liked to vagueness of the D&D hit point system. For one, you are as fresh and able on your last, single hit point as you were at full, even though the difference might be several score.
Second, I found them to be grotesquely out of whack with regards to monsters and even mundane animals. How many hit points do you think a cow should have? Would a cow be killed more or less easily than, say, Mike Tyson? Do you think that a hero with a sword in full plate mail would last 30 seconds against a grizzly bear? I do not. The closest I came to finding a resolution was The Grim and Gritty combat system.
I never tried it, though.
Because a two handed sword had enough reach to balance it versus the dagger
I do not believe so. Not in terms of how many times you can attempt to use it in a given period of time. The balancing difference between them is the damage output.
I had different rules for reach. For weapons that were substantially longer - spears and pole arms - they automatically got initiative over shorter weapons when the two parties were approaching each other. If they hit, the target was prevented from advancing further. So if you had a spear and your opponent did not, you could
keep your opponent at bay indefinitely. It was my own proto-version of 3e's attack of opportunity.