I think there have been some attempts to explain, none of them fully satisfying.
But there have also been systems that create vectors with that property more directly instead of as a side-effect of a neural net. Examples include Stanford's GloVe, or Omer Levy's Hyperwords, which is made entirely from old-school ideas such as mutual information and dimensionality reduction, and was for a while the best system if you limited it to fixed training data (still Wikipedia).
A gripe: reviewers don't even seem to like explanations. If you can explain your system with well-understood operations, it's boring and "not novel". But when Google publishes magical mystery vectors, they lap it up.
But there have also been systems that create vectors with that property more directly instead of as a side-effect of a neural net. Examples include Stanford's GloVe, or Omer Levy's Hyperwords, which is made entirely from old-school ideas such as mutual information and dimensionality reduction, and was for a while the best system if you limited it to fixed training data (still Wikipedia).
A gripe: reviewers don't even seem to like explanations. If you can explain your system with well-understood operations, it's boring and "not novel". But when Google publishes magical mystery vectors, they lap it up.