I have to agree. I had a love affair with unicode put directly in LaTeX markup (delta, integral signs, element-of, etc...) and it was very fun at first. Then I had to send the paper to a collaborator while writing a follow-up article together. I ended up removing all the unicode, and in subsequent work I didn't do it any more.
Why would you want to put math symbols in LaTeX using unicode? What would be the advantage over using a package like amsmath or even the native math environment? I'm genuinely curious.