Well, that kind of reply is just rude, and personally I'd probably ignore it and leave the meeting declined. If it comes up later, "Your reply about the meeting? No, I don't think...oh, I did see it, right. I assumed there was more info coming; I'm totally focussed on $X right now, you know."