big set back

I am so mad at myself! I had to text my ex about a bill. That was it. Well things started progressing from there. I stupidly told him I had been on a date. In his twisted mind he wanted to know all about sexual details! He said I sounded happy that he should just leave me alone to do what I needed to do. Then he asks if we can do something sexual one more time. I told him I would be out of town. He calls the next morning and says I stayed up all night and feel like I should just leave you alone to be happy! I just don't get it. Are these more mind games. He dumped me in the first place and doesn't want a relationship with me so why does he do this. Of course it is my fault too to be manipulated by it. ugggggg!