Feeling guilty - why IS that?

After a month of NC, the ex N contacted me Friday and asked me to dinner. I had plans that night (I think the Universe was speaking...I was headed to my first Al-anon meeting when I got his text). I responded that I had plans and wasn't available. He replied that he and some friends were doing karaoke on NYE if I wanted to join. I didn't respond. He contacted me on NYE and said a mutual friend was asking if I was coming to sing karaoke? I responded that I had dinner plans and would not be available. A few hours later, I got another text saying, "Please come. We need more good-looking girls here." I responded that I was already home and didn't want to get back out. No response and nothing today.

So, my question is this: Why on earth do I feel so guilty today about saying no to him last night? Why do I feel WRONG about doing something that was RIGHT for me? Is my mind still warped from two years under his control?