Divorced my N in 2010, still think about him!

Finding this website this week has helped me understand what I am going through, which even after years of counseling I have never been able to understand. I met my N in 2007, married him, and then in 2011 finally had the courage to leave. I feel like my N has destroyed me as a person, who I was before I met him.
I was 25 when I met him, he swept me off my feet. We were inseparable, best friends. He showered me with love, gifts, trips, everything you can think of. We moved in together and got married within a year, and then the hell began. Nothing I did was right, I was belittled, called names, none of my friends were good enough. I was never allowed to talk about children and was told if I were to get pregnant I would have to get an abortion immediately. This from the man who in the beginning told me he loved kids and wanted a family. We bought a fixer upper, which I thought was to fix up for our home, but after a few months found out he wanted to flip it and start an investment business. When I tried to give my opinion, it was never allowed. I wasn’t allowed to see the checkbook or credit card statements. It was like I was supposed to just do whatever he said. I was a pastry chef which wasn’t good enough, so I went back to finish my bachelor’s degree, and once I finished he told me he didn’t want me to work, I was to be a stay at home wife. If I mopped the floors I was berated for not ironing first. Nothing. I. Did. Was. Right. In return, I became resentful, and never felt interested in having sex with him. This would cause daily blow up fights which would result in one of us leaving. I remember at one point sitting in a park alone thinking I would die by the time I was 30 from stress if I didn’t get out of this. To make matters worse, his family was the same way, manipulative, belittling, and argumentative. I was not raised this way which made everything that much worse. Most people would say I was crazy for staying, but it was like a roller coaster, some days were amazing and euphoric, and the bad days were BAD.
We attempted counseling with 3 different counselors. At each session my N would tell the counselor that it was my fault because I have anxiety issues. Avoidance was his favorite game. His family was the same, they never did anything wrong. It was sickening. After every counseling session we would fight. I finally got the courage to leave and stay with my parents, which I wound up going back to him 3 different times before I left for good. I moved back to my parent s in March of 2011. Never got any of my furniture from the house, I wanted to cut all ties. I realized if I saw him I would want to try to work things out and I knew it was a viscous cycle.
Later that year I met my future husband, who treats me with respect, and we now have a child together. Things have been great, except I still haven’t been able to let go of my N. I felt like he was the love of my life, but that he destroyed our relationship. I am filled with regret for every letting myself marry someone like him, or be absorbed into his crazy world. Before I met him I was happy with very little effort, now I feel like I need so much more to be happy even though I have a wonderful husband. It is all very confusing. I agree that holding onto memories is a way to hold on to the person, and I do want to let go. I think part of it is that he will never understand what he did or how he is, and that drives me crazy. Help!!!