Your absolutely right MovingForward88, he has had the easy part of your relationship and sounds awful to say but he probably always will when it comes to matters of his adultery. People who have affairs have little empathy for anyone but themselves, he will say the right things for awhile but he is not truly sorry and never will, he just wants to put the mess behind so he doesn't have to deal with it anymore.
Several years ago I was in a situation with a female co worker who started to become attached to me and hinted she wanted more. As attractive as she was I would have felt awful if my wife had found out and didn't want to risk the life that I had. Years after that my wife in a similar situation chose the affair because there was nobody more important in the world than she.
My father was a serial cheater, my poor mother would always find evidence like ticket stubs or earing's in the car that weren't hers. But she always forgave him and he never stopped until he became to old to cheat. Now him and my mom are still married and they get along OK but what kind of happiness can they truly have with all that baggage behind them.
I got divorced months after my dday and there were times I wish I would have worked it out with her, because it does get lonely. But I always think of my mom and don't want to spend the rest of my life waiting for the next evidence to pop up.