You're not sabotaging anything - HE sabotaged your marriage and you're just trying to clean up the mess! No matter how good the times can be now, NOTHING can change the fact that your vows have been violated and heart shattered.
I find myself opening up his phone records all the time, and it's like suicide. I don't know why I do it; I should delete them forever. But anger is safe. And I know that for me, when things are good between us, I'm not letting myself completely embrace it because it makes me feel vulnerable. I don't trust the "good" anymore because I don't even know if it's real!
I hate saying it because I remember hating when people said this to me, but it is still SO early. Don't be hard on yourself. Don't worry about attacking him right now and needing to feel the anger. Your anger is righteous indignation. It's a natural response to being hurt. You need to feel the anger or you won't heal. Scream if you have to! He needs to know how badly you've hurt him!