I don't feel qualified either but I also want to jump in with the others in support of you.
This is hard. It's like a war that you have rounded a corner on (reconciliation) but there is still much damage & destruction to clean up.
Can you get any time to yourself in the midst of all of this? Can you get any help? Friends-family? A support group is good.
Dont be afraid to ask yourself hard questions. You sound like you want to say this is a deal breaker. Yes?
Give yourself permission to ask all the questions from your husband & yourself. I am keeping a diary. It helps me keep track, I have to concentrate to do it, it makes me feel proactive about my situation. I think half the battle is figuring some things out. I'm not saying "do as I do" just I have found it helpful to me.
My heart goes out to you SSmile. I support you!