I understand what you are saying. The whole situation is FUBAR.
Although I am a man, my situation was reversed. I nurtured our young children. My wife had the A.
So, I understand. My assessment is this.
Your H and you are very different people.
YOU:
You value family. You value being married. You commit yourself to your children. You are comforted when your family is together, even though nothing "exciting" is going on. You gain strength from a happy, growing family.
YOUR H:
He enjoys the good parts of the family, but doesn't give during the hard parts. Likes being married, until there's something he doesn't like. Misses the excitement of being single. Is depleted by the responsibilities of the family. He seizes an opportunity to have sex outside the marriage, with less concern than a teenager kissing someone other than the present they are going steady with.
So, yes, he's selfish. But, I think it's much more than that. I believe he values marriage and family less, selfishness aside.
I will NEVER……N….E…V…E…R, understand how someone can casually have a marriage-ending affair, followed by a complete lack of gut wrenching remorse, when they have a 2 year old and a 4 year old at home, not to mention a loving spouse.
Marriage vows are short, but they are powerful. Sickness and Health. Better of worse. Forsaking all others.
Makes me want to go out and have my own A. Then maybe, I too will be just as fun and carefree as the "other" women.
So, he has dealt you a hand which contains a bunch of shit, but at least it contains your self respect. You have done nothing wrong. He has. Whatever you do, don't throw away your self respect card by having an A on your own. It wouldn't hurt him, in any case. It would just validate what he did. It's natural to think about it, but when you think it through, it wouldn't do anything constructive. The opposite is true.
He had better be begging with every ounce of his soul for you to give him another chance. If not, I'd drop him like a hot rock.