This is normal.
It's because we truly did believe our vows. It feels wrong to finish the relationship we said would be for life.
And not to sound legalistic but it's not our breach of contract. We're off the hook after infidelity in most religious/moralistic practices, and we were never on a permanent hook legally in the first place in most of the Western World for any reason.
In fact, what you're doing is what he should have done if he had had the guts to finish one relationship before starting another.
Trying to have the cake and eat it too (which is what they do when they say one thing but do another) is worse, and should actually remove the guilt. In my STBXWW case, she can go find OM and ask him for money now if she's running low I will pay my monthlies to her on time because that's my moral character, but not a penny more. If she wants something sentimental, she better change and make it verifiable first, then we can talk about how to rebuild trust.
@Pass: Love your rock bottom tagline.