My parents had a temple marriage - I'm 5th generation, born in covenant.
My father had repeated affairs. My mother had 1 revenge affair (that I know of). They were married for 22 years before they separated, and ultimately divorced. My father never rec'd any type of church discipline, and my mother stopped attending after becoming the painful brunt of gossip.
My father's father also had repeated affairs. Eventually he left his eternal bride (who was in a wheelchair due to debilitating RA) and their 3 almost-grown children for a much younger woman -- whom he eventually converted, along with her 5 children.
He was excommunicated initially, but returned after a year. You wouldn't believe the amount of praise he rec'd from church leadership for bringing her and her children to the church. He even made the front page of the Church News (this was in the 1970's)!
Never mind the family he left behind to do it.
My experience with the church and infidelity is that it's easily rugswept.
I've since left the church, and I married a man who was never a member.
There are countless Christian leaders who have fallen into the trap of infidelity. Countless Mormon leaders. Countless Atheists...school teachers, doctors, and other professionals whom people would expect to adhere to a a monogamous moral compass.
Personally, I don't think it has anything to do with religion or spirituality.
It has everything to do with the inability to set personal boundaries and adhere to them.