I also struggle with the fact that my WH knew that my father's A shattered me and my family. My dad left us for his OW, and I never wanted the same thing to happen to my kids.
However, the big difference between my WH and my dad: WH has remorse, he's doing the work, he's being everything that a recovering wayward should be.
When the thoughts of my WH being just like my dad creep in, I tell myself "They are NOT the same person, AON, your dad was a cheater and a coward, he ran rather than face the music; your husband is doing the opposite of what your dad did, he's working hard to repair the damage, he's facing his demons!"
Sounds a little nuts, but I say it out loud, and like it's somebody else knocking some sense into me. It works!