As the OW I can relate to your feelings of hopeless guilt. I tried to apologize to my APs BW too soon after D Day and to say it didnt go well was an understatement. I think I was just trying to make my guilt go away without really feeling what I was saying. And she knew total BS when she saw it.
It is especially tough if they are trying to reconcile which makes you the big bad wolf in all of it.
I am now over 2 years out and I still struggle with the guilt although it comes and goes now. But sometimes it hits me how destructive my actions were and I literally just want to curl up into a ball and die.
I too celebrated their pregnancy (while secretly wished a girl on them because I knew a boy would make him stick out the marriage) I too held their newborn daughter and wished for the hope that this meant we might be together one day.
Quite frankly my actions disgust me and I have learned that living with that guilt is just part of the consequences I have to pay.
Probably not the answer you wanted but just letting you know that I get understand your conflict.
Me - WS
H - BH
D day - July 2011 after a 4 year relationship with OM
Reconciled and renewed our vows on our 22 Anniversary in June 2012