I am sorry for your frustration, R and R. I don't have much advice on the negative thoughts about yourself and his words, but Perv said similar things. He said he felt guilty not being with OW if he was where he should be and with "us" and at the house he built. He sure doesn't seem guilty about leaving us, the family he built and pregnant wife.
You know, though, I am learning very, very slowly that it really isn't me and I doubt it really is you, either. It's each of them and their own crap and they do and say hurtful things even after they've both gone away from us.
Yes, I get the venom also and OW is on a pedestal so high she may fall off. What I've heard about it is that it's because she accepted him at a time when he did really sh....things to other people in his life and so that makes her a gold star.
In your situation, you were the spouse, had the ties and WH had "responsibility" for you and any kids you may have...with the others, he has total freedom to come and go and maybe it feels differently when they do it? Ow in his case-your Wh-may be like the one Perv chose and be stupid and not question anything he does, so the praise goes up and when we question, down we go in their heads.
I don't know if that makes any sense, but it's happening here.
I'm sorry for your struggle too.
When the negative thoughts come for me, FWIW, I tell myself little things like pep talks. Like, "I didn't do anything wrong. I stayed the course. I fought for what I believed. I fought for family and future. This was done to me. I didn't do it."
And then I list things like accomplishments in a day or block of time and write them down, for it helps know that something good went on with my day even without him in it. I'm not stupid and don't have to depend on anyone at all for the most part and we can't let them trick us or the games trick our minds. We just can't, R and R, we have to stand up to it.
I liken it to a bully at school, of whom I had several. And I read. I read all I can on narcissism and passive agreession and why.
And yes, to your initial question, I think that we act or behave or speak differently among different people. I think for most of us it's not intentional, just relationship based.
We aren't the same people coming out these "situations" either and may never be again.