We told them that if they were willing to go to counseling with us we would give them an opportunity to have healthy communication. You can imagine how well this went over with NPD substance abusers.
It's a super long story, but in the end we just stick to our guns and take our space. I recommend giving them as little fodder as possible. If you can both get away with declining being in their presence indefinitely, I say go for it. Spend your energy on the people who contribute positively to your existence. You shouldn't have to apologize to anyone for not making time to deal with their crap.
I'll tell you, it's still really hard. SIL has said some AWFUL things about us, and needs to blame me for all her family's problems or she would have to face up to the fact that she has been drinking some messed up kool aid her whole life. (Her parents were furious when Crazz moved out at TWENTY FIVE years old. They all said I was a bad influence when I invited him to a baseball game, because THEY only watch football. Oh, and I was too good to get drunk at dinner.)
I get validation every time anyone else in Crazz's life says that they have watched him grow and mature in the last ten years and are proud of the man he has become. In every other relationship I was in the family seemed to love me. This was the first time I had a door slammed in my face because they didn't like that I had my own life that was nothing like theirs.
ALLL this being said, it still breaks my heart that my DD4 does not know her aunt or grandfather on that side. It feels so strange to me, but Crazz has more resolve than I do to keep away from them and their vortex of judgment and misery.
Your FWH may be uncomfortable about the idea, but he really needs to take a hard look at the long term here. People like this aren't going to change without big impetus, and your feelings alone aren't going to be enough for them. There's no fixing them - there's only getting yourselves to the healthiest space possible. Don't call it "getting rid." Call it "keeping their toxicity at arms length."