Oh, I detatched from my FWH for some time. I had to, for my own mental health. He knew I was doing that too. I had made the decision to be there, while I detatched, because he was so clinically depressed that I was actually not convenced that he would not kill himself. So while I stayed, to work on R and keep tabs on him, I was also not fully present for him and I really didn't give a rat's patootie about that. He was left in limbo-land, wondering if, when his ADs kicked in, I would pack up or pack him up. And to quote your WH, it wasn't my problem.
And it's not YOUR problem if your detatchment makes him feel uneasy. So Sad, Too Bad. If he complains about you doing what you need to do to keep yourself together, being the somewhat nasy person i can be, I'd probably throw that phrase back in his teeth.