I'm sorry. That must feel so terrible.
H and I are HB but not at the rate you are (!), so H is keeping up. However, back when H was lying to me about his PA, there was one night he couldn't perform. I was very suspicious of a PA (H had admitted only EA at the time) and it was devastating to think that he wasn't turned on by me, that he was thinking of her, blah, blah. He now says it was guilt from what he did and that he was lying, but who knows?
You say your H is depressed. I know when I'm depressed, I'm not feeling too sexy. But this reaction when you are feeling so insecure bc of the damn A is extra difficult for you. But I don't think it is really you he's rejecting, even though it feels like it is. His problem is with himself. Have you tried holding each other instead of sex? For me, the HB is about 2 things: 1. reclaiming my space as his sexual partner (feeling wanted); 2. feeling secure and safe. What if you tried physical closeness but without the sex? Maybe that would put him more at ease and the sex would come more naturally?