I was in a similar boat. I have always had lower than my H tho, but the A added so many new layers of issues around sex.
First I had to pinpoint what my issues were. For me there were several I had to work thru one at a time. For awhile it was the feeling of being compared to the OW, I hated feeling like I was in some sort of competition ( it was in my head but it felt real ) talking it over with my H didn't help b/c of course he denied it and I didn't believe him. What ended it helping was talking to my IC about it and journaling about it. Ultimately I had to shift my perspective, I knew the OW but I think my view applies to most AP, it's not real, they put on an act to elicit a response. So if I wanted to I could put on the most awesome act ever, I'm capable but I don't want to live like that, I don't want to almost trick my H into thinking I'm some sex goddess, so I decided I would be ok with being who I am, maybe I'm not whatever she was in that department but I'm better b/c I'm me and I'm honest and I"m real. I can live with that. So that helped.
Mind movies, I had to power thru that. I'd would let my H know what might happen and I explained why and I asked him to just hold me and be there for me. He did and that helped. Ultimately I knew that we had committed to R so I had to get to a point where I didn't cry the whole time, it wasn't good for either of us. For sure, time helped b/c as H did kind things and was supportive I was able to focus on those good things and keep my mind away from the bad during that time.
the last thing I can think of was resentment, a part of me felt like he didn't deserve to get me, like he could treat me so badly and as a reward he gets to have sex! That was tough. I think that is part of the bigger issue of R, I think many of us have some issues with giving them a chance after what they have done and we have to make peace with that in our own way. for me it always comes down to I want to be married to my H, I love him and if he is treating me kindly and respectfully and lovingly then I will stay. He used up every chance he will ever get from me with his A.
I guess the summary is fixing our sex lives after A is going to take work on both of your parts, I don't think it can just fix itself. Pinpoint where the exact issues lie and then do the work, with your H and probably your IC, to work through the issues one at a time.