He has a history of cheating and he's getting frustrated with you? You need to heal. You need to talk this through with him and he needs to be remorseful and empathetic to your feelings.
Is he doing any work on himself? What will prevent him from cheating again?
I think you're not feeling connected to him because you haven't been allowed to safely work through your feelings. It's his job to create a safe environment for you to do this. And it's certainly his job to work on fixing himself before he expects anything from you!
"That's the thing about pain, it demands to be felt."
What he did is considered cheating, right?
He has not voiced his frustration.
Like I said in my other response, he needs to make a safe environment for you to feel comfortable talking things through with him.
I'm glad you're in MC. Is he owning his shit? Is he questioning whether or not it was cheating if no PA? Are you talking about your feeling of disconnect with him because of your unresolved feelings?
It takes time, but please don't rugsweep. That never works. Sometimes we pretend things are okay because we so badly want them to be.
he's saying all the right things.
To answer your topic title question....no, you are not being too hard on him.
As Authenicnow said rugsweeping does not help. He needs to deal with the why's of this and his past PA. If not you will most likely be hurt again.
You need to establish clear boundaries, and requirements of behavior from him for you both to heal and get past this.
It's ok to feel a disconnect right now. He needs to understand that he broke your trust again. He needs to deal with the consequences of his actions. Is he being remorseful for what he did, or is he just sorry he was caught? This is a fine line, but a big difference in so far as successful recovery.