Gently, one of the things I learned on here that proved true for me is that if your gut is telling you there is more, then there usually is. At one point in my story, the "more" was WS's ability to admit that she probably did more than she was able to remember and revealing the ways she probably interacted knowing the space she was in at the time. Before that she insisted that even though she didn't remember, she didn't initiate or fully engage in the sexual discussions she had with her AP. It was difficult to hear, but a relief.
You are not expecting too much, but you might be expecting too much of him right now. If what you expect is what you need to heal and he's not willing or able to do it, then you have some difficult decision to make. How long are you willing to wait for him to "get it?" In order to R we need to recognize, express and expect what we need to stay in the relationship. He needs to know what your deal breakers are, and truly know that you will hold him accountable. If he doesn't change what will you do?
You mentioned that he's been distant "lately." What was he like before then? How long has it been?