I told him that my expectation of him is that he is allowed to say anything or talk about anything he wants about me with X. If there is anything that I don't want him to know, then he won't know about it.
That's a great way to put it. My girls are only 6 and 3.5 but already feel at odds about what they're 'allowed' to tell each of us. I've told them if it's true they can talk about whatever they like if they want to. I don't ask them to not tell their dad stuff and I don't grill them about him. They mention things and I try to focus on the 'them' in the story.
My 6 y/o calls her dad a sticky beak because he grills them about everything. We don't talk about what he said or what she said, we talk about how it makes her feel. She already notices these differences - how I wish she had nothing to notice but it does reinforce that I've chosen the right path here.
I'm actually not interested in what he's goes on but there are things they say of their own volition that concern me as a mum. I made a decision early on to stop trying to deal with this stuff with him - there is nothing I can do. I can only help them navigate it as best as is possible.
I think that's what you've done for your son here.
It certainly is a learning curve. Not one I wished for myself or my girls but it is what it is. I'm up to the task.