Stuff like this is hard bc APs lie to each other just as much as they lie to us. So, it's possible that he just said that to shut her up. But I wouldn't give him that easy out.
I think I would just show him the quote and ask him to explain it to you.
WH had a conversation with OW where he told her he would never leave me. He said it was very important to him to make that clear (gee, what a stand up guy ) Anyway, she said, Yeah, but what if SHE leaves YOU. And then he told her he would. She also made him promise that if they got caught they would continue to see each other.
He dropped her on DDay like a hot potato and never looked back. And I told him he could leave with impunity...I would never disparage him, keep the kids from him, etc. That if he in any way whatsoever didn't want to be there, then I wanted him to go and I would make it easy for him.
He didn't go.
They lie to each other.
Don't know if this was one of them or not, but in any case he owes you an explanation.