You do not have to stay for abusive interactions.
I would probably create the boundary of "I will talk about anything with you as long as we are able to have healthy communication. I understand I have caused the rage but ai don’t think it makes either of us feel better when it escalates like that. I am going to have to take a step back and let you get more under control but then I will be back to discuss anything you want.
And you do that when it gets that way. And later, go back to him and say " if you feel calmer I am here to talk to spend time together or whatever you need now."
This way you are providing you both more safety, but providing reassurance that you are willing to have difficult conversations as long as the communication can be safe and healthy for both.
Be there for him, answer his questions, even bring up the affair for him when you can.
I say that last part because sometimes the bs feels like a burden to keep bringing up these difficult conversations. By doing that task for them, it like a little pressure release valve.
These two things done together provides assurance to him that you are willing to do what it takes to work through the issues.
The only way I wouldn’t do any of that is if you feel leaving when the conversation spirals if you think he could be violent. And in that case, there is a bigger issue and you need to take other steps. That doesn’t sound at all like what you are saying, but I think it’s worth stating.