I'm not trying to tell you how to manipulate the situation, but if you are honestly feeling at all threatened (physically or emotionally), you may have grounds to keep him out of the house. You have to understand that you have alot of power in this situation to keep yourself from harm.
I'm not sure what the worse thing that could happen could be, but it's likely a misdemeanor that your lawyer could help you fight.
Not to make this about me, but I don't know why some WS's don't leave. It seems like most do - but it appears you and I won the lottery because mine won't leave either and it is infuriating at times.
Do no harm. But take no shit.
I'd make sure you get an emergency motion going for sole possession. That's what my lawyer did for me with all his bull he was pulling.