I'd call his bluff. He'll get laughed out of court. He's an ass. Just walk away, with confidence, and say do what you have to do. My bet is, he's all hot air.
WH says marriage is over: May 15, 2009.
EA#2 July 20, 2009. Legally sep: Aug 16, 2009. DIVORCED!!!! Signed Nov 23, final Dec 24, 2010, adultery listed.
Next time he came to pick up DD, I looked him right in his fucking face (and I NEVER talk to him, ever, even during handoff of DD) and said "You better cut this fucking shit out about how you're going to take her away from me. You hearing me?" well, he was shocked and just mumbled something about how I need to be reasonable, etc. etc. So far he hasn't threatened it again.
He's all talk. Don't give in and don't let him know he upset you. Keep waiting for the court notices that never come.