I wonder if it is not his mother but the OW who is willing to watch the child...?
Is she still in the picture?
His angle might be to ride off in the the sunset with his insta-family, but never under-estimate the length some will go to to avoid paying child support.
Have your daycare write that up now. And if you haven't yet retained a lawyer, start doing consults now (the first might not be a good fit! I meet with 4 before I found a keeper!) You need advice on how to CYA, just so you don't walk into a trap he might set.
If he asks for the child's pediatrician's name - document the question. Keep to email, or text for that reason - documentation.
I wonder if you could have the marriage annulled, given his deception... but as to custody, if he has only ever seen the child for visitation 4 days a month - he has a uphill battle.
btw - I could be wrong about this, but when I read former betrayed spouse I matched it in my head with a former wayward spouse... as in a couple that is reconciling... which was initially confusing.
And be aware that if you do move, the court may state that you have to pay the child's travel cost, not sure how that would work under these circumstances - another question for the Lawyer...