Read your D paperwork carefully. Thee may be a clause about moving more than 50 or 100 miles. There should also be specific information about visitation and holidays. Follow it to a T. If it specifies he gets visitation 1 day a week and every 1st and 3rd weekend, then that is what he gets. If he wants the kids at another time, too bad, so sad. If he is supposed to give you a month notice for summer visitation, then stick to that boundary.
Moving without telling you (or giving you no notice) was a dick thing to do. A cowardly thing. Conflict avoidance to the max.
My X moved overseas without tell me or DD (who was 15 at the time). I figured it out almost immediately but DD did not. But when she found out she was pissed. (note this asshole previously got M and did not tell us )
What your conflict avoiding X does not understand is that there are consequences to his actions and no about of avoidance will make them go away. If he wants to change the visitation schedule, then he can get a L and ask the court to change it. Thing is, conflict avoiders rarely do this because they will then have to face the conflict. Until there is a new court order, follow the current court order completely.