If he doesn't communicate enough for setting up a McDonalds meeting, something else you can do is "at the curb". Be outside with your DS at the curb when he arrives - bring out his toys, lemonade and snacks for some "front porch time" if you have to. Give him the child while outside. Then go inside and lock the door. He can play there, observed, for the 4 hours.
What he's doing is peeking into your house to spy on you, getting all the way in your shower to see if you've been sharing it, what pills you may take or not, or simply just showing you "what a big man he is" (because he's an idiot and he feels he has to have one up on you inside his tiny little snake-mind). It doesn't matter whether or not you're in a NB, seeing anyone or anything of the sort. It's his dysfunction, his detachment problems and it shouldn't be allowed. He needs to drop the codependence. And for that he needs limits.
If you have to think of it like this, you'd actually be kind of doing him a kindness by pushing him away. It's no longer appropriate, and he knows it at some level. The faster he gets it the easier it will be for him to move on (and incidentally, get another source of NSupply, which is good for you).
He is D now and he's not your friend, he doesn't get to enter any part of your house, let alone the privacy of your rooms and bathroom.
Heck my own D isn't final yet and I feel I'm intruding even when I must drive by her cross street!
Strength,