I'm sorry it's been hard lately. Sending hugs to you! You've been doing so well about so much of this...but I'm afraid you're doing the wrong thing now.
I think maybe this is the issue: both you and he are confused about the different duties of wife and mother. He has fired you as a wife, but he still expecting you to do some of the wifely duties. After you to decide not to be together, if you follow the traditional route of 50/50 visitation, your job is to give your daughter all of the love, care and support you can in the hours that you are allotted to have her. Your job is to show up at the scheduled times and to be reliable and responsible for her in the hours that you have agreed to. Of course, that doesn't mean you stop loving her or stop calling her or keeping in touch with her during that time, but she is not your responsibility during those hours.
It sounds like when you two were married you had an agreement that you would take up the slack when he had to have business meetings or go away for travel. That is normal in a married couple's relationships. It is one of the things you do for your spouse because your spouse does things for you. I used to do the same thing in my marriage, but when I got a divorce, I was no longer available to take up that slack. My ex used to have to work late (lol...actually legitimately a lot of the time) and travel, and although I didn't like him being gone that much, I accepted the inconvenience of not having much time for myself. That was a wifely duty, not a motherly duty.
It would be extremely unfair if he would expect you to keep performing that wifely duty when he has fired you from the job. Is it okay that he is giving the affection and attention to the OW while still expecting you to take up the slack whenever he needs it? Of course it isn't!
Please don't let your concern for your daughter's well-being weaken you on this point. If you married him and had children with him, he must have at least basically decent parenting skills. By stepping back and requiring him to step up, you are doing good for every single person in your situation. You are helping him learn how to prioritize his time and dedicate more of that time to his child. You are helping your daughter to establish a better more reliable relationship with her father. And equally importantly, you are keeping yourself from going insane during this time. You are setting standards and you're requiring other people to meet the standards. Well actually, you being tough about this is bad for one person in your situation...the OW. She doesn't love your daughter, so when your H had to break plans with her, limit their alone time and vacations, that's going to annoy her. Bonus points!
Every time that you fold and take responsibility for your daughter during the times that you have not already agreed to, you are allowing him to spend unrestricted time with the OW. You are allowing their relationship to blossom freely and without limitations. You are allowing him to choose the other woman over his own daughter. None of that is acceptable. If he's being selfish and childish right now, the only way you can stop this from happening is to establish a set schedule and require him to stick with it. That means if he has to travel for work he's going to have to tell them that he either can't travel or he's going to have to travel on the days when he doesn't have his child. Or he will have to find a reliable family member or possibly a college student to take care of your child during that time.
Eventually a year or two down the road once he has gotten in the habit of being responsible, you can revisit the idea of being more open with the schedule. That's how it was for me and my ex. We started out very strict so that he could fully understand what it means to be a real parent 50% of the time. He stepped up and it surprised me because he was so used to letting me be responsible.
Now that he is doing his job, I feel much happier and more at peace in my heart, and I am much more willing to accommodate him when he has special scheduling needs. He also knows that I mean business and he would never think of doing anything like scheduling a vacation without asking me first. If he did schedule a vacation like that, I would tell him that he needs to cancel it or rebook it because I already have obligations.
I agree with the other posters that what this means is you need to start having some guidelines. It's not the house that's really holding you together. It's that you are still involved with each other in so many ways. Conversations that you have and trying to deal with all of this childcare is too hard on you right now. Those things are remnants of your husband and wife relationship. Just out of curiosity can you not start the divorce before the house is sold? Why does selling the house affect anything?
Whatever the status of the divorce , as soon as his little vacation is over, I would sit him down and clearly state that things are going to change. I would come up with an ironclad visitation agreement and tell him that you expect him to fulfill his obligations as far as that visitation schedule. Tell him that if you need someone to watch your child during times when he has other things, that is now his concern to arrange safe, loving care for her because he has fired you from that job. And before you say,"Oh no! I could never let somebody else take care of my child when my husband is running off and leaving her!" remember, if you find some cool college student to watch your child for a couple of days, she's probably going to think of that as an amazing adventure. As much as we want to think we're the only ones that can love our child properly , unfortunately he is placed you in a situation where you won't always have that control in the future.
It will feel uncomfortable standing up for yourself in the first couple of months if you still tries to take advantage of it, but if you consistently and calmly remind him of the schedule and tell him that he is obligated to follow it, eventually things will change. He will begin to understand that it is no longer your duty to take over whenever he has things come up. It will get a lot easier over time if you handle the situation consistently. Also, remind him that if he is not following the visitation schedule, this is something that will look bad in court when you do go for the final divorce.
Sorry this is so long, but this is something I feel really passionate about. One of the gifts that your husband has given you as a result of this affair is that now you have the right to have the private time for yourself that you probably never had before. Maybe you don't even want it, but it is going to be good for you. That is the time you can use to take care of yourself, to rest and relax, and to build those other interests that will make you feel like a whole person again. I can tell you I feel like a 100% better parent when I have had a little time away from my children. I am much more patient and happy.
So please don't be worried about what he thinks or the marriage counselor thinks. When it comes down to it, you know what is right. That is why this is bothering you and your heart so much. You know you're being taken advantage of and nobody likes that. Ultimately, standing up for yourself and setting some guidelines will help all of you to be healthier, happier , and to have a better coparenting relationship in the future