I'm sorry you're going through this emotional torture, which is what it is.
But if you want R it sounds like things may work out.
If you get him any gift at all, make sure it's Linda MacDonald's book, "How to Help Your Spouse Heal From Your Affair."
You have to be so careful how to let him know you'd consider taking him back. He's not safe by any means.
Maybe the affair became too cumbersome and not worth the effort anymore. But he still works with the OW. That means he has to demonstrate a willingness to prove to you that he ended it to your satisfaction. He has to come up with a plan to go NC with her, not too easy if they work together.
He lied to you for so long, it's not second nature for him, it's first. It's habitual and instinctual.
You've got to take a very hard line with him. Definitely IC for him. What the H was he thinking? NC and a timeline followed up with a poly. Full transparency for all phones and social media. He does the work first, you then decide if his efforts meet your exacting standards. You are no pushover.
I hate to bring these cruel things up, but he could have looked at the financial situation and decided it was cheaper to move back, find another friend and start the entire secret life again. He may count on you being easy and accommodating. He has to do the work to woo YOU back. If he doesn't, he was never going to be a husband to you, a faithful one anyway.
I would make him aware of the possibility of you finding someone and sleeping with the new man.
He probably never imagined his good wife in another man's arms and this thought ought to make him throw up.
Depending on how young your smallest is, maybe ask him if he'd ever consider giving up his parental rights so a new man in your life might adopt. Just to get him thinking.
He's got to start seeing you as the prize. He's got to be willing to work much harder to get you back.
It does seem to me like he might be the kind to give up when the going gets too hard. ( His kvetching about too late) He needs to buck up and become a man for once in his life.
As for himself, what single woman of quality out there is going to spend her time with a guy whose resume includes screwing a coworker for three years while married?
He's damaged goods and it's to your advantage if he knows it.
I hope things work out for you, but not at the cost of him getting the itch again in two or three years.
Make him do the heavy lifting. He'll come to like your new found authority. They always admire the one who has the power and isn't afraid to use it.
And you have more power than you think.