Honey, I'm so sorry you have found yourself in this nightmare.
First off, honey, he's not remorseful, not at all, he's in damage control because he got caught. He wants a wife and a girlfriend. His words are just that, words, his actions tell the real story, he's still contacting her. You can bet at the first chance, he'll see her again, and not just over the fence.
I'm going to bump Katherine41 post of "20/20 hindsight, what I wish I'd done". Please look for it and read it, so much wisdom there. Read it and implement it, save yourself from further pain.
Knowing what you know about his continued contact with the OW, can you get him to move out for a while, to allow you space to think about your future?
If she has a BH, please, tell him immediately, now, today. Don't tell either your WS or the OW that you're going to, just do it. He has a right to know what's going on in his marriage too.
Don't think you are harming another family, that was already done by your WS and the OW, you're just taking care of business, managing the consequences.
Please, find and read the 180, read it, live it, it's designed to make you stronger and better able to make decisions for your future. It's in the healing library, top left hand corner of the page, if your marriage is to have any chance of surviving, you have to control the outcome from here on. This is all about you now honey.
Hugs honey, lots of hugs, you must be in so much pain.