Honestly honey, you need to stop talking to him.
No amount of begging, rationalizing, pleading, guilting is going to push him away from his other woman. If anything, that is pushing him and her closer. They are bonding over your pain. You are their common enemy. You begging is "affair fuel" for them.
Trust me, when you go NC, file for divorce, SEEK TEMPORARY CHILD SUPPORT and stop answering his calls, texts, email about anything not child related.... It help the waywards see what they are about to lose.
YOU CAN NOT COMPETE WITH NEW LOVE.
She is shiny, fresh and new. Everything about her is "perfect". He can not see the flaws until he realizes you are gone. You need to "disappear" and let him face reality with his new toy.
I am willing to bet that if you gave them 6 months of no drama, no contact, no issues- he will come back when he realizes he doesn't want to be a father to four kids that aren't his.
No contact except for kids and bills. No more initiating talks to him unless you really have to. No more responding unless it is about kids, finances. No more pleading, begging, or rationalizing. He is in "love", nothing you say makes sense to him and is only pushing him to his affair partner more.