Welcome to the club no one ever plans to join. You will find many wonderful people here who have had similar experiences, they will offer their words of wisdomw, and share their thoughts and concerns.
I suggest you read in the library up on the left side of your page there is a tab. There is a ton of useful information in there to help you find your feet, and figure out how to navigate this shitstorm.
I would also suggest that you copy this post, and place it in the Just Found Out Category. There you will find many others that are new to their situation, and are more for figuring out what to do next.
Now to answer your question, is staying the right choice?
Of course only you can answer that, but it sounds as though your partner has done ZERO to heal from this, has not changed his behaviors in any way shape or form, and certainly does not place your well being at the top of his list. There is not wonder that you are afraid he will do it again, as Dr. Phil says the best predictor of future behavior is past behavior. His past behavior screams of his inablility to be monogomous with you.
He cheated early on, but you accept blame, because he needed closure? Really? How about he behave abhorently, and you allowed it because you wanted to hang on to him.
Then you find out about a second relationship, which led to a 3rd, but with the same person, and now a 4th with the other chicky he was texting, so he has had some level of emotional relationship/incident 4 times in the 4 years you have been together.
There are a few things that we all agree we need for a relationship to heal and be healthy after infidelity.
1. Honesty and Transparency - Is he giving you this? Is he willingly sharing his phone, his computer, and emails with you? Does he willingly account for his time that he is away from you? If not then you can hang your hat that he is not behaving, and is not being honest and transparent.
2. No Contact - He needs to establish, and maintain No Contact (NC) with these other women. He needs to this before you even consider trying to Reconcile (R).
3. The Wayward (WS) has to really do some hard work and look at themselves to figure out why they feel the need to cheat, what is broken in them that causes the desire to have this external validation? If he isn't doing this, then considering R with him is just setting yourself up for more heartbreak and abuse, and yes infidelity is abuse.
Honestly honey, I would say with the given information, and the fact that you are NOT married to this guy (and I am not one to encourage this when I see hope)I would run, run far, run fast, run hard. Get away from this toxic man, and then do some real work on yourself, to figure out why you think it's ok to allow a man to treat you in such a disrespectful manner.