I knew to leave when that sounded so much better and healthier and easier than sticking around and trying to fix the broken disgusting pile of crap that was my marriage (that and a million other reasons, but who's counting..). Only you can decide when it's time to pull the plug..
But I wouldn't let your fear of being lonely stop you from leaving. I'd MUCH rather be alone than constantly wondering whether I was being lied to or manipulated..
And you never know what windows will open when you finally shut and lock that door..
You say you're unhappy, so it really isn't a lose/lose. It's a lose if you stay and a possible win if you go..
Honestly, the fact that he was searching for men as well tells me you are never going to be everything he wants and "needs." I don't have a problem with homosexuality or bisexuality, but I do have a big problem with people who are not monogamous after they have promised to be..
Big hugs to you.. ((((confused))))