I'm sorry you find yourself in this forum. Posting here for the first time takes a lot of courage so well done.
You are now a madhatter, when both spouse's have cheated in the same relationship. There is a thread in the ICR forum specifically for those with MH status.
I think the first thing to do is figure out if your boyfriend's A was a dealbreaker for you. It doesn't sound like you've healed yourself from the trauma of his infidelity, get into counselling and work on yourself. If you really can't accept it (I don't think anyone ever gets over it) and heal from it, then you need to tell him that you're done and end the relationship.
I am surprised at your decision not to confess. Being able to see the BS side of this means you should get why it's so important that your boyfriend knows.
Counselling will also help you figure out why you cheated. At the moment you're blameshififting onto your boyfriend. It's not his fault, just like it wasn't your fault that your boyfriend or your XWH cheated. That stuff works both ways!
There is no such thing as getting even. An eye for an eye and the whole world goes blind, right?
Infidelity is unfair. His cheating is unfair and so is yours, you haven't settled the score or made things even, you've made things a million times harder. But you already know this deep down.
As for the AP, do you know for certain that he is single? There might be a wife or a girlfriend, in which case you are the OW in their relationship. How do you feel about that?
You know that affairs are built on lies and none of the feelings are real. You state several times that you are not a cheater and this is out of character, do you really see a future with a guy who would actively help you to betray yourself in the worst possible way?
You have a very hard road ahead of you, painful self realisations, hard work and changes to make. But we are here for you, keep posting.Me - 27
Him - 27
My Ddays - Jan 2010 & 12/04/14
His Dday - 23/12/13
Chin up. Unwavering. Fight. I can do this.