No, you are not pushing him to the edge! You are allowing him to manipulate you and define your reality. Do not let his pity party derail you. You are entitled to the truth and living in the fun house (the one with the distorted mirrors) will make you feel crazy! He isn't being truthful, he hasn't dug deep to find the truth. He just wants you to drop it and move on. If you do that, there will be more of the same in the future. This isn't a good dynamic for you. Could you limit contact with him? Start really working on yourself and figure out why you let your SO define your reality?
Honey, you can't keep things going the way that they are. You are going to have a breakdown.