First of all, respond to her facebook message to you. Tell her plainly that this trip is for you and your partner only because the two of you need some time alone. Anyone would understand this. Don't discuss it with your partner first. Just tell her that and see what happens. See if she objects and see if your partner objects.
If she objects, they are either in a PA or else in an EA and she wants it to become a PA. If he objects, they are either in a PA or else in an EA and he wants it to become a PA.
As for him going to her house while you are at work -- I'm sorry to say it, but as far as I'm concerned, there's your proof that it's a PA right there.
Your mom says ignore her and go anyway? Ask yourself, though, which would make you the most miserable, going and being tormented throughout the entire vacation or putting your foot down that she not go.
His comment about not telling her to go because it's a public place is very telling. Yes, it's a public place, but if she knows you want it to be a vacation for you and your partner, there could only be one reason for her to go, and that is to spoil it for you because she wants him. He knows this. To me, if he stands his ground on her going, that would be enough to tell him to go with her. Then you can take your son and go elsewhere for a camping trip. You'll enjoy it a lot more. Once you get home, pack his stuff, set it outside and be done with him. You know he lied to you about her guessing about the camping trip. You know he told her. Why would he lie about it unless they were in an affair?
Your mom may be trying to help, but it is impossible for anyone to understand what you're going through unless they've gone through it themselves. It isn't as easy as ignoring her or waiting until the end of summer to do something about it. It sounds like she doesn't realize that for you to tolerate this situation for the summer would make you miserable. If she realized it, she would give you different advice, but she just can't understand. For you to tolerate this would only wear you down further, and by the end of summer, you wouldn't have the gumption left to do anything. You would be in a state of depression. Don't let that happen.
And he does have a very explosive temper. I know if I put up a fuss he will get mad. I've gotten into a bad place where I shut up and not say anything to keep him quiet. He is very much like a 15yr old in a 44 yr old man ... He has worn me down and I can't do it anymore.
So what if he gets mad?! You have to come to a deliberate decision that you do not care if he gets mad.
This is emotional abuse. Getting angry is a way to manipulate you into doing what he wants. In this case, he wants you to keep quiet and just be happy watching him cheat on you. Don't let him wear you down.
You don't have to get mad at him to confront him. Simply tell him calmly that you know he's in an affair and you want him gone. Simple as that. No arguing, no theatrics, just gone. His response will shock you.
Is he physically abusive to you?
If so, it's needless to say that you need to get away from him now for good. If not, when he gets mad, it is not true anger you're seeing and hearing. It's a show, put on as a control tactic, to control your actions. Don't let him get away with it. You need to get firm and tell him it's over now and tell him straight up that you are not afraid of his anger. If you are not ready to end it with him, at least stand your ground about the two of them going No Contact in any way, shape or form.
You can't control him or what he does with her. You can control your response to it. Give him an ultimatum and be ready to stand by it.