This is one of the toughest things to resolve....what is a "healthy normal" after adultery.
Simple, everyday interactions such as your neighbors laid back gathering become a huge "what if" struggle.
I think we can all relate to this.
My wifes affair involved another small business owner. She met him through work and texting was done a LOT while she was working....something that I was not encouraged to do because my wife was working and it doesn't look very professional for her to be texting, nor did she have time TO text while working. So what she considered "normal and acceptable" in her role as a spouse was very different than what was "normal and acceptable" as an AP. Truth is....there was more room for interaction between her and I while she was working, but her interaction level with her AP DID jeopardize her professional career. Point is our pre-A "normal" was not healthy, nor was her activity in her A "normal". A new normal must be defined.
This is you guys defining what "normal" is for you guys now. I suspect your pre-A "normal" missed the mark of being healthy too.
NOW my wife is growing her business and is active in a small business group. Pre-A blakesteele would have rejoiced over both her business growth and her professional involvment. Post-A blakesteele has much anxiety over this. Her meeting one on one with other male business professionals is very tough for me to deal with. She assures me with words that she will NEVER act inappropriately with these other men. Words that are necessary but do not "complete" the need for security within me now.
So I get your anxiety.
Best advice I can offer you is to find ways to sit with your feelings....find the strongest, most painful feelings you can regarding you attending this party. Sit with those as long as you think you can stand it. Then sit with them a bit more. Then find ways to express the core painful feeling you have to your husband and stop there.
Make sure they are primary feelings....anger is not a primary feeling. Anger masks primary feelings.
Stop well short of telling your husband what he should or should not do. He is a big boy....trust that he can hear your feelings and find ways to interact that validate and connect to you with this information.
In other words....express how this sitch makes you feel, then allow HIM to decide what HE can do about it.
Caution: Don't let him take the easy way out by putting a question to you....don't accept his response of "Well, what would you like me to do about this?" That puts the burden of meeting your needs back on you. You need to express those needs and then he needs to figure out what he can do to tend to those needs.
This is NOT a game. This is what therapy and books have told me time and again to do. Often you might think you know what you need...but if you consistently express needs AND tell others how to meet those needs you will miss bonding at the true deep level we all desire to bond at. Plus it stands the real chance of creating resentment in the relationship.
Resentment in you that you HAVE to tell your husband what to do.
Resentment in him that he is being told what to do.
The key is to not expect him to read your mind.
You express a need.
He responds to that expression of a need.
You express the feeling his choices evoke in you.
He responds to that expression...
....and so on.
He has to do the same thing.
This is the dance of intimacy.
As a former CoD I know how difficult this is.
Keep the faith.
This is a good opportunity to pracitice the "new normal" you are trying for!
God is with us all.