Any SIer will tell you that exposure kills affairs. The OW's husband deserves to know what he is dealing with, and who his wife is.
I understand your concern, but actions have consequences, although physical violence is certainly not the answer.
However, there are things you can do to protect yourselves, and vigilance is key. Be aware of what's going on around you when you two are out and about, and if he turns up at your home, don't answer the door, call the police.
You're right Cali, personal safety is important, take it from someone who has an unstable OW abusing her every day. That said, I don't regret letting her BP know about the A for a second.