Welcome JSAM! I'm so sorry for the reason that you are here, but know that you have found a wonderful and supportive community at SI.
My story is similar to yours. You can read about it in detail in my profile if you are interested (click the smiley face in the upper right side of this post).
The one thing I would urge you to do is to not sweep this under the rug....thinking it was a near miss rather than a direct hit. The fact is that whether it was physical or not, it was an A.
I say this from personal experience. A year or two prior to my WH's EA he had a "questionable" relationship with another COW. My WH did not think there was anything wrong with it bc he was absolutely not attracted to her at all and supposedly she was a lesbian (or so she told him but then later said she was dating men again). I know that he really didn't see her as anything more than a friend, but I saw the texts between them and she was definitely flirty with my H. Long story short, that should have been a MAJOR wake up call to both of us - he had poor boundaries and liked validation from coworkers. Instead, my WH thought he did nothing wrong, and low and behold when the exact same patterns came into play with the next COW, he had a 5 month EA. He said he justified it as "just friends" but after they kissed he knew that he could no longer minimize that he was having an affair.
My one piece of advice is that you and your H read "Not Just Friends" by Shirley Glass together and discuss the book. It is excellent, based on scientific research, and an easy read.
Best of luck to you. Sending you strength and (((hugs))).