Justsomelady, I'll write a bit more of things I wish I knew a long time ago. It might or might not apply to you, no worries if it doesn't.
I'm thinking about what SaddestDad wrote. It is important that you are honest, truthful, and don't lie, hide, minimize, deflect, or blame him in any way. When I wrote that you should take care with your words, I didn't mean in ANY way to not be honest and truthful. You must be, even if it is hard for you and painful for him.
Here's where I find it a bit tricky, and I would welcome any BS or WS opinions because I think about this a lot. The truth is easy - the facts of the situation are clear. You and your co-worker engaged in conversations and interactions that you would not have done in front of your spouses, on a regular basis. You began to look forward to those interactions and you are worried about what that means. You are stopping the interactions. This has happened before, and you want to really understand and repair what it is in you that led to it.
But being honest is harder. You can be perfectly honest and say things that lead to a misunderstanding of the truth. For example - this is from my own experience - I said "the affair had nothing to do with you." What my husband heard was "you are so unimportant to me that I will engage in activities that are extremely hurtful to you without thinking of you." Ouch. What I MEANT was "The affair was not your fault, you couldn't have prevented it and you can't fix it BUT you and our marriage are SO important to me that I will do everything in my power to change the parts of me that led to the affair and keep you and our marriage central in my life."
There are a bunch of threads in General that are "stupid things waywards say" or variations on that and many (not all) of the stupid things are things that I think were honestly meant but coming from a place of very distorted thinking and misunderstanding about how they would be heart. I don't mean to suggest that you lie, minimize, or hide to spare his feelings, but think about how he will hear it and make sure what you are saying is fully truthful, and drill down into what's underneath statements where you mean one thing and he hears another.
Another thing to take care with - your feelings are liars. You may honestly feel something but it is coming from a place of distorted thinking. So this:
I like to think he is noble, as I’ve observed him to be with others, and is a funny, smart, confident guy.
Is your feelings making him out to be a better person than he really is, because then the ego-kibbles that he feeds you are worth more. Fortunately, you haven't totally lost touch with reality
But can’t be too noble if he is so interested in me and we both have kids and spouses.
No kidding!
But it's a good clue for YOU to figure out what you need, not because of who that man is (he's by definition a jerk if he's looking to married women for attention). Confidence is one's sense of their own power. I think it speaks to what you need that it's what you are drawn to. You don't have a sense of your own power and you probably think your husband doesn't have a sense of his own power. So build yourself up, and build up your husband. True confidence comes from clearly seeing one's value and one's contribution. If you see that, you'll see past his pseudo-confidence.
For your husband - it might help for a while to put him on a throne (so to speak). To show him that he is your highest priority, at any time of day or night. For a while I called and texted my husband regularly, hourly, to let him know I was thinking of him. Our sex life went into overdrive which means a lot to him. I slacked off at home, at work, in my social life so that I could be super attentive to him (it was fine, the kids ate a lot of takeout and had babysitters so we could be alone together or I could travel with him, I got by at work doing the minimum, the house was untidy. None of that matters when you are building your marriage, and I'm fixing up the other stuff now that my husband is more secure).
You'll figure out what your husband needs. He might want to talk about it a lot and know what you are learning and thinking. He might want to never talk about it and you can figure out how often to bring it up (weekly? monthly?) to let him know that he is still your priority, you are fixing yourself, and a strong marriage is what you want. I hope that he shares enough of his feelings and insecurities you can be both proactive and responsive - but if he doesn't, you can read about the BS/male perspective in general on SI to stay in the ballpark on how he's feeling and thinking.
Anwyay, enough for now :) The process with your husband may be messy. He may be hurt and scared, you may doubt why you are blowing things up, and you may wonder if you did the right thing. From where I'm sitting, it looks like you have a big opportunity to face up to and fix yourself without doing the horrible damage that a further EA or a PA can cause, so that's a blessing. And you have a lot going for you, as I said before. Honesty, not being defensive, finding support on SI, etc. I hope you can lean into it and use it as a chance to find a more emotionally intimate marriage and deeper self awareness.