Anger is normal and you have to go through it.
For me once the shock wore off and the "fight" to keep my husband was gone - I was pissed.
Pissed that he had put us in this situation, pissed he hurt me, pissed he disregarded me and our family. Just pissed for having to deal with it all.
So you are right on target with the anger. Let it flow.
Even if he is sorry he also needs to understand the magnitude of emotions you are feeling due to his horrible choices.
Don't beat yourself up for it. It is okay. Deep breaths and keep moving.
We are rooting for you.