That's not what forgiving yourself means. Forgiving yourself does NOT mean forgetting, just like it doesn't mean forgetting for the BS when they forgive the WS.
I was a horrible mother to my oldest child. I was borderline abusive a few times with him. It was all I knew, I was severely abused as a child myself. I knew it was wrong, but I didn't have the tools to change. So when I couldn't do better with the limited knowledge about how to deal with him that I had, I would take it out on him. I felt so bad for such a long time, I really hated myself for the way I treated him. I got help eventually, and I have worked very hard on myself. I have also forgiven myself for the way I treated my son.
That does not mean I don't still feel terrible about it when I think of it, or that I don't still feel shame or hurt over my own actions. It means I am able to view that as a part of my past that I have learned from and very much remember so that I don't make those same mistakes again. I am human, I made mistakes and still do today, but I have forgiven myself of those mistakes and no longer punish myself for them. That is vastly different than what you described.
In order for your WH to heal, he does need to work on forgiving himself and moving past his mistakes just as much as we BS's do. Many people argue that forgiveness isn't necessary to R, so be it. However living in a cycle of pain, punishment, anger, and grief isn't necessary either. Coming to terms with the past and learning from it is what matters. So if you don't want to be stuck in that cycle of pain, anger, punishment, etc., then you might want your WH to forgive himself also. My H cheated on me in part because he was living with the guilt and shame of cheating on an ex of his that he never resolved. You better bet your bottom dollar I wanted him to resolve that guilt and shame so he didn't do that to me again.