Blake,
Wow. This matters. A lot. No, you don't have to let it define you, but it sure may be helpful in explaining a lot, so don't diminish it just yet.
Others have relayed much that I agree with, so I won't repeat their advice here. I am a CSA survivor, too, and one book I found very helpful is "The Courage to Heal: A Guide for Women Survivors of Childhood Sexual Abuse." (Bass and Davis) Yeah, I know it says "women," but there is tons here that could apply and you may find helpful. They define the term "abuse" very broadly, so regardless of the label, there is a lot of good stuff here - including things for your spouse to read, to help them understand you better. It's also good to read as you first start remembering things; it may help you work through some things on your own. Good fodder for IC.
Here are a few points regarding your post, that perhaps others haven't yet raised directly:
First, from what I can tell, your parents didn't give you a choice in this. They didn't say "hey Blake, you're the right age to learn about sex; would you like to talk about it now?" Rather, they waited until you were barely dressed and waylaid you, bringing you in without warning. If they were truly only teaching you, then why did they wait until you were most vulnerable (barely dressed)? Please don't underestimate the shock factor.
When I spoke to each of my three kids about sex, they came to me first with questions, and signaled they were ready to talk. I asked them if that is what they wanted. I didn't wait until they were in a state of undress themselves! Really, what was the point of that?
Second, are you sure that's all that really happened? That you're not blocking anything? You mentioned that you and your brother can't remember whole blocks of time. Keep the talks with him coming; you may piece together things that help.
Finally, and related to point two, this sounds like somewhat of a super-charged atmosphere, sexually. Please don't underestimate this impact, either. For example, once that boundary was hopped, you may next have thought it not a big deal to "play doctor" with an older male teenage neighbor because your parents inadvertently groomed you for it. As you replay Blake-as-slightly-older-teen, can you recollect anything else?
I am definitely not a "recovered memories" advocate, but I do think we naturally forget or minimize things as we move along through life. Kids can have a skewed way of viewing things.
I wish you well, and please continue to engage your wife in this; you may find her a big help. My WH has been a rock about my CSA stuff. It has really helped us bond sexually. His acceptance and comfort helped me build trust in him, believe it or not.