I think you need to confront your wife now. Yes, the healthcare professionals tell you that you must keep your wife from physical and mental stress. BUT, they are not walking in your shoes. Since you are the caretaker, this has got to be eating a hole right through you. You know your wife and how she might react. Do it in a way appropriate to your situation. It's possible she may be dying to tell you with her health situation being what it is. It appears she's noticed something is up with you as she's noticed you are appearing withdrawn.
Although I am a BH myself, my experience with your type of situation stems from my mother having an affair with her boss when I was in high school. I also have an older sister who cheated long term on her husband. Both affairs were basically rug swepted. This history had much to do with me divorcing my wife without any thought or reconciliation.
My mothers affair appears similar to your wife's as it seems to have gone on 3 or so years. Appears to have died on its own, except my dad found out about a year later at the end of my freshman year at the Air Force Academy. He was the last to know too, and his reaction towards the so called "friends" very similar to yours. My siblings and I were shocked when this came out. We knew something was up, but mom having an affair was the farthest thing from our minds. It was shocking to find out so many supposed close friends knew about it and were able to keep the secret.
You said earlier that your daughter noticed her mom being detached during her high school years. My mom missed my last half of my high school years. Football games, baseball games, I made very good grades and had a couple of chances at athletic scholarships. The best offer was an appointment to the Air Force Academy from my SAT scores. She barely took notice. Nothing like when my two older sisters went through. My younger sister who was an up and coming student in her own right was virtually ignored by our mother during this time. This did not go unnoticed by me or my little sister. Neither of us obviously had the life experience to understand, but we both knew something was not right. She was just too busy working all kinds of hours so we thought. My dad bought into it as well. Looking back, it seems so preposterous. My mom was a typest/file clerk. What could she possibly be doing working those hours on the programs she said she worked on? We know now.
My parents stayed together and put on an amazing front for years. However, my younger sister and I never truly rebonded with our mother. Mom never acknowledged her affair had any effect on me or my siblings. She just ignored what happened and tried to carry on as though nothing happened.
My moms health failed her about 5 years ago with breast cancer. My dad was forced into the caregiver role in her last year or so before she passed away in 2011. It had an awful effect on him having to be the dutiful husband even years after my moms affair ended. They never did anything to facilitate healing from her cheating. My younger sister and I remained detached from our mother and her legacy is forever stained in our eyes.
It appears your daughter is holding on to some animosity from those years and she doesn't understand why her mom was like she was. You know now, and it's eating you up inside. For both of your sakes and to give your a wife a chance to come clean while the opportunity exists, you need to confront her on this. Do it in a way you feel is appropriate for the situation. Cue your daughter in on it as well. It may answer some nagging questions in her mind too. At the same time it may preserve your wife's legacy for your daughter and future grandchildren. It may even make being caregiver a little easier for you if she is repentant upon confrontation.
Just a thought from the kids viewpoint of it.