My WH has done the same and it is devastating. Is there any way you could do telephone counseling or take the baby with you to IC? I've taken my daughter with me before when I didn't have childcare (she was 6 months old the last time I took her). It is well worth trying to do that so you can go.
Also, what is your situation like? Have you confronted your husband? Have you been tested for STD's, etc.?