This business is my retirement plan. It's been in the works for many years and I've put so much time, labor and money into it. It's like putting money in a savings account that will earn high interest but only if you keep it for enough years without disturbing it. I can't carry it alone, either.
You may have some tough decisions ahead of you, one being removing yourself from any hint of a personal relationship and keeping things just business, only business, and as others said, buying him out or whatever. Perhaps you can split up the properties as equitably as possible, but ask an attorney first.
Mr. Creepy Creep might get all agitated if you bring this up to him out of the blue or if he thinks you're lowering the boom.
In addition to seeing an attorney since your assets are mixed with his and you're not married, you might want to consider a forensic accountant if need be, AND look into seeing the pros and cons of having all the investment properties etc in a LLC. Protecting yourself from being named in a harassment suit is super important too.