STBXH has showed up at my workplace with DD recently, more than once.
So yes, I go to great length not to tell them where I'm going. I'm an open-book person, so learning to be covert is yet another core value that he is forcing against what I believe.
What's also hard is the things he's done have some aura of being "unsafe", so when people don't have actual experience in it, they immediately think he could be physically unsafe. I don't think so, I think he's too cowardly.
Yes, it's really, really hard because we a trusting Bs's are easier to catch off guard. I'm learning, though, this and many other life lessons that I never would have thought I would need to know.
I'm sorry he followed you to the library and disturbed your time with DD and what you were trying to accomplish.
I've been bullied by STBXH about "interfering" and "meddling" when he is angry, yet he is free to spend hours of their visits texting me and taking up my time!
I think figuring out how to not tell him your locations are a good idea.