I used to watch Sailor Moon as a kid, and back then, you couldn't tell anyone (except my best friend who was also into it for some reason).
A few months back I had a fit of nostalgia, so I started downloading SM and rewatching it, but being in college now, and having a roommate, I had to be pretty discreet. Whenever he came into the apartment I had to close the Windows Media Player window pretty awkwardly. I think he thought I was jacking off to pr0n.
On a more general note, I think anime itself is a guilty pleasure. It makes me cry knowing that I'm watching at best B-grade material when I could be reading some intellectual novel and expanding my mind. What is perhaps even worse, is realizing that due to an uneventful and uninteresting life, I must rely on anime as a form of escapist entertainment to provide the material for my own autistic social fantasies. It's all very depressing when you think about it that way, but it doesn't seem to stop me from enjoying shows like Karin, Shakugan no Shana, and KGNE.