There's nothing wrong with looking at things critically; having fun and using your brain aren't mutually exclusive. I mean, I tend to be very critical and break everything down in my reviews, but that doesn't mean I don't do of the silly "omg she's so cute/cool" stuff that people do too.
I am also a very casual fan; I haven't even watched that many anime and don't really care either.
The only problem is when something insults your intelligence and it stops you from enjoying it on either level. That's when it's bad.