Ideally, what most people have said here is true: that there is no age limit in watching anime, and that the society's opinions shouldn't affect you.
However, when you are living in a place like Japan (10 years ago, at least), the so-called "social acceptability" can
actually matter. Taken from
this thread, which no doubt some of you have come across already, otakus (term used to describe hardcore fanatics, but is now used more often to describe anime/galge fanatics) actually risked
losing their jobs if they were to be exposed as an otaku.
This wouldn't be that big of a problem if society treated hardcore fans as the only form of anime-related otakus, but rather, society at that point treated anyone above the age of...20(?) who watches any other anime besides
Sazae-san and
Doraemon to be an otaku. It IS that scary, and the fact that there was a child-sexual abuse criminal who was identified as an otaku (Whether him being an anime-otaku being true or not, being unknown), made the impression/stereotype of otakus to fall further into the pits, as if hentai/galge didn't make that bad enough.
In the eyes of the average human being (lets face it, the negative stigma attached to anime isn't exclusive to Americans), anime is either seen as:
1) Childish - just because it is animated, or the mentality that anime and cartoons are no different, even though as
Vexx has mentioned, Looneytoons (or whatever their original names were) were originally made for adults in the first place.
2) Porn - Hentai, galge (A.K.A ergo-ge/erotic games, gal-games, or the outside-Japan definition - hentai game),
and even ecchi. As if in denial, most people who aren't anime fans will hop on the bandwagon and start writing petitions for banning anime when they happen to come across the mentioned items, and choose to ignore other examples that fans present to them. Since the average person makes up for majority (And majority = win), they will usually have their way. (Albeit not to the extend of banning anime altogether, but something negative enough)