The lolita-fashion movement has its direct origin in Isao Kaneko's
PINK HOUSE, which has had nothing with otaku culture by itself. It's true. But it is also fact that some of the earliest female otakus in the 80's enthusiastically accepted the brand, and even today both male and female otakus cherish a strange love for the descendants of Kaneko's works. It's not wonder if a person, whether a native Japanese or not, thinks the lolita belongs to the otaku world.
Eiji Otsuka once critically studied the relationship between lolitas and otakus; see,
Mental History of Otaku: Analysis on the 1980's, pp.139-146; and
Folkloristics of Girls, if you please.
