Chaste? hardly, celibate maybe. He is oblivious to their special attraction maybe, but in hypocritical term, is charmed by them anyway.
It's not like she has zero chance, but when the protagonist wonder that she's beyond his reach, then her chance are definitely behind the rest (at best comparable to Rin and Cecilia). Unless this is lighthearted poor & rich romance fantasy, small chance it'll happen.
