I was just wandering during a rewacht/read....
Why? Actually he was'nt so immune to allure in some situations before and after that moment. So maybe he tried to ignore it purposely, because it affect his vigilantness, his concentration or whatever.
Is there any hint or something in the novels or somewhere else??