Well, my only possible choice is having ByakuShiki not letting him at all, as in, you can't remove it without breaking its frame and hurt Ichika, and the some.
Suck to be too serious, I wish I could just use 'harem plot, bear with the nonsense' but no... not my style.