Finally found what I was looking for. But re-reading it, perhaps I did read too much into the dialog.
I don't know if I can put up manga pages, and since I can't be arsed uploading images anyway, I'll just point out that it was in chapter 281, page 2. Yakumo explains to the 1-D girls about the night she confessed. She says Harima avoided what she wanted to say (which to me sounds intentional), then goes on to say that made her think they don't 'harmonize' well (she knows there's a gap there).
It doesn't say he avoided her words in order not to hurt her, as I 'ad-libbed(sp?)' before
, but I don't see why that can't be the case, especially since he intentionally avoided her words.
Of course now I'm in a somber mood, having re-read those chapters. Also re-realising that Harima never called Yakumo by her name, it's always Imouto-san (at least I believe this is the case). Which in itself is a big indicator of how Harima feels about Tenma((and consequently Yakumo).
And of course, I read chapters 282 and 283.....
