^Except that Dousan died because of sickness later in the novel. Who's to say that Imagawa would not be dead too, so that the 'flow of history will be balanced' as mentioned by Hanbei.
I still believe it is a dream though, and all it takes to send Yoshiharu back will be by killing him
