Right, I didn't say Yoitsu was "nothing but trees" (though it may be only that now)
... only that whatever was there was destroyed centuries before. Whether there is anything there "now" (at the time of Horo and Lawrence) is, at yet, unknown though Lawrence seems to believe there is not much to be found. There may be nothing, there may be ruins, there may be a new town built on the old, there may be nothing but that which destroyed it all, but they're on their way to find out.