Yeah it's impossible that the golden land is on that reef, unless you imagine that the underground passage connects to it.
According to Kawabata it was impossible to use a boat in such condition, using a small boat or swimming would have been a suicide, also Eva and Rosa would have come back completely soaked if they swam, and certainly I don't think there were not one but two boats for them to take a trip.
@Renall: There's the kanji of "person" there, but the word itself doesn't necessarily involves human beings only. This has been explained by Klashikari here:
http://forums.animesuki.com/showthread.php?t=82557
Also this is riddle, and riddles use metaphors.