You might try local country bookstores that carry japanese novels (like Kinokuniya Bookstores - they're scattered about the US). That might include comic shops that specialize in imports.
Other online possibilities include
www.nippon-export.com (they work in English) and if you email them directly, they'll often hunt up what you need.
Its actually not terribly difficult to navigate amazon.co.jp as long as you're very familiar with the Amazon ordering process and use the "in english" buttons. If you can find the title in japanese, then just cut'n'paste it into the search field and that will usually get you the results you want.
That's what I used to do before I figured out enough japanese and how to type in it to expedite the process (even if you type in Japanese and let the system "auto-kanji" sommmmmetimes it picks the wrong kanji so ymmv).
Frankly, I just keep a text file of the titles I'm interested in to cut'n'paste for speed.